however, this doesnt seem uniform with all prisoners from what i see.
From Wikipedia: Under US federal law, all physically abled inmates who are not a security risk or have a health exception are required to work, either for UNICOR or at some other prison job.
There are state prisons and federal prisons, which explains why there are so many differences across the country. Each state has different rules and practices.
who gets assigned to where and how?
As I mentioned, it will vary by state or if it's federal prison. Let's look at Georgia (because as a visitor to the country I saw a fucking chain gang working on a road and that sure left an impression).
It is the responsibility of the Warden/ Superintendent, assisted by designated staff members as appropriate, to evaluate properly each newly arrived inmate as to his (her) work capability and capacity prior to his (her) assignment to a work detail. Work classification factors shall include, but not be limited to, consideration of all available data concerning the inmate's physical and mental condition, attitude, security rating, the nature of his (her) crime, the length of his (her) sentence, his (her) treatment plan, and his (her) personal preference if job options for which he (she) is eligible are available at the institution. The inmate's work assignment shall be determined by the data developed, including a personal interview, and shall be circumscribed as directed by the institutional physician based on the inmate's medical condition.
Inmates may work on private property in situations in which the Commissioner deems a project to be public work, i.e., a project in which the sole benefit flows to the State, or in the case of a natural disaster or a nuclear emergency. Determinations of a public work shall be made on a case-by-case basis. Participation in response to natural disasters and nuclear emergencies shall be in accordance with approved plans.
Alabama (for example) allows prisoners to work for private companies, either outside of the prison or via prison "in sourcing". Worker selection is along the same lines.
are they just selecting specifically black people in these types of programs?
It is not limited by race. One of the reasons it is considered an extension of chattel slavery practices, is the now "ended" convict leasing system. This system gave financial incentive to have a large prison population. Combined with just plain white supremacy, this was a reason for criminalizing otherwise innocuous behavior (loitering, morality laws) and how they are disproportionately applied to black people.