Study: Risk Assessment Software Twice as Likely to Mislabel Black Defendants as High Risk

risk assessment 1African-Americans in the U.S. criminal justice system, already burdened by the long history of bias against them, face yet another barrier to equality: risk assessment software.

Institutions across the nation are using computer programs to predict recidivism, an offender’s return to crime following punishment for a prior offense. Each offender receives a score used to help justice officials determine bond amounts, sentence lengths, and rehabilitation needs. The assessments are currently reviewed by judges in nine states during criminal sentencing, according to Propublica, a Manhattan-based, non-profit news organization.

Investigative reporters examined the algorithm developed by popular research and consulting firm, Northpointe Inc., and found that Black defendants were almost twice as likely to garner future criminal predictions as white defendants.

Northpointe scores are calculated with the help of a survey, which includes questions such as “Was one of your parents ever sent to jail or prison?” and “How many of your friends/acquaintances are taking drugs illegally?” Not one of the 137 questions asks the race of the defendant. The scores predict whether the defendant will be arrested again in the next two years.

The journalists collected risk scores assigned to over 7,000 offenders arrested in Broward County, Florida, in 2013 and 2014 to see how many actually went on to commit crimes within two years.

Unsurprisingly, the results yielded significant racial inequities.

Black defendants who did not go on to commit new crimes were mislabeled as high-risk 45 percent of the time (23 percent for whites).

White defendants who did re-offend were incorrectly labeled low-risk 48 percent of the time (28 percent for Blacks).

Black defendants were twice as likely to be mistakenly flagged as high-risk for violent crimes. And 63 percent of white violent re-offenders were wrongly classified as low-risk for violent recidivism.

Even when controlling for prior and future crimes, age and gender variables, Black defendants were 77 percent more likely to be assigned higher risk scores than white defendants.

Northpointe contested the results, attacking the method of analysis in a letter to the publication.

“Northpointe does not agree that the results of your analysis, or the claims being made based upon that analysis, are correct or that they accurately reflect the outcomes from the application of the model,” the company said.

Experts look to recidivism as the hallmark of national crime rates in any country. It is one of the most fundamental concepts in criminal justice, according to the National Institute of Justice. The U.S. government has devoted millions of dollars to researching the phenomenon.

Propublica‘s findings go against the Bureau of Justice’s widely reported statistic that over half of state prisoners return to prison within five years of their release. Other researchers have tested the BJS data on recidivism.

A 2014 study led by William Rhodes, lead economist at public policy firm Abt Associates, found that following incarceration, two out of three released inmates never return to prison, and those who do represent only 11 percent of the prison population.

Rhodes attributed the discrepancy to a failure to control for the overrepresentation of repeat offenders in prisons. The scientist made the analogy to a local mall surveyor. These surveys overwhelmingly reflect the opinions of people who regularly visit the mall as opposed to someone who may visit once every few months.

“Offenders who repeatedly return to prison are like frequent mall visitors – they are overrepresented in samples used to estimate the rate at which offenders return to prison,” Rhodes wrote in a piece for USApp- the London School of Economics’ daily blog on American politics and policy.

“If a statistician fails to weight his statistics to correct for this overrepresentation, offenders will appear to be highly recidivistic: One of every two will return to prison within five years. If the statistician correctly weights her statistics, offenders appear less recidivistic: Two of every three will never return.”

Back to top