COMPAS (software)

• 1. Introduction

  ›

  COMPAS is short for Correctional Offender Management Profiling for Alternative Sanctions. Courts in New York, Wisconsin, California, and Florida's Broward County have used it to weigh a defendant's odds of committing another crime. A private company built the algorithm behind that risk score, and judges have leaned on it during sentencing and pretrial release decisions. What goes into that number, and who is allowed to examine it, are two of the questions this program follows. A third is why two respected studies of the same software reached opposite verdicts.

• 2. Origins And Design

  ›

  In 1998, Northpointe, Inc. created the software that would become COMPAS. The company merged with other justice technology firms in January 2017 to form equivant, which owns the tool today. Analysts classify it as a fourth-generation risk assessment instrument. That category weighs fixed data, such as a person's prior criminal record, alongside dynamic factors researchers call 'criminogenic' needs. Those needs include a person's current social environment and employment status. COMPAS was first built for correctional rehabilitation planning, but its use spread into judicial sentencing. The Wisconsin Department of Corrections adopted the software statewide in 2012, a decision that later produced the legal challenge known as Loomis v. Wisconsin. That statewide rollout would put the software's inner workings before the Wisconsin Supreme Court within a few years.

• 3. Three Scales Of Risk

  ›

  Northpointe built three separate scales: one for pretrial misconduct, one for general recidivism, and one for violent recidivism. Each draws on a different mix of information. According to the research behind the pretrial scale, a person's current charges, pending charges, and prior arrest history carry particular weight. Previous pretrial failure, residential stability, employment status, community ties, and substance abuse also feed into that score. The general recidivism scale is different. It is calculated after someone has already been through a COMPAS assessment. It draws on a person's criminal history and associates, drug involvement, and any record of juvenile delinquency. The violent recidivism scale looks at a different set of factors. It considers a person's history of violence, history of non-compliance, and vocational or educational problems. It also factors in their age at intake and their age at first arrest. Those pieces combine through a formula that multiplies each factor by its own weight.

  Northpointe explains how those weights are set. Each one is 'determined by the strength of the item's relationship to person offense recidivism that we observed in our study data.' The COMPAS Practitioner's Guide describes the underlying constructs as being 'of very high relevance to recidivism and criminal careers.' That question, whether a weighted formula beats a judge's own instincts, is what drew courts to tools like this in the first place.

• 4. Why Courts Turned To Algorithms

  ›

  Judges are more likely to hand down a lenient decision right after eating a meal, a pattern researchers call the hungry judge effect. Supporters of tools like COMPAS argue that algorithms can correct for exactly that kind of inconsistency, alongside other predictable biases in judges' reasoning. The case for using risk assessment software also rests on a desire for objective, evidence-based sentencing and a more efficient court system. Other approaches to assessing risk exist, though they have proven difficult to implement in practice. Whether that argument survives contact with the algorithm's own inner workings is a separate question this program takes up next.

• 5. The Black Box Problem

  ›

  COMPAS's algorithm is a trade secret. The people it scores are not allowed to examine how their number came about. Researchers have described that opacity as a violation of due process. They have also found that simple, transparent algorithms can predict about as accurately as COMPAS's proprietary formula. Without access to Northpointe's code, analysts have relied on the publicly available questionnaires instead. They have also built their own reverse-engineered approximations of the software from publicly available data.

  The related LSI-R algorithm illustrates a real risk of relying on data-driven scores. Its initial version was trained mainly on Caucasian offenders, which left it less valid for black and Latino offenders. Machine-learning systems generally are only as good as the data that trains them, so when the underlying data is biased, the results are too. Researchers note that algorithms can carry other forms of bias, though these draw less attention because racial bias dominates the debate.

  Critics have gone further, arguing that COMPAS risk assessments violate the 14th Amendment's equal protection guarantee. They contend the algorithm is racially discriminatory, produces disparate treatment, and is not narrowly tailored to its purpose. Those equal protection claims set the stage for an empirical test of whether the algorithm actually was racially biased.

• 6. Two Definitions Of Fairness

  ›

  In 2016, ProPublica published an investigation that found COMPAS was racially biased against black defendants. Northpointe countered that the algorithm predicted recidivism accurately regardless of race. Both claims turned out to be true, because they rested on two mutually exclusive definitions of fairness. ProPublica measured the rate of classification errors, meaning how often the software's prediction about a defendant turned out to be wrong. Northpointe measured the accuracy of prediction, meaning whether the algorithm treated all defendants equally. A tool that is fair by one of those metrics will be biased by the other.

  A 2018 study by Dressel and Farid compared COMPAS against people with little or no criminal justice background. It found the software somewhat more accurate than individual predictions, but less accurate than the judgment of grouped predictions. A later review pushed back, writing that these results "seemed like a specific occurrence and less reflective of general and real conditions." That review found algorithms performed better than humans once conditions were closer to the real world. A replication study, for instance, found the algorithms did better when the base rate of rearrest was low. Dressel and Farid, by contrast, had assumed recidivism and non-recidivism were about equally likely.

  Risk assessment tools do not explicitly incorporate race into their calculations, since doing so would likely violate the US Constitution. Factors such as education level or employment status are correlated with race, though, so algorithms that use them still produce different results across racial groups.

  A 2024 analysis of COMPAS's practical impact in Broward County tested one of the tool's proposed benefits directly: a reduction in incarceration rates. The analysis found that COMPAS's use lowered rates of confinement across demographic groups. It also found that the tool widened the gap between racial groups. Those numbers set up the next question directly: how the Wisconsin Supreme Court would rule on whether judges could keep relying on scores like these.

• 7. The Court Weighs In

  ›

  In July 2016, the Wisconsin Supreme Court ruled on the question directly. Judges could still weigh a COMPAS score during sentencing, the court said. But that score now had to come with a warning spelling out the tool's 'limitations and cautions.' That requirement means every score a Wisconsin court sees today arrives with a warning attached. It has to spell out exactly what the number can and cannot tell a judge.