Short answers, pulled from the story.
Fairness in machine learning refers to efforts to detect and correct algorithmic bias in automated decision-making systems. Decisions made by such models can be considered unfair when they are influenced by sensitive characteristics such as gender, ethnicity, sexual orientation, or disability. The field grew sharply after 2016, partly in response to public debate over racially biased risk assessment software used in US courts.
ProPublica's 2016 analysis of COMPAS, a recidivism prediction tool used in US courts, found that black defendants were almost twice as likely as white defendants to be incorrectly labeled as higher risk. White defendants were more often mislabeled as lower risk when they went on to reoffend. The maker of COMPAS, Northepointe Inc., disputed the findings; ProPublica refuted that challenge.
The three main criteria are independence, separation, and sufficiency. Independence requires that predictions be statistically independent of sensitive attributes. Separation allows correlation between predictions and sensitive attributes only when the true outcome variable justifies it. Sufficiency requires that individuals with the same predicted outcome have equal probability of that prediction being correct, regardless of group. Satisfying all three simultaneously, called total fairness, is mathematically impossible in most real-world settings.
In 2015, Google Photos mistakenly labeled a black couple as gorillas, and Flickr's auto-tag feature labeled some black people as apes and animals. A 2018 study of three commercial gender classification systems found all three were most accurate for light-skinned males and least accurate for dark-skinned females. In 2020 Twitter's image cropping tool was shown to prefer lighter-skinned faces, and in 2022 the creators of DALL-E 2 acknowledged that its generated images were significantly stereotyped by gender and race.
Bias mitigation uses three main approaches: preprocessing, inprocessing, and postprocessing. Preprocessing modifies training data before model training, for example by reweighing data points so weighted discrimination across groups falls to zero. Inprocessing adds fairness constraints to the training objective, such as requiring equal false positive rates across groups. Postprocessing adjusts a trained model's output thresholds to equalize error rates across groups, sometimes using a ROC curve to find the right setting.
Mathematical fairness definitions require placing individuals into predefined social groups, which can be too coarse to capture how discrimination actually operates. Different fairness criteria are often mathematically incompatible, making it impossible to satisfy all of them at once. Human operators can also undermine an algorithm's designed fairness by accepting its recommendations only when those recommendations align with their own existing biases, a gap that technical solutions alone cannot close.