How opaque models make consequential decisions — and why 'the math' is not neutral.
Algorithms increasingly decide who gets a loan, a job interview, a longer sentence, or more police attention. O'Neil calls the harmful ones 'weapons of math destruction': opaque (you can't see or contest the logic), scaled (one flawed model judges millions), and self-reinforcing (their predictions shape the data that trains the next version). The deeper point is that a model encodes choices — which data, which proxy for success, which errors are tolerable — so 'let the data decide' often means 'let past inequities decide, with a veneer of objectivity.' The literacy goals: demand transparency and the right to appeal, ask what the model optimizes and who bears its errors, and resist the authority-by-mathematics that makes a contestable judgment feel like settled fact.
Investigate with the AI detective