When models forecast crime and risk, whose past becomes whose future — and the feedback loops that bake in bias.
Predictive policing uses historical crime data to forecast where (or who) to police; risk-assessment tools score defendants for bail and sentencing. The documented problem is the feedback loop: if past policing over-targeted certain neighborhoods, the data reflects enforcement patterns, not ground-truth crime — so the model directs more police there, generating more recorded incidents, which 'confirms' the prediction. O'Neil's framework names why these are dangerous: opaque (you can't see or contest the logic), scaled (one model judges many), and self-reinforcing. ProPublica's investigation of the COMPAS risk tool found racial disparities in its error patterns (a finding the vendor disputed — the methodological debate itself is instructive). The point is not that data is useless but that 'let the algorithm decide' can launder historical injustice as mathematical objectivity. Accountability — auditing, transparency, the right to appeal a machine's judgment — is the antidote.
Investigate with the AI detective