Cathy O’Neil worked in quantitative finance before this, which matters, because she’s not writing this as an outsider throwing rocks at big data from a safe distance, she’s writing it as someone who helped build these systems and watched them go wrong from the inside. Her acronym for the harmful ones is WMDs, weapons of math destruction, and the pitch is straightforward: any model can be biased, but the dangerous ones are opaque, at scale, and have no feedback loop to correct themselves when they’re hurting people.

I went in expecting a broadside against big data generally and it isn’t that, it’s more careful than that, she draws a real distinction between models that help and models that quietly punish, teacher-evaluation algorithms that fire good teachers based on noise, recidivism scores that launder existing bias into a number that looks objective, hiring software that screens out anyone who doesn’t resemble the people already in the room. The examples in the first couple chapters are strong enough that some later chapters can’t help but feel like variations on the same theme rather than new ground.

What stuck with me most is her point that data isn’t neutral just because it’s data, algorithms encode assumptions whether or not anyone admits to making them, and “objective” often just means “opaque enough that nobody can argue with it.” That’s the sentence I think about most, honestly, more than any individual case study in the book.

This should be required reading for anyone whose job touches models that make decisions about actual people, and I say that as someone who isn’t in that field and still found it clarifying. It runs a little repetitive by the back half, and the tone tips openly into advocacy in places, which some readers will bristle at and I mostly didn’t mind, given how well it’s earned.