How do we decide how long a person stays in jail? Which teachers get fired and which get to keep their job? How is our credit rating linked to the ads we see online and the amount of money we pay for car insurance? There’s one answer for all these questions: Algorithms. There are thousands of mathematical models built that track each and every one of us, putting us in neat little boxes for purposes of safety, advertising, and a dozen other broad causes. But are those algorithms always fair, or even coherent? Cathy O’Neil is here to argue that many of the algorithms running invisibly in the background of our day-to-day lives are ‘weapons of math destruction’, vicious computer codes that can destroy lives as a byproduct of completely failing to measure the thing they were created to measure.
If you’re a fan of Freakonomics, Weapons of Math Destruction is for you. As with Dubner and Levitt’s popular book, O’Neil finds a way to take a complex subject most of us know little about – economics there; math here – and makes it approachable, easy-to-understand, and entertaining. She does so by grounding her examples not in formula after formula, but in anecdotes and stories that explain the outcomes. In one page, O’Neil can take vast swaths of data, find a common thread within them, and explain how that thread affects all of us. Chapter 5, for example, details the intersection between Big Data and criminal justice, and O’Neil introduces us to crime prevention software like PredPol, meant to help predict ‘high crime’ areas so the police can better allocate their resources. On its surface, it seems like a strong, functional program, but O’Neil – using her key components for a ‘weapon of math destruction’ – lays out succinctly how easily the system breaks. If more cops go to one area, there will be more arrests for minor crimes like drinking on the porch. Those minor crimes, which happen everywhere, get fed back into the system, which reads only that area as being even more dangerous, which demands more police, which fuels more arrests. Rather than predicting and preventing violent crime, it’s a system for arresting people in specific area codes–which, she later demonstrates, can have a dramatic effect on recidivism calculations, insurance rates, and more. These sorts of vicious cycles are at the heart of many weapons of math destruction.
As with Freakonomics, however, the book is only as strong as the stories it finds to tell. Most chapters of Weapons of Math Destruction are quite strong, but there are a couple arguments, specifically towards the end of the book, that felt a little loose and disjointed from the urgency of the early chapters. It was easy to see the urgency of reforming the algorithms that led to the 2008 housing crisis, or the ones that cost people their jobs because another teacher – who was rewarded – cheated. It’s harder to see the urgency behind some of the ‘almost-WMDs’, behind things like political mailers being micro-targeted more efficiently. As the examples become looser, I think the structure of the chapters and the arguments within them become a little sloppier. Similarly, a few of O’Neil’s examples require accepting assumptions about reality that you may not agree with, such as blaming the massive increase in college tuition costs on ever-expanding facilities, something much of the data doesn’t necessarily support. O’Neil does pick up for the book’s final chapters, though, which offer some ideas as to how we can move forward more efficiently. After one or two down-chapters, it was nice to end on a strong note.
Still, despite some flaws, Weapons of Math Destruction is a timely book that introduced me to a complex but vital issue in a reader-friendly manner. O’Neil has a great talent for explaining complicated concepts without ever coming across as lecturing or dull, and that talent made it hard for me to put this book down. It is neither a secret nor a surprise to learn how much algorithms rule our day-to-day lives; what is surprising is how poorly built so many of them are. I went into this book knowing next to nothing about the subject, and left it wishing there were twice as much here. That, to me, is the sign of a great nonfiction book: So good, it makes me want to dive in even deeper to a subject I normally wouldn’t think twice about. As the Cult of Big Data gets stronger and stronger and the desire to believe that algorithms can solve all our problems becomes more and more prevalent, O’Neil’s book may just seem far ahead of its time.
Alexander Morrison is a writer and educator in the Midwest. He divides his time pretty evenly between reading, writing, film, and Overwatch, so you can tell he’s pretty well rounded. You can read his thoughts about love & sex in pop culture at Cinema Romantique, or follow him on Twitter at Ikiruined.
A personal copy of the book was used for this review. No review copies were provided.