Simpson Says

In an interesting, if unavailing, Boston Globe op-ed, Aubrey Clayton offers a valuable contribution to the discussion of the racism of police killings. As it’s kind of a big subject at the moment, with nearly everyone who’s math challenged certain that police are slaughtering black people, it provides some worthwhile concerns.

Unfortunately, Clayton opens by begging the question.

There is overwhelming evidence of racial bias in the criminal justice system, in everything from policing to sentencing. Nonetheless, the ongoing protests against racism and police brutality have prompted a familiar, fallacious reply from armchair statisticians in op-eds, social media, and police departments: that racial bias in the use of force by police is a myth, easily debunked with statistics.

By using the conclusion to open, one might assume there’s no particular reason to read further, since the outcome is already a certainty, proved by a column by Radley Balko that states:

In any case, after more than a decade covering these issues, it’s pretty clear to me that the evidence of racial bias in our criminal justice system isn’t just convincing — it’s overwhelming.

Seems mathematically conclusive, and, as Clayton asserts, all the studies to the contrary are fallacious.

Could the reaction to high-profile killings like those of Breonna Taylor and George Floyd be a matter of confirmation bias? Could the narrative of police racism be disproved with a tweet-sized calculation?

No.

Who could have seen that “no” coming? But aside from the absolute certainty presented, what basis is there to back it up? Simpson’s Paradox.

Math is hard. Statistics are hard and boring. Still, if we’re to credit them when some wag challenges your incredibly emotional anecdote from which you demand the extrapolation that everything is racist, it’s worthwhile to consider some of these terribly boring details.

The key point is that not all encounters with police are equally deadly. In any given kind of encounter with the police, a Black person can be likelier to be killed than a white person even if the overall rate of deaths per encounter appears lower for Black people. This would happen because Black people have many more interactions with police in non-deadly situations — a dynamic exacerbated by racism. And all those extra encounters dilute the rate.

On the one hand, there is a valuable point here, there are millions of interactions between police and the public, and the infinitesimal number of people physically harmed or killed as a result, But that’s just one piece of the puzzle. Some interactions involve police investigating serious crimes and some involve police tossing black kids to make their numbers. They are not equivalent interactions. Some involve cops being professional. Some involve cops being animals. They are not equivalent interactions. Some involve people grabbing a cop’s taser and running and some involve people complying and being beaten anyway. They are not equivalents. Well, you get the point.

Consider two extremes of police encounters: traffic stops and active shooter scenarios. Suppose, hypothetically, that a white suspect is killed by police in one out of 100,000 traffic stops and nine out of 10 shootings. And imagine that Black suspects are killed by police after 20 out of 1,000,000 traffic stops and in 10 out of 10 active shooter incidents. In each kind of incident, Black suspects are killed more often than white suspects. In aggregate, though, the percentage is higher for white people: 10 out of 100,010 white people are killed vs. 30 out of 1,000,010 Black people, because the white people tend to encounter the police in more grave situations.

Clayton recognizes that these aren’t the only scenarios where cops interact with the public, but nonetheless employs Simpson’s Paradox to dismiss studies that don’t support her conclusions.

That’s why one study, frequently cited as evidence that Black people are killed just as often (or less often) as others in similar situations, has been critiqued by other researchers who noted that “its approach is mathematically incapable of supporting its central claims.”

While some researchers critiquing a “frequently cited” study doesn’t help in determining whether the study or the critique is right, it does alert us to the fact that empirical studies, whether or not they confirm our bias, can be questioned. It’s curious that Clayton neglect to mention Harvard’s Roland Fryer study, but it didn’t make the cut.

Empiricism, real or imagined, has become a critical weapon in the battle of proving our priors, so people latch on to stats and raw numbers, to be used in whatever ways confirm what we’re absolutely certain needs to be confirmed, even though most of us couldn’t tell a good study from rape culture at dog parks. Statistics are hard, and accounting for valid variables is even harder.

On the other hand, without empirical analysis, we’re left with anecdotes from which we’re asked to induce that this one sad story, assuming it’s been told accurately which might ask too much of the story teller, is representative of most, even all, stories and should therefore guide our policy views on the entire subject.

That, of course, is unhelpful, as each side tells the story it wants to use to manipulate our emotions and create the frame of reference for us to consider what follows. If it’s a bad cop story, cops are bad. If it’s a bad perp story, perps are bad. Neither story does much to inform us about how to guide our views. Worse, we tend to only read those stories that appeal to our bias, so we read 1000 bad cop stories and believe, with absolute certainty, that bad cops are the overarching problem. They are, of course, but not for that reason.

So how do we avoid Simpson’s Paradox, not to mention the fact that statistics is hard? Every situation is, to some extent, sui generis. Even if there are a thousand images of cops being needless violent toward protesters, or looters grabbing the leftovers at Macy*s, it doesn’t answer the question of whether Derek Chauvin’s knee on George Floyd’s neck was the cause of death.

I’m not good with stats, and math was never my strength. But I care about facts, one case at a time, and try not to deny facts that fail to align with the fuzzy normative beliefs I have lurking around in the back of my head. Rather than do mental gymnastics to prove what I believe to be true, I accept the premise that while others better equipped to fight over empirical analysis will reject each other’s analysis, I will limit my purview to the facts of each instance of police interaction and do my best to determine what happened in that instance and what, if anything, can be learned from it.

Even though I’ve had decades longer than Radley dealing with these issues, and I’ve no doubt that pernicious racism exists and is reflected in various ways and at various points throughout the system, nothing is “pretty clear” to me, no less overwhelming, about it when any particular scenario unfolds.