"Harvard Study on Police Shootings and Race Offers Shocking Conclusion"

Harvard economist Roland Fryer Jr. published an article in The National Bureau of Economic Research (NBER) with the title: “An Empirical Analysis of Racial Differences in Police use of Force.” This article was published in July 2016 in the midst of a heated debated about race and policing in America.

From the paper’s conclusion:

...this paper takes first steps into the treacherous terrain of understanding the nature and extent of racial differences in police use of force. On non-lethal uses of force, there are racial differences – sometimes quite large – in police use of force, even after accounting for a large set of controls designed to account for important contextual and behavioral factors at the time of the police-civilian interaction. Interestingly, as use of force increases from putting hands on a civilian to striking them with a baton, the overall probability of such an incident occurring decreases dramatically but the racial difference remains roughly constant. Even when officers report civilians have been compliant and no arrest was made, blacks are 21.3 (0.04) percent more likely to endure some form of force. Yet, on the most extreme use of force – officer-involved shootings – we are unable to detect any racial differences in either the raw data or when accounting for controls.

This study spoke directly to our nations anxieties about race and policing in the wake of the highly visible police shootings of Philando Castile and Alton Sterling and the vocal protests of the Black Lives Matter movement and others. Unsurprisingly, this result was picked up by major news outlets including the New York Times with the headline "Surprising New Evidence Shows Bias in Police use of Force but not in Shootings." A Breitbart headline: "National Bureau of Economic Research: ‘No Racial Differences’ in Officer-Involved Shootings." On my Facebook feed, I saw the click-bait version of this event: “Harvard Study on Police Shootings and Race Offers Shocking Conclusion.” From the body of that Tribunist article:

The results? Not what Black Lives Matter would have you believe. The study found no indication of racial bias associated with incidents in which cops fired their guns. The study concluded that police officers who had not been attacked were more likely to shoot white suspects. This goes completely against the mythology.

Productive social policies are dependent on having a clear and data-informed view of the world, so what is the significance of this result? Here are my thoughts:

1. The disproportionate use of non-lethal force against African-Americans is as problematic and disturbing as a disproportionate use of lethal force against African-Americans. The paper found that a black person was 24% more likely to have a weapon pointed at them than a white person in the same situation and were 18% more likely to be pushed to the ground. That’s a harrowing result. Ultimately, Black Lives Matter is a group of people who are currently alive and speaking out about their experiences with police that didn't end in them being killed so Fryer's results certainly vindicate that narrative.
2. It's a well known and easily verifiable result that a black or Latino person is far more likely to be stopped and searched by police than a white person. More confrontations with police means a greater likelihood of having a deadly encounter with police, so the problem of disproportionate police stops is a critical component to understanding the racial discrepancies in lethal police encounters. Fryer's article makes no mention of this consideration and as a result, provides a very incomplete picture of the reality on the ground. According to a Center for Constitutional Rights report titled “Racial Disparities in NYPD Stops-and-Frisks,” the NYPD initiated nearly 1.6 million stops of New Yorkers between 2005 and 2008, and a black or Latino person was about 13 times more likely to be stopped and frisked by police than a white person. Other dramatic reports about this disparity can be found here and here. In the case of Philando Castile, the New York Times reported that he was pulled over forty-nine times by police before his deadly encounter. South Carolina Senator Tim Scott gave a deeply personal speech on the Senate floor in which he describes having been stopped seven times by police in the course of one year as an elected official (NPR).
3. I don't think Fryer's conclusion is compelling and I agree with Harvard Social Scientist Justin Feldman who completely rejects and dismantles Fryer’s conclusion in a Harvard faculty blog post here. Some excerpts: 

Fryer’s analysis is highly flawed...It suffers from major theoretical and methodological errors, and he has communicated the results to news media in a way that is misleading. While there have long been problems with the quality of police shootings data, there is still plenty of evidence to support a pattern of systematic, racially discriminatory use of force against black people in the United States. 

There should be no argument that black and Latino people in Houston are much more likely to be shot by police compared to whites. I looked at the same Houston police shooting dataset as Fryer for the years 2005-2015, which I supplemented with census data, and found that black people were over 5 times as likely to be shot relative to whites. Latinos were roughly twice as likely to be shot versus whites. 

The method that Fryer employs has, for the most part, been used to study traffic stops and stop-and-frisk practices. In those cases, economic theory holds that police want to maximize the number of arrests for the possession of contraband (such as drugs or weapons) while expending the fewest resources. If they are acting in the most cost-efficient, rational manner, the officers may use racial stereotypes to increase the arrest rate per stop. This theory completely falls apart for police shootings, however, because officers are not trying to rationally maximize the number of shootings. 

Economic theory aside, there is an even more fundamental problem with the Houston police shooting analysis. In a typical study, a researcher will start with a previously defined population where each individual is at risk of a particular outcome. For instance, a population of drivers stopped by police can have one of two outcomes: they can be arrested, or they can be sent on their way. Instead of following this standard approach, Fryer constructs a fictitious population of people who are shot by police and people who are arrested. The problem here is that these two groups (those shot and those arrested) are, in all likelihood, systematically different from one another in ways that cannot be controlled for statistically (UPenn Professor Uri Simonsohn expands on this point here). Fryer acknowledges this limitation in a brief footnote, but understates just how problematic it is. 

Fryer’s study is far from the first to investigate racial bias or discrimination in police shootings. A number of studies have placed officers in shooting simulators, and most have shown a greater propensity for shooting black civilians relative to whites. Other research has found that cities with black mayors and city councilors have lower rates of police shootings than would otherwise be expected. A recent analysis of national data showed wide variation in racial disparities for police shooting rates between counties, and these differences were not associated with racial differences in crime rates. This is just a small sample of the dozens of studies on police killings published since the 1950s, most of which suggests that racial bias is indeed a problem. 

4. In his paper, Fryer writes about the lack of good data for use in studying these issues and gives the following caveat for his results: "...all but one dataset was provided by a select group of police departments. It is possible that these departments only supplied the data because they are either enlightened or were not concerned about what the analysis would reveal. In essence, this is equivalent to analyzing labor market discrimination on a set of firms willing to supply a researcher with their Human Resources data!" In other words, Fryer openly admits that his paper was constructed from data volunteered by a small subset of police departments that are probably a lot more progressive in their policing than the vast majority of police departments that are not willing to be transparent.

On the Representation of Academic Results in the Media

There is a major problem with the way the media reports on academic studies in our society. This problem is most glaring in the arena of health journalism where, if you trust the news, everything you eat is both healthy and unhealthy for you (Vox's excellent analysis). Unlike the general public, the academic community can recognize the value (or lack thereof) of a single study and contextualize within a larger body of work. Academics understand that a single study, especially if the result is novel and surprising, is not a reliable source of "truth," and needs to be well corroborated. Any lone academic result can be misleading due to small data problems, an inability to corroborate the result in subsequent studies, methodological shortcomings and a myriad sources of error that affect every scientific study. Over time and through the work of many researchers, critics, countless publications, and meta-surveys on a topic, an academic consensus begins to emerge and impress itself on the academic landscape. The context and limitations of an individual study get lost when reported in the media.

As a society, we should engage in dialog that will move us toward a future in which the media is more accountable for the way the way academic results are presented and journalists are better informed about how to interpret those results. Until then, there is no substitute to learning how to access, read and criticize primary academic materials to better understand the academic landscape and its political import.

In conclusion: we live in the age of click-bait. Publishers make money from the amount of clicks that they get on a Facebook post leading to a magnification of the most surprising, outlandish and ideologically compelling results at the expense of nuance and serious analysis.