It's a Crime What Some
People Do With Statistics

By Arnold Barnett
Copyright 2000 Wall Street Journal
August 30, 2000

The truism that statistics can be misleading has no more content than the statement that paragraphs can be misleading. But certain statistics that are indeed highly misleading have made their way into debates about crime, punishment and race. The resulting misconceptions have intensified already bitter disputes, and can only sow confusion among voters.

For example, in the controversy over whether innocent people are being executed, a 1-in-7 ratio has attained prominence. Newsweek sought to explain the ratio when it stated that "for every seven executions nationwide since the death penalty was reinstated in 1976, one death-row inmate has been set free." William F. Buckley Jr. probably reflected the common understanding of this statistic when he wrote that "if the figures work out retroactively, then one out of seven (of the 640) executed Americans was, in fact, innocent."

Greatly upset by the ratio, the Economist noted that "if an airline crashed once for every seven times it reached its destination, it would surely be suspended immediately." A bit of probing makes clear, however, that the ratio makes no sense.

There is an obvious interest in the error rate for capital-sentencing, which is the number of innocents sentenced to death divided by the total number of people thus sentenced. Also of importance is the error rate for actual executions: the number of innocents executed divided by the total number executed. In an ideal world, both these rates would be zero.

The 1-to-7 ratio, however, represents neither of these rates but rather a confused amalgam of their components. It divides the number of known innocents freed from death row by the number of executions. In other words, it divides the numerator of the error rate for capital-sentencing by the denominator of the rate for executions. Such a calculation is of no value: It is akin to computing an earnings-per-share statistic by dividing the earnings of one company by the number of shares of a completely different one.

Suppose that there are 2,000 people on death row and that, over a given period, one of them is found innocent and freed while one is executed. The only reliable inference from these statistics is the obvious point that, during this period, both executions and known sentencing errors were extremely rare. To divide one by the other while ignoring the 2,000 altogether does not demonstrate that executions are fraught with errors; it is a meaningless act that yields no insight.

Another confusing statistic appeared several months ago when the New York Times described a Columbia University/New York State study about police stops and searches of New York City residents. The "most basic finding" of the study, the Times reported, was that blacks were stopped six times as often on a per-capita basis as whites. And, "even when the numbers are adjusted to reflect higher crime rates in some minority neighborhoods," blacks were stopped 23% more often than whites.

Hold on a minute. The original black/white stop ratio was six (as opposed to the value of one, which would mean equal stopping rates). After an adjustment that the researchers thought appropriate, the ratio fell to 1.23. Thus, instead of 600 blacks stopped for every 100 whites in comparable groups of equal size, 123 blacks were stopped. The disparity still exists, but it is far smaller. Put in percentage terms, the black/white excess fell from 500% to 23% (i.e. declined by a factor of 20).

It is unclear whether readers of the Times grasped this last point because, instead of working consistently with ratios or with percentages, the Times started with the former and then shifted to the latter. Matters were especially confusing because the Times narrative repeatedly suggested that the adjustment had reaffirmed the "basic" finding rather than nearly overturned it.

Over at National Review, an author noted that homicide in the U.S. plummeted in the 1990s, while executions soared, and discerned a deterrent effect of capital punishment. But this aggregate correlation misses a crucial local detail: Recent drops in killing have been greatest in places (e.g., New York City, Boston) where no death sentences have been carried out during the past three decades. Unless one believes than an execution in Virginia that goes unreported in the Bronx nonetheless prevents some killings there, one should be wary of statistics that pool Virginia executions with Bronx murders.

There is more. A full-page ad from the American Civil Liberties Union, placed in several prominent magazines, showed a picture of Martin Luther King Jr. next to one of Charles Manson. The accompanying text declared that "the man on the left is 75 times more likely to be stopped by the police while driving than the man on the right." The basis of this finding was that "in Florida 80% of those stopped and searched were black and Hispanic, while they constituted only 5% of all drivers."

This analysis is baffling. It is hard to imagine that the 5% figure is accurate: Government statistics indicate that blacks and non-black Hispanics constitute 29% of all Florida residents, and that these groups drive approximately 20% of the state's vehicle miles. Moreover, applying the statistics to a comparison between Dr. King (who presumably represents innocence) and Manson (who presumably represents guilt) requires a strong tacit assumption: that race was the only determinant of auto stops in Florida. A car would not be stopped, for example, merely because it was going 110 miles per hour down Interstate 95. Simply stating such a premise suggests its absurdity.

We should not overreact to such frightful statistical "analyses." Some of them might reflect not deliberate distortion but rather innocent intellectual disorder. Furthermore, the fact that certain numbers are flawed need not invalidate the general point they try to advance. It could well be that there are some innocent people on death row, much as race could play an indefensible role in some police stops. Such possibilities should be investigated in sensible and unbiased ways.

In the meantime, certain widely cited statistics should be sent into exile.

Arnold Barnett is a professor of management science at the Massachusetts Institute of Technology.