It's a Crime What Some
People Do With Statistics
By Arnold Barnett
Copyright 2000 Wall Street Journal
August 30, 2000
The truism that statistics can be
misleading has no more content than the statement that paragraphs can be
misleading. But certain statistics that are indeed highly misleading have
made their way into debates about crime, punishment and race. The resulting
misconceptions have intensified already bitter disputes, and can only sow
confusion among voters.
For example, in the controversy over whether innocent people are being
executed, a 1in7 ratio has attained prominence. Newsweek sought to
explain the ratio when it stated that "for every seven executions
nationwide since the death penalty was reinstated in 1976, one deathrow
inmate has been set free." William F. Buckley Jr. probably reflected the
common understanding of this statistic when he wrote that "if the figures
work out retroactively, then one out of seven (of the 640) executed
Americans was, in fact, innocent."
Greatly upset by the ratio, the Economist noted that "if an airline
crashed once for every seven times it reached its destination, it would
surely be suspended immediately." A bit of probing makes clear, however,
that the ratio makes no sense.
There is an obvious interest in the error rate for capitalsentencing,
which is the number of innocents sentenced to death divided by the total
number of people thus sentenced. Also of importance is the error rate for
actual executions: the number of innocents executed divided by the total
number executed. In an ideal world, both these rates would be zero.
The 1to7 ratio, however, represents neither of these rates but rather
a confused amalgam of their components. It divides the number of known
innocents freed from death row by the number of executions. In other words,
it divides the numerator of the error rate for capitalsentencing by the
denominator of the rate for executions. Such a calculation is of no value:
It is akin to computing an earningspershare statistic by dividing the
earnings of one company by the number of shares of a completely different
one.
Suppose that there are 2,000 people on death row and that, over a given
period, one of them is found innocent and freed while one is executed. The
only reliable inference from these statistics is the obvious point that,
during this period, both executions and known sentencing errors were
extremely rare. To divide one by the other while ignoring the 2,000
altogether does not demonstrate that executions are fraught with errors; it
is a meaningless act that yields no insight.
Another confusing statistic appeared several months ago when the New
York Times described a Columbia University/New York State study about
police stops and searches of New York City residents. The "most basic
finding" of the study, the Times reported, was that blacks were stopped six
times as often on a percapita basis as whites. And, "even when the numbers
are adjusted to reflect higher crime rates in some minority neighborhoods,"
blacks were stopped 23% more often than whites.
Hold on a minute. The original black/white stop ratio was six (as
opposed to the value of one, which would mean equal stopping rates). After
an adjustment that the researchers thought appropriate, the ratio fell to
1.23. Thus, instead of 600 blacks stopped for every 100 whites in
comparable groups of equal size, 123 blacks were stopped. The disparity
still exists, but it is far smaller. Put in percentage terms, the
black/white excess fell from 500% to 23% (i.e. declined by a factor of
20).
It is unclear whether readers of the Times grasped this last point
because, instead of working consistently with ratios or with percentages,
the Times started with the former and then shifted to the latter. Matters
were especially confusing because the Times narrative repeatedly suggested
that the adjustment had reaffirmed the "basic" finding rather than nearly
overturned it.
Over at National Review, an author noted that homicide in the U.S.
plummeted in the 1990s, while executions soared, and discerned a deterrent
effect of capital punishment. But this aggregate correlation misses a
crucial local detail: Recent drops in killing have been greatest in places
(e.g., New York City, Boston) where no death sentences have been carried
out during the past three decades. Unless one believes than an execution in
Virginia that goes unreported in the Bronx nonetheless prevents some
killings there, one should be wary of statistics that pool Virginia
executions with Bronx murders.
There is more. A fullpage ad from the American Civil Liberties Union,
placed in several prominent magazines, showed a picture of Martin Luther
King Jr. next to one of Charles Manson. The accompanying text declared that
"the man on the left is 75 times more likely to be stopped by the police
while driving than the man on the right." The basis of this finding was
that "in Florida 80% of those stopped and searched were black and Hispanic,
while they constituted only 5% of all drivers."
This analysis is baffling. It is hard to imagine that the 5% figure is
accurate: Government statistics indicate that blacks and nonblack
Hispanics constitute 29% of all Florida residents, and that these groups
drive approximately 20% of the state's vehicle miles. Moreover, applying
the statistics to a comparison between Dr. King (who presumably represents
innocence) and Manson (who presumably represents guilt) requires a strong
tacit assumption: that race was the only determinant of auto stops in
Florida. A car would not be stopped, for example, merely because it was
going 110 miles per hour down Interstate 95. Simply stating such a premise
suggests its absurdity.
We should not overreact to such frightful statistical "analyses." Some
of them might reflect not deliberate distortion but rather innocent
intellectual disorder. Furthermore, the fact that certain numbers are
flawed need not invalidate the general point they try to advance. It could
well be that there are some innocent people on death row, much as race
could play an indefensible role in some police stops. Such possibilities
should be investigated in sensible and unbiased ways.
In the meantime, certain widely cited statistics should be sent into
exile.
Arnold Barnett is a professor of management
science at the Massachusetts Institute of Technology.
