I'm behind the curve a bit here, but I've seen and heard a bunch of
people making really sleazy arguments about the current financial stimulus
package working its way through congress, and those arguments are a perfect
example of one of the classic ways of abusing statistics. I keep mentioning metric errors - this is another kind of metric error. The difference between this and some of the other examples that I've shown is that this is deliberately dishonest - that is, instead of accidentally using the wrong metric to get a wrong answer, in this case, we've got someone deliberately taking one metric, and pretending that it's an entirely different metric in order to produce a desired result.
As I said, this case involves the current financial stimulus package that's working its way through congress. I want to put politics aside here: when it comes to things like this financial stimulus, there's plenty of room for disagreement.
Economic crises like the one we're dealing with right now are really uncharted territory - they're very rare, and the ones that we have records of have each had enough unique properties that we don't have a very good collection of evidence
to use to draw solid conclusions about recoveries from them work. This isn't like
physics, where we tend to have tons and tons of data from repeatable experiments; we're looking at a realm where there are a lot of reasonable theories, and there isn't enough evidence to say, conclusively, which (if any) of them is correct. There are multiple good-faith arguments that propose vastly different ways of trying
to dig us out of this disastrous hole that we're currently stuck in.
Of course, it's also possible to argue in bad faith, by
creating phony arguments. And that's the subject of this post: a bad-faith
argument that presents real statistics in misleading ways.
When you see statistics used in arguments, the number one thing that you
should always be careful about is to make sure that you understand exactly what the statistic means. One of the classic ways of misleading people with statistics is take take a valid statistic measuring some quantity, and present it as if it represented an entirely different quantity.
The current example of this comes from a variety of people who've been citing the work of a conservative anti-Keynesian writer named Amity Shlaes. Ms.
Shlaes has put forward an argument that the New Deal was a failure, and that it
didn't really reduce unemployment. The full argument is in her book,
The Forgotten Man: A New History of the Great Depression; a short version of the
particular claim that I'm objecting to can be found in a Wall Street Journal Op-Ed
Now, in school, we've all seen the history of the Great Depression. We've all
heard about how the New Deal put so many people to work, doing all sorts of things - building roads, bridges, buildings, aircraft carriers, etc. All told, the WPA (the organization that the government used to fund all of the New Deal work programs) employed something over three million people. So how did employing three
million people not help improve the unemployment situation? It seems ridiculous on the face of it - did employing three million people somehow destroy more than three million jobs, without anyone noticing that?
The answer is that when Ms. Shlaes talks about unemployment, she's not really talking about unemployment. She's careful to not really cite unemployment rates. What she does is cite the number of permanent jobs.
The WPA programs in the New Deal were designed as a deliberate stop-gap measure. That is, there were millions of unemployed people in America, who weren't able to afford the basic necessities, like food and housing. The point of the WPA was to get those people working, immediately, doing something to earn a wage, so that they could feed their families, pay their rent, etc. It wasn't to give them permanent jobs working for the government, but to give them paying jobs until
the economy recovered enough that private employers would start hiring again.
So in other words, Ms. Shlaes deceptively picks a statistic that excludes the jobs created by the New Deal, and then uses that to argue that the New Deal didn't create any jobs. If you actually state that properly, it becomes transparently ridiculous: "If you exclude all of the jobs created by the New Deal, then the New Deal didn't create any jobs".
You can make the argument that they way the government tried to help the
economy delayed that recovery. I don't find the argument particularly convincing, but that's probably as much my philosophical bias as anything else. There are some strong arguments that the Roosevelt economic stimulus programs were the right thing; and there are some strong arguments that it was the wrong thing. Both arguments are, largely, speculative - there just isn't enough data about this kind of situation to be able to really know which argument is right - both arguments follow logically from their assumptions, but we don't know which set of assumptions is right in this situation.
But to argue that unemployment was not reduced by hiring three million people? That's idiocy. And Ms. Shlaes (and most of the people citing her) know it. In fact, she basically admits it herself, but handwaves her way past it: "To be sure, Michael Darby of UCLA has argued that make-work jobs should be counted. Even so, his chart shows that from 1931 to 1940, New Deal joblessness ranges as high as 16% (1934) but never gets below 9%. Nine percent or above is hardly a jobless target to which the Obama administration would aspire."
Read that carefully. She's admitting that WPA programs reduced
unemployment by nearly half. (And even that's using skewed figures. Different ways of estimating unemployment during the Depression range as high as 25%.) But even in
the midst of her argument about how the New Deal didn't decrease unemployment,
she's admitting that it reduced unemployment quite dramatically.
As I said, this is a typical way of using statistics in a misleading way. Pick a statistic that measures quantity A, and use it as if it measures quantity B. You can see arguments like this all over the place.
To cite another example, this time from the other side of the political spectrum: when criticizing the Bush administration's fiscal policies, you
constantly hear people talking about the Clinton surplus. They tell you that
under Bill Clinton's fiscal policies, the federal government's budget went
from operating at a huge deficit to a huge surplus.
The problem is, there was no surplus. There was never a real surplus
under President Clinton. It's once again a game of switching metrics.
The government collects takes to pay for government programs. There are
multiple kinds of taxes - personal income taxes, corporate taxes, capital gains
taxes, inheritance taxes, social security taxes, etc. Some of the money collected
as those taxes goes into a general funds pool - the basic pool from which the
government operates. Some of those taxes go into special funds, which are supposed to be used only for a specific purpose. An example of the latter kind of tax
is social security, which is only supposed to be used to pay social security
benefits. Any excess in social security is supposed to be put away for
When people talk about the Clinton surplus, what they mean is that by
the end of Clinton's term in office, the total collection of funds from all taxes during a fiscal year exceeded the total funds spent by the federal government during that fiscal year. Sounds great, right?
Only the special purpose taxes, like social security, were included in the tax collections. But in order to use social security funds for the general
budget, the goverment has to borrow money from social security. It issues
government bonds - the exact same bonds that make up the federal deficit. But instead of selling those bonds to private purchasers, it sells them to itself. But it's debt, nonetheless.
When someone talks about the surplus, they're playing a misleading
game by using invalid metrics. But by citing one quantity (total government income including money borrowed from social security), and pretending that it represented a different quantity (total government income from general taxes), they can dishonestly claim to have balanced the federal budget, and produced a surplus.
This tactic - using the wrong measure - is incredibly widespread, because it's
incredibly effective. It's easy to use to deceive people, because most people won't pay attention to the exact definition used in the statistic - they'll focus on the number. And if someone objects that the statistic is wrong, it's easy to twist the
argument into a debate about the number, rather than about the meaning - which, in turn, makes it look to people observing the argument, like there's a legitimate debate.
The lesson here should be clear. Always, always make sure that you understand
exactly what a statistic really measures. When someone makes an argument
based on statistics, make sure that the statistics they cite really do measure what
the arguer claims they measure.