News from the lab
What makes economics less than a proper science, sceptics often argue, is that you cannot do controlled experiments. Many economists disagree
EXPERIMENTAL economics, as it is called, is a growth industry. The idea of quizzing people in controlled settings about actual or hypothetical decisions, then coming to conclusions about the way they make them, goes back a long way. But its heyday in modern economics began in the 1950s, when Maurice Allais, one of France’s greatest economists (and winner of the Nobel Prize in 1988), asked people to choose among certain gambles, and then examined their decisions. He discovered that people acted in ways that were inconsistent with the standard theory of utility—a basic building block of economics, which predicts how people make choices subject to their limited resources. Later, more careful experiments found, by and large, the same thing: people do not behave as economics says they are supposed to.
Spurred by these findings, the experimental tendency in economics has expanded prodigiously. All kinds of further puzzles and anomalies have come to light. And the experimentalist literature has moved beyond utility theory to bargaining, public goods, industrial organisation, auctions, asset markets and more. Is this a sensible path to take? Can lab work make economics a better science? Or is it all, as many mainstream economists appear to think, a waste of time? A symposium in a recent issue of the Economic Journal takes up these questions.
Experimental economics has raised interesting issues and stimulated thought and work. It has added to the sum of economic knowledge. But that is not to say that it has made economics a better science, or more like the proper science it would like to be. Consider one example.
Experimental economists have paid a lot of attention to so-called preference reversals. Typically, in a variation of the Allais experiment, subjects are asked to consider two different lotteries—one with a big probability of winning a small prize and another with a smaller probability of winning a big one. Then two questions are put to the subjects. The first asks which lottery ticket they would prefer to own; the second asks how much they would be willing to pay for each kind of ticket. The result is curious. Suppose, depending on the prizes and the odds, that half of the subjects say they prefer the first kind of ticket. Then it usually turns out that, with the same prizes and odds, far more than half go on to attach higher prices to the second kind of ticket. This is irrational. It appears that people draw on different information in deciding which ticket to choose as opposed to how much they are willing to pay.
This is a serious embarrassment to economic theorists. Many different and increasingly complicated versions of this experiment have now been carried out. Some used actual money, some didn’t. (When real rather than hypothetical cash is used, the number of reversals tends to go up.) Ever more ingenuity has been spent on making sure that the subjects were properly motivated and conscientious. Some ways of setting the experiment up do reduce the number of preference reversals, but the basic finding seems to hold. One set of authors concluded that the reversal effect was inconsistent not only with utility theory but also with every existing theory of economic decision-making. In the light of this, a variety of new theories (some devised by psychologists rather than economists) have been proposed.
Which all seems quite scientific: theories falsified, new ones offered up for falsification. The trouble is, mainstream economics has not abandoned utility theory: it does not regard it as falsified, after all. Most economists, it is probably fair to say, would simply rather not contemplate the failure of such cherished assumptions. Many would defend utility theory (as well as other aspects of their standard model of behaviour) as a useful approximation, pointing out that exceptions to approximations can always be found. Pressed, however, they might take the battle to the experimenters and argue that experiments do not provide an accurate representation of how people behave in the “real world”.
That is a fair point. The very idea of an experiment is artificial. Subjects know they are being watched: they will think about what the experimenters expect them to do, about what they “should” do, as well as about what they themselves want to do. The incentives may be smaller than in real-world decisions: so rational choices may be inadequately rewarded and irrational ones inadequately punished.
Most important, it is hard for experiments to create realistic opportunities for learning and adapting. Pick any of these experiments and one’s reaction on reading the subjects’ instructions is usually confusion. Outright mistakes must be common. When experiments allow for this by letting subjects learn, they do it through sheer repetition. Repeated trials are likely to be better than one-off experiments in this respect: people make fewer straightforward mistakes at the end of 20 reps than at the beginning. But these trials are still far removed from normal experience. For most people, as one critic points out, “Groundhog Day” is a surreal nightmare rather than a plausible depiction of real life.
Experiments are very often interesting and idea-provoking and probably require no further justification. Mainstream economists should no doubt pay closer attention. But whether experiments improve the scientific credentials of the discipline must be very much in doubt. In the end, the main problem is not that designing good experiments is hard. It is simply that, unlike physics, economics yields no natural laws or universal constants. That is what makes decisive falsification in economics so difficult. And that is why, with or without experiments, economics is not and never can be a proper science.
The Economic Journal symposium was in the issue of February 1999. The indispensable reference on the subject, with essays surveying the entire field, is “The Handbook of Experimental Economics” edited by John Hagel and Alvin Roth, published by Princeton University Press in 1995.