Yesterday the Royal Swedish Academy of Sciences announced the winners of the 2019 Nobel Prize in Economics: the trio of Abhijit Banerjee, Esther Duflo and Michael Kremer "for their experimental approach to alleviating global poverty." Banerjee and Duflo are a married couple at MIT, and Michael Kremer is from Harvard. Duflo is the youngest woman, and only the second woman ever, to win the award. If you are a nerd, you may recognize Banerjee and Duflo from their book Poor Economics, and their work in popularizing the use of RCTs (Randomized Control Trials) through their mega-influential institute at MIT the Abdul Latif Jameel Poverty Action Lab (J-PAL). All three newly minted Laureates are, according to everybody on EconTwitter this morning, extremely nice, good natured people always willing to help junior scholars out and the fact that so many have felt it necessary to point that out as a redeeming quality really underscores how many jerks end up being economists.

So what does an “experimental approach to alleviating global poverty” mean? Well, to answer that you have to put on your Socrates hat and really think about the study of economics. It is called a science, and indeed the Royal Swedish Academy of Sciences apparently agrees it is a science, but much of what economists study is either unobservable, or is often interpreted after the fact by testing a theory using historical case studies. It is exceedingly difficult to design and implement real-time economic experiments, for the simple fact that economies are large and unwieldy, effects are difficult to measure, and variables are hard or even impossible to isolate. Who knows if economic growth is being driven by interest rates, FDI, institutional design, demographic change, geography, cultural values or some unobserved variable? The truth is, nobody really knows.

We can look at a particular economic outcome (inequality, for instance) and theorize about why it is happening. We can then test that theory using historical data and some mathematical models, if we are so inclined. But because there are so many factors that could possibly be impacting inequality, it is difficult to answer conclusively what is causing it or how best to alleviate it. This problem becomes especially acute when your goal is to identify a general causal mechanism that is valid across all economic and social systems and maybe even across time. In the 1990s, mainstream economics became pretty convinced that free markets and assumptions about rationality were the antidote to global poverty, but the mustard is increasingly coming off that hot dog.

Kremer, Banerjee and Duflo’s approach to this puzzle was to narrow their focus and apply a more rigorous methodology, modeled on biological science, to experimental case studies. By doing so, they could empirically measure in a structured way how different groups responded to specific policy interventions, and thus better gauge the effectiveness of actual policies. Social scientists are often wary of causal effects, but a well-designed study can greatly improve the confidence of your estimate. Randomized Control Trials are nothing new - they are common in medical science, when testing the effectiveness of a new medication for instance. You take two groups of people, you give one the medicine and the other a placebo and you see if the treatment group responds differently than the control group and in what way.

But Randomized Control Trials are very difficult to pull off to test economic theories. They are rife with ethical concerns, require the cooperation of host governments (much of the work these three have done has been in Africa and India), and they need a lot of funding and a lot of time. It’s much easier for an economist to pull some data from a World Bank database and run it through a regression than to spend years juggling the logistical and financial demands of collecting detailed survey data on the effect of microfinance or de-worming pills in dozens of Indian or Kenyan villages.

RCTs have come to dominate much of the discourse in development economics these days, as they are an effective and proven way to measure the impact of specific policy interventions and that kind of data is essential for allocating resources more effectively. By aping the rigor of medical studies, they lend the field of economics a kind of empirical grounding that it is often accused of lacking. These are also the kinds of studies that big institutions, financial backers and governments can get behind - because you are testing the effect of a narrow policy intervention, the results will be useful right away to design and implement better policies. For all of these reasons, and many more which I do not have the space to cover here, Kremer, Banerjee and Duflo were awarded the Nobel Prize this year.

Not everyone is a fan, though.

The first criticism of their work is an ethical one. In order to measure the effect of a policy intervention, you must withhold it from the control group. So, if you are testing to see what happens when you provide de-worming pills to a certain segment of the population, there is also a control group who are being deliberately deprived of those pills in order to establish a baseline for comparison. Some might argue that this flirts with ethical concerns.

The second criticism is that their approach is, in fact, too narrow. RCTs are good at answering what effect a specific policy has on a particular population. And I think as a tool for policy-makers to evaluate specific policies they are excellent. When proven policies are scaled up in a smart way, they can bee deeply impactful. But many (including myself) would argue that economic development, more broadly conceived, must be approached in a more expansive manner. To figure out why certain economic systems function the way they do, and what can make them function more equitably or grow faster, you have to look at the big picture: historical path dependence, institutional design, the way social incentives are structured, the interests of political actors. It’s not all about developing an effective policy intervention and then carefully measuring the effect it produces; there are many other factors that deserve to be considered.

The problem is that an institutional approach to economic development does not lend itself to RCTs. You can’t take one country and re-structure its political and economic institutions and then compare it to another similar country with unchanged institutions. Some argue that by focusing on the narrow, technical aspects of policy interventions Kremer, Banerjee and Duflo are brushing past more important and weighty issues like how societieis think about poverty, and how and why economic resources are allocated in the way they are. And only once you start to engage with those questions, can you really address the root causes of poverty.

This leads to another criticism of their work, which is that it often lacks external validity. A policy that works in India to increase school attendance, may not be effective at all in Indonesia or Kenya (in fact, it probably won’t be). In my opinion, this is largely unimportant - as long as you acknowledge it, which they do. A narrowly designed policy that produces good results makes for perfectly good research, even if it cannot be generalized. But I do worry that the growing popularity of RCTs with funders, NGOs and governments may squeeze out other approaches to understanding the substance of economics. We need to keep in mind that RCTs, while very good at what they are designed to do, are merely one out of many possible tools.

A fourth criticism, mainly from the heterodox economic community, is that the work of Duflo, Banerjee and Kremer, while being hailed as “innovative” and “experimental” is actually neither. It is still principled on the use of behavioral economics and assumptions about rationality, and how certain changes can tweak actor behavior by adjusting incentives. This is not, at its core, all that revolutionary. It would really only be considered revolutionary in the mainstream economics community, where the idea of doing any kind of empirical field work is still considered novel and probably a waste of time by most. Their real contribution, then, may have been to raise the profile of empirical field work, and allow it to be taken more seriously. As long as is rigorously structured. And modeled on the kind of randomized control trials usually used to test cancer drugs.