don't be a daron acemoglu

a little philosophy of social science

Daron Acemoglu is one of the most highly cited scientists of all time. He has 172 papers with at least 172 citations, which is staggering. A large number of his papers individually have more impact, when measured by citations, than most scientists’ entire careers. And yet.

I think Daron Acemoglu’s research is in the cursed quadrant in the graphic below: extremely highly cited and extremely low quality. This may sound tough, but I’m going to make the case. Note that we are living in the intellectual realm here, where ideas are fair game for tough criticism. I’m sure Acemoglu personally is a nice guy and would be fun to get drinks with. But when we write down our ideas and submit them to the ~marketplace~ of ideas, our ideas (like this blog post) are fair game for criticism. And by my count Acemoglu’s ideas are a mixture of extremely influential and not very useful.

Beyond the low quality of the research itself, by using a giant microphone to channel enormous scientific attention to less-than-useful ideas, Acemoglu is robbing the social sciences of progress. There’s a famous sociological argument that high-status scientists can speed the progress of science by focusing an entire field on important questions. But the flipside of that is when the high status scientist focuses people on the wrong thing, entire fields can waste time. Such is the case here.

Previously in this “don’t be a…” series, we looked at Noam Chomsky (confident, wrong) and Judith Butler (lots of words, says nothing). Daron Acemoglu is prolific with paper thin ideas, and a for that reason we should strive to avoid emulating him. His ideas aren’t wrong, they just don’t matter. When we think about what kind of science (and work more generally) we want to do, we want to produce ideas that generate additional good research and stand up to real-world scrutiny. Acemoglu’s ideas unfortunately do neither, yet are hyper-evolved to survive in the particular competitive landscape presented by academia.

let’s talk about institutions

As a sociologist, the area of Acemoglu’s work I am most familiar with is his work on long-run institutions and economic growth, and most of this post will be about that. Acemoglu has written too many things for me to cover all of it here, and because the stuff I know the best is not very good, I am not motivated to dive into the rest with my limited time.

Acemoglu and James A. Robinson wrote a well-received book called Why Nations Fail, which is based on a series of academic papers authored by Acemoglu and others. The book has over 16,000 academic citations, so this is not a pop-science book, and it is based on a series of peer-reviewed academic articles with many tens of thousands of citations themselves. So this is not a case of a scientist doing excellent work and then becoming famous and going off the rails. This is the work that really made Acemoglu’s career.

And it is not very good. If you don’t believe me, listen to Bill Gates, who writes in a book review

Ultimately, though, the book is a major disappointment. I found the authors’ analysis vague and simplistic. Beyond their “inclusive vs. extractive” view of political and economic institutions, they largely dismiss all other factors—history and logic notwithstanding. Important terms aren’t really defined, and they never explain how a country can move to have more “inclusive” institutions.

This is completely correct. The thing the book is trying to explain is why some nations do well in the long run and some don’t. The basic idea the authors propose is that countries that have “inclusive” institutions do well and those that have “extractive” ones don’t. You can think of this roughly as cases where the government is inclusive of the people vs cases where the government is adversarial with the people and tries to shake them down for money. OK, fair enough.

But… that’s the whole theory! There’s no like breaking down what makes a country extractive vs inclusive, it’s all very vague. Basically, if you had inclusive institutions 400 years ago, you had economic growth, and that continued through to today, and now you’re a rich country. But if you had extractive institutions 400 years ago, you had no growth, and that also continued to today, and now you’re a poor country.

That’s the theory. That’s it. Paper thin.

It doesn’t tell us much about how institutions work and it’s not actionable to make institutions better. You pretty much need your idea to do one of those things, and this does neither!

From the abstract of Acemoglu’s most cited paper (18k cites!), we can see the theory stated a different way,

We exploit differences in European mortality rates to estimate the effect of institutions on economic performance. Europeans adopted very different colonization policies in different colonies, with different associated institutions. In places where Europeans faced high mortality rates, they could not settle and were more likely to set up extractive institutions. These institutions persisted to the present.

Again, that’s it. The theory is that if Europeans (who apparently all had good institutions in the colonial era?!) faced low mortality rates, then they set up good institutions in colonies, and then those good institutions persist to today. But where they faced high mortality, they set up bad institutions, and those also persist.

Talk about fatalism. If you live in a country with bad institutions, why even get up in the morning? There’s nothing you can do! You have bad institutions, you’re condemned to an eternity of poverty.

We just summarized the main idea in two works with over 34,000 academic citations between them. And it’s sort of like, what do we do with this idea? Why does it matter?

why this not great idea became famous

The reason this argument that can be summarized in a few paragraphs (and does not go deeper than that in the 600 page book that propounds it) sparked a mini-revolution in economics is related to the empirical turn in economics in the late 1990’s and through the aughts. In the 1980’s economics was highly theoretical, with economists basically just doing lots of boring algebra with some calculus thrown in there and lots of Greek letters to give it all an imprimatur of seriousness. No data or anything like that.

An enterprising set of primarily microeconomists changed that by developing a set of causal inference techniques that let economists do empirical research to study causal effects from non-experimental data. Acemoglu’s research uses a causal inference method called instrumental variables (IV) which was very hip at the time but has since been overtaken by difference-in-difference methods and actual experiments (that’s me, I’m an experimentalist!).

So in this context—an entire field rediscovering empirical research and the promise of new methods to infer causality—Acemoglu devised a research strategy which promised to explain long-run economic growth. This is a very, very important question for economists and for the rest of us on planet Earth! Economic growth is the reason we have nice things. Understanding it better is one of the core questions of the entire field of economics.

However, the IV method that Acemoglu had at his disposal can only answer a limited set of questions, and knowing those limitations he could only study a theory such as “if there was growth at time t, then there would be growth at time t + 1”. So his entire research design had to fit into that framework.

My guess is that the basic idea behind the institutions research was just a search for “what predicted growth at some point a long time ago” with the assumption that growth yesterday predicts growth today. And that search turned up mortality rates in the early colonial era as something that sorted colonies into high and low growth ones. That’s a nifty piece of history to find, for sure. But it tells us almost nothing about the mechanics of long-term economic growth. It tells us maybe why growth occurred where it did, but it does not tell us what the components of growth are. And the whole point of the research is why nations fail (or succeed), not why European colonists chose particular colonization strategies hundreds of years ago.

when the method makes the theory

The social sciences, particularly the largely non-experimental ones such as economics, have a tough time. They are the “hard” sciences in the sense that they are difficult. It is often difficult to establish basic things like “does X thing increase or decrease growth.” Many things are situational, specific to certain times and contexts. This drives non-experts crazy as experts correctly hem and haw.

And while we try to study these things, we have deeply imperfect methodologies and data at our disposal. Part of the danger of imperfect methodologies is not just that we can’t answer questions well, it’s that we tailor our questions only to things our methodologies can plausibly answer. Now, this is not bad in principle—we want to be able to match method and question so we can do good research. But it leads to cases like we see here with Acemoglu where we mistake what question the method can answer for the question we actually want to answer.

Acemoglu’s method could answer a question like “does having low mortality in the colonial era predict growth today?” and instead that answer was interpreted erroneously by the researchers as answering the question “do inclusive institutions generate growth?” In fact, there is nothing about institutions in the nuts and bolts of Acemoglu’s research. There is no definition of an inclusive institution as a statistical profile of, say, bureaucracy setup and civil society participation. It’s just: growth yesterday means growth today! This is not a theory of institutions, it’s a theory of persistent growth.

And so we have this field-defining paper which does not answer the question it purports to answer and also likely focuses future researchers on the wrong stuff. Not great! When we do social science, we must be critically aware that our theories are interpretations of stylized social facts, and we must return to that interpretative step over and over to make sure we’re doing it right.