In June of 1967, George Akerlof submitted a paper to an academic economics journal in the hopes of getting published. “The Market for ‘Lemons'” examined how asymmetric information affects markets, taking the used car business as an example. The first journal rejected the submission out of hand, writing that it “did not publish papers on subjects of such triviality.” It was rejected by two more journals, one of which noted presciently that if Akerlof’s arguments were correct, “economics would be different.” The paper was eventually published by the Quarterly Journal of Economics in 1970. Akerlof would go on to win the Nobel Prize for this work and it is now considered a seminal article in the economics canon.
Akerlof’s experience has been frequently recounted, as a fluke, or a funny anecdote about how the work of a brilliant economist almost never saw the light of day. But as HCEO Co-Director James J. Heckman and his co-author, CEHD Predoctoral Fellow Sidharth Moktan, show in their new HCEO working paper, economics publishing is still frequently defined by a narrow range of journals. Heckman and Moktan were interested in how publishing and the promotion of academic economists are influenced by the Top 5 (T5) economics journals: The American Economic Review, Econometrica, the Journal of Political Economy, the Quarterly Journal of Economics, and the Review of Economic Studies. The authors note that publishing in these journals has become a professional standard, with T5 publication counts often tied to tenure decisions.
“The promotion and advancement power of the T5 in the top 35 U.S. economics departments is unquestionable,” the authors write.
In order to examine the relationship between T5 publications and tenure decisions, Heckman and Moktan analyze the job histories of tenure-track economists hired by the top 35 U.S. economics departments between 1996 and 2010. This time frame allows the authors to assess the impacts of published papers but is also recent enough to describe the current economics environment.
“We show that, compared to publication in other outlets, publishing in the top 5 increases your chances of tenure by quite a lot,” Moktan says. The authors find that publishing three T5 articles is associated with a 370% increase in the rate of receiving tenure, compared to candidates with similar levels of publications who do not place any in the T5. Candidates with one or two T5 articles “are estimated to experience increases in the rate of receiving tenure of 90% and 260% respectively, while the estimated effects of non-top 5 publications “pale in comparison.” The authors also note that requirements for T5 publication decline with department quality and that “the impact on tenure of T5 publication increases with declines in department quality.”
These findings are corroborated using a survey of current assistant and associate professors in the top 50 economics departments. Survey respondents were asked about their beliefs of how tenure decisions are made within their departments, with an emphasis on the role played by T5 publications. The majority of respondents reported that the quantity of T5 publications is “the most important source of influence on tenure and promotion decisions.” The importance of T5 publication was ranked ahead of seven other areas of performance, including teaching and external letters of review.
In the survey, the authors also tried to tease out whether faculty believe the T5 effect is due to these publications being better quality. To do this, they asked respondents to predict which of two hypothetical candidates were more likely to receive tenure. The candidates had equal portfolios, except that one had published in T5 journals and the other had not. “On average, junior faculty at the top 50 departments believe that their department would award tenure to the T5 candidate instead of the non-T5 candidate at least 89 times out of 100,” the authors write.
“That just shows that the influence of the top 5 is perceived to operate independently of article quality,” Moktan says.
The authors also try to suss out whether T5 articles are superior to other publications by comparing citations counts of work published in T5 journals to 25 other journals between 2000 and 2010. They find that “non-T5 non-survey journals publish a significant volume of influential research in economics, frequently outperforming some of the less-influential T5 journals.” Moktan notes that when looking just at mean citations, T5 journals do tend to generate more citations. But when they looked at by-article citations, he says “there was a big overlap in the distribution.” The paper notes that “papers published in non-T5 journals account for more than 70% of RePEc’s most-cited articles in the past 10 and 20 years, respectively,” and that “among the 20 most cited articles by RePEc, 35% were not published in the T5.”
Heckman and Moktan point out that many of the most influential economists frequently publish outside of the T5 journals, often in non-T5 working paper series. “Scholars who themselves primarily publish in, read, and cite papers from non-T5 field journals appraise the quality of prospective candidates for promotion and hiring using their T5 publications,” they write. “The appraisers do not practice what they preach.”
It is worth noting that the amount of space available in T5 publications has decreased over the years, as “journal space is fixed in supply and papers have become longer.” This creates a more competitive environment, the authors write, which “implies that the cost and effort going into T5 publishing has increased with the decline in the probability of success.” T5 acceptance rates declined from 15% in 1980 to 6% in 2012.
“Without doubt, publication in the Top Five is a powerful determinant of tenure in academic economics that influences the choice of topics on which young economists work and squeezes papers into bite-sized journal-friendly fragments,” the authors write.
The emphasis on T5 publications holds several important implications for economics as a profession. Significantly, two of the T5 journals are in-house, meaning that editors belong to a specific university and tend to have much longer tenures than other journals. “One must allow for the possibility of strong network bias against tenure-track faculty who lack connections with T5 editors, regardless of whether such bias stems from blatant editorial corruption or from the above conjectured impact-maximizing behavior of editors who seek quality papers,” the authors write.
“Corresponding with that, they tend to publish many more papers by people who are affiliated with the university,” Moktan says. “There’s a lot of influence given to just a handful of people.”
As this research shows, the T5 operate independent of quality, so incentivizing faculty to pursue these publications affects their research agendas and “discourages large-scale, data-intensive empirical projects.” As the example of Akerlof’s study demonstrates, faculty are not necessarily incentivized to pursue innovate and creative lines of research.
“Our findings should spark a serious conversation in the profession about how to develop implementable alternatives to judge quality research,” the authors write. They note that solutions would at least reduce the role T5 publications play in tenure and promotion decisions. “Under any event, the profession should deemphasize crass careerism and promote creative activity,” they write. “Short tenure clocks and reliance on the Top 5 to certify quality does just the opposite.”
Thank you to our funder, the Institute for New Economic Thinking, for their support of this research.