Editor’s Corner

October 2019

Travis J. Lybbert, AJAE Editor

In my time as AJAE editor over the past four years, an interesting crisis for our profession – and, indeed, for the wider social sciences. The replication crisis and associated crisis of confidence in empirical analysis reflect concerns about how research is conducted and published, including researcher degrees of freedom, p-hacking, the “garden of forking paths,” and the file drawer problem. The crisis hit close to home for the AAEA with our own “pizzagate” and, in its wake, 17 retractions and the high profile resignation of Brian Wansink from Cornell University.

As this crisis unfolded, we as editors frequently discussed appropriate responses for the AJAE and how we might increase research transparency. These discussions led to some substantive revisions to the AJAE Editorial Policies to include explicit adherence to Transparency and Openness Promotion “Level 1” standards and to editor participation in special sessions on the topic at the AAEA annual meetings. In 2020, we will also introduce a revised disclosure form that requires authors to report their complete consulting history over the prior five years (rather than just financial support narrowly related to the submitted manuscript).

We have elevated our data and code sharing policy from a suggestion to an expectation, as is now the norm for other top journals. Whether due to this revision or to a rapidly diffusing professional norm, the share of empirical articles in the AJAE that are accompanied by data and code to replicate the analysis has increased from 8% in 2015 to nearly 20% today. With support from the AAEA, we also recently launched a ‘Data and Code Verification Pilot’ that supports a small team of analysts that use these files to verify “push button” replicability of the tables and figures in the manuscript.

Addressing the crisis of confidence of the past decade will require more than changes in journal policies and editorial practices, but the AJAE Editorial Board is striving to play a constructive role in building greater transparency and restoring confidence in empirical analysis. Ultimately, however, alleviating the effects of professional “sinning in the basement” requires strict adherence to standards of professional integrity, which begins with the training and mentoring of the next generation of applied economists.