Computational Reproducibility in Finance: Evidence from 1,000 Tests

A-Tier
Journal: The Review of Financial Studies
Year: 2024
Volume: 37
Issue: 11
Pages: 3558-3593

Authors (11)

Christophe Pérignon (not in RePEc) Olivier Akmansoy (not in RePEc) Christophe Hurlin (not in RePEc) Anna Dreber (not in RePEc) Felix Holzmeister (Leopold-Franzens-Universität I...) Jürgen Huber (not in RePEc) Magnus Johannesson (Stockholm School of Economics) Michael Kirchler (not in RePEc) Albert J Menkveld (Tinbergen Instituut) Michael Razen (not in RePEc) Utz Weitzel (not in RePEc)

Score contribution per author:

0.366 = (α=2.01 / 11 authors) × 2.0x A-tier

α: calibrated so average coauthorship-adjusted count equals average raw count

Abstract

We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.

Technical Details

RePEc Handle
repec:oup:rfinst:v:37:y:2024:i:11:p:3558-3593.
Journal Field
Finance
Author Count
11
Added to Database
2026-01-25