Selective reporting of placebo tests in top economics journals

C-Tier
Journal: Economic Inquiry
Year: 2024
Volume: 62
Issue: 3
Pages: 921-932

Authors (3)

Anna Dreber (not in RePEc) Magnus Johannesson (Stockholm School of Economics) Yifan Yang (not in RePEc)

Score contribution per author:

0.335 = (α=2.01 / 3 authors) × 0.5x C-tier

α: calibrated so average coauthorship-adjusted count equals average raw count

Abstract

Placebo tests provide incentives to underreport statistically significant tests, a form of reversed p‐hacking. We test for such underreporting in 11 top economics journals between 2009 and 2021 based on a pre‐registered analysis plan. If the null hypothesis is true in all tests, 2.5% of them should be significant at the 5% level with an effect in the same direction as the main test (and 5% in total). The actual fraction of statistically significant placebo tests with an effect in the same direction is 1.29% (95% CI [0.83, 1.63]), and the overall fraction of statistically significant placebo tests is 3.10% (95% CI [2.2, 4.0]).

Technical Details

RePEc Handle
repec:bla:ecinqu:v:62:y:2024:i:3:p:921-932
Journal Field
General
Author Count
3
Added to Database
2026-01-25