Unpacking P-hacking and Publication Bias

S-Tier
Journal: American Economic Review
Year: 2023
Volume: 113
Issue: 11
Pages: 2974-3002

Score contribution per author:

2.011 = (α=2.01 / 4 authors) × 4.0x S-tier

α: calibrated so average coauthorship-adjusted count equals average raw count

Abstract

We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review; i.e., marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.

Technical Details

RePEc Handle
repec:aea:aecrev:v:113:y:2023:i:11:p:2974-3002
Journal Field
General
Author Count
4
Added to Database
2026-01-24