Cited by 333Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsscientometrics and bibliometrics research

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Brodeur, Abel, Cook, Nikolai, Heyes, Anthony·2020·American Economic Review

DOI 10.1257/aer.20190687

Open this paper in CoChat — free account

Abstract

The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-in-differences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over 21,000 hypothesis tests published in 25 leading economics journals, we find that the extent of p-hacking and publication bias varies greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that (i) papers published in the Top 5 journals are different to others; (ii) the journal “revise and resubmit” process mitigates the problem; (iii) things are improving through time. (JEL A14, C12, C52)

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1257/aer.20190687

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“18See Ioannidis et al. (2017) for an investigation of statistical power and bias in economics.”
Background
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr043 · Cited by 2,135
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
The American Economic Review
(2009) · American Economic Review · doi:10.1257/aer.99.1.i · Cited by 1,764
The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Weak Instruments in Instrumental Variables Regression: Theory and Practice
Andrews, I., Stock, J. H., & Sun, L. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080218-025643 · Cited by 849
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection*
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605

Flagged first, then by citations

Show all 41 references

Research works printed but not linked: 3

Blanco-Perez Cristina, 2019, IZA World of Labor, 2019, 467
Leamer Edward E, 1983, American Economic Review, 73, 31
McCloskey Donald N, 1985, American Economic Review, 75, 201

Sorted by citations

Items 101–200 of 332

The impact of financial inclusion initiatives: Evidence from meta‐analysis
Biru, A. G., Molla, A., Ferraro, S., & Navon, Y. (2024) · Journal of International Development · doi:10.1002/jid.3943 · Cited by 11
Hic Sunt Dracones: On the Risks of Comparing the ITCV With Control Variable Correlations
Lonati, S., & Wulff, J. N. (2024) · Journal of Management · doi:10.1177/01492063241293126 · Cited by 11
Subjective modeling choices and the robustness of impact evaluations in conservation science
Desbureaux, S. (2021) · Conservation Biology · doi:10.1111/cobi.13728 · Cited by 11
Experimenting with financial professionals
Huber, C., König-Kersting, C., & Marini, M. M. (2024) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2024.107329 · Cited by 11
Competition Laws, Governance, and Firm Value
Levine, R., Lin, C., & Xie, W. (2021) · Social Science Research Network · doi:10.3386/w28908 · Cited by 8
We Need to Talk About Mechanical Turk: What 22,989 Hypothesis Tests Tell Us About Publication Bias and P-Hacking in Online ExperimentsPreprint
Brodeur, A., Cook, N., & Heyes, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4188289 · Cited by 11
Self-citation
Spurious precision in meta-analysis of observational research
Iršová, Z., Bom, P. R. D., Havránek, T., & Rachinger, H. J. (2025) · Nature Communications · doi:10.1038/s41467-025-63261-0 · Cited by 14
Paying for expertise: The effect of experience on insurance demand
Anand, V., Leverty, J. T., & Wunder, K. (2020) · Journal of Risk & Insurance · doi:10.1111/jori.12331 · Cited by 10
Estimating the replicability of technology education research
Buckley, J., Hyland, T., & Seery, N. (2022) · International Journal of Technology and Design Education · doi:10.1007/s10798-022-09787-6 · Cited by 11
More Than Half of Statistically Significant Research Findings in the Environmental Sciences are Actually NotPreprint
Deressa, T. K., Stern, D. I., Vangronsveld, J., Minx, J. C., Lizin, S., Malina, R. M., & Bruns, S. B. (2023) · doi:10.32942/x24g6z · Cited by 13
Are lessons being learnt from the replication crisis or will the revolution devour its children? Open Q science from the editor's perspective
Hüttel, S., & Heß, S. (2024) · Q Open · doi:10.1093/qopen/qoae019 · Cited by 10
How improper dichotomization and the misrepresentation of uncertainty undermine social science research
Rigdon, E. E. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.114086 · Cited by 9
US school finance: Resources and outcomes
Handel, D., & Hanushek, E. A. (2023) · Handbook of the economics of education · doi:10.1016/bs.hesedu.2023.03.003 · Cited by 12
Marginal propensity to consume and unemployment: A meta-analysis
Sokolova, A. (2023) · Review of Economic Dynamics · doi:10.1016/j.red.2023.08.005 · Cited by 9
The impact of announcements of regulatory and law enforcement penalties on stock market valuation of US banks from 2000 to 2022
Brož, V. (2024) · Journal of Financial Regulation and Compliance · doi:10.1108/jfrc-01-2024-0007 · Cited by 8
Socioeconomic impacts of land restoration in agriculture: A systematic review
Malan, M., Berkhout, E., Duchoslav, J., Voors, M. J., & van der Esch, S. (2024) · Q Open · doi:10.1093/qopen/qoae022 · Cited by 8
Systematic review and meta-analysis on antimicrobial resistance and drug resistance in Saudi Arabia
Kiran, M. A., Alghamdi, S. A., Ashgar, S. S., Alhindi, Z., Al‐Abdullah, N., & Dablool, A. S. (2024) · Journal of Global Antimicrobial Resistance · doi:10.1016/j.jgar.2024.07.013 · Cited by 8
External Validity and Evidence Accumulation
Slough, T. L., & Tyson, S. A. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009375856 · Cited by 7
We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online ExperimentsPreprint
Brodeur, A., Cook, N., & Heyes, A. (2022) · doi:10.31222/osf.io/a9vhr · Cited by 8
Self-citation
Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?Preprint
Brodeur, A., Cook, N., Hartley, J., & Heyes, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4180594 · Cited by 10
Self-citation
Financial Incentives and Performance: A Meta-Analysis of Economics EvidencePreprint
Cala, P., Havranek, T., Irsova, Z., Matoušek, J., Irsova, Z., & Novák, J. (2022) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/wbe9k · Cited by 9
Experiments in finance
Kirchler, M., & Weitzel, U. (2023) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2023.106975 · Cited by 7
Crowdsourced data in public administration research: A review and look to the future
Stritch, J. M., Pedersen, M. J., & Pezo, I. (2024) · Public Administration Review · doi:10.1111/puar.13823 · Cited by 8
The dilemma between fertility and work: How did the Universal Two-Child policy affect Chinese women’s labor income?
Chen, Y., & Wang, Z. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308709 · Cited by 7
How Evidential Pluralism Can Help Clarify the Nature of the Relationship Between Unemployment and Entrepreneurship
Saridakis, G. (2024) · Journal of Entrepreneurship and Innovation in Emerging Economies · doi:10.1177/23939575241257137 · Cited by 7
What taxpayers, governments and tax economists do – and what they should do
Slemrod, J. (2023) · Fiscal Studies · doi:10.1111/1475-5890.12324 · Cited by 7
Rural electrification, the credibility revolution, and the limits of evidence-based policy
Peters, J., & Schmidt, C. Μ. (2025) · Environment and Development Economics · doi:10.1017/s1355770x24000263 · Cited by 7
Interpretable Research Replication Prediction via Variational Contextual Consistency Sentence Masking
Luo, T., Meng, R., Wang, X., & Liu, Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.305 · Cited by 10
Critical Values Robust to P-Hacking
McCloskey, A., & Michaillat, P. (2024) · The Review of Economics and Statistics · doi:10.1162/rest_a_01456 · Cited by 11
Forking Paths in Empirical StudiesPreprint
Coqueret, G. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3999379 · Cited by 8
Mind Your SortsPreprint
Soebhag, A., van Vliet, B., & Verwijmeren, P. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4136672 · Cited by 7
Do Pre-Registration and Pre-Analysis Plans Reduce P-Hacking and Publication Bias?Preprint
Brodeur, A., Cook, N., Hartley, J., & Heyes, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4188287 · Cited by 7
Self-citation
Publication bias in the social sciences since 1959: Application of a regression discontinuity framework
Jerke, J., Velicu, A., Winter, F., & Rauhut, H. (2025) · PLoS ONE · doi:10.1371/journal.pone.0305666 · Cited by 7
The Impact of Infrastructure on Development Outcomes: A Meta-Analysis
Foster, V., Gorgulu, N., Jain, D., Straub, S., & Vagliasindi, M. (2025) · The World Bank Research Observer · doi:10.1093/wbro/lkaf003 · Cited by 11
What Impacts Should We Expect From Tutoring at Scale? Exploring Meta-Analytic Generalizability
Kraft, M. A., Schueler, B. E., & Falken, G. T. (2026) · Review of Educational Research · doi:10.3102/00346543261446660 · Cited by 18
Yes, money matters, but the details can make all the difference
McGee, J. B. (2023) · Journal of Policy Analysis and Management · doi:10.1002/pam.22519 · Cited by 6
Improved marginal likelihood estimation via power posteriors and importance sampling
Li, Y., Wang, N., & Yu, J. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.11.009 · Cited by 6
The Case for Preregistering Quasi-Experimental Program and Policy Evaluations
Dee, T. S. (2025) · Evaluation Review · doi:10.1177/0193841x251326738 · Cited by 9
Ideological bias in the production of research findings
Borjas, G. J., & Breznau, N. (2026) · Science Advances · doi:10.1126/sciadv.adz7173 · Cited by 5
Increasing Take‐Up of Social Benefits: A Meta‐Analysis of Field Experiments
Bendtsen, K. (2026) · Journal of Policy Analysis and Management · doi:10.1002/pam.70085 · Cited by 5
No ground truth? No problem: Improving administrative data linking using active learning and a little bit of guile
Tahamont, S., Jelveh, Z., McNeill, M., Yan, S., Chalfin, A. J., & Hansen, B. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283811 · Cited by 5
How Credible Is the Credibility Revolution?
Lang, K. (2024) · Journal of Labor Economics · doi:10.1086/732772 · Cited by 5
Measuring transparency in the social sciences: political science and international relations
Scoggins, B., & Robertson, M. P. (2024) · Royal Society Open Science · doi:10.1098/rsos.240313 · Cited by 7
Election Outcomes and Affective Polarization in the United States
Phillips, J. B., & Warner, S. B. (2026) · Political Research Quarterly · doi:10.1177/10659129251411892 · Cited by 5
Do robots boost productivity? A quantitative meta‐study
Schneider, F. (2025) · Journal of Economic Surveys · doi:10.1111/joes.70042 · Cited by 8
Breaking bad? Playing the fool and constructing the ‘bad researcher’ in entrepreneurship
Harrison, R. (2024) · Journal of Business Venturing Insights · doi:10.1016/j.jbvi.2024.e00484 · Cited by 4
A protocol for structured robustness reproductions and replicability assessments
Peters, J., Brodeur, A., Dreber, A., Johannesson, M., Neubauer, F., & Rose, J. A. R. (2025) · Q Open · doi:10.1093/qopen/qoaf004 · Cited by 5
Self-citation
Methods to Estimate Causal Effects. An Overview on IV, DiD and RDD and a Guide on How to Apply them in Practice
Collischon, M. (2022) · Soziale Welt · doi:10.5771/0038-6073-2022-4-713 · Cited by 4
Contexts of Convenience: Generalizing from Published Evaluations of School Finance Policies
Handel, D., & Hanushek, E. A. (2024) · Evaluation Review · doi:10.1177/0193841x241228335 · Cited by 4
The cost of environmental inequality: Evidence from offsite investment
Zhou, M., & Jiang, K. (2025) · Borsa Istanbul Review · doi:10.1016/j.bir.2025.01.014 · Cited by 4
Dismissing the use of P-values and statistical significance testing in scientific research: new methodological perspectives in toxicology and risk assessment
Berselli, N., Filippini, T., Adani, G., & Vinceti, M. (2021) · Elsevier eBooks · doi:10.1016/b978-0-323-85215-9.00002-7 · Cited by 6
Estimating the extent of inflated significance in economicsPreprint
Bruns, S. B., Deressa, T. K., Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2022) · doi:10.31222/osf.io/h29xn · Cited by 4
Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screeningPreprint
Peters, J., Vance, C., & Bensch, G. (2022) · doi:10.31219/osf.io/sw6kd · Cited by 4
Heterogeneity in effect size estimates: Empirical evidence and practical implicationsPreprint
Holzmeister, F., Johannesson, M., BOHM, R. M., Dreber, A., Huber, J., & Kirchler, M. (2024) · doi:10.31222/osf.io/583un · Cited by 4
Does immigration affect native wages? A meta-analysis
Aubry, A., Héricourt, J., Marchal, L., & Nedoncelle, C. (2026) · Labour Economics · doi:10.1016/j.labeco.2025.102815 · Cited by 3
Predicting replication outcomes and generalizability of results
Dreber, A. (2025) · European Review of Agricultural Economics · doi:10.1093/erae/jbaf035 · Cited by 3
Servitization and firm performance: a p-curve analysis
Latan, H., Jabbour, A. B. L. D. S., Jabbour, C. J. C., Ali, M., & Filho, M. G. (2024) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-07-2023-0392 · Cited by 3
Researchers' degrees of flexibility: Revisiting COVID‐19 policy evaluations
Weill, J., Stigler, M., Deschênes, O., & Springborn, M. (2024) · Economic Inquiry · doi:10.1111/ecin.13273 · Cited by 3
The effects of ethnic sentiment and social differentiation on pastoralists’ willingness to turn out of pasture
Li, P., Xie, Y., Li, X., & Xia, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-91059-z · Cited by 3
Além da manipulação: ciência aberta e a nova era da confiabilidade científica
Limongi, R., Rogers, P., Bandeira, P., & Lula, C. D. S. G. (2025) · Internext · doi:10.18568/internext.v20i1.819 · Cited by 3
Kernel Regression Coefficients for Practical Significance
Vinod, H. D. (2022) · Journal of risk and financial management · doi:10.3390/jrfm15010032 · Cited by 3
Quantitative Macroeconomics: Lessons Learned from Fourteen Replications
Kirkby, R. (2022) · Computational Economics · doi:10.1007/s10614-022-10234-w · Cited by 4
Business cycle characteristics of Mediterranean economies: a secular trend and cycle dynamics perspective
Solms, A., & Süßmuth, B. (2022) · International Economics and Economic Policy · doi:10.1007/s10368-022-00544-7 · Cited by 3
Meta-Analysis
Picchio, M. (2022) · Springer eBooks · doi:10.1007/978-3-319-57365-6_350-1 · Cited by 3
Inaccurate forecasting of a randomized controlled trial
Ahrenshop, M., Golden, M. A., Gulzar, S., & Sonnet, L. (2023) · Journal of Experimental Political Science · doi:10.1017/xps.2023.28 · Cited by 3
Emission reduction effects of electric vehicle promotion and application pilots: Cooperative pilots vs single pilots
Zhao, X., Li, X., Liu, Y., & Mao, Y. (2025) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2025.104456 · Cited by 3
Agricultural and applied economists' views on pre‐registration and pre‐analysis plans for empirical research
Grebitus, C., & Hu, W. (2025) · Journal of the Agricultural and Applied Economics Association · doi:10.1002/jaa2.70011 · Cited by 3
Spillover Effects of the China‐ASEAN Free Trade Agreement on Agricultural Trade: A Structural Gravity Model Approach
Chang, T., Zhu, W., & Chen, Y. (2025) · Agribusiness · doi:10.1002/agr.22066 · Cited by 3
Estimating the extent and sources of model uncertainty in political science
Ganslmeier, M., & Vlandas, T. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2414926122 · Cited by 3
Dynamic interactionism between research fraud and research culture: a commentary to Harvey’s analysis
Orhan, M. A. (2021) · Quality in Higher Education · doi:10.1080/13538322.2021.1857900 · Cited by 3
Recent developments in inference: practicalities for applied economics
Michler, J. D., & Josephson, A. (2022) · Edward Elgar Publishing eBooks · doi:10.4337/9781800372054.00019 · Cited by 5
The Economics of Computational ReproducibilityPreprint
Colliard, J.-É., Hurlin, C., & Hurlin, C. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3418896 · Cited by 3
Welfare Effects of Capital Controls
Andreasen, E., Bauducco, S., & Dardati, E. (2021) · doi:10.18235/0003307 · Cited by 3
When Does Monetary Policy Sway House Prices? A Meta-AnalysisPreprint
Bajzík, J., Ehrenbergerová, D., & Havránek, T. (2021) · RePEc: Research Papers in Economics · doi:10.31222/osf.io/npeqs · Cited by 3
The Economics of Computational ReproducibilityPreprint
Hurlin, C., Colliard, J.-É., & Pérignon, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4811513 · Cited by 3
A Protocol for Structured Robustness Reproductions and Replicability AssessmentsPreprint
Peters, J., Brodeur, A., Dreber, A., Johannesson, M., Neubauer, F., & Rose, J. A. R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5135862 · Cited by 3
Self-citation
Salary inequality in young professors: evidence from public U.S. economic department
Bian, Y., Kong, W., & Zhang, Q. (2024) · Applied Economics · doi:10.1080/00036846.2024.2304096 · Cited by 2
Liberal Democracy in Eclipse: The Economy Post-February 2020, Nourished by Four Odds—Disease Phobia, Profanity in the Media, Unethical Profit Seekers and Tail Risk
Majumder, D. (2024) · Theoretical Economics Letters · doi:10.4236/tel.2024.142034 · Cited by 2
An Investigation of p-Hacking in E-Commerce A/B Testing
Miller, A. P., & Hosanagar, K. (2025) · Information Systems Research · doi:10.1287/isre.2024.0872 · Cited by 2
P -hacking and Significance Stars
Naguib, C. (2025) · The Economic Journal · doi:10.1093/ej/ueaf073 · Cited by 2
Humans in the Loop: The Next Frontier in the Credibility Revolution
Stevenson, M. T., & Fischman, J. B. (2026) · Journal of Economic Literature · doi:10.1257/jel.20261775 · Cited by 2
Epidemiology and development economics two sides of the same coin in impact evaluation
Vigneri, M., Clarke, M. J., Exley, J., Tugwell, P., Welch, V. A., & White, H. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.029 · Cited by 2
Will I tell you that you are smart (dumb)? Deceiving Others about their IQ or about a Random Draw
Burro, G., & Castagnetti, A. (2021) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2022.101920 · Cited by 2
Economics Peer-Review: Problems, Recent Developments, and Reform Proposals
Siemroth, C. (2023) · The American Economist · doi:10.1177/05694345241269484 · Cited by 3
Long-term effects of the Targeting the Ultra Poor program—A reproducibility and replicability assessment of Banerjee et al. (2021)
Rose, J. A. R., Neubauer, F., & Peters, J. (2024) · Q Open · doi:10.1093/qopen/qoae031 · Cited by 2
Unbiased Interest Modeling in Sequential Basket Analysis: Addressing Repetition Bias with Multi-Factor Estimation
Deng, Z., Li, J., Liu, W., & Zhao, J. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3718100 · Cited by 2
The Achievement Effects of Scaling Early Literacy Reforms
Novicoff, S., & Dee, T. S. (2025) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737251349178 · Cited by 2
Unlucky Number 13? Manipulating Evidence Subject to Snooping
Hassler, U., & Pohle, M. (2022) · International Statistical Review · doi:10.1111/insr.12488 · Cited by 4
Rounding and Other Pitfalls in Meta-Studies on P-Hacking and Publication Bias: A Comment on Brodeur Et Al. (2020)Preprint
Kranz, S., & Pütz, P. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3848786 · Cited by 2
Star Wars at Central BanksPreprint
Gorajek, A., Bank, J., Staib, A., Malin, B., & Fitchett, H. (2021) · doi:10.21034/sr.620 · Cited by 2
Hedge Fund Performance: A Quantitative SurveyPreprint
Yang, F., Havránek, T., Iršová, Z., & Novak, J. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4151821 · Cited by 2
Unpacking P-Hacking and Publication Bias
Brodeur, A., Carrell, S., Figlio, D., & Lusher, L. R. (2023) · National Bureau of Economic Research · doi:10.3386/w31548 · Cited by 2
Self-citation
Where Have All the Alphas Gone? A Meta-Analysis of Hedge Fund PerformancePreprint
Yang, F., Havránek, T., Iršová, Z., & Novák, J. (2024) · doi:10.31222/osf.io/ps2yn · Cited by 2
Questionable Research Practices and Open Science in Quantitative CriminologyPreprint
Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2024) · CrimRxiv · doi:10.21428/cb6ab371.0d3ec44f · Cited by 3
The Endogenous Nature of War and Its Economic Consequences.Preprint
Adamson, J., & Stephenson, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4869123 · Cited by 2
The Perks and Perils of Machine Learning in Business and Economic ResearchPreprint
Dudda, T. L., & Hornuf, L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4981802 · Cited by 2
Do Experimental Asset Market Results Replicate? High‐Powered Preregistered Replications of 17 Key Results
Huber, C., Holzmeister, F., Johannesson, M., König-Kersting, C., Dreber, A., Huber, J., & Kirchler, M. (2026) · The Journal of Finance · doi:10.2139/ssrn.5048949 · Cited by 3
A Meta-Meta-Analysis of Behavior Change Interventions: Two Tales of Behavior ChangePreprint
Kaiser, T., Kloidt, J., Mata, J., & Hertwig, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5251012 · Cited by 2
Power and bias in industrial relations research
Doucouliagos, A., Doucouliagos, H., & Stanley, T. D. (2023) · British Journal of Industrial Relations · doi:10.1111/bjir.12746 · Cited by 3
Investigating Potential Interactive Effects Between Limbal Rings and Facial Attractiveness
Howard, M. C., & Brown, M. (2024) · Evolutionary Psychological Science · doi:10.1007/s40806-024-00413-3 · Cited by 1

Authors: Brodeur, A., Cook, N., & Heyes, A.
Affiliations: University of Ottawa (CA)
ORCID: Brodeur, Abel
Cook, Nikolai
Heyes, Anthony
Venue: American Economic Review
Publisher: American Economic Association
Published: 2020-11-01
Volume: 110
Issue: 11
Pages: 3634-3660
ISSN: 0002-8282 (ISSN-L)
DOI: 10.1257/aer.20190687
OpenAlex: W3047016911
Cited by: 333
References: 41
Citations by year: 2026: 44 · 2025: 50 · 2024: 91 · 2023: 49 · 2022: 47 · 2021: 41 · 2020: 7
FWCI: 25.39
Type: journal article
Language: en
Open access: closed
Locations: American Economic Review (publishedVersion)
Open Research Exeter (University of Exeter) (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Figshare (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords: p-hacking, publication bias, credibility revolution, causal inference, instrumental variables, difference-in-differences, regression discontinuity design, randomized controlled trials, hypothesis testing, economics journals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks