Cited by 333Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsscientometrics and bibliometrics research

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Brodeur, Abel, Cook, Nikolai, Heyes, Anthony·2020·American Economic Review

DOI 10.1257/aer.20190687

Open this paper in CoChat — free account

Abstract

The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-in-differences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over 21,000 hypothesis tests published in 25 leading economics journals, we find that the extent of p-hacking and publication bias varies greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that (i) papers published in the Top 5 journals are different to others; (ii) the journal “revise and resubmit” process mitigates the problem; (iii) things are improving through time. (JEL A14, C12, C52)

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1257/aer.20190687

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“18See Ioannidis et al. (2017) for an investigation of statistical power and bias in economics.”
Background
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr043 · Cited by 2,135
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
The American Economic Review
(2009) · American Economic Review · doi:10.1257/aer.99.1.i · Cited by 1,764
The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Weak Instruments in Instrumental Variables Regression: Theory and Practice
Andrews, I., Stock, J. H., & Sun, L. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080218-025643 · Cited by 849
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection*
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605

Show all 41 references

Research works printed but not linked: 3

Blanco-Perez Cristina, 2019, IZA World of Labor, 2019, 467
Leamer Edward E, 1983, American Economic Review, 73, 31
McCloskey Donald N, 1985, American Economic Review, 75, 201

Sorted by citations

RCTs to Scale: Comprehensive Evidence From Two Nudge Units
DellaVigna, S., & Linos, E. (2022) · Econometrica · doi:10.3982/ecta18709 · Cited by 510
Financial Education Affects Financial Knowledge and Downstream Behaviors
Kaiser, T., Lusardi, A., Menkhoff, L., & Urban, C. (2022) · Journal of Financial Economics · doi:10.1016/j.jfineco.2021.09.022 · Cited by 508
The Short-Term Economic Consequences of Covid-19: Exposure to Disease, Remote Work and Government Response
Beland, L., Brodeur, A., & Wright, T. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0270341 · Cited by 321
Self-citation
Can Policies Stall the Fertility Fall? A Systematic Review of the (Quasi‐) Experimental Literature
Bergsvik, J., Fauske, A., & Hart, R. K. (2021) · Population and Development Review · doi:10.1111/padr.12431 · Cited by 259
Causality redux: The evolution of empirical methods in accounting research and the growth of quasi-experiments
Armstrong, C., Kepler, J. D., Samuels, D., & Taylor, D. J. (2022) · Journal of Accounting and Economics · doi:10.1016/j.jacceco.2022.101521 · Cited by 243
The COVID-19 pandemic and the 2020 US presidential election
Baccini, L., Brodeur, A., & Weymouth, S. J. (2021) · Journal of Population Economics · doi:10.1007/s00148-020-00820-3 · Cited by 199
Self-citation
Nonreplicable publications are cited more than replicable ones
Serra-Garcia, M., & Gneezy, U. (2021) · Science Advances · doi:10.1126/sciadv.abd1705 · Cited by 177
Estimating the Armington elasticity: The importance of study design and publication bias
Bajzik, J., Havranek, T., Irsova, Z., & Schwarz, J. (2020) · Journal of International Economics · doi:10.1016/j.jinteco.2020.103383 · Cited by 150
Technology and jobs: A systematic literature review
Hötte, K., Somers, M. A., & Theodorakopoulos, A. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122750 · Cited by 185
Meta-analysis of Empirical Estimates of Loss Aversion
Brown, A. L., Imai, T., Vieider, F. M., & Camerer, C. F. (2024) · Journal of Economic Literature · doi:10.1257/jel.20221698 · Cited by 180
Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, P. G. S., & Barnett, A. G. (2023) · BMC Medicine · doi:10.1186/s12916-023-03048-6 · Cited by 149
Measuring capital-labor substitution: The importance of method choices and publication bias
Gechert, S., Havránek, T., Iršová, Z., & Kolcunová, D. (2021) · Review of Economic Dynamics · doi:10.1016/j.red.2021.05.003 · Cited by 139
Big little lies: a compendium and simulation of p -hacking strategies
Stefan, A. M., & Schönbrodt, F. D. (2023) · Royal Society Open Science · doi:10.1098/rsos.220346 · Cited by 177
Methodological Variation in Empirical Corporate Finance
Mitton, T. (2021) · Review of Financial Studies · doi:10.1093/rfs/hhab030 · Cited by 99
The Distribution of School Spending Impacts
Jackson, C. K., & Mackevicius, C. (2021) · doi:10.3386/w28517 · Cited by 89
Individual discount rates: a meta-analysis of experimental evidence
Matousek, J., Havránek, T., & Iršová, Z. (2021) · Experimental Economics · doi:10.1007/s10683-021-09716-9 · Cited by 86
Political Language in Economics
Jelveh, Z., Kogut, B., & Naidu, S. (2014) · The Economic Journal · doi:10.1093/ej/ueae026 · Cited by 89
Unemployment and health: A meta‐analysis
Picchio, M., & Ubaldi, M. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12588 · Cited by 99
Does Democracy Matter?
Gerring, J., Knutsen, C. H., & Berge, J. (2022) · Annual Review of Political Science · doi:10.1146/annurev-polisci-060820-060910 · Cited by 76
What Impacts Can We Expect from School Spending Policy? Evidence from Evaluations in the United States
Jackson, C. K., & Mackevicius, C. L. (2023) · American Economic Journal Applied Economics · doi:10.1257/app.20220279 · Cited by 91
Heterogeneity in effect size estimates
Holzmeister, F., Johannesson, M., BOHM, R. M., Dreber, A., Huber, J., & Kirchler, M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2403490121 · Cited by 89
Carbon taxes and emission trading systems: Which one is more effective in reducing carbon emissions?—A meta-analysis
Ahmad, M., Li, X. F., & Wu, Q. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.143761 · Cited by 72
How puzzling is the forward premium puzzle? A meta-analysis
Zigraiova, D., Havranek, T., Irsova, Z., & Novak, J. (2021) · European Economic Review · doi:10.1016/j.euroecorev.2021.103714 · Cited by 71
The Populist Backlash Against Globalization: A Meta-Analysis of the Causal Evidence
Scheiring, G., Serrano‐Alarcón, M., Moise, A. D., McNamara, C. L., & Stückler, D. (2024) · British Journal of Political Science · doi:10.1017/s0007123424000024 · Cited by 90
Meta‐analysis of social science research: A practitioner's guide
Iršová, Z., Doucouliagos, H., Havránek, T., & Stanley, T. D. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12595 · Cited by 76
Detecting p‐Hacking
Elliott, G., Kudrin, N., & Wüthrich, K. (2022) · Econometrica · doi:10.3982/ecta18583 · Cited by 79
Regression discontinuity designs in agricultural and environmental economics
Wuepper, D., & Finger, R. (2023) · European Review of Agricultural Economics · doi:10.1093/erae/jbac023 · Cited by 55
Examining the generalizability of research findings from archival data
Delios, A., Clemente, E. G., Wu, T., Tan, H., Wang, Y., Gordon, M., Viganola, D., Chen, Z., Dreber, A., Johannesson, M., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2120377119 · Cited by 66
Publication and Attenuation Biases in Measuring Skill Substitution
Havránek, T., Iršová, Z., Laslopová, Ľ., & Zeynalova, O. (2022) · The Review of Economics and Statistics · doi:10.1162/rest_a_01227 · Cited by 49
Do economists replicate?
Peters, J., Fiala, N., & Neubauer, F. (2023) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2023.05.009 · Cited by 55
Questionable Research Practices and Open Science in Quantitative Criminology
Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2021) · Journal of Quantitative Criminology · doi:10.1007/s10940-021-09525-6 · Cited by 66
Quantitative Political Science Research Is Greatly Underpowered
Arel‐Bundock, V., Briggs, R. C., Doucouliagos, H., Aviña, M. M., & Stanley, T. D. (2024) · The Journal of Politics · doi:10.1086/734279 · Cited by 54
Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments
Muralidharan, K., Romero, M., & Wüthrich, K. (2020) · The Review of Economics and Statistics · doi:10.1162/rest_a_01317 · Cited by 102
Non-standard errors in asset pricing: Mind your sorts
Soebhag, A., van Vliet, B., & Verwijmeren, P. (2024) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2024.101517 · Cited by 47
The Null Result Penalty
Chopra, F., Haaland, I., Roth, C., & Stegmann, A. (2023) · The Economic Journal · doi:10.1093/ej/uead060 · Cited by 57
Unpacking p-Hacking and Publication Bias
Brodeur, A., Carrell, S., Figlio, D., & Lusher, L. R. (2023) · American Economic Review · doi:10.1257/aer.20210795 · Cited by 47
Self-citation
Replications in agricultural economics
Finger, R., Grebitus, C., & Henningsen, A. (2023) · Applied Economic Perspectives and Policy · doi:10.1002/aepp.13386 · Cited by 43
Rain, Rain, Go Away: 195 Potential Exclusion-Restriction Violations for Studies Using Weather as an Instrumental VariablePreprint
Mellon, J. (2021) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/9qj4f · Cited by 49
Preference-based instrumental variables in health research rely on important and underreported assumptions: a systematic review
Widding‐Havneraas, T., Chaulagain, A., Lyhmann, I., Zachrisson, H. D., Elwert, F., Markussen, S., McDaid, D., & Mykletun, A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.06.006 · Cited by 40
Causal Machine Learning and its use for public policy
Lechner, M. (2023) · Zeitschrift für schweizerische Statistik und Volkswirtschaft/Schweizerische Zeitschrift für Volkswirtschaft und Statistik/Swiss journal of economics and statistics · doi:10.1186/s41937-023-00113-y · Cited by 57
Unemployment Scarring Effects: An Overview and Meta-analysis of Empirical Studies
Filomena, M. (2023) · Italian Economic Journal · doi:10.1007/s40797-023-00228-4 · Cited by 40
A framework for evaluating reproducibility and replicability in economics
Dreber, A., & Johannesson, M. (2024) · Economic Inquiry · doi:10.1111/ecin.13244 · Cited by 49
Rain, rain, go away: 194 potential exclusion‐restriction violations for studies using weather as an instrumental variable
Mellon, J. (2024) · American Journal of Political Science · doi:10.1111/ajps.12894 · Cited by 38
Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies
Chang, X., Gao, H., & Li, W. (2024) · Journal of Accounting Research · doi:10.1111/1475-679x.12579 · Cited by 35
Nudging for Tax Compliance: A Meta-Analysis
Antinyan, A., & Asatryan, Z. (2024) · The Economic Journal · doi:10.1093/ej/ueae088 · Cited by 35
The Significance of Data-Sharing Policy
Askarov, Z., Doucouliagos, A., Doucouliagos, H., & Stanley, T. D. (2022) · Journal of the European Economic Association · doi:10.1093/jeea/jvac053 · Cited by 38
Big Little Lies: A Compendium and Simulation of p-Hacking StrategiesPreprint
Stefan, A. M., & Schönbrodt, F. D. (2022) · doi:10.31234/osf.io/xy2dk · Cited by 35
When Does Monetary Policy Sway House Prices? A Meta-Analysis
Ehrenbergerová, D., Bajzík, J., & Havránek, T. (2022) · IMF Economic Review · doi:10.1057/s41308-022-00185-5 · Cited by 38
False Discovery in A/B Testing
Berman, R., & den Bulte, C. V. (2020) · Management Science · doi:10.1287/mnsc.2021.4207 · Cited by 36
Relative Risk Aversion: A Meta‐Analysis
Elminejad, A., Havránek, T., & Iršová, Z. (2025) · Journal of Economic Surveys · doi:10.1111/joes.12689 · Cited by 43
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Comment
Kranz, S., & Pütz, P. (2022) · American Economic Review · doi:10.1257/aer.20210121 · Cited by 31
Is Investment in Preprimary Education Too Low? Lessons from (Quasi) Experimental Evidence across Countries
Holla, A., Bendini, M., Dinarte, L., & Trako, I. (2021) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-9723 · Cited by 31
Student Employment and Education: A Meta-Analysis
Kroupová, K., Havránek, T., & Iršová, Z. (2024) · Economics of Education Review · doi:10.1016/j.econedurev.2024.102539 · Cited by 57
On the reliability of published findings using the regression discontinuity design in political science
Stommes, D., Aronow, P. M., & Sävje, F. (2023) · Research & Politics · doi:10.1177/20531680231166457 · Cited by 35
The p-Value Debate and Statistical (Mal)practice – Implications for the Agricultural and Food Economics Community
Heckelei, T., Hüttel, S., Rommel, J., & Rommel, J. (2023) · German Journal of Agricultural Economics · doi:10.30430/gjae.2023.0231 · Cited by 38
Rain, Rain, Go Away: 137 Potential Exclusion-Restriction Violations for Studies Using Weather as an Instrumental VariablePreprint
Mellon, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3715610 · Cited by 28
Experimental and behavioral economics to inform agri-environmental programs and policies
Palm‐Forster, L. H., & Messer, K. D. (2021) · Handbook of agricultural economics · doi:10.1016/bs.hesagr.2021.10.006 · Cited by 26
Mass Reproducibility and Replicability: A New HopePreprint
Brodeur, A., Mikola, D., & Cook, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4790780 · Cited by 31
Self-citation
The effectiveness of cash and cash plus interventions on livelihoods outcomes: Evidence from a systematic review and meta-analysisPreprint
Leight, J., Hirvonen, K., & Zafar, S. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/dnc2r · Cited by 27
Intertemporal substitution in labor supply: A meta-analysis
Elminejad, A., Havránek, T., Horváth, R., & Iršová, Z. (2023) · Review of Economic Dynamics · doi:10.1016/j.red.2023.10.001 · Cited by 25
Promoting Reproducibility and Replicability in Political Science
Brodeur, A., Esterling, K., Peters, J., Bueno, N., DESPOSATO, S. W., Dreber, A., Genovese, F., GREEN, D. P., Hepplewhite, M., de la Guardia, F. H., et al. (2024) · Research & Politics · doi:10.1177/20531680241233439 · Cited by 30
Self-citation
The Potency of Quality Incentives: Evidence from the Indonesian Dairy Value Chain
Treurniet, M. (2021) · American Journal of Agricultural Economics · doi:10.1111/ajae.12176 · Cited by 25
A Framework for Evaluating Reproducibility and Replicability in EconomicsPreprint
Dreber, A., & Johannesson, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4458153 · Cited by 26
Ulysses' pact or U lysses' raft: Using pre‐analysis plans in experimental and nonexperimental research
Janzen, S., & Michler, J. D. (2021) · Applied Economic Perspectives and Policy · doi:10.1002/aepp.13133 · Cited by 21
P ‐hacking in meta‐analyses: A formalization and new meta‐analytic methods
Mathur, M. B. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1701 · Cited by 25
Replicability in Brain Imaging
Kelly, R. E., & Hoptman, M. J. (2022) · Brain Sciences · doi:10.3390/brainsci12030397 · Cited by 24
P-Hacking, Data Type and Data-Sharing Policy
Brodeur, A., Cook, N., & Neisser, C. (2022) · The Economic Journal · doi:10.1093/ej/uead104 · Cited by 21
Self-citation
Meta-Analysis of Empirical Estimates of Loss-AversionPreprint
Brown, A. L., Imai, T., Vieider, F. M., & Camerer, C. (2020) · Social Science Research Network · doi:10.31222/osf.io/hnefr · Cited by 161
Persistence studies: a new kind of economic history?
Cioni, M., Federico, G., & Vasta, M. (2022) · Review of Regional Research · doi:10.1007/s10037-022-00167-0 · Cited by 20
Detecting p-hackingPreprint
Elliott, G., Kudrin, N., & Wüthrich, K. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1906.06711 · Cited by 20
What meta‐research has taught us about research and changes to research practices
Ioannidis, J. P. A. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12666 · Cited by 20
Deindustrialisation and the post-socialist mortality crisis
Scheiring, G., Azarova, A., Irdam, D., Doniec, K., McKee, M., Stückler, D., & KING, L. P. (2022) · Cambridge Journal of Economics · doi:10.1093/cje/beac072 · Cited by 17
The case for mixed methods research: Embracing qualitative research to understand the (informal) economy
Cartwright, E. J., & Igudia, E. (2023) · Review of Development Economics · doi:10.1111/rode.13069 · Cited by 31
Is economics self‐correcting? Replications in the American Economic Review
Peters, J., Fiala, N., & Neubauer, F. (2024) · Economic Inquiry · doi:10.1111/ecin.13222 · Cited by 17
Towards improved and more transparent ethics in randomised controlled trials in development social science
Evans, D. K. (2023) · Journal of Development Effectiveness · doi:10.1080/19439342.2023.2196978 · Cited by 17
Estimating the Impact of Weather on Agriculture
Michler, J. D., Josephson, A., Kilic, T., & Murray, S. (2021) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-9867 · Cited by 22
Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?Preprint
Brodeur, A., Cook, N., Hartley, J., & Heyes, A. (2022) · doi:10.31222/osf.io/uxf39 · Cited by 17
Self-citation
How Does Data Access Shape Science? The Impact of Federal Statistical Research Data Centers on Economics Research
Nagaraj, A., & Tranchero, M. (2023) · National Bureau of Economic Research · doi:10.3386/w31372 · Cited by 17
Financial crime and punishment: A meta‐analysis
de Batz, L., & Evžen Kočenda, A. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12580 · Cited by 17
Selective reporting of placebo tests in top economics journals
Dreber, A., Johannesson, M., & Yang, Y. (2024) · Economic Inquiry · doi:10.1111/ecin.13217 · Cited by 18
U.S. School Finance: Resources and Outcomes
Handel, D., & Hanushek, E. A. (2022) · National Bureau of Economic Research · doi:10.3386/w30769 · Cited by 18
Selective and (mis)leading economics journals: Meta‐research evidence
Askarov, Z., Doucouliagos, A., Doucouliagos, H., & Stanley, T. D. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12598 · Cited by 19
Revisiting the Income Inequality-Crime Puzzle
Pazzona, M. (2024) · World Development · doi:10.1016/j.worlddev.2023.106520 · Cited by 17
Reconsidering macroeconomic policy prescriptions with meta-analysis
Gechert, S. (2022) · Industrial and Corporate Change · doi:10.1093/icc/dtac005 · Cited by 15
When Shooting for the Stars Becomes Aiming for Asterisks: P -Hacking in Family Business Research
Brinkerink, J. (2021) · Entrepreneurship Theory and Practice · doi:10.1177/10422587211050354 · Cited by 14
Tempest in a teacup: An analysis of p-Hacking in organizational research
Gupta, A., & Bosco, F. A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281938 · Cited by 17
Methods Used in Economic Research: An Empirical Study of Trends and Levels
Paldam, M. (2021) · Economics · doi:10.1515/econ-2021-0003 · Cited by 14
Questionable Research Practices and Open Science in Quantitative CriminologyPreprint
Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2021) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/bwm7s · Cited by 12
Income growth and inequality reduction through rural industrial integration: Evidence from small-scale farmers in China
Jian-hua, Z., Zijin, Z., & Yanxi, S. (2025) · Journal of Asian Economics · doi:10.1016/j.asieco.2025.102009 · Cited by 13
Reproducibility and robustness of economics and political science research
Brodeur, A., Mikola, D., Cook, N., Fiala, L., Brailey, T., Briggs, R. C., de Gendre, A., Dupraz, Y., Gabani, J., Gauriot, R., et al. (2026) · Nature · doi:10.1038/s41586-026-10251-x · Cited by 18
Self-citation
Causal Analysis of Policy Effects on Fertility
Hart, R. K., Bergsvik, J., Fauske, A., & Kim, W. (2024) · Springer eBooks · doi:10.1007/978-3-319-57365-6_451-1 · Cited by 14
The cost of operational complexity: A causal assessment of pre-fire mitigation and wildfire suppression
Young, J. D., Belval, E. J., Gannon, B. M., Wei, Y., O’Connor, C. D., Dunn, C. J., Pietruszka, B. M., Calkin, D. E., & Thompson, M. P. (2024) · Forest Policy and Economics · doi:10.1016/j.forpol.2024.103351 · Cited by 15
Does agricultural cooperative membership improve farm productive performance? A meta‐regression analysis
Ayouba, K., Duvivier, C., & Bussière, C. (2024) · Agricultural Economics · doi:10.1111/agec.12862 · Cited by 19
The evolving ethics of analysis, publication, and transparency in applied economics
Lybbert, T. J., & Buccola, S. T. (2020) · Applied Economic Perspectives and Policy · doi:10.1002/aepp.13131 · Cited by 15
Hedge Fund Performance: A Quantitative Survey
Yang, F., Havránek, T., Iršová, Z., & Novák, J. (2022) · Journal of Economic Surveys · doi:10.1111/joes.12574 · Cited by 13
The Returns to Education: A Meta‐Study
Clark, G., & Nielsen, C. A. A. (2026) · Kyklos · doi:10.1111/kykl.70041 · Cited by 12
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply
Brodeur, A., Cook, N., & Heyes, A. (2022) · American Economic Review · doi:10.1257/aer.20220277 · Cited by 12
Self-citation
Ownership Concentration and Firm Value: New Evidence from Owner Stakes in IPOs
Larraín, B., Roosenboom, P., Sertsios, G., & Francisco Urzúa, I. (2023) · Management Science · doi:10.1287/mnsc.2021.01039 · Cited by 11
Registered reports and associated benefits for agricultural economics
Arpinon, T., & Lefebvre, M. (2024) · Q Open · doi:10.1093/qopen/qoae011 · Cited by 11
Conventional wisdom, meta‐analysis, and research revision in economics
Gechert, S., Mey, B., Opatrný, M., Havránek, T., Stanley, T. D., Bom, P. R. D., Doucouliagos, H., Heimberger, P., Iršová, Z., & Rachinger, H. J. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12630 · Cited by 11

Authors: Brodeur, A., Cook, N., & Heyes, A.
Affiliations: University of Ottawa (CA)
ORCID: Brodeur, Abel
Cook, Nikolai
Heyes, Anthony
Venue: American Economic Review
Publisher: American Economic Association
Published: 2020-11-01
Volume: 110
Issue: 11
Pages: 3634-3660
ISSN: 0002-8282 (ISSN-L)
DOI: 10.1257/aer.20190687
OpenAlex: W3047016911
Cited by: 333
References: 41
Citations by year: 2026: 44 · 2025: 50 · 2024: 91 · 2023: 49 · 2022: 47 · 2021: 41 · 2020: 7
FWCI: 25.39
Type: journal article
Language: en
Open access: closed
Locations: American Economic Review (publishedVersion)
Open Research Exeter (University of Exeter) (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Figshare (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords: p-hacking, publication bias, credibility revolution, causal inference, instrumental variables, difference-in-differences, regression discontinuity design, randomized controlled trials, hypothesis testing, economics journals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks