Cited by 333Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsscientometrics and bibliometrics research

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Brodeur, Abel, Cook, Nikolai, Heyes, Anthony·2020·American Economic Review

DOI 10.1257/aer.20190687

Open this paper in CoChat — free account

Abstract

The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-in-differences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over 21,000 hypothesis tests published in 25 leading economics journals, we find that the extent of p-hacking and publication bias varies greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that (i) papers published in the Top 5 journals are different to others; (ii) the journal “revise and resubmit” process mitigates the problem; (iii) things are improving through time. (JEL A14, C12, C52)

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1257/aer.20190687

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“18See Ioannidis et al. (2017) for an investigation of statistical power and bias in economics.”
Background
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr043 · Cited by 2,135
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
The American Economic Review
(2009) · American Economic Review · doi:10.1257/aer.99.1.i · Cited by 1,764
The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Weak Instruments in Instrumental Variables Regression: Theory and Practice
Andrews, I., Stock, J. H., & Sun, L. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080218-025643 · Cited by 849
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection*
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605

Show all 41 references

Research works printed but not linked: 3

Blanco-Perez Cristina, 2019, IZA World of Labor, 2019, 467
Leamer Edward E, 1983, American Economic Review, 73, 31
McCloskey Donald N, 1985, American Economic Review, 75, 201

Sorted by citations

Items 201–300 of 332

Experimentation Culture: Defend Learning against Intellectual Laziness
Deniz, B. C. (2024) · Management and business review. · doi:10.1177/26941058241301626 · Cited by 1
Less Eligibility, Welfare, and Punishment: The Econometric Evidence
Bonnet, F. (2025) · Crime and Justice · doi:10.1086/735664 · Cited by 1
Impact evaluations in data-scarce environments: The case of stress-tolerant rice varieties in Bangladesh
Michler, J. D., Rafi, D. A. A., Giezendanner, J., Josephson, A., Pede, V. O., & Tellman, B. (2025) · Journal of Development Economics · doi:10.1016/j.jdeveco.2025.103648 · Cited by 2
Operating Systems For Conscious Learning
Weng, J. (2025) · International Conference on Machine Learning Technologies (ICMLT) · doi:10.1109/icmlt65785.2025.11193293 · Cited by 1
The delayed acceptance of female research in economics
Bruns, S. B., Doucouliagos, A., Doucouliagos, H., König, J., Stanley, T. D., & Zigová, K. (2025) · European Economic Review · doi:10.1016/j.euroecorev.2025.105204 · Cited by 1
Metrics methods for scholarship on teaching and learning in economics
Stock, W. A. (2025) · The Journal of Economic Education · doi:10.1080/00220485.2025.2588275 · Cited by 1
Beyond infrastructure: evaluating the financial inclusion effects of China’s BRI participation across partner economies
Chedid, C., Teruel, M., & Ferré, M. (2026) · Development Studies Research · doi:10.1080/21665095.2026.2633999 · Cited by 1
Financial Incentives and Performance: A Meta-Analysis of Experiments in Economics
Cala, P., Havránek, T., Iršová, Z., Lušková, M., Matousek, J., & Novák, J. (2026) · Journal of Political Economy Microeconomics · doi:10.1086/743543 · Cited by 1
Empiryczna rewolucja. Nagroda Banku Szwecji im. Alfreda Nobla w 2021 r.: David Card, Joshua Angrist, Guido Imbens
Buttler, D. (2022) · Ekonomista · doi:10.52335/dvqigjykff51 · Cited by 1
Åpen tilgang
Wenaas, L. (2022) · Nytt Norsk Tidsskrift · doi:10.18261/nnt.39.1.6 · Cited by 1
A systematic review of conference papers presented at two large Japanese psychology conferences in 2013 and 2018: did Japanese social psychologists selectively report p < 0.05 results without peer review?
Hiraishi, K., Miura, A., Higuchi, M., Fujishima, Y., Nakamura, D., & Suyama, M. (2024) · PeerJ · doi:10.7717/peerj.16763 · Cited by 2
Intergenerational Transformation of Empirical Research in Economics
Sawada, Y. (2024) · Springer eBooks · doi:10.1007/978-981-97-1887-0_2 · Cited by 1
Multibenchmark reality checks
Arbués, I., & Matilla‐García, M. (2024) · Economic Modelling · doi:10.1016/j.econmod.2024.106848 · Cited by 1
Fitting z-curves to estimate the size of the UESD file drawer and the replicability of published findings
Frese, J. (2024) · Research & Politics · doi:10.1177/20531680241277634 · Cited by 1
Falling and failing (to learn): Evidence from a nation-wide cybersecurity field experiment with SMEs
Gonzalez-Jimenez, D., Capozza, F., Dirkmaat, T., van de Veer, E., van Druten, A., & Baillon, A. (2025) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2024.106868 · Cited by 1
Recent Developments in Inference: Practicalities for Applied Economics
Michler, J. D., & Josephson, A. (2021) · UA Campus Repository (The University of Arizona) · doi:10.48550/arxiv.2107.09736 · Cited by 1
Bootstrap Version of Rao–Blackwellization to Two-Step and Instrumental Variable Estimators
Vinod, H. D. (2022) · Journal of Quantitative Economics · doi:10.1007/s40953-022-00303-0 · Cited by 1
Corporate Finance p-HackingPreprint
Mitton, T. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3304875 · Cited by 1
Incentive-Compatible Critical ValuesPreprint
McCloskey, A., & Michaillat, P. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4021797 · Cited by 1
Treatment for natural experiments: How to improve causal estimates using conceptual definitions and substantive interpretationsPreprint
Öberg, S. (2021) · doi:10.31235/osf.io/pkyue · Cited by 1
Application of Difference-in-Differences Strategies in Finance: The Case of Natural Disasters and Bank ResponsesPreprint
Barth, J. R., Lee, K., Shen, X., & Yoon, Y. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3764159 · Cited by 1
Behavioral Interventions and Students' Success at University - Evidence from Randomized Field Experiments
Brade, R. (2022) · doi:10.53846/goediss-9055 · Cited by 1
Cherry PickingPreprint
Lang, M., & Qiu, W. (2021) · doi:10.31222/osf.io/as9zd · Cited by 1
Why is the statistical revolution not progressing? Vicious cycle of the scientific reformPreprint
Białek, M., Misiak, M., & Dziekan, M. (2021) · doi:10.31219/osf.io/gmfs9 · Cited by 1
Application of Difference-in-Differences Strategies in Finance: The Case of Natural Disasters and Bank Responses
Barth, J. R., Lee, K. B., Shen, X., & Yoon, Y. S. (2022) · Springer eBooks · doi:10.1007/978-3-030-91231-4_77 · Cited by 1
How Does Data Access Shape Science? The Impact of Federal Statistical Research Data Centers on Economics ResearchPreprint
Nagaraj, A., & Tranchero, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4483672 · Cited by 1
Criminal justice from a public choice perspective: an introduction to the special issue
Adamson, J., & Rentschler, L. (2023) · Public Choice · doi:10.1007/s11127-023-01089-2 · Cited by 1
Contexts of Convenience: Generalizing from Published Evaluations of School Finance Policies
Handel, D., & Hanushek, E. A. (2023) · National Bureau of Economic Research · doi:10.3386/w31653 · Cited by 1
Bayesian nonparametric latent class analysis with different item typesPreprint
Qiu, M., Paganin, S., Ohn, I., & Lin, L. (2023) · doi:10.31219/osf.io/djzn7 · Cited by 1
How does the Accounting Research Community Think About Open Science?Preprint
Gassen, J., Klug, J., & van Pelt, V. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5004669 · Cited by 1
Replicating Anomalies Using the Aggregate AveragePreprint
Guo, J., Korniotis, G. M., Kumar, A., & Li, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4990134 · Cited by 1
Ideological Bias in Estimates of the Impact of ImmigrationPreprint
Borjas, G. J., & Breznau, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5069454 · Cited by 1
Ideological Bias in Estimates of the Impact of ImmigrationPreprint
Borjas, G. J., & Breznau, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5069989 · Cited by 1
The Sources of Researcher Variation in EconomicsPreprint
Huntington‐Klein, N., Pörtner, C. C., Acharya, Y., Adamkovic, M., Adema, J. A. H., Agasa, L. O., Ahmad, I., Akbulut‐Yuksel, M., Andresen, M. E., Angenendt, D., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5152665 · Cited by 1
p-Hacking as a Questionable Research Practice in Industrial and Organizational Psychology
Cocoș, B. (2021) · doi:10.47040/sd/sdpsych.v12i1.119
Crime et famines. Europe, Afrique, Asie, XIX e et XX e siècles
Bonnet, F. (2021) · Déviance et Société · doi:10.3917/ds.454.0131
Reproduction of 'Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics'
Zhang, M. (2021) · doi:10.48152/ssrp-3vwp-3685
‘Trust Us’: Open Data and Preregistration in Political Science and International RelationsPreprint
Scoggins, B., & Robertson, M. P. (2022) · doi:10.31222/osf.io/8h2bp
Faculty Opinions recommendation of Nonreplicable publications are cited more than replicable ones.
Brembs, B. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.740150836.793586309
Application of Difference-in-Differences Strategies in Finance: The Case of Natural Disasters and Bank Responses
Barth, J. R., Lee, K. B., Shen, X., & Yoon, Y. S. (2021) · Springer eBooks · doi:10.1007/978-3-030-73443-5_77-1
Risk, reproducibility, and reproduction : Essays on scholar’s analytic decisions and consumers' product purchases
Christensen, J. D. (2023) · Acta universitatis agriculturae Sueciae · doi:10.54612/a.7etj27hhft
Age, Wealth, and the MPC in Europe - A Supervised Machine Learning ApproachPreprint
Dutt, S., & Radermacher, J. W. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4360002
Non-Experimental Data, Hypothesis Testing, and the Likelihood Principle: A Social Science PerspectivePreprint
Engsted, T., & Schneider, J. W. (2023) · doi:10.31235/osf.io/nztk8
The multiversal methodology as a remedy of the replication crisisPreprint
Cantone, G. G. (2023) · doi:10.31222/osf.io/kuhmz
Pre-Registered Analysis Plan: Is the Inter-Researcher Variability in Social Scientific Results Explicable? An Adversarial Collaboration and Joint Effort to Parse Model and Estimate DispersionPreprint
Breznau, N., Auspurg, K., Brüderl, J., Holzmeister, F., Nilsonne, G., Aczél, B., Clark, C. J., Nguyen, H. H. V., Varga, M. A., & Uhlmann, E. L. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/h9t5c
P-Hacking Made Easy
Herndon, J. (2023) · Journal of Economics Teaching · doi:10.58311/jeconteach/d09c7366b1439d6cd0f16a520fedbde8ce64af60
Statistical reporting errors in economicsPreprint
Bruns, S. B., Herwartz, H., Ioannidis, J. P. A., Islam, C.-G., & Raters, F. H. C. (2023) · doi:10.31222/osf.io/mbx62
Revisiting ‘Growth and Inequality in Public Good Provision’—Reproducing and Generalizing Through Inconvenient Online ExperimentationPreprint
Roggenkamp, H. (2024) · doi:10.31219/osf.io/6rn97
Non-Experimental Data, Hypothesis Testing, and the Likelihood Principle: A Social Science Perspective
Engsted, T., & Schneider, J. W. (2024) · Foundations and Trends® in Econometrics · doi:10.1561/0800000048
Reproduction of 'Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics'
(2024) · doi:10.48152/ssrp-z5sm-w854
An Equivalence Test Approach to Rounding Parameter Estimates
Astier, N., & Wolak, F. A. (2024) · National Bureau of Economic Research · doi:10.3386/w32124
Reproduction of 'Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics'
Huang, C. (2024) · doi:10.48152/ssrp-zcvz-a685
Examining Selection Pressures in the Publication Process through the Lens of Sniff Tests
Snyder, C. M., & Zhuo, R. (2024) · The Review of Economics and Statistics · doi:10.1162/rest_a_01410
Green Self-Image, Pro-Environmental Behavior and Life Satisfaction: A Mediation AnalysisPreprint
Binder, M., Blankenberg, A., & Nickel, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4805593
From Behavioral Economics to Behavioral Science: A Message from the “New Humans” to the “Modern Humans”
Yamamura, E. (2024) · Springer eBooks · doi:10.1007/978-981-97-1887-0_8
Challenges in Empirical Research in Economics: The Way Forward
Otsuka, K., Higuchi, Y., & Suzuki, A. (2024) · Springer eBooks · doi:10.1007/978-981-97-1887-0_4
Conclusion
Otsuka, K., Kurosaki, T., Sawada, Y., & Sonobe, T. (2024) · Springer eBooks · doi:10.1007/978-981-97-1887-0_11
A note on the discussion regarding terrorism and land use in agriculture
Mußhoff, O., Ölkers, T., & Kirchner, E. (2024) · Land Use Policy · doi:10.1016/j.landusepol.2024.107236
Dynamic capabilities, firm performance and firm adaptation in New Zealand
Ng, T. (2024) · doi:10.26686/wgtn.26370319
VaR Comparison for Fractal Diffusion System using R/S Method
Zhang, L. (2024) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs61882.2024.10602981
Volatility Index, Exchange Rate, Economic Growth On Stock Indexes
Junaid, A., Muslim, M., Wilda, & Said, D. (2024) · Jurnal Akuntansi · doi:10.24912/ja.v28i3.2459
Robustifying and simplifying high-dimensional regression with applications to yearly stock return and telematics data
Marchese, M., Martínez‐Miranda, M. D., Nielsen, J. P., & Scholz, M. (2024) · Financial Innovation · doi:10.1186/s40854-024-00657-9
Why are replication rates so low?
Vu, P. (2024) · Journal of Econometrics · doi:10.1016/j.jeconom.2024.105868
Ethical Guidelines
Franses, P. H. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009428033.003
Analysis of Gross Margin of Women Melon Processors With and Without Improved Melon Seed Shelling Technology: A Study in Benue and Niger StatesPreprint
Oyediran, W. O., Fakoya, E. O., Banmeke, T. O. A., Alaka, F. A., & Omoare, A. M. (2024) · Qeios · doi:10.32388/j6iogp
On Overfitting in Machine Learning ContestsPreprint
Qiu, X., Xu, H., & Yu, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5012282
Multiverse Robustness tests in Macroeconomic Calibration ModelsPreprint
Lilletvedt, J. S., Johannesson, M., Kvarven, A., & Strømland, E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5038496
Is Patience Malleable Via Educational Intervention? Evidence on the Role of Age in Field ExperimentsPreprint
Kaiser, T., Menkhoff, L., & Oberrauch, L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5060455
Heir presence? The impact of offspring gender composition on long-term agricultural investment behavior
Chen, P., Zhang, L., & Ruiz‐Menjivar, J. (2025) · Renewable Agriculture and Food Systems · doi:10.1017/s1742170524000358
Geometric and statistical curvatures of the skew- t distributions and their application
Wu, Q., & Hu, H. (2025) · Communication in Statistics- Theory and Methods · doi:10.1080/03610926.2025.2457494
Underpowered studies and exaggerated effects: A replication and re‐evaluation of the magnitude of anchoring effects
Li, T., Weigel, C., Ferraro, P. J., & Messer, K. D. (2025) · Economic Inquiry · doi:10.1111/ecin.13279
The Economics of Encouragement: Can A Simple Email Shape Major Choice?Preprint
Edwards, O., & Meer, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5215989
The Economics of Encouragement: Can A Simple Email Shape Major Choice?
Edwards, O., & Meer, J. (2025) · National Bureau of Economic Research · doi:10.3386/w33676
Yes, Virginia, there are superstar money managers
Dimitrov, V., & Jain, P. C. (2025) · Review of Quantitative Finance and Accounting · doi:10.1007/s11156-025-01416-2
Job Market Stars: Statistical Significance and Academic Hiring in EconomicsPreprint
Kattan, L., Musumeci, M., & Brodeur, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5252971
Self-citation
The economics of encouragement: Can a simple email shape major choice?
Edwards, O., & Meer, J. (2025) · Economics of Education Review · doi:10.1016/j.econedurev.2025.102680
Influential analysis of smart transportation policy on common prosperity using casual mediation effect inference
Li, Q., Yu, X., Xu, H., Wang, D., Guo, Z., & Liu, L. (2025) · Case Studies on Transport Policy · doi:10.1016/j.cstp.2025.101529
Media Stars: Statistical Significance and Research ImpactPreprint
Brodeur, A., Cook, N., Heyes, A., & Wright, T. J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5371748
Self-citation
Machine confirming: Validating financial theories with transfer learning
Zhao, X., Li, Y., & Wang, T. (2025) · International Review of Financial Analysis · doi:10.1016/j.irfa.2025.104485
Reputation, Risk, and VisibilityPreprint
Lukyanov, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5436718
Scientific Peer Review: Institutional Workings and PerformancePreprint
Boudreau, K. J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5528578
The (Statistical) Power of Incentives
Alekseev, A. (2025) · Journal of the Economic Science Association · doi:10.1017/esa.2025.10019
Get a Skill, Get a Job, Get Ahead? Evaluating the effects of Virginia’s workforce-targeted free college program
Bonilla, S., & Sparks, D. (2026) · Economics of Education Review · doi:10.1016/j.econedurev.2026.102798 · Cited by 1
Short-Run Multi-Outcome Effects of Nightlife Regulation in San JuanPreprint
Arroyo, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5670691
Running replicable experiments
Coffman, L. C., & Dreber, A. (2025) · Elsevier eBooks · doi:10.1016/bs.hbem.2025.09.010
Intuitive prosociality: heterogeneous treatment effects or false positive?
Kvarven, A., Strømland, E., & Torsvik, G. (2025) · Journal of Economic Psychology · doi:10.1016/j.joep.2025.102873
Additional file 1 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615050
Additional file 6 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615065
Additional file 4 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615059
Additional file 7 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615068
Additional file 2 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615053
Additional file 3 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615056
Additional file 5 of Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, G., & Barnett, A. G. (2024) · Figshare · doi:10.6084/m9.figshare.26615062
The Power of Incentives
Alekseev, A. (2022) · University of Regensburg Publication Server (University of Regensburg) · doi:10.5283/epub.53030
Open-Skull Post-Selection Misconduct versus Closed-Skull Sole-Learner
Weng, J. (2025) · International Conference on Image Processing, Intelligent Control and Computer Engineering (IPICE) · doi:10.1145/3768184.3768255
A meta‐analysis on the productive value of crop biodiversity
Bareille, F., & Largier, B. (2026) · American Journal of Agricultural Economics · doi:10.1002/ajae.70041 · Cited by 3
Methods for Studying Union Effects: A Review and Comparative Analysis of Empirical Industrial Relations Literature
Han, K., Zhang, T., & Lamare, J. (2026) · Industrial relations journal · doi:10.1111/irj.70028
A solution to the p-hacking problem with a general robust significance test under variance uncertainty
Li, X., Yang, S., & Yao, J. (2026) · Statistical Papers · doi:10.1007/s00362-025-01788-6
Trusted-Execution Environment (TEE) for Solving the Replication Crisis in AcademiaPreprint
Li, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6078228
The Economic Significance of Statistical Insignificance in Structural Entry ModelsPreprint
Yang, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6315878

Authors: Brodeur, A., Cook, N., & Heyes, A.
Affiliations: University of Ottawa (CA)
ORCID: Brodeur, Abel
Cook, Nikolai
Heyes, Anthony
Venue: American Economic Review
Publisher: American Economic Association
Published: 2020-11-01
Volume: 110
Issue: 11
Pages: 3634-3660
ISSN: 0002-8282 (ISSN-L)
DOI: 10.1257/aer.20190687
OpenAlex: W3047016911
Cited by: 333
References: 41
Citations by year: 2026: 44 · 2025: 50 · 2024: 91 · 2023: 49 · 2022: 47 · 2021: 41 · 2020: 7
FWCI: 25.39
Type: journal article
Language: en
Open access: closed
Locations: American Economic Review (publishedVersion)
Open Research Exeter (University of Exeter) (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Figshare (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords: p-hacking, publication bias, credibility revolution, causal inference, instrumental variables, difference-in-differences, regression discontinuity design, randomized controlled trials, hypothesis testing, economics journals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks