The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Are all Economic Hypotheses False?
Abstract
We develop an estimator that allows us to calculate an upper bound to the fraction of unrejected null hypotheses tested in economics journal articles that are in fact true. Our point estimate is that none of the unrejected nulls in our sample is true. We reject the hypothesis that more than one-third are true. We consider three explanations for this finding: that all null hypotheses are mere approximations, that data-mining biases reported standard errors downward, and that journals tend to publish papers that fail to reject their null hypotheses only when the null hypotheses are likely to be false. While all these explanations are important, the last seems best able to explain our findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Let's Take the Con out of Econometrics
Leamer, E. E. (2026) · Journal of Business Cycle Research · doi:10.1007/s41549-026-00137-2 · Cited by 2,202
An Introduction to Bayesian Inference in Econometrics.
Sargan, J. D., & Zellner, A. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344559 · Cited by 1,914
Mccloskey, D. N. (2008) · The New Palgrave Dictionary of Economics · doi:10.1007/978-1-349-58802-2_1435 · Cited by 1,816
Specification Searches: Ad Hoc Inference with Non-Experimental Data.
Richard, J.-F., & Leamer, E. E. (1981) · Economica · doi:10.2307/2553491 · Cited by 1,114
Econometrics: Alchemy or Science?: Essays in Econometric Methodology
Hendry, D. F. (2000) · doi:10.1093/0198293542.001.0001 · Cited by 634
An Introduction to Bayesian Inference in Econometrics
Good, I. J. (1975) · Technometrics · doi:10.1080/00401706.1975.10489285 · Cited by 501
Reporting the Fragility of Regression Estimates
Leamer, E., & Leonard, H. (1983) · The Review of Economics and Statistics · doi:10.2307/1924497 · Cited by 322
Unit roots in real GNP: Do we know, and do we care?
Christiano, L. J., & Eichenbaum, M. (1990) · Carnegie-Rochester Conference Series on Public Policy · doi:10.1016/0167-2231(90)90021-c · Cited by 265
Is the Stabilization of the Postwar Economy a Figment of the Data
Romer, C. (2016) · American Economic Review · Cited by 260
Cited by
Do Investment-Cash Flow Sensitivities Provide Useful Measures of Financing Constraints?
Kaplan, S. N., & Zingales, L. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555163 · Cited by 6,770
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Using Randomization in Development Economics Research: A Toolkit
Duflo, E., Glennerster, R., & Kremer, M. (2006) · National Bureau of Economic Research · doi:10.3386/t0333 · Cited by 1,191
Multinational Companies and Productivity Spillovers: A Meta‐Analysis
Gorg, H., & Strobl, E. (2001) · The Economic Journal · doi:10.1111/1468-0297.00669 · Cited by 1,039
Hou, K., Xue, C., & Zhang, L. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhy131 · Cited by 989
Stanley, T. D. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00250.x · Cited by 742
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605
Chapter 61 Using Randomization in Development Economics Research: A Toolkit
Duflo, E., Glennerster, R., & Kremer, M. (2007) · Handbook of Development Economics · doi:10.1016/s1573-4471(07)04061-2 · Cited by 545
Estimating vertical spillovers from FDI: Why results vary and what the true effect is
Havranek, T., & Irsova, Z. (2011) · Journal of International Economics · doi:10.1016/j.jinteco.2011.07.004 · Cited by 535
Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis
Doucouliagos, H., & Stanley, T. D. (2009) · British Journal of Industrial Relations · doi:10.1111/j.1467-8543.2009.00723.x · Cited by 507
Transparency, Reproducibility, and the Credibility of Economics Research
Christensen, G., & Miguel, E. (2018) · Journal of Economic Literature · doi:10.1257/jel.20171350 · Cited by 446
Star Wars: The Empirics Strike Back
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2016) · American Economic Journal: Applied Economics · doi:10.1257/app.20150044 · Cited by 430
Identification of and correction for publication bias
Andrews, I., & Kasy, M. (2019) · American Economic Review · doi:10.1257/aer.20180310 · Cited by 388
MEASURING INTERTEMPORAL SUBSTITUTION: THE IMPORTANCE OF METHOD CHOICES AND SELECTIVE REPORTING
Havránek, T. (2015) · Journal of the European Economic Association · doi:10.1111/jeea.12133 · Cited by 365
Determinants of Horizontal Spillovers from FDI: Evidence from a Large Meta-Analysis
Iršová, Z., & Havránek, T. (2013) · World Development · doi:10.1016/j.worlddev.2012.07.001 · Cited by 361
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A., Cook, N., & Heyes, A. (2020) · American Economic Review · doi:10.1257/aer.20190687 · Cited by 333
Chapter 16 Schools, Teachers, and Education Outcomes in Developing Countries
Glewwe, P., & Kremer, M. (2006) · Handbook of the Economics of Education · doi:10.1016/s1574-0692(06)02016-2 · Cited by 323
Multinationals’ profit response to tax differentials: Effect size and shifting channels
Heckemeyer, J. H., & Overesch, M. (2017) · Canadian Journal of Economics/Revue canadienne d'économique · doi:10.1111/caje.12283 · Cited by 322
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Quasi-experimental and experimental approaches to environmental economics
Greenstone, M., & Gayer, T. (2009) · Journal of Environmental Economics and Management · doi:10.1016/j.jeem.2008.02.004 · Cited by 295
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Gerber, A., & Malhotra, N. (2008) · Quarterly Journal of Political Science · doi:10.1561/100.00008024 · Cited by 284
ARE ALL ECONOMIC FACTS GREATLY EXAGGERATED? THEORY COMPETITION AND SELECTIVITY
Doucouliagos, C., & Stanley, T. D. (2013) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2011.00706.x · Cited by 276
Viewpoint: Replication in economics
Hamermesh, D. S. (2007) · Canadian Journal of Economics/Revue canadienne d'économique · doi:10.1111/j.1365-2966.2007.00428.x · Cited by 273
Cross-country heterogeneity in intertemporal substitution
Havranek, T., Horvath, R., Irsova, Z., & Rusnak, M. (2015) · Journal of International Economics · doi:10.1016/j.jinteco.2015.01.012 · Cited by 273
An estimate of the science-wise false discovery rate and application to the top medical literature
Jager, L. R., & Leek, J. T. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt007 · Cited by 179
Demand for gasoline is more price-inelastic than commonly thought
Havranek, T., Irsova, Z., & Janda, K. (2012) · Energy Economics · doi:10.1016/j.eneco.2011.09.003 · Cited by 164
Adopt or innovate: Understanding technological responses to cap-and-trade
Calel, R. (2020) · American Economic Journal: Economic Policy · doi:10.1257/pol.20180135 · Cited by 166
Havranek, T., & Sokolova, A. (2020) · Review of Economic Dynamics · doi:10.1016/j.red.2019.05.004 · Cited by 141
False (and Missed) Discoveries in Financial Economics
HARVEY, C. R., & LIU, Y. (2020) · The Journal of Finance · doi:10.1111/jofi.12951 · Cited by 118
Significance testing in empirical finance: a critical review and assessment
Kim, J. H., & Ji, P. I. (2015) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2015.08.006 · Cited by 86
Grant, D., & Green, W. B. (2013) · Empirical Economics · doi:10.1007/s00181-012-0578-0 · Cited by 37
“College grades do matter to employers (Chia and Miller 2008; Grant 2007; Wise 1975), and the monetary returns to additional study effort, as implied by prior research, are substantial. Jones and Jackson’s (1990) estimates indicate that a one letter-grade improvement in one college course raises wages by .23%. A very s…”
InfluentialBlack, B., Hollingsworth, A., Nunes, L., & Simon, K. (2022) · Journal of Public Economics · doi:10.1016/j.jpubeco.2022.104713 · Cited by 31
Metadata
- Authors
- De Long, J. B., & Lang, K.
- Venue
- Journal of Political Economy
- Publisher
- University of Chicago Press
- Published
- 1992-12
- Volume
- 100
- Issue
- 6
- Pages
- 1257-1272
- ISSN
- 0022-3808 (ISSN-L)
- 1537-534X
- OpenAlex
- W2079187306
- Cited by
- 295
- References
- 12
- FWCI
- 1.99
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.