The DOI matches its Crossref record; 227 of the 227 references it cites were found in a registry or an index.
Transparency, Reproducibility, and the Credibility of Economics Research
Open accessAbstract
There is growing interest in enhancing research transparency and reproducibility in economics and other scientific fields. We survey existing work on these topics within economics and discuss the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in the discipline. We next discuss recent progress in this area, including through improved research design, study registration and pre-analysis plans, disclosure standards, and open sharing of data and materials, drawing on experiences in both economics and other social sciences. We discuss areas where consensus is emerging on new practices, as well as approaches that remain controversial, and speculate about the most effective ways to make economics research more credible in the future. ( JEL A11, C18, I23)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
3 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
RetractedWhen contact changes minds: An experiment on transmission of support for gay equality
LaCour, M. J., & GREEN, D. P. (2014) · Science · doi:10.1126/science.1256151 · Cited by 136
RetractedRetracted : Correction or comparison? The effects of prime awareness on social judgments
Maringer, M., & Stapel, D. A. (2009) · European Journal of Social Psychology · doi:10.1002/ejsp.569 · Cited by 26
RetractedControlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
Journal of economic literature
Armstrong, J. S. (1996) · International Journal of Forecasting · doi:10.1016/0169-2070(96)88197-0 · Cited by 9,923
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
k-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571
The colonial origins of comparative development: An empirical investigation
Acemoglu, D., Johnson, S., & Robinson, J. A. (2001) · American Economic Review · doi:10.1257/aer.91.5.1369 · Cited by 8,517 · cited 1×
“…and Sergenti (2004), Ciccone (2011) and Miguel and Satyanath (2011) on civil conflict and GDP growth using rainfall as an instrumental variable; and Acemoglu, Johnson, and Robinson (2001), Albouy (2012), and Acemoglu, Johnson, and Robinson (2012) on institutions and GDP growth with settler…”
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Journal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
The Colonial Origins of Comparative Development: An Empirical Investigation: Reply
Acemoglu, D., Johnson, S., & Robinson, J. A. (2012) · American Economic Review · doi:10.1257/aer.102.6.3077 · Cited by 6,047
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Moffitt, R. A. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,694
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504
Burnside, C., & Dollar, D. (2000) · American Economic Review · doi:10.1257/aer.90.4.847 · Cited by 3,728
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Multiple significance tests: the Bonferroni method
Bland, J. M., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.310.6973.170 · Cited by 3,341
Adaptive linear step-up procedures that control the false discovery rate
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006) · Biometrika · doi:10.1093/biomet/93.3.491 · Cited by 3,228
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“Easterbrook et al. (1991) reviewed the universe of protocols submitted to the Central Oxford Research Ethics Committee, and both Turner et al. (2008) and Kirsch et al. (2008) employ the universe of tests of certain anti-depressant drugs submitted to the FDA, and all found significantly higher…”
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Experimental Analysis of Neighborhood Effects
Kling, J. R., Liebman, J. B., & Katz, L. F. (2007) · Econometrica · doi:10.1111/j.1468-0262.2007.00733.x · Cited by 2,719
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 2×
“39 Recently, we, the authors of this article, were part of an interdisciplinary group of researchers that developed a detailed set of journal guidelines called the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al. 2015).”
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Preventive Medicine · doi:10.1016/j.ypmed.2007.08.012 · Cited by 2,698
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Economic Shocks and Civil Conflict: An Instrumental Variables Approach
Miguel, E., Satyanath, S., & Sergenti, E. (2004) · Journal of Political Economy · doi:10.1086/421174 · Cited by 2,460
Self-citationKirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050045 · Cited by 2,442
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Anderson, M. L. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000841 · Cited by 2,410
Boutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292
Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities
Miguel, E., & Kremer, M. (2004) · Econometrica · doi:10.1111/j.1468-0262.2004.00481.x · Cited by 2,268
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Card, D., & Krueger, A. B. (2000) · American Economic Review · doi:10.1257/aer.90.5.1397 · Cited by 2,206
… and the Cross-Section of Expected Returns
Harvey, C. R., Liu, Y., & Zhu, H. (2016) · Review of Financial Studies · doi:10.1093/rfs/hhv059 · Cited by 2,119
Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach
Sala-i-Martin, X., Doppelhofer, G., & Miller, R. I. (2004) · American Economic Review · doi:10.1257/0002828042002570 · Cited by 2,112
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., & Loder, E. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.002 · Cited by 1,998
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“…of Health, and John, Loewenstein, and Prelec (2012) in psychology, as well as a meta-analysis of 18 surveys of academic misbehavior, do not paint a very rosy picture, with 2% of respondents admitting to data fabrication, and 34% admitting to lesser forms of academic misconduct (Fanelli 2009).”
A Reality Check for Data Snooping
White, H. (2000) · Econometrica · doi:10.1111/1468-0262.00152 · Cited by 1,851
Wacholder, S., Chanock, S., Garcia-Closas, M., El ghormli, L., & Rothman, N. (2004) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djh075 · Cited by 1,754
Social Security, Induced Retirement, and Aggregate Capital Accumulation
Feldstein, M. (1974) · Journal of Political Economy · doi:10.1086/260246 · Cited by 1,703
Teoria Statistica Delle Classi e Calcolo Delle Probabilità
(2010) · Encyclopedia of Research Design · doi:10.4135/9781412961288.n455 · Cited by 1,698
The Oregon Health Insurance Experiment: Evidence from the First Year*
Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., & Oregon Health Study Group (2012) · The Quarterly Journal of Economics · doi:10.1093/qje/qjs020 · Cited by 1,657
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · cited 6×
“Figure 1 reproduces some of the main patterns from Franco, Malhotra, and Simonovits (2014), as described in (Mervis 2014b).”
InfluentialP-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
Reporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension
Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., & CONSORT PRO Group, F. T. (2013) · JAMA · doi:10.1001/jama.2013.879 · Cited by 1,386
Imitation Costs and Patents: An Empirical Study
Mansfield, E., Schwartz, M., & Wagner, S. (1981) · The Economic Journal · doi:10.2307/2232499 · Cited by 1,316
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314 · cited 10×
“One of the most important recent studies is Camerer et al. (2016), which repeated 18 behavioral economics lab experiments originally published between 2011 and 2014 in the American Economic Review and the Quarterly Journal of Economics to assess their replicability.”
InfluentialNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Electrochemically induced nuclear fusion of deuterium
Fleischmann, M., & Pons, S. (1989) · Journal of Electroanalytical Chemistry and Interfacial Electrochemistry · doi:10.1016/0022-0728(89)80006-3 · Cited by 1,284
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
A Test for Superior Predictive Ability
Hansen, P. R. (2005) · Journal of Business & Economic Statistics · doi:10.1198/073500105000000063 · Cited by 1,230
Minimum wage effects across state borders: Estimates using contiguous counties
Dube, A., Lester, T. W., & Reich, M. (2010) · Review of Economics and Statistics · doi:10.1162/rest_a_00039 · Cited by 1,221
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff
Herndon, T., Ash, M., & Pollin, R. (2014) · Cambridge Journal of Economics · doi:10.1093/cje/bet075 · Cited by 1,197
Procedures for Comparing Samples with Multiple Endpoints
O'Brien, P. C. (1984) · Biometrics · doi:10.2307/2531158 · Cited by 1,162
The Oregon Experiment — Effects of Medicaid on Clinical Outcomes
Baicker, K., Taubman, S. L., Allen, H. L., Bernstein, M., Gruber, J. H., Newhouse, J. P., Schneider, E. C., Wright, B. J., Zaslavsky, A. M., & Finkelstein, A. N. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1212321 · Cited by 1,150
Does Competition Among Public Schools Benefit Students and Taxpayers?
Hoxby, C. M. (2000) · American Economic Review · doi:10.1257/aer.90.5.1209 · Cited by 1,089
Multinational Companies and Productivity Spillovers: A Meta‐Analysis
Gorg, H., & Strobl, E. (2001) · The Economic Journal · doi:10.1111/1468-0297.00669 · Cited by 1,039
Data-snooping, technical trading rule performance and the Bootstrap
Sullivan, R., Timmermann, A., & White, H. (1999) · The Journal of Finance · doi:10.1111/0022-1082.00163 · Cited by 1,028 · cited 1×
“This method was implemented on a large number of trading rules in Sullivan, Timmermann, and White (1999) and a similar method that addresses the presence of poorly performing or irrelevant alternatives was developed in Hansen (2005).”
Moher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027
Scandinavian Journal of Statistics
(2010) · Scandinavian Journal of Statistics · doi:10.1111/(issn)1467-9469 · Cited by 976
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964
Model uncertainty in cross‐country growth regressions
Fernández, C., Ley, E., & Steel, M. F. J. (2001) · Journal of Applied Econometrics · doi:10.1002/jae.623 · Cited by 891
Aid, Policies, and Growth: Comment
Easterly, W., Levine, R., & Roodman, D. (2004) · American Economic Review · doi:10.1257/0002828041464560 · Cited by 861
Small Telescopes: Detectability and the Evaluation of Replication Results
Simonsohn, U. (2015) · Psychological Science · doi:10.1177/0956797614567341 · Cited by 742
Stanley, T. D. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00250.x · Cited by 742
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730
p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717
Registered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716
A note on graphical presentation of estimated odds ratios from several clinical trials
Galbraith, R. F. (1988) · Statistics in Medicine · doi:10.1002/sim.4780070807 · Cited by 663
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Using Regional Variation in Wages to Measure the Effects of the Federal Minimum Wage
Card, D. (1992) · ILR Review · doi:10.1177/001979399204600103 · Cited by 633
The Encompassing Principle and its Application to Testing Non-Nested Hypotheses
Mizon, G. E., & Richard, J.-F. (1986) · Econometrica · doi:10.2307/1911313 · Cited by 616
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605
The Impact of Legalized Abortion on Crime
Donohue, J. J., & Levitt, S. D. (2001) · The Quarterly Journal of Economics · doi:10.1162/00335530151144050 · Cited by 591
Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015) · American Economic Review · doi:10.1257/aer.p20151023 · Cited by 584
Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment
Taubman, S. L., Allen, H. L., Wright, B. J., Baicker, K., & Finkelstein, A. N. (2014) · Science · doi:10.1126/science.1246183 · Cited by 577
Registered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Hall, P. A. (2013) · Encyclopedia of Behavioral Medicine · doi:10.1007/978-1-4419-1005-9_1175 · Cited by 558
Promoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationTargeting the Poor: Evidence from a Field Experiment in Indonesia
Alatas, V., Banerjee, A., Hanna, R., Olken, B. A., & Tobias, J. (2012) · American Economic Review · doi:10.1257/aer.102.4.1206 · Cited by 536
Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis
Doucouliagos, H., & Stanley, T. D. (2009) · British Journal of Industrial Relations · doi:10.1111/j.1467-8543.2009.00723.x · Cited by 507 · cited 3×
“𝛽𝛽1 here can also be interpreted as the true effect (called the precision effect test, PET) free of publication bias, and Doucouliagos and Stanley (2009) find no evidence of a true effect of the minimum wage on unemployment.”
Comment on “Estimating the reproducibility of psychological science.”
Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016) · Science · doi:10.1126/science.aad7243 · Cited by 474
Revisiting the Minimum Wage—Employment Debate: Throwing Out the Baby with the Bathwater?
Neumark, D., Salas, J. M. I., & Wascher, W. (2014) · ILR Review · doi:10.1177/00197939140670s307 · Cited by 474
META‐ANALYSIS OF ECONOMICS RESEARCH REPORTING GUIDELINES
Stanley, T. D., Doucouliagos, H., Giles, M., Heckemeyer, J. H., Johnston, R. J., Laroche, P., Nelson, J. P., Paldam, M., Poot, J., Pugh, G., et al. (2013) · Journal of Economic Surveys · doi:10.1111/joes.12008 · Cited by 463
Rose, A. K., & Stanley, T. D. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00251.x · Cited by 463
Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan*
Casey, K., Glennerster, R., & Miguel, E. (2012) · The Quarterly Journal of Economics · doi:10.1093/qje/qje027 · Cited by 453
Self-citationReinhart, C. M., & Rogoff, K. S. (2010) · American Economic Review · doi:10.1257/aer.100.2.573 · Cited by 438
King, G. (1995) · PS Political Science & Politics · doi:10.1017/s1049096500057607 · Cited by 436
Star Wars: The Empirics Strike Back
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2016) · American Economic Journal: Applied Economics · doi:10.1257/app.20150044 · Cited by 430
Evaluating Behaviorally Motivated Policy: Experimental Evidence from the Lightbulb Market
Allcott, H., & Taubinsky, D. (2015) · American Economic Review · doi:10.1257/aer.20131564 · Cited by 427
Longo, D. L., & Drazen, J. M. (2016) · New England Journal of Medicine · doi:10.1056/nejme1516564 · Cited by 419
Why has the number of scientific retractions increased?
Steen, R. G., Casadevall, A., & Fang, F. C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068397 · Cited by 415
Size matters: the standard error of regressions in the American Economic Review
Ziliak, S. T., & McCloskey, D. N. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.024 · Cited by 408
HIPAA regulations - a new era of medical-record privacy?
Annas, G. J. (2003) · New England Journal of Medicine · doi:10.1056/nejmlim035027 · Cited by 402
Do Minimum Wages Reduce Employment? A Case Study of California, 1987–89
Card, D. (1992) · ILR Review · doi:10.1177/001979399204600104 · Cited by 390
A Meta-Analytic Assessment of the Effect of Immigration on Wages
Longhi, S., Nijkamp, P., & Poot, J. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00255.x · Cited by 388
Happiness and Time Preference: The Effect of Positive Affect in a Random-Assignment Experiment
Ifcher, J., & Zarghamee, H. (2011) · American Economic Review · doi:10.1257/aer.101.7.3109 · Cited by 369
THREE ECONOMETRIC METHODOLOGIES: A CRITICAL APPRAISAL1
Pagan, A. (1987) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.1987.tb00022.x · Cited by 366
Modeling Publication Selection Effects in Meta-Analysis
Hedges, L. V. (1992) · Statistical Science · doi:10.1214/ss/1177011364 · Cited by 359
Neumark, D., & Wascher, W. (2000) · American Economic Review · doi:10.1257/aer.90.5.1362 · Cited by 358
Multiplicity in randomised trials II: subgroup and interim analyses
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66516-6 · Cited by 357
Lovell, M. C. (1983) · The Review of Economics and Statistics · doi:10.2307/1924403 · Cited by 354
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000104 · Cited by 343
Using prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331
Reporting the Fragility of Regression Estimates
Leamer, E., & Leonard, H. (1983) · The Review of Economics and Statistics · doi:10.2307/1924497 · Cited by 322
Clinical Trial Registration — Looking Back and Moving Ahead
Laine, C., Horton, R., DeAngelis, C. D., Drazen, J. M., Frizelle, F. A., Godlee, F., Haug, C., Hébert, P. C., Kotzin, S., Marusic, A., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejme078110 · Cited by 320
Promises and Perils of Pre-Analysis Plans
Olken, B. A. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.61 · Cited by 299
Humphreys, M., Sanchez de la Sierra, R., & van der Windt, P. (2013) · Political Analysis · doi:10.1093/pan/mps021 · Cited by 298
Are all Economic Hypotheses False?
De Long, J. B., & Lang, K. (1992) · Journal of Political Economy · doi:10.1086/261860 · Cited by 295
A Meta-Analysis of β-Convergence: the Legendary 2%
Abreu, M., de Groot, H. L. F., & Florax, R. J. G. M. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00253.x · Cited by 294
The Colonial Origins of Comparative Development: An Empirical Investigation: Comment
Albouy, D. Y. (2012) · American Economic Review · doi:10.1257/aer.102.6.3059 · Cited by 291
Gerber, A., & Malhotra, N. (2008) · Quarterly Journal of Political Science · doi:10.1561/100.00008024 · Cited by 284
The Behavior of the P-Value When the Alternative Hypothesis is True
Hung, H. M. J., O'Neill, R. T., Bauer, P., & Kohne, K. (1997) · Biometrics · doi:10.2307/2533093 · Cited by 281
The Desperate Need for Replications
Hunter, J. E. (2001) · Journal of Consumer Research · doi:10.1086/321953 · Cited by 280
Replication in Psychological Science
Lindsay, D. S. (2015) · Psychological Science · doi:10.1177/0956797615616374 · Cited by 280
ARE ALL ECONOMIC FACTS GREATLY EXAGGERATED? THEORY COMPETITION AND SELECTIVITY
Doucouliagos, C., & Stanley, T. D. (2013) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2011.00706.x · Cited by 276
Viewpoint: Replication in economics
Hamermesh, D. S. (2007) · Canadian Journal of Economics/Revue canadienne d'économique · doi:10.1111/j.1365-2966.2007.00428.x · Cited by 273
Why Growing Retractions Are (Mostly) a Good Sign
Fanelli, D. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001563 · Cited by 272
Differential Privacy for Statistics: What we Know and What we Want to Learn
Dwork, C., & Smith, A. (2010) · Journal of Privacy and Confidentiality · doi:10.29012/jpc.v1i2.570 · Cited by 264
Uses and Abuses of Empirical Evidence in the Death Penalty Debate
Donohue, J., & Wolfers, J. (2006) · National Bureau of Economic Research · doi:10.3386/w11982 · Cited by 262
Baicker, K., Finkelstein, A., Song, J., & Taubman, S. (2014) · American Economic Review · doi:10.1257/aer.104.5.322 · Cited by 258
What Do Unions Do to Productivity? A Meta‐Analysis
Doucouliagos, C., & Laroche, P. (2003) · Industrial Relations: A Journal of Economy and Society · doi:10.1111/1468-232x.00310 · Cited by 257
Eich, E. (2014) · Psychological Science · doi:10.1177/0956797613512465 · Cited by 255
Sharing Clinical Trial Data: A Proposal From the International Committee of Medical Journal Editors
Taichman, D. B., Backus, J., Baethge, C., Bauchner, H., de Leeuw, P. W., Drazen, J. M., Fletcher, J., Frizelle, F. A., Groves, T., Haileamlak, A., et al. (2016) · JAMA · doi:10.1001/jama.2015.18164 · Cited by 254
Tantalus on the Road to Asymptopia
Leamer, E. E. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.31 · Cited by 252
Social Security and Private Saving: New Time-Series Evidence
Leimer, D. R., & Lesnoy, S. D. (1982) · Journal of Political Economy · doi:10.1086/261077 · Cited by 251
Gerber, A. S., & Malhotra, N. (2008) · Sociological Methods & Research · doi:10.1177/0049124108318973 · Cited by 246
Patil, P., Peng, R. D., & Leek, J. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616646366 · Cited by 243
Policy Evaluation in Uncertain Economic Environments
Brock, W. A., Durlauf, S. N., & West, K. D. (. D. (2003) · Brookings Papers on Economic Activity · doi:10.1353/eca.2003.0013 · Cited by 242
Crowdsourced research: Many hands make tight work
Silberzahn, R., & Uhlmann, E. L. (2015) · Nature · doi:10.1038/526189a · Cited by 239
Betrayers of the Truth: Fraud and Deceit in the Halls of Science
Broad, W. J. (1983) · Cited by 232
Economic Shocks and Civil Conflict: A Comment
Ciccone, A. (2011) · American Economic Journal: Applied Economics · doi:10.1257/app.3.4.215 · Cited by 228
Interactive notebooks: Sharing the code
Shen, H. (2014) · Nature · doi:10.1038/515151a · Cited by 224
Let's Practice What We Preach: Turning Tables into Graphs
Gelman, A., Pasarica, C., & Dodhia, R. (2002) · The American Statistician · doi:10.1198/000313002317572790 · Cited by 223
The Last Word on the Wage Curve?
Nijkamp, P., & Poot, J. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00254.x · Cited by 222
Does Capital Punishment Have a Deterrent Effect? New Evidence from Postmoratorium Panel Data
Dezhbakhsh, H. (2003) · American Law and Economics Association · doi:10.1093/aler/ahg021 · Cited by 222
Hedges, L. V., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165338 · Cited by 221
Verifying the Solution from a Nonlinear Solver: A Case Study
McCullough, B. D., & Vinod, H. D. (2003) · American Economic Review · doi:10.1257/000282803322157133 · Cited by 220
Blind analysis: Hide results to seek the truth
MacCoun, R., & Perlmutter, S. (2015) · Nature · doi:10.1038/526187a · Cited by 218
Retractions in the scientific literature: do authors deliberately commit research fraud?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.038125 · Cited by 217
Just Post It: The Lesson From Two Cases of Fabricated Data Detected by Statistics Alone
Simonsohn, U. (2013) · Psychological Science · doi:10.1177/0956797613480366 · Cited by 216
D. Chambers, C., 1 Cardiff University Brain Research Imaging Centre, S. O. P., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P. (2014) · AIMS Neuroscience · doi:10.3934/neuroscience.2014.1.4 · Cited by 213
Doucouliagos, C. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00252.x · Cited by 213
Testing competing models of loss aversion: an adversarial collaboration
Bateman, I., Kahneman, D., Munro, A., Starmer, C., & Sugden, R. (2005) · Journal of Public Economics · doi:10.1016/j.jpubeco.2004.06.013 · Cited by 208
Normative Dissonance in Science: Results from a National Survey of U.S. Scientists
Anderson, M. S., Martinson, B. C., & De Vries, R. (2007) · Journal of Empirical Research on Human Research Ethics · doi:10.1525/jer.2007.2.4.3 · Cited by 208
The value of a statistical life: A meta-analysis with a mixed effects regression model
Bellavance, F., Dionne, G., & Lebeau, M. (2009) · Journal of Health Economics · doi:10.1016/j.jhealeco.2008.10.013 · Cited by 192
Response to Comment on “Estimating the reproducibility of psychological science”
Anderson, C. J., Bahník, Š., Barnett-Cowan, M., Bosco, F. A., Chandler, J., Chartier, C. R., Cheung, F., Christopherson, C. D., Cordes, A., Cremata, E. J., et al. (2016) · Science · doi:10.1126/science.aad9163 · Cited by 191
PICTURE THIS: A SIMPLE GRAPH THAT REVEALS MUCH ADO ABOUT RESEARCH
Stanley, T. D., & Doucouliagos, H. (2010) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2009.00593.x · Cited by 185
Scientific misbehavior in economics
Necker, S. (2014) · Research Policy · doi:10.1016/j.respol.2014.05.002 · Cited by 177
Control of the false discovery rate under dependence using the bootstrap and subsampling
Romano, J. P., Shaikh, A. M., & Wolf, M. (2008) · Test · doi:10.1007/s11749-008-0126-6 · Cited by 177
BLIND ANALYSIS IN NUCLEAR AND PARTICLE PHYSICS
Klein, J. R., & Roodman, A. J. (2005) · Annual Review of Nuclear and Particle Science · doi:10.1146/annurev.nucl.55.090704.151521 · Cited by 174
McCullough, B. D., McGeary, K. A., & Harrison, T. D. (2006) · Journal of Money, Credit, and Banking · doi:10.1353/mcb.2006.0061 · Cited by 173
Female hurricanes are deadlier than male hurricanes
Jung, K., Shavitt, S., Viswanathan, M., & Hilbe, J. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1402786111 · Cited by 169
The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey
Hirshleifer, S., McKenzie, D., Almeida, R., & Ridao-Cano, C. (2016) · The Economic Journal · doi:10.1111/ecoj.12211 · Cited by 167
Does Competition Among Public Schools Benefit Students and Taxpayers? Comment
Rothstein, J. (2007) · American Economic Review · doi:10.1257/aer.97.5.2026 · Cited by 167
Testing for Publication Bias in Political Science
Gerber, A. S., GREEN, D. P., & Nickerson, D. W. (2001) · Political Analysis · doi:10.1093/oxfordjournals.pan.a004877 · Cited by 162
Re-examining Economic Shocks and Civil Conflict
Miguel, E. A., & Satyanath, S. (2011) · American Economic Journal Applied Economics · doi:10.1257/app.3.4.228 · Cited by 157
Self-citationPre-Analysis Plans Have Limited Upside, Especially Where Replications Are Feasible
Coffman, L. C., & Niederle, M. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.81 · Cited by 156
Are estimates of the value of a statistical life exaggerated?
Doucouliagos, H., Stanley, T. D., & Giles, M. J. (2011) · Journal of Health Economics · doi:10.1016/j.jhealeco.2011.10.001 · Cited by 156
Estimation of Random-Coefficient Demand Models: Two Empiricists' Perspective
Knittel, C. R., & Μεταξόγλου, Κ. (2013) · The Review of Economics and Statistics · doi:10.1162/rest_a_00394 · Cited by 148
Do economics journal archives promote replicable research?
McCullough, B. D., McGeary, K. A., & Harrison, T. D. (2008) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.1111/j.1540-5982.2008.00509.x · Cited by 146
Aiken, A. M., Davey, C., Hargreaves, J., & Hayes, R. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv127 · Cited by 141
The Impact of Legalized Abortion on Crime: Comment*
Foote, C. L., & Goetz, C. (2008) · The Quarterly Journal of Economics · doi:10.1162/qjec.2008.123.1.407 · Cited by 134
Toward Fairness in Data Sharing
The International Consortium of Investigators for Fairness in Trial Data Sharing (2016) · New England Journal of Medicine · doi:10.1056/nejmp1605654 · Cited by 132
Denton, F. T. (1985) · The Review of Economics and Statistics · doi:10.2307/1928442 · Cited by 131
Registration of observational studies
Loder, E., Groves, T., & Macauley, D. (2010) · BMJ · doi:10.1136/bmj.c950 · Cited by 131
Open Access to Data: An Ideal Professed but Not Practised
Andreoli-Versbach, P., & Mueller-Langer, F. (2014) · Research Policy · doi:10.1016/j.respol.2014.04.008 · Cited by 124
Making Prospective Registration of Observational Research a Reality
Dal-Ré, R., Ioannidis, J. P., Bracken, M. B., Buffler, P. A., Chan, A.-W., Franco, E. L., La Vecchia, C., & Weiderpass, E. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3007513 · Cited by 123
Protocol for CONSORT-SPI: an extension for social and psychological interventions
Montgomery, P., Grant, S., Hopewell, S., Macdonald, G. M., Moher, D., Michie, S., & Mayo-Wilson, E. (2013) · Implementation Science · doi:10.1186/1748-5908-8-99 · Cited by 123
Reflections on Econometric Methodology*
Phillips, P. C. B. (1988) · Economic Record · doi:10.1111/j.1475-4932.1988.tb02075.x · Cited by 119
Strengthening Research through Data Sharing
Warren, E. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1607282 · Cited by 116
A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections
Monogan, J. E. (2013) · Political Analysis · doi:10.1093/pan/mps022 · Cited by 116
Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz*
Donohue, J. J., & Levitt, S. D. (2008) · The Quarterly Journal of Economics · doi:10.1162/qjec.2008.123.1.425 · Cited by 110
Privacy and Data-Based Research
Heffetz, O., & Ligett, K. (2014) · The Journal of Economic Perspectives · doi:10.1257/jep.28.2.75 · Cited by 106
The Income Elasticity of Money Demand: A Meta-Analysis of Empirical Results*
Knell, M., & Stix, H. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00257.x · Cited by 105
Malin, B., Benitez, K., & Masys, D. R. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004622 · Cited by 104
Fink, G., McConnell, M. A., & Vollmer, S. (2014) · Journal of Development Effectiveness · doi:10.1080/19439342.2013.875054 · Cited by 103
Publication selection and the income elasticity of the value of a statistical life
Doucouliagos, H., Stanley, T. D., & Viscusi, W. K. (2013) · Journal of Health Economics · doi:10.1016/j.jhealeco.2013.10.010 · Cited by 101
The Role of Publication Selection Bias in Estimates of the Value of a Statistical Life
Viscusi, W. K. (2015) · American Journal of Health Economics · doi:10.1162/ajhe_a_00002 · Cited by 96
On reproducible econometric research
Koenker, R. W., & Zeileis, A. (2009) · Journal of Applied Econometrics · doi:10.1002/jae.1083 · Cited by 94
The role of data/code archives in the future of economic research
Anderson, R. G., Greene, W. H., McCullough, B. D., & Vinod, H. D. (2008) · Journal of Economic Methodology · doi:10.1080/13501780801915574 · Cited by 93
Whose Line Is It? Plagiarism in Economics
Enders, W., & Hoover, G. A. (2004) · Journal of Economic Literature · doi:10.1257/0022051041409066 · Cited by 90
Model Uncertainty and the Deterrent Effect of Capital Punishment
Cohen‐Cole, E., Durlauf, S. N., Fagan, J. A., & Nagin, D. S. (2008) · American Law and Economics Review · doi:10.1093/aler/ahn001 · Cited by 89
Strategic Disclosure of Research Results: The Cost of Proving Your Honesty
Henry, E. (2009) · The Economic Journal · doi:10.1111/j.1468-0297.2009.02265.x · Cited by 87
Ulrich, R., & Miller, J. O. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000086 · Cited by 87
Drazen, J. M. (2016) · New England Journal of Medicine · doi:10.1056/nejme1601087 · Cited by 82
Data Availability in Political Science Journals
Gherghina, S., & Κατσανίδου, Α. (2013) · European Political Science · doi:10.1057/eps.2013.8 · Cited by 81
Does Competition Among Public Schools Benefit Students and Taxpayers? Reply
Hoxby, C. (2007) · American Economic Review · doi:10.1257/aer.97.5.2038 · Cited by 81
Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study
Findley, M. G., Jensen, N. M., Malesky, E. J., & Pepinsky, T. B. (2016) · Comparative Political Studies · doi:10.1177/0010414016655539 · Cited by 80
Open Access Economics Journals and the Market for Reproducible Economic Research
McCullough, B. D. (2009) · Economic Analysis and Policy · doi:10.1016/s0313-5926(09)50047-1 · Cited by 78
Academic economists behaving badly? A survey on three areas of unethical behavior
List, J. A. (2001) · Economic Inquiry · doi:10.1093/ei/39.1.162 · Cited by 75
A meta-analysis of the export growth hypothesis
Mookerjee, R. (2006) · Economics Letters · doi:10.1016/j.econlet.2005.12.020 · Cited by 74
Lee, S., & Shaikh, A. M. (2013) · Journal of Applied Econometrics · doi:10.1002/jae.2327 · Cited by 73
Cited by
Hou, K., Xue, C., & Zhang, L. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhy131 · Cited by 989
RCTs to Scale: Comprehensive Evidence From Two Nudge Units
DellaVigna, S., & Linos, E. (2022) · Econometrica · doi:10.3982/ecta18709 · Cited by 480
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A., Cook, N., & Heyes, A. (2020) · American Economic Review · doi:10.1257/aer.20190687 · Cited by 333
Model Averaging and Its Use in Economics
Steel, M. F. J. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191385 · Cited by 321
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Metadata
- Authors
- Christensen, G., & Miguel, E.
- Venue
- Journal of Economic Literature
- Publisher
- American Economic Association
- Published
- 2018-09-01
- Volume
- 56
- Issue
- 3
- Pages
- 920-980
- ISSN
- 0022-0515 (ISSN-L)
- OpenAlex
- W2561942675
- Cited by
- 446
- References
- 227
- FWCI
- 78.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.