The DOI matches its Crossref record; 227 of the 227 references it cites were found in a registry or an index.
Transparency, Reproducibility, and the Credibility of Economics Research
Open accessAbstract
There is growing interest in enhancing research transparency and reproducibility in economics and other scientific fields. We survey existing work on these topics within economics and discuss the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in the discipline. We next discuss recent progress in this area, including through improved research design, study registration and pre-analysis plans, disclosure standards, and open sharing of data and materials, drawing on experiences in both economics and other social sciences. We discuss areas where consensus is emerging on new practices, as well as approaches that remain controversial, and speculate about the most effective ways to make economics research more credible in the future. ( JEL A11, C18, I23)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
3 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
RetractedWhen contact changes minds: An experiment on transmission of support for gay equality
LaCour, M. J., & GREEN, D. P. (2014) · Science · doi:10.1126/science.1256151 · Cited by 136
RetractedRetracted : Correction or comparison? The effects of prime awareness on social judgments
Maringer, M., & Stapel, D. A. (2009) · European Journal of Social Psychology · doi:10.1002/ejsp.569 · Cited by 26
RetractedControlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
Journal of economic literature
Armstrong, J. S. (1996) · International Journal of Forecasting · doi:10.1016/0169-2070(96)88197-0 · Cited by 9,923
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Cited by
Hou, K., Xue, C., & Zhang, L. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhy131 · Cited by 989
RCTs to Scale: Comprehensive Evidence From Two Nudge Units
DellaVigna, S., & Linos, E. (2022) · Econometrica · doi:10.3982/ecta18709 · Cited by 480
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A., Cook, N., & Heyes, A. (2020) · American Economic Review · doi:10.1257/aer.20190687 · Cited by 333
Model Averaging and Its Use in Economics
Steel, M. F. J. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191385 · Cited by 321
“iscard estimates inconsistent with the competitionfragility hypothesis, one of the two main hypotheses in this area. For an in-depth discussion of publication bias and meta-analyses in economics, see Christensen and Miguel (2018). 71 nants of output changes after crises and conclude that for high-income countries the…”
BackgroundResultSwedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
The GDP-Temperature relationship: Implications for climate change damages
Newell, R. G., Prest, B. C., & Sexton, S. E. (2021) · Journal of Environmental Economics and Management · doi:10.1016/j.jeem.2021.102445 · Cited by 280
Are Referees and Editors in Economics Gender Neutral?*
Card, D., DellaVigna, S., Funk, P., & Iriberri, N. (2020) · The Quarterly Journal of Economics · doi:10.1093/qje/qjz035 · Cited by 279
Reporting guidelines for meta-analysis in economics
Havránek, T., Stanley, T. D., Doucouliagos, H., Bom, P., Geyer‐Klingeberg, J., Iwasaki, I., Reed, W. R., Rost, K., & van Aert, R. C. M. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12363 · Cited by 271
“…assessed the incidence and impact of publication bias and p-hacking in economics (Brodeur et al., 2016; Bruns and Ioannidis, 2016; Ioannidis et al., 2017; Brodeur et al., 2018; Christensen and Miguel, 2018), and attention has turned to journal editorial policies (Blanco-Perez and Brodeur, 2020).”
BackgroundRemittances and economic growth: A meta-analysis
Cazachevici, A., Havranek, T., & Horvath, R. (2020) · World Development · doi:10.1016/j.worlddev.2020.105021 · Cited by 262
Estimating the Armington elasticity: The importance of study design and publication bias
Bajzik, J., Havranek, T., Irsova, Z., & Schwarz, J. (2020) · Journal of International Economics · doi:10.1016/j.jinteco.2020.103383 · Cited by 150
Influence of land tenure interventions on human well-being and environmental outcomes
Tseng, T.-W. J., Robinson, B. E., Bellemare, M. F., BenYishay, A., Blackman, A., Boucher, T., Childress, M., Holland, M. B., Kroeger, T., Linkow, B., et al. (2020) · Nature Sustainability · doi:10.1038/s41893-020-00648-5 · Cited by 140
The Distribution of School Spending Impacts
Jackson, C. K., & Mackevicius, C. (2021) · doi:10.3386/w28517 · Cited by 89
“Studies that find null results may be less likely to be published than studies that find signif- icant effects (Franco et al. (2014), Christensen and Miguel (2018)).”
BackgroundHow puzzling is the forward premium puzzle? A meta-analysis
Zigraiova, D., Havranek, T., Irsova, Z., & Novak, J. (2021) · European Economic Review · doi:10.1016/j.euroecorev.2021.103714 · Cited by 71
Integrating Qualitative Methods and Open Science: Five Principles for More Trustworthy Research*
Humphreys, L., Lewis, N. A., Sender, K., & Won, A. S. (2021) · Journal of Communication · doi:10.1093/joc/jqab026 · Cited by 69
Instrumental Variables and Causal Mechanisms: Unpacking The Effect of Trade on Workers and Voters
Dippel, C., Gold, R., Heblich, S., & Pinto, R. (2017) · doi:10.3386/w23209 · Cited by 59
Gordon, M., Viganola, D., Dreber, A., Johannesson, M., & Pfeiffer, T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0248780 · Cited by 44
Can Restorative Justice Conferencing Reduce Recidivism? Evidence From the Make-it-Right Program
Shem-Tov, Y., Raphael, S., & Skog, A. (2021) · National Bureau of Economic Research · doi:10.3386/w29150 · Cited by 15
Metadata
- Authors
- Christensen, G., & Miguel, E.
- Venue
- Journal of Economic Literature
- Publisher
- American Economic Association
- Published
- 2018-09-01
- Volume
- 56
- Issue
- 3
- Pages
- 920-980
- ISSN
- 0022-0515 (ISSN-L)
- OpenAlex
- W2561942675
- Cited by
- 446
- References
- 227
- FWCI
- 78.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.