Reproducible Research: Moving toward Research the Public Can Really Trust
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
A community of scientists arrives at the truth by independently verifying new observations. In this time-honored process, journals serve 2 principal functions: evaluative and editorial. In their evaluative function, they winnow out research that is unlikely to stand up to independent verification; this task is accomplished by peer review. In their editorial function, they try to ensure transparent (by which we mean clear, complete, and unambiguous) and objective descriptions of the research. Both the evaluative and editorial functions go largely unnoticed by the public--the former only draws public attention when a journal publishes fraudulent research. However, both play a critical role in the progress of science. This paper is about both functions. We describe the evaluative processes we use and announce a new policy to help the scientific community evaluate, and build upon, the research findings that we publish.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Health care providers and policy makers are increasingly recommending the use of prediction models within clinical practice guidelines to inform decision making at various stages in the clinical pathway (30,31).”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [31]
In the citing paper
“It is a general requirement of reporting of research that other researchers can, if required, replicate all the steps taken and obtain the same results [32].”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [32]
In the citing paper
“It is a general requirement of reporting of research that other researchers can, if required, replicate all the steps taken and obtain the same results.32”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [32]
Citations
Cites
1 retracted
Horton, R. C. (2006) · The Lancet · doi:10.1016/s0140-6736(06)68120-8 · Cited by 22 · cited 1×
“Lancet editors retracted a paper on oral cancer when the editors learned that the researchers had fabricated their data (2).”
BackgroundRetractedMoher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Moher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
De Angelis, C. D., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2005) · New England Journal of Medicine · doi:10.1056/nejme058127 · Cited by 354
Self-citationManuscript Quality before and after Peer Review and Editing at Annals of Internal Medicine
Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-1-199407010-00003 · Cited by 324 · cited 1×
“These outcomes of editing and peer review have been documented in empirical studies at Annals (13) and, more recently, in a preliminary report of a study that included several journals (14).”
BackgroundSelf-citationReproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292 · cited 2×
“Recognizing this, Peng and colleagues describe forms of data sharing that are between unlimited access and no access (17).”
BackgroundFontanarosa, P., Flanagin, A., & DeAngelis, C. D. (2005) · JAMA · doi:10.1001/jama.294.1.110 · Cited by 134 · cited 1×
“Our colleagues at JAMA have established a policy requiring an independent academic statistician to corroborate analyses of industryfunded studies (19).”
BackgroundExpression of Concern Reaffirmed
Curfman, G. D., Morrissey, S., & Drazen, J. M. (2006) · New England Journal of Medicine · doi:10.1056/nejme068054 · Cited by 109
Is This Clinical Trial Fully Registered?
Angelis, C. D. D., Drazen, J. M., Frizelle, F., Haug, C. J., Hoey, J., Horton, R., Kotzin, S., Laine, C. M., Marušić, A., Overbeke, A. J. P. M., et al. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-2-200507190-00016 · Cited by 62
Self-citation
Research works printed but not linked: 1
- Horton R, 2006, Lancet, 367, 382
Cited by
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
“It is a general requirement of reporting of research that other researchers can, if required, replicate all the steps taken and obtain the same results (32).”
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
“It is a general requirement of reporting of research that other researchers can, if required, replicate all the steps taken and obtain the same results (Laine et al, 2007).”
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
“It is a general requirement of reporting of research that other researchers can, if required, replicate all the steps taken and obtain the same results [32].”
Result
Metadata
- Authors
- Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C.
- Affiliations
- American College of Physicians (US)
- Johns Hopkins University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2007-03-20
- Volume
- 146
- Issue
- 6
- Pages
- 450-453
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2063463192
- Cited by
- 213
- References
- 19
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 3 · 2023: 7 · 2022: 6 · 2021: 34 · 2020: 6 · 2019: 9 · 2018: 16 · 2017: 6 · 2016: 10 · 2015: 25 · 2014: 19 · 2013: 11 · 2012: 13
- FWCI
- 25.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- reproducible research, peer review, journal editorial policy, research evaluation, scholarly publishing, research transparency, research trust
- MeSH terms
- Periodicals as Topic, Publishing, Reproducibility of Results, Scientific Misconduct, Conflict of Interest, Peer Review, Research, Ethics, Research, Editorial Policies, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reproducible Research: Moving toward Research the Public Can Really Trust. https://app.cochat.ai/papers/10.7326/0003-4819-146-6-200703200-00154
- The paper
- Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C. (2007). Reproducible Research: Moving toward Research the Public Can Really Trust. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-146-6-200703200-00154
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.