Cited by 75Academic Writing and PublishingAcademic Publishing and Open AccessMeta-analysis and systematic reviews

The Hidden Research Paper

Horton, Richard·2002·JAMA

Abstract

CONTEXT: To determine whether the views expressed in a research paper are accurate representations of contributors' opinions about the research being reported. METHODS: Purposive sampling of 10 research articles published in The Lancet; qualitative analysis of answers to 6 questions about the meaning of the study put to contributors who were listed on the byline of these articles. Fifty-four contributors listed on the bylines of the 10 articles were evaluated, and answers to questions were compared between contributors within research groups and against the published research report. RESULTS: A total of 36 (67%) of 54 contributors replied to this survey. Important weaknesses were often admitted on direct questioning but were not included in the published article. Contributors frequently disagreed about the importance of their findings, implications, and directions for future research. I could find no effort to study systematically past evidence relating to the investigators' own findings in either survey responses or the published article. Overall, the diversity of contributor opinion was commonly excluded from the published report. I found that discussion sections were haphazardly organized and did not deal systematically with important questions about the study. CONCLUSIONS: A research paper rarely represents the opinions of those scientists whose work it reports. The findings described herein reveal evidence of (self-)censored criticism, obscured meanings, confused assessment of implications, and failures to indicate directions for future research. There is now empirical support for the introduction of structured discussion sections in research papers. Editors might also explore ways to recover the plurality of contributors' opinions.

Integrity signals

Registry

Crossref · 10.1001/jama.287.21.2775

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Surveys have found that discussion sections are often dominated by incomplete or biased assessments of the study's results and their implications, and rhetoric supporting the authors' findings.

    “Surveys have found that discussion sections are often dominated by incomplete or biased assessments of the study's results and their implications, and rhetoric supporting the authors' findings [ 192 , 193 ].”
  • A survey of authors who had published original research articles in The Lancet found that important weaknesses of the study were reported by the investigators in the survey questionnaires, but not in the published article.

    “A survey of authors who had published original research articles in The Lancet found that important weaknesses of the study were reported by the investigators in the survey questionnaires, but not in the published article [ 192 ].”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Horton, R.
Corresponding author
Horton, Richard
Venue
JAMA
Published
2002-06-05
Volume
287
Issue
21
Pages
2775
ISSN
  • 0002-9955 (ISSN-L)
  • 0098-7484
OpenAlex
W2095959788
Cited by
75
References
15
Citations by year
2024: 2 · 2023: 3 · 2022: 4 · 2021: 1 · 2019: 4 · 2018: 2 · 2017: 2 · 2016: 1 · 2015: 3 · 2014: 4 · 2013: 3 · 2012: 2
FWCI
17.75
Type
journal article
Language
en
Open access
closed
Locations
  • JAMA (publishedVersion)
  • PubMed (publishedVersion)
  • DigitalGeorgetown (Georgetown University Library) (submittedVersion)
Topics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
self-censorship, discussion section, research implications, Lancet
MeSH terms
Manuscripts, Medical as Topic, Publishing, Writing, Peer Review, Research, Editorial Policies, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review