CoChat

Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report

Open access
Clarke, Mike, Hopewell, Sally, Chalmers, Lain · 2007 · Journal of the Royal Society of Medicine
Cited by 129

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

Summary Objective: Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported. Design: Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals. Setting: Reports of randomized trials in five general medical journals. Participants: Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine. Interventions: None. Main outcome measures: The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed. Results: We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the Discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration—quantitative or qualitative—of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials. Conclusions: There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/014107680710011415

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 7 references

Research works printed but not linked: 21

  1. 10.1371/journal.pmed.0010035 · DOI
  2. 10.1016/S0140-6736(98)07618-1 · DOI
  3. 10.1001/jama.280.3.280 · DOI · Paper page
  4. 10.1001/jama.287.21.2799 · DOI · Paper page
  5. Hill AB, 1965, BMJ, 4, 870
  6. 10.1191/1740774505cn090oa · DOI · Paper page
  7. 10.1016/S0140-6736(05)66846-8 · DOI · Paper page
  8. 10.1016/S0140-6736(06)68734-5 · DOI · Paper page
  9. 10.1001/jama.293.19.2367 · doi:10.1001/jama.293.19.2367
  10. 10.1136/bmj.38427.404259.8F · DOI · Paper page
  11. 10.1016/S0140-6736(05)66617-2 · DOI · Paper page
  12. 10.7326/0003-4819-142-10-200505170-00005 · DOI · Paper page
  13. 10.1016/S0140-6736(05)66545-2 · DOI · Paper page
  14. 10.1001/jama.293.18.2245 · DOI · Paper page
  15. 10.1056/NEJMsa042830 · DOI · Paper page
  16. 10.1056/NEJMoa041109 · DOI · Paper page
  17. 10.1001/jama.293.17.2118 · DOI · Paper page
  18. 10.1016/S0140-6736(05)66388-X · DOI · Paper page
  19. 10.1016/S0140-6736(05)66426-4 · DOI · Paper page
  20. 10.1056/NEJMoa043596 · DOI · Paper page
  21. 10.1001/jama.293.17.2109 · DOI · Paper page

Cited by

See all 129

Metadata

Authors
Clarke, M., Hopewell, S., & Chalmers, L.
Venue
Journal of the Royal Society of Medicine
Published
2007-04
Volume
100
Issue
4
Pages
187-190
ISSN
  • 0141-0768 (ISSN-L)
  • 1758-1095
OpenAlex
W2072165789
Cited by
129
References
7
FWCI
3.03
Type
journal article
Language
en
Open access
green
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. https://app.cochat.ai/papers/10.1177/014107680710011415
The paper
Clarke, M., Hopewell, S., & Chalmers, L. (2007). Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. Journal of the Royal Society of Medicine. https://doi.org/10.1177/014107680710011415

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.