Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
BACKGROUND: To allow an accurate evaluation of abstracts of systematic reviews, the PRISMA Statement recommends that the limitations of the evidence (e.g., risk of bias, publication bias, inconsistency, imprecision) should be described in the abstract. We aimed to evaluate the impact of adding such limitations sections on reader's interpretation. METHOD: We performed a two-arm parallel group randomized controlled trial (RCT) using a sample of 30 abstracts of systematic reviews evaluating the effects of healthcare intervention with conclusions favoring the beneficial effect of the experimental treatments. Two formats of these abstracts were derived: one reported without and one with a standardized limitations section written according to the PRISMA statement for abstracts. The primary outcome was readers' confidence in the results of the systematic review as stated in the abstract assessed by a Likert scale from 0, not at all confident, to 10, very confident. In total, 300 participants (corresponding authors of RCT reports indexed in PubMed) were randomized by a web-based randomization procedure to interpret one abstract with a limitations section (n = 150) or without a limitations section (n = 150). Participants were blinded to the study hypothesis. RESULTS: Adding a limitations section did not modify readers' interpretation of findings in terms of confidence in the results (mean difference [95% confidence interval] 0.19 [-0.37-0.74], p = 0.50), confidence in the validity of the conclusions (0.07 [-0.49-0.62], p = 0.80), or benefit of the experimental intervention (0.12 [-0.42-0.44], p = 0.65).This study is limited because the participants were expert-readers and are not representative of all systematic review readers. CONCLUSION: Adding a limitations section to abstracts of systematic reviews did not affect readers' interpretation of the abstract results. Other studies are needed to confirm the results and explore the impact of a limitations section on a less expert panel of participants. TRIAL REGISTRATION: ClinicalTrial.gov (NCT01848782).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, the Annals of Internal Medicine requires the reporting of a limitation in the abstract (75, 76).”
From Misrepresentation and distortion of research in biomedical literature (2018) · cites this paper as [76]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“To improve the transparency of abstracts [9,10], methodologists, researchers and editors have established recommendations for the presentation of systematic reviews in the PRISMA statements [11] with an extension for reporting abstracts [12].”
BackgroundGRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · cited 1×
“Background Systematic reviews are the cornerstone of therapeutic evaluation [1].”
BackgroundCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Yeh, M.-L. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,829
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250 · cited 1×
“of trials included, median; [Q1-Q3]; (min-max) 13; [6–23]; (4–54)”
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184 · cited 1×
“Clinicians, decision makers and researchers use them to keep up-to-date with current medical literature, develop clinical practice guidelines and sometimes plan future research [2-4].”
BackgroundSeventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“Clinicians, decision makers and researchers use them to keep up-to-date with current medical literature, develop clinical practice guidelines and sometimes plan future research [2-4].”
Background
Cited by
Misrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273
“For example, the Annals of Internal Medicine requires the reporting of a limitation in the abstract (75, 76).”
BackgroundSelf-citationFunctionality of sugars in foods and health
Clemens, R. A., Jones, J. M., Kern, M., Lee, S., Mayhew, E. J., Slavin, J. L., & Zivanovic, S. (2016) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.12194 · Cited by 224
Carl, J., Barratt, J., Arbour‐Nicitopoulos, K. P., Barnett, L. M., Dudley, D. A., Holler, P., Keegan, R. J., KWAN, M. Y. W., Scurati, R., Sum, R. K. W., et al. (2023) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-023-01423-3 · Cited by 54
“There is some evidence that such information not only promotes transparency and scientific progress [93], but also that the disclosure of limitations does not substantially affect the reader’s confidence in the results and conclusions [94].”
BackgroundREAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
Smith, J. J., Amershi, S., Barocas, S., Wallach, H., & Vaughan, J. W. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533122 · Cited by 52
“Due to the severity of the potential harms that may arise if research findings are misinterpreted or taken out of context and the need to guard against both unintentional misreporting and deliberate spinning of research claims [3], some publication venues in fields like biomedicine require limitations to be stated in…”
BackgroundPetkovic, J., Welch, V. A., Jacob, M. H., Yoganathan, M., Ayala, A. P., Cunningham, H., & Tugwell, P. (2018) · Campbell Systematic Reviews · doi:10.4073/csr.2018.8 · Cited by 21
Metadata
- Authors
- Yavchitz, A., Ravaud, P., Hopewell, S., Baron, G., & Boutron, I.
- Affiliations
- Inserm (FR)
- Corresponding author
- Yavchitz, Amélie
- Venue
- BMC Medical Research Methodology
- Published
- 2014-11-24
- Volume
- 14
- Issue
- 1
- Pages
- 123-123
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC4247631
- OpenAlex
- W2109435591
- Cited by
- 11
- References
- 33
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 1 · 2022: 2 · 2019: 1 · 2018: 2 · 2016: 1
- FWCI
- 0.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Columbia Academic Commons (Columbia University)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- PRISMA statement, reader confidence, risk of bias, publication bias, health interventions, interpretation of research findings
- MeSH terms
- Abstracting and Indexing, Adult, Female, Health Personnel, Humans, Male, Middle Aged, Meta-Analysis as Topic, Bias, Outcome Assessment, Health Care, Publication Bias, Editorial Policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial. https://app.cochat.ai/papers/10.1186/1471-2288-14-123
- The paper
- Yavchitz, A., Ravaud, P., Hopewell, S., Baron, G., & Boutron, I. (2014). Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-14-123
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
