Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Reporting quality of systematic reviews' (SRs) abstracts is important because this is often the only information about a study that readers have. The aim of this study was to assess adherence of SR abstracts in the field of anesthesiology with the reporting checklist PRISMA extension for Abstracts (PRISMA-A) and to analyze to what extent will the use of PRISMA-A yield concordant ratings in two raters without prior experience with the checklist. METHODS: We analyzed reporting quality of SRs with meta-analysis of randomized controlled trials of interventions published in the field of anesthesiology from 2012 to 2016 by using 12-item PRISMA-A checklist. After calibration exercise, two authors without prior experience with PRISMA-A scored the abstracts. Primary outcome was median adherence to PRISMA-A checklist. Secondary outcome was adherence to individual items of the checklist. We analyzed whether there was improvement in reporting of SR abstracts over time. Additionally, we analyzed discrepancies between the two raters in scoring individual PRISMA-A items. RESULTS: Our search yielded 318 results, of which we included 244 SRs. Median adherence to PRISMA-A checklist was 42% (5 items of 12). The majority of analyzed SR abstracts (N = 148, 61%) had a total adherence score under 50%, and not a single one had adherence above 75%. Adherence to individual items was very variable, ranging from 0% for reporting SR funding, to 97% for interpreting SR findings. Overall adherence to PRISMA-A did not change over the analyzed 5 years before and after publication of PRISMA-A in 2013. Even after calibration exercise, discrepancies between the two raters were found in 275 (9.3%) out of 2928 analyzed PRISMA-A items. Cohen's Kappa was 0.807. In the item about the description of effect there were discrepancies in 59% of the abstracts between the raters. CONCLUSION: Reporting quality of systematic review abstracts in the field of anesthesiology is suboptimal, and did not improve after publication of PRISMA-A checklist in 2013. We need stricter adherence to reporting checklists by authors, editors and peer-reviewers, and interventions that will help those stakeholders to improve reporting of systematic reviews. Some items of PRISMA-A checklist are difficult to score.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, Maticic et al.[15] from 2012 to 2016 reported on abstracts of systematic review and MA published in all journals within the Journal Citation Reports (JCR) category Anesthesiology based on the JCR impact factor for year 2016 using the PRISMA abstract checklist 2013 and found that item 1, Item 10 were aligned with the findings of the present research.”
From Quality of Reporting Abstracts of Systematic Reviews and Meta-Analysis in Preventive Dentistry: An Assessment Using the PRISMA 2020 Extension (2026) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, assessments of the completeness of reporting of SRs/MAs abstracts have been undertaken in other fields e.g. dental specialties [3,5], anesthesiology [17], or general medical literature [18,19].”
From Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist (2023) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Pain Management: A Fundamental Human Right
Brennan, F., Carr, D. B., & Cousins, M. (2007) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000268145.52345.55 · Cited by 1,014 · cited 1×
“Since pain management is considered fundamental human right [9], it could be argued that importance of proper reporting in the field of anesthesiology and pain is extremely important for both patients and health care workers.”
BackgroundPRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts
Beller, E. M., Glasziou, P. P., Altman, D. G., Hopewell, S., Bastian, H., Chalmers, I., Gøtzsche, P. C., Lasserson, T., Tovey, D., & for the PRISMA for Abstracts Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001419 · Cited by 706 · cited 7×
“However, PRISMA-A was published until 2013 [3], so the authors of the abstracts analyzed by Kazerani et al.”
BackgroundMethodInfluentialBigna, J. J. R., Um, L. N., & Nansseu, J. R. N. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0356-8 · Cited by 51
Hopewell, S., Boutron, I., Altman, D. G., & Ravaud, P. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.006 · Cited by 47 · cited 7×
“were item 1 (title) with 89% of adherence and sub-item 4 (number and type of included studies) with 81% of adherence [8].”
BackgroundMethodResultInfluentialŠarić, L., Vucic, K., Dragicevic, K., Vrdoljak, M., Jakus, D., Vuka, I., Kadić, A. J., Saldanha, I. J., & Puljak, L. (2018) · European Journal of Pain · doi:10.1002/ejp.1289 · Cited by 42 · cited 1×
“We also found that RCT abstracts presented in four consecutive World Congresses on Pain are not properly reported and not necessarily dependable sources of information, as there are frequently discrepant results between RCT presented at a conference and published subsequently in a peer-reviewed journal [13].”
ResultSelf-citationPuljak, L. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.002 · Cited by 36
Self-citationImproving the quality and reporting of systematic reviews⋆
Eccleston, C., Moore, R. A., Derry, S., Bell, R. F., & McQuay, H. (2010) · European Journal of Pain · doi:10.1016/j.ejpain.2010.05.015 · Cited by 31 · cited 1×
“indicated in 2010, there are tools that can help authors improve quality and reporting of SRs [14].”
Background
Research works printed but not linked: 1
- CRD: Center for Reviews and Dissemination, University of York. Undertaking Systematic Reviews of Research on Effectiveness. Research report. CRD Report 2001, 4(2). York: NHS Center or Reviews and Dissemination; 2001.
Cited by
A tutorial on methodological studies: the what, when, how and why
Mbuagbaw, L. C. E., Lawson, D. O., Puljak, L., Allison, D. B., & Thabane, L. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01107-7 · Cited by 200
Self-citationHelbach, J., Hoffmann, F., Pieper, D., & Allers, K. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.12.019 · Cited by 70
Definition, harms, and prevention of redundant systematic reviews
Puljak, L., & Lund, H. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02191-8 · Cited by 45
Self-citationJones, C. D., Rulon, Z., Arthur, W., Ottwell, R., Checketts, J. X., Detweiler, B., Calder, M., Adil, A., Hartwell, M. L., Wright, D., et al. (2021) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2020.11.026 · Cited by 41
Li, T., Hua, F., Dan, S., Zhong, Y., Levey, C., & Song, Y. (2020) · Journal of Dentistry · doi:10.1016/j.jdent.2020.103471 · Cited by 38
Metadata
- Authors
- Maticic, K., Krnic Martinic, M., & Puljak, L.
- Affiliations
- Catholic University of Croatia (HR)
- Corresponding author
- Puljak, Livia
- ORCID
- Puljak, Livia
- Venue
- BMC Medical Research Methodology
- Published
- 2019-02-14
- Volume
- 19
- Issue
- 1
- Pages
- 32-32
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC6376734
- OpenAlex
- W2934309103
- Cited by
- 43
- References
- 13
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 4 · 2023: 5 · 2022: 5 · 2021: 12 · 2020: 5 · 2019: 2
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- reporting quality, reporting checklist, inter-rater reliability, anesthesiology, meta-analysis, reporting guidelines adherence, funding disclosure, peer review, editorial policy, Cohen's kappa
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience. https://app.cochat.ai/papers/10.1186/s12874-019-0675-2
- The paper
- Maticic, K., Krnic Martinic, M., & Puljak, L. (2019). Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-019-0675-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.