Is Quality and Completeness of Reporting of Systematic Reviews and Meta-Analyses Published in High Impact Radiology Journals Associated with Citation Rates?
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
PURPOSE: The purpose of this study is to determine whether study quality and completeness of reporting of systematic reviews (SR) and meta-analyses (MA) published in high impact factor (IF) radiology journals is associated with citation rates. METHODS: All SR and MA published in English between Jan 2007-Dec 2011, in radiology journals with an IF >2.75, were identified on Ovid MEDLINE. The Assessing the Methodologic Quality of Systematic Reviews (AMSTAR) checklist for study quality, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for study completeness, was applied to each SR & MA. Each SR & MA was then searched in Google Scholar to yield a citation rate. Spearman correlation coefficients were used to assess the relationship between AMSTAR and PRISMA results with citation rate. Multivariate analyses were performed to account for the effect of journal IF and journal 5-year IF on correlation with citation rate. Values were reported as medians with interquartile range (IQR) provided. RESULTS: 129 studies from 11 journals were included (50 SR and 79 MA). Median AMSTAR result was 8.0/11 (IQR: 5-9) and median PRISMA result was 23.0/27 (IQR: 21-25). The median citation rate for SR & MA was 0.73 citations/month post-publication (IQR: 0.40-1.17). There was a positive correlation between both AMSTAR and PRISMA results and SR & MA citation rate; ρ=0.323 (P=0.0002) and ρ=0.327 (P=0.0002) respectively. Positive correlation persisted for AMSTAR and PRISMA results after journal IF was partialed out; ρ=0.243 (P=0.006) and ρ=0.256 (P=0.004), and after journal 5-year IF was partialed out; ρ=0.235 (P=0.008) and ρ=0.243 (P=0.006) respectively. CONCLUSION: There is a positive correlation between the quality and the completeness of a reported SR or MA with citation rate which persists when adjusted for journal IF and journal 5-year IF.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas
Garfield, E. (1955) · Science · doi:10.1126/science.122.3159.108 · Cited by 2,334
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
Meho, L. I., & Yang, K. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20677 · Cited by 974
Users' Guides to the Medical Literature: IX. A Method for Grading Health Care Recommendations
Guyatt, G. H. (1995) · JAMA · doi:10.1001/jama.1995.03530220066035 · Cited by 846
Callaham, M. (2002) · JAMA · doi:10.1001/jama.287.21.2847 · Cited by 644 · cited 1×
“A prior investigation of studies that were originally submitted to an emergency medicine specialty meeting found study design and quality did not correlate with post-publication citation rates [161].”
BackgroundMoher, D., Altman, D. G., Liberati, A., & Tetzlaff, J. (2011) · Epidemiology · doi:10.1097/ede.0b013e3181fe7825 · Cited by 627
(2000) · doi:10.1016/s0261-3794(99)00028-1 · Cited by 620
Reproductive outcomes in women with congenital uterine anomalies: a systematic review
Chan, Y. Y., Jayaprakasan, K., Tan, A., Thornton, J. G., Coomarasamy, A., & Raine‐Fenning, N. J. (2011) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.10056 · Cited by 564 · cited 1×
“129 studies from 11 journals were identified that met our inclusion criteria [30–158] (S2 Table).”
BackgroundKung, J. (2010) · The Open Dentistry Journal · doi:10.2174/1874210601004020084 · Cited by 329
Menke, J. (2010) · Radiology · doi:10.1148/radiol.10091938 · Cited by 319
Association of Journal Quality Indicators With Methodological Quality of Clinical Research Articles
Lee, K. P. (2002) · JAMA · doi:10.1001/jama.287.21.2805 · Cited by 284 · cited 1×
“This supports prior findings that studies of higher levels of evidence are cited more often [5,6,163].”
ResultHudelist, G., English, J., Thomas, A. E., Tinelli, A., Singer, C. F., & Keckstein, J. (2011) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.8858 · Cited by 270 · cited 1×
“129 studies from 11 journals were identified that met our inclusion criteria [30–158] (S2 Table).”
BackgroundDave, M., Elmunzer, B. J., Dwamena, B. A., & Higgins, P. D. R. (2010) · Radiology · doi:10.1148/radiol.10091953 · Cited by 265
Falagas, M. E., Zarkali, A., Karageorgopoulos, D. E., Bardakas, V., & Mavros, M. N. (2013) · PLoS ONE · doi:10.1371/journal.pone.0049476 · Cited by 168 · cited 1×
“Several factors shown to correlate with more citations include: study design and study topic in the urology literature [5]; high levels of evidence, large sample size, multi-institutional studies and conflict of interest disclosure in the orthopedic surgery literature [6]; extended description of statistical analysis…”
Background
Cited by
Factors Affecting Number of Citations: A Comprehensive Review of the Literature
Tahamtan, I., Safipour Afshar, A., & Ahamdzadeh, K. (2016) · Scientometrics · doi:10.1007/s11192-016-1889-2 · Cited by 912
“…et al. 2010; Patsopoulos et al. 2005; Peng and Zhu 2012; Piwowar et al. 2007; Ralston et al. 2008; Royle et al. 2013; Schneider and Henriksen 2013; Subotic and Mukherjee 2014; Van Der Pol et al. 2015; van Leeuwen and Moed 2005; Vanclay 2013; Vaughan and Shaw 2005; Weale et al. 2004; Winker 2011).”
BackgroundPage, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
Salameh, J.-P., Bossuyt, P. M., McGrath, T. A., Thombs, B. D., Hyde, C. J., Macaskill, P., Deeks, J. J., Leeflang, M., Korevaar, D. A., Whiting, P., et al. (2020) · BMJ · doi:10.1136/bmj.m2632 · Cited by 719
Self-citationSystematic Review of the Literature: Best Practices
Gupta, S., Rajiah, P., Middlebrooks, E. H., Baruah, D., Carter, B. W., Burton, K. R., Chatterjee, A. R., & Miller, M. M. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.04.025 · Cited by 105
Pitfalls of Systematic Reviews and Meta-Analyses in Imaging Research
McInnes, M. D. F., & Bossuyt, P. M. M. (2015) · Radiology · doi:10.1148/radiol.2015142779 · Cited by 99
Self-citation
Metadata
- Authors
- van der Pol, C. B., McInnes, M. D. F., Petrcich, W., Tunis, A. S., & Hanna, R.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-16
- Volume
- 10
- Issue
- 3
- Pages
- e0119892
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2018907126
- Cited by
- 56
- References
- 14
- FWCI
- 1.10
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.