Pitfalls of Systematic Reviews and Meta-Analyses in Imaging Research
Open accessThe DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Systematic reviews of imaging research represent a tool to better understand test accuracy or the efficacy of interventions. Like any type of research, appropriate methods must be applied to optimize quality. The purpose of this review is to outline common pitfalls in performing systematic reviews of imaging research, with a focus on challenges particular to performing reviews of diagnostic accuracy studies. The following challenges are highlighted: posing relevant review questions, conducting comprehensive literature searches, assessing for bias in included studies, testing for heterogeneity and publication bias, pooling results across studies, and forming appropriate conclusions. By guiding authors on how to overcome these, the hope is that published reviews in imaging research will be of higher quality and have a positive impact on clinical practice. In addition, the review aims to educate readers of reviews so they become aware of crucial elements of systematic reviews that could bias review results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592
Self-citationReitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
Self-citationTowards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Self-citationTowards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Self-citation
Research works printed but not linked: 8
- Ebell MH, 2004, J Fam Pract, 53, 111
- Macaskill P, 2010, Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0. The Cochrane Collaboration
- Beynon R, 2013, Cochrane Database Syst Rev, 9
- Hopewell S, 2007, Cochrane Database Syst Rev
- Citrome L, 2014, Innov Clin Neurosci, 11, 42
- Bossuyt P, 2013, Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.9. The Cochrane Collaboration
- Taylor T, 2014, Cochrane Database Syst Rev, 3, CD009694
- Josephson CB, 2014, Cochrane Database Syst Rev, 9
Cited by
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., and the PRISMA-DTA Group, Clifford, T., Cohen, J. F., Deeks, J. J., Gatsonis, C., et al. (2018) · JAMA · doi:10.1001/jama.2017.19163 · Cited by 3,210
“Systematic reviews of diagnostic test accuracy synthesize data from primary studies to provide insight into the ability of medical tests to detect a target condition; they also can provide estimates of test performance, allow comparisons of the accuracy of different tests, and facilitate the identification of sources…”
InfluentialSelf-citationvan der Pol, C. B., Lim, C. S., Sirlin, C. B., McGrath, T. A., Salameh, J.-P., Bashir, M. R., Tang, A., Singal, A. G., Costa, A. F., Fowler, K., et al. (2019) · Gastroenterology · doi:10.1053/j.gastro.2018.11.020 · Cited by 334
Self-citationMaw, A. M., Hassanin, A., Ho, P. M., McInnes, M. D. F., Moss, A., Juarez-Colunga, E., Soni, N. J., Miglioranza, M. H., Platz, E., DeSanto, K., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0703 · Cited by 309
Self-citationMcGrath, T. A., Alabousi, M., Skidmore, B., Korevaar, D. A., Bossuyt, P. M. M., Moher, D., Thombs, B., & McInnes, M. D. F. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0590-8 · Cited by 142
“Thus, some items in the original PRISMA checklist may not apply to DTA reviews, and some essential items necessary for reporting DTA systematic reviews may be lacking [2, 6, 13, 14].”
BackgroundMethodInfluentialSelf-citationSuh, C. H., & Park, S. H. (2016) · Korean Journal of Radiology · doi:10.3348/kjr.2016.17.1.5 · Cited by 144
“Korean J Radiol 17(1), Jan/Feb 2016 kjronline.”
BackgroundMyelin quantification with MRI: A systematic review of accuracy and reproducibility
van der Weijden, C. W. J., García, D. V., Borra, R., Thurner, P., Meilof, J. F., van Laar, P.-J., Dierckx, R., Gutmann, I. W., & de Vries, E. F. J. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117561 · Cited by 137
Kwee, R. M., & Kwee, T. C. (2019) · European Journal of Radiology · doi:10.1016/j.ejrad.2019.04.023 · Cited by 126
Diagnostic accuracy of point-of-care ultrasound for shock: a systematic review and meta-analysis
Yoshida, T., Yoshida, T., Noma, H., Nomura, T., Suzuki, A., & Mihara, T. (2023) · Critical Care · doi:10.1186/s13054-023-04495-6 · Cited by 120
van der Pol, C. B., McInnes, M. D. F., Salameh, J., Levis, B., Chernyak, V., Sirlin, C. B., Bashir, M. R., Allen, B. C., Burke, L. M. B., Choi, J., et al. (2021) · Radiology · doi:10.1148/radiol.2021211244 · Cited by 116
Self-citationSystematic Review of the Literature: Best Practices
Gupta, S., Rajiah, P., Middlebrooks, E. H., Baruah, D., Carter, B. W., Burton, K. R., Chatterjee, A. R., & Miller, M. M. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.04.025 · Cited by 105
Woo, S., Atun, R. A., Ward, Z. J., Scott, A. M., Hricak, H., & Vargas, H. A. (2020) · European Radiology · doi:10.1007/s00330-020-06909-3 · Cited by 100
“We only meta-analytically pooled endpoints that had at least four available studies, as pooling small numbers of studies has been recognized as misleading [14].”
BackgroundFarahani, S. J., & Baloch, Z. W. (2018) · Diagnostic Cytopathology · doi:10.1002/dc.24097 · Cited by 99
“asymmetry in the funnel plot of the original data could be attributable to the high level of heterogeneity among the included studies.(23)”
BackgroundGandhi, N., Krishna, S., Booth, C. M., Breau, R. H., Flood, T. A., Morgan, S. C., Schieda, N., Salameh, J., McGrath, T. A., & McInnes, M. D. F. (2018) · British Journal of Urology · doi:10.1111/bju.14366 · Cited by 96
Self-citationWoo, S., Suh, C. H., Kim, S. Y., Cho, J. Y., & Kim, S. H. (2017) · European Radiology · doi:10.1007/s00330-017-4958-x · Cited by 95
“The degree of heterogeneity in meta-analyses is important, as it may affect the general applicability of the results [14, 15].”
BackgroundAlabousi, M., Zha, N., Salameh, J., Samoilov, L., Sharifabadi, A. D., Pozdnyakov, A., Sadeghirad, B., Freitas, V., McInnes, M. D. F., & Alabousi, A. (2020) · European Radiology · doi:10.1007/s00330-019-06549-2 · Cited by 86
Self-citationConnolly, M. J., McInnes, M. D. F., El-Khodary, M., McGrath, T. A., & Schieda, N. (2017) · European Radiology · doi:10.1007/s00330-017-4785-0 · Cited by 78
“The risk for bias across studies (publication bias) was not assessed since there is no generally accepted method for diagnostic accuracy studies and the number of included studies was low [21, 22].”
MethodSelf-citationChau, M. T., Todd, G., Wilcox, R., Agzarian, M., & Bezak, E. (2020) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2020.07.002 · Cited by 77
Perry, L. A., Lucarelli, T., Penny‐Dimri, J. C., McInnes, M. D. F., Mondello, S., Bustamante, A., Montaner, J., Foerch, C., Kwan, P., Davis, S. M., et al. (2018) · International Journal of Stroke · doi:10.1177/1747493018806167 · Cited by 70
Self-citationAlsbrook, D. L., Napoli, M. D., Bhatia, K., Desai, M. J., Hinduja, A., Rubiños, C., Mansueto, G., Singh, P., Domeniconi, G. G., Ikram, A., et al. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12031015 · Cited by 65
“The possible reasons for such inconsistent results are different sample sizes of the studies deriving variable statistical power, lack of true source of heterogeneity, presence of population stratification, unforeseen epistasis, and miscalculating gene-environmental interactions [31].”
BackgroundShaish, H. A., Kang, S. K., & Rosenkrantz, A. B. (2016) · Abdominal Radiology · doi:10.1007/s00261-016-0848-y · Cited by 65
“The ‘‘PICO’’ question to be addressed by the review was [25] (P) Patients with prostate cancer; (I) Intervention—quantitative ADC measurements obtained from diffusionweighted MRI; (C) Comparison—none; (O) Outcome/ Target Condition—presence of high- or low-risk tumor based on histopathologic findings from either prosta…”
BackgroundHaghshomar, M., Antonacci, D., Smith, A. D., Thaker, S., Miller, F. H., & Borhani, A. A. (2024) · Radiology · doi:10.1148/radiol.241171 · Cited by 67
McGrath, T. A., McInnes, M. D. F., Korevaar, D. A., & Bossuyt, P. M. M. (2016) · Radiology · doi:10.1148/radiol.2016152229 · Cited by 57
Self-citationSystematic Reviews and Meta-Analyses of Diagnostic Test Accuracy: The PRISMA-DTA Statement
Frank, R. A., Bossuyt, P. M., & McInnes, M. D. F. (2018) · Radiology · doi:10.1148/radiol.2018180850 · Cited by 58
Self-citationUdare, A., Agarwal, M., Alabousi, M., McInnes, M. D. F., Rubino, J. G., Marcaccio, M. J., & van der Pol, C. B. (2021) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.27606 · Cited by 57
Self-citationDiagnostic Performance of MRI for Esophageal Carcinoma: A Systematic Review and Meta-Analysis
Lee, S. L., Yadav, P., Starekova, J., Christensen, L., Chandereng, T., Chappell, R. J., Reeder, S. B., & Bassetti, M. F. (2021) · Radiology · doi:10.1148/radiol.2021202857 · Cited by 55
Heterogeneity in Systematic Reviews of Medical Imaging Diagnostic Test Accuracy Studies
White, S., Phua, Q. S., Lu, L., Yaxley, K. L., McInnes, M. D. F., & To, M. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.0649 · Cited by 56
Self-citationSingh, R., Wilson, M. P., Katlariwala, P., Murad, M. H., McInnes, M. D. F., & Low, G. (2020) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000001724 · Cited by 51
Self-citationSherbaf, F. G., Sair, H. I., Shakoor, D., Fritz, J. M., Schwaiger, B. J., Johnson, M. H., & Demehri, S. (2021) · Radiology · doi:10.1148/radiol.2021203624 · Cited by 50
Salameh, J., McInnes, M. D. F., McGrath, T. A., Salameh, G., & Schieda, N. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20527 · Cited by 50
Self-citationMcGrath, T. A., McInnes, M. D. F., Langer, F. W., Hong, J., Korevaar, D. A., & Bossuyt, P. M. (2017) · European Journal of Radiology · doi:10.1016/j.ejrad.2017.05.032 · Cited by 50
Self-citationPelekos, G., Acharya, A., Tonetti, M. S., & Bornstein, M. M. (2018) · Clinical Oral Implants Research · doi:10.1111/clr.13143 · Cited by 47
“Two aspects, “risk of bias” and “applicability to the research question,” were assessed by customizing the QUADAS- 2 tool (Whiting et al., 2011) suitable for systematic reviews that address diagnostic accuracy (McInnes & Bossuyt, 2015).”
MethodAbdalla, G., Dixon, L. V. M., Sanverdi, E., Machado, P., Kwong, J. S. W., Panovska‐Griffiths, J., Rojas‐García, A., Yoneoka, D., Veraart, J., Cauter, S. V., et al. (2020) · Neuroradiology · doi:10.1007/s00234-020-02425-9 · Cited by 42
A brief introduction of meta‐analyses in clinical practice and research
Wang, X., Zhang, X., Li, Z., Zhong, W., Yang, P., & Mao, C. (2021) · The Journal of Gene Medicine · doi:10.1002/jgm.3312 · Cited by 45
“accuracy (DTA) provides insight into the ability of medical tests to detect the target diseases(41); it also can provide estimates of test performance, allow comparisons of the accuracy of different tests and facilitate the identification of sources of variability.(42) For example, the FilmArray® (Biomerieux, Marcy-l'…”
BackgroundDawit, H., Lam, E., McInnes, M. D. F., van der Pol, C. B., Bashir, M. R., Salameh, J., Levis, B., Sirlin, C. B., Chernyak, V., Choi, S. H., et al. (2024) · Radiology · doi:10.1148/radiol.231501 · Cited by 39
Self-citationUltrasound for diagnosing radiographically occult scaphoid fracture
Kwee, R. M., & Kwee, T. C. (2018) · Skeletal Radiology · doi:10.1007/s00256-018-2931-7 · Cited by 39
Ultrasonography in diagnosing clinically occult groin hernia: systematic review and meta-analysis
Kwee, R. M., & Kwee, T. C. (2018) · European Radiology · doi:10.1007/s00330-018-5489-9 · Cited by 38
Best practices for MRI systematic reviews and meta‐analyses
McGrath, T. A., Bossuyt, P. M., Cronin, P., Salameh, J., Kraaijpoel, N., Schieda, N., & McInnes, M. D. F. (2018) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26198 · Cited by 38
“Similarly, if a high-quality systematic review has recently been published on the topic (or a protocol is registered and one is under way), duplicating these efforts may not be contributory.(12) The ideal scenario is one in which: 1) there is clinical equipoise; 2) there are a large number of well-designed, well-repor…”
BackgroundSelf-citationSalameh, J.-P., McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Frank, R., Dehmoobad Sharifabadi, A., Kraaijpoel, N., Levis, B., & Bossuyt, P. M. (2019) · Clinical Chemistry · doi:10.1373/clinchem.2018.292987 · Cited by 37
Self-citationGomes, Y. E., Chau, M. T., Banwell, H. A., & Causby, R. S. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05831-7 · Cited by 34
Pötsch, N., Sodano, C., & Baltzer, P. (2024) · Radiology · doi:10.1148/radiol.232508 · Cited by 33
Caruana, C. B., Jackson, S. M., Khuong, J. N., Campbell, R. J., Liu, Z., Ramson, D. M., Douglas, N. W. R., Kok, J., Perry, L. A., & Penny‐Dimri, J. C. (2020) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2020.03.039 · Cited by 31
Dinkelacker, V., Dupont, S., & Samson, S. (2016) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2016.08.028 · Cited by 29
“Note however, that among the available studies, several did not find correlations between structural connectivity and cognitive performance [79,83,84], and others were possibly not published because of lack of positive results [85].”
Walker, D. T., Davenport, M. S., McGrath, T. A., McInnes, M. D. F., Shankar, T., & Schieda, N. (2020) · Radiology · doi:10.1148/radiol.2020192855 · Cited by 31
Self-citationImpact and perceived value of journal reporting guidelines among Radiology authors and reviewers
Dewey, M., Levine, D., Bossuyt, P. M., & Kressel, H. Y. (2019) · European Radiology · doi:10.1007/s00330-018-5980-3 · Cited by 30
Self-citationKrishna, S., Schieda, N., Kulkarni, G. S., Shanbhogue, K., Baroni, R. H., & Woo, S. (2022) · American Journal of Roentgenology · doi:10.2214/ajr.21.27063 · Cited by 28
Goins, S. M., Jiang, H., van der Pol, C. B., Salameh, J., Lam, E., Adamo, R. G., McInnes, M. D. F., Costa, A. F., Tang, A., Alhasan, A. S., et al. (2023) · Radiology · doi:10.1148/radiol.231656 · Cited by 26
Self-citationRapid-sequence MRI for evaluation of pediatric traumatic brain injury: a systematic review
Kessler, B. A., Goh, J. L., Pajer, H. B., Asher, A. M., Northam, W. T., Hung, S., Selden, N. R. W., & Quinsey, C. S. (2021) · Journal of Neurosurgery Pediatrics · doi:10.3171/2021.2.peds20852 · Cited by 25
Bellini, D., Panvini, N., Laghi, A., Marin, D., Patel, B. N., Wang, C. L., Carbone, I., & Mileto, A. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20625 · Cited by 25
Gupta, P. C., Bansal, A., Das, G. C., Kumar, P. M., Chaluvashetty, S. B., Bhujade, H., Gulati, A., & Kalra, N. (2021) · European Radiology · doi:10.1007/s00330-021-07837-6 · Cited by 24
Macroscopic Fat in Adrenocortical Carcinoma: A Systematic Review
Ranathunga, D. S., Cherpak, L. A., Schieda, N., Flood, T. A., & McInnes, M. D. F. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.19.21851 · Cited by 24
Self-citationMammographic compression practices of force‐ and pressure‐standardisation protocol: A scoping review
Serwan, E., Matthews, D., Davies, J., & Chau, M. T. (2020) · Journal of Medical Radiation Sciences · doi:10.1002/jmrs.400 · Cited by 23
“Although potentially indicative of publication bias, this could not be assessed as no accepted method exists for its evaluation of diagnostic test accuracy studies.(27) Furthermore, the selection of sources may display potential subjectivity as only two reviewers were involved in the process; reproducible criteria wer…”
MethodAdamo, R. G., van der Pol, C. B., Alabousi, M., Lam, E., Salameh, J., Abedrabbo, N., Lerner, E., Naringrekar, H., Bashir, M. R., Costa, A. F., et al. (2025) · Radiology · doi:10.1148/radiol.243450 · Cited by 25
Self-citationDiagnostic Accuracy of Imaging Findings in Pleural Empyema: Systematic Review and Meta-Analysis
Zettinig, D., D’Antonoli, T. A., Wilder-Smith, A. J., Bremerich, J., Roth, J. A., & Sexauer, R. (2021) · Journal of Imaging · doi:10.3390/jimaging8010003 · Cited by 21
“The publication bias could only be assessed to a limited extent, as there is no generally accepted method for diagnostic accuracy studies and the number of studies included was low [22].”
MethodSalinas-Miranda, E., Birosh, A., McInnes, M. D. F., Breau, R. H., Lam, E., McGrath, T. A., Flood, T. A., & Schieda, N. (2025) · American Journal of Roentgenology · doi:10.2214/ajr.24.32328 · Cited by 21
Self-citationKim, T., Woo, S., Han, S., Suh, C. H., Lee, D. H., & Lee, J. M. (2019) · European Radiology · doi:10.1007/s00330-019-06499-9 · Cited by 19
Alabousi, M., Alabousi, A., McGrath, T. A., Cobey, K. D., Budhram, B., Frank, R. A., Nguyen, F., Salameh, J., Sharifabadi, A. D., & McInnes, M. D. F. (2018) · European Radiology · doi:10.1007/s00330-018-5567-z · Cited by 22
Self-citationAlabousi, M., Zha, N., & Patlas, M. N. (2020) · American Journal of Roentgenology · doi:10.2214/ajr.20.24636 · Cited by 19
Kwee, R. M., & Kwee, T. C. (2022) · European Radiology · doi:10.1007/s00330-021-08457-w · Cited by 19
Reporting bias in imaging: higher accuracy is linked to faster publication
Sharifabadi, A. D., Korevaar, D. A., McGrath, T. A., van Es, N., Frank, R. A., Cherpak, L. A., Dang, W., Salameh, J., Nguyen, F., Stanley, C., et al. (2018) · European Radiology · doi:10.1007/s00330-018-5354-x · Cited by 21
Self-citationPan, J., Tao, Y., Chi, X., Yang, L., Zhao, Y., & Chen, F. (2022) · European Radiology · doi:10.1007/s00330-022-08665-y · Cited by 18
“Methodological guidance for this study followed best practices in diagnostic test accuracy systematic reviews [20].”
MethodMata‐Mbemba, D., Rohringer, T. J., Ibrahim, A. Y., Adams-Webber, T., Moineddin, R., Doria, A. S., & Vali, R. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225663 · Cited by 18
“To assess the methodology and risk of bias of included studies, Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) was not used because none of these studies used a reference imaging test (micro computed tomography) or bony biopsy to compare with [21,22].”
MethodKim, T., Woo, S., Halpenny, D. F., Kim, Y. J., Yoon, S. H., & Suh, C. H. (2020) · European Journal of Radiology · doi:10.1016/j.ejrad.2020.108978 · Cited by 17
“The degree of heterogeneity in meta-analyses is important because it affects the general applicability of pooled results [30].”
BackgroundZhang, L., Gerson, L. B., & Maluf‐Filho, F. (2018) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2018.03.001 · Cited by 17
CT features of toxic megacolon: A systematic review
Eghbali, E., Milani, A. A., Shirmohamadi, M., & Hosseinifard, H. (2020) · Radiography · doi:10.1016/j.radi.2020.10.019 · Cited by 16
Fasen, B. A. C. M., Heijboer, R. J. J., Hulsmans, F. J., & Kwee, R. M. (2020) · European Journal of Radiology · doi:10.1016/j.ejrad.2020.109458 · Cited by 14
Are Study and Journal Characteristics Reliable Indicators of “Truth” in Imaging Research?
Frank, R. A., McInnes, M. D. F., Levine, D., Kressel, H. Y., Jesurum, J. S., Petrcich, W., McGrath, T. A., & Bossuyt, P. M. (2017) · Radiology · doi:10.1148/radiol.2017170586 · Cited by 15
Self-citationBellini, D., Rivosecchi, F., Panvini, N., Rengo, M., Caruso, D., Carbone, I., Ferrari, R., Paolantonio, P., & Laghi, A. (2019) · World Journal of Gastroenterology · doi:10.3748/wjg.v25.i31.4555 · Cited by 14
Yaxley, K. L., & To, M. (2020) · Insights into Imaging · doi:10.1186/s13244-020-00936-w · Cited by 13
Tang, R., Pu, J., & Huang, Z. (2025) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-025-07312-0 · Cited by 12
“Furthermore, publication bias was not formally analyzed, as no standard method for evaluating publication bias in meta-analyses of diagnostic test accuracy has been established [24, 25].”
Abedrabbo, N., Lam, E., McInnes, M. D. F., Dawit, H., Kadi, D., van der Pol, C. B., Salameh, J., Levis, B., Naringrekar, H., Lerner, E., et al. (2025) · Radiology · doi:10.1148/radiol.242278 · Cited by 14
Self-citationGoins, S. M., Jiang, H., van der Pol, C. B., Salameh, J., Lam, E., Adamo, R. G., McInnes, M. D. F., Costa, A. F., Clarke, C. G. D., Choi, S. H., et al. (2023) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.29167 · Cited by 12
Self-citationAlabousi, M., Gauthier, I. D., Li, N., dos Santos, G. M., Golev, D., Patlas, M. N., & Alabousi, A. (2019) · Emergency Radiology · doi:10.1007/s10140-019-01696-x · Cited by 12
Kabir, R., Parsa, A. D., Syed, H. Z., Bai, A. C. M., Hussain, R., Khan, M. F., Parvin, S., Vinnakota, D., Sathian, B., Sivasubramanian, M., et al. (2024) · Asian Journal of Public Health and Nursing. · doi:10.62377/j544ed47 · Cited by 14
“Hence, a PRISMA extension for ScRs was vital to offer the recording of search strategy, study protocol and guidance (McInnes & Bossuyt, 2015).”
MethodFrank, R. A., Salameh, J., Islam, N., Yang, B., Murad, M. H., Mustafa, R. A., Leeflang, M., Bossuyt, P. M., Takwoingi, Y., Whiting, P. F., et al. (2023) · Radiology · doi:10.1148/radiol.221437 · Cited by 12
Self-citationElmi, N., McEvoy, D., McInnes, M. D. F., Alabousi, M., Hecht, E. M., Luk, L., Asghar, S., Jajodia, A., de Carvalho, T. L., Warnica, W., et al. (2023) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.29168 · Cited by 10
“20,21 Guidance and reporting criteria referenced the Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement (Appendix S1 in the Supplemental Material).”
Self-citationHong, J. U., Kim, J. H., Lee, K. H., Lee, M., Hyun, I. Y., Cho, S. G., Kim, Y. J., Lee, H. Y., & Kim, G. R. (2019) · Medicine · doi:10.1097/md.0000000000015785 · Cited by 10
“However, authors who want to restrict language should clearly cite the excluded studies.([2,38])”
BackgroundMethodInfluentialAdamo, R. G., Lam, E., Salameh, J., van der Pol, C. B., Goins, S. M., Dawit, H., Costa, A. F., Levis, B., Singal, A. G., Chernyak, V., et al. (2024) · Canadian Association of Radiologists Journal · doi:10.1177/08465371241306297 · Cited by 8
“30,31 Reporting was conducted via the Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data (PRISMA-IPD) and PRISMA for Diagnostic Test Accuracy (DTA) statements.”
Self-citationPitfalls of Systematic Reviews and Meta-Analyses
Leo, G. D., & Sardanelli, F. (2016) · Radiology · doi:10.1148/radiol.2016152558 · Cited by 7
CONNOLLY, M. J., Rasuli, P., & McInnes, M. D. F. (2016) · Journal of Interventional Cardiology · doi:10.1111/joic.12288 · Cited by 6
Self-citationLi, Q., Hou, W., Li, L., Xu, J., Ren, Y., Zou, K., Tian, R., & Sun, X. (2022) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26043 · Cited by 4
“A study was defined as a systematic review if it had all of the following five key characteristics according to the Cochrane Handbook10: (1) a clearly stated set of objectives with predefined eligibility criteria for studies; (2) explicit, reproducible methodology; (3) a systematic search that attempted to identify al…”
MethodReporting Guidelines for Imaging Research
McInnes, M. D. F., Lim, C. S., van der Pol, C. B., Salameh, J., McGrath, T. A., & Frank, R. A. (2018) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2018.11.008 · Cited by 7
Self-citationBellini, D., Panvini, N., Carbone, I., Rengo, M., Wang, C. L., & Mileto, A. (2020) · Journal of Computer Assisted Tomography · doi:10.1097/rct.0000000000001105 · Cited by 4
“The risk for bias across studies (ie, “publication bias”) was not assessed because there is no generally accepted method for this task and the number of included studies was low.(10) Summary Measures and Statistical Methods The primary end point in this systematic review and metaanalysis was to evaluate sensitivity, s…”
MethodOlesen, A. S. O., Miger, K. C., Thune, J. J., Nielsen, O. W., Jakobsen, J. C., & Grand, J. (2025) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.70042 · Cited by 2
“For diagnostic accuracy, a hierarchical summary receiver operating characteristic (HSROC) model will be used to synthesize the data [51].”
Performance of CEUS LI-RADS v2017 major feature combinations: individual patient data meta-analysis
Awad, R., Naringrekar, H., Adamo, R. G., Lam, E., Alabousi, M., Al-Ghita, M. K., Wilson, S. R., Salameh, J., Bashir, M. R., Costa, A. F., et al. (2026) · Abdominal Radiology · doi:10.1007/s00261-026-05417-0 · Cited by 2
“Methodologic guidance was per best practices in diagnostic test accuracy systematic reviews [27 , 28 ].”
Self-citationTerayama, T., Taniguchi, T., Imai, R., Anan, K., Yoshida, T., Andō, K., Okamori, S., & Okada, Y. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01869-1 · Cited by 2
“Assessment of reporting bias We will not assess publication or reporting bias as there is no accepted method that can be used for its evaluation in a meta-analysis of diagnostic test accuracy studies [33].”
MethodEvidence-Based Medicine in the Domain of Nuclear Medicine, the Fifty-Year View
Zuckier, L. S. (2019) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2019.07.007 · Cited by 2
Li, Q., Li, L., Wang, R., Zou, K., Tian, R., & Sun, X. (2021) · Clinical and Translational Imaging · doi:10.1007/s40336-021-00433-0 · Cited by 2
Prognostic Value of CMR-Verified Myocardial Scarring in Cardiac Sarcoidosis
Roos, A., van den Berg‐Huysmans, A. A., & Schoones, J. W. (2017) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2016.07.018 · Cited by 2
Lam, E., Boutron, I., Adamo, R. G., van der Pol, C. B., Alabousi, M., Salameh, J. P., Naringrekar, H., Bashir, M. R., Costa, A. F., Osman, H., et al. (2026) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.70592 · Cited by 1
Self-citationSystematic Review Classification
McInnes, M. D. F. (2017) · American Journal of Roentgenology · doi:10.2214/ajr.16.17644 · Cited by 1
Self-citationEvidence-Based Imaging of the Acute Abdomen: Where Is the Evidence?
Kielar, A. Z., Walsh, C. B., & McInnes, M. D. F. (2018) · Springer eBooks · doi:10.1007/978-3-319-70778-5_1 · Cited by 1
Self-citationYe, K., Kuang, Y., Liang, J., Xu, X., Shi, C., Cheng, Q., Jiang, Z., Xiang, Z., & Luo, L. (2021) · Journal of Biosciences and Medicines · doi:10.4236/jbm.2021.94005 · Cited by 2
What to Consider in a Systematic Literature Review: Three Examples from Design Science Research
Ermel, A. P. C., Lacerda, D. P., Morandi, M. I. W. M., & Gauss, L. (2021) · Springer eBooks · doi:10.1007/978-3-030-75722-9_8 · Cited by 3
Medical Writing: Systematic Reviews and Meta-analyses
Tiziani, S., & Pape, H. (2017) · Springer eBooks · doi:10.1007/978-3-319-69350-7_5
Beyond the AJR: Beware of Publication Bias
Kwee, R. M., & Kwee, T. C. (2022) · American Journal of Roentgenology · doi:10.2214/ajr.22.28675
Wang, L., Gao, X., & Huang, Z. (2024) · Journal of Radiation Research and Applied Sciences · doi:10.1016/j.jrras.2024.101046
Wang, X., Xu, Y., & Jing, J. (2025) · Current Medicinal Chemistry · doi:10.2174/0109298673374240250812022451
Gomes, Y. E., Chau, M., Banwell, H. A., & Causby, R. S. (2022) · Figshare · doi:10.6084/m9.figshare.21200746
Terayama, T., Taniguchi, T., Imai, R., Anan, K., Yoshida, T., Ando, K., Okamori, S., & Okada, Y. (2021) · Figshare · doi:10.6084/m9.figshare.17210244
Metadata
- Authors
- McInnes, M. D. F., & Bossuyt, P. M. M.
- Affiliations
- University of Ottawa (CA)
- Venue
- Radiology
- Published
- 2015-10
- Volume
- 277
- Issue
- 1
- Pages
- 13-21
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2114832664
- Semantic Scholar
- 3a7a8f4cad83a1bb9a35d26da9ac6cb134fbbae8
- Cited by
- 99
- References
- 53
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 6 · 2023: 7 · 2022: 7 · 2021: 13 · 2020: 15 · 2019: 12 · 2018: 18 · 2017: 8 · 2016: 6
- FWCI
- 1.76
- Type
- journal article
- Language
- en
- Open access
- green
- pure.amsterdamumc.nl
- Licence
- other-oa
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- systematic reviews, meta-analysis, imaging research, diagnostic accuracy, review methodology, bias assessment, publication bias, heterogeneity, literature searching, review questions
- MeSH terms
- Diagnostic Imaging, Humans, Review Literature as Topic, Meta-Analysis as Topic, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.