Technical Note: MRQy — An open‐source tool for quality control of MR imaging data
Open accessAbstract
Purpose: There is an increasing availability of large imaging cohorts [such as through The Cancer Imaging Archive (TCIA)] for computational model development and imaging research. To ensure development of generalizable computerized models, there is a need to quickly determine relative quality differences in these cohorts, especially when considering MRI datasets which can exhibit wide variations in image appearance. The purpose of this study is to present a quantitative quality control tool, MRQy, to help interrogate MR imaging datasets for: (a) site‐ or scanner‐specific variations in image resolution or image contrast, and (b) imaging artifacts such as noise or inhomogeneity; which need correction prior to model development. Methods: Unlike existing imaging quality control tools, MRQy has been generalized to work with images from any body region to efficiently extract a series of quality measures (e.g., noise ratios, variation metrics) and MR image metadata (e.g., voxel resolution and image dimensions). MRQy also offers a specialized HTML5‐based front‐end designed for real‐time filtering and trend visualization of quality measures. Results: MRQy was used to evaluate (a) n = 133 brain MRIs from TCIA (7 sites) and (b) n = 104 rectal MRIs (3 local sites). MRQy measures revealed significant site‐specific variations in both cohorts, indicating potential batch effects. Before processing, MRQy measures could be used to identify each of the seven sites within the TCIA cohort with 87.5%, 86.4%, 90%, 93%, 90%, 60%, and 92.9% accuracy and the three sites within the rectal cohort with 91%, 82.8%, and 88.9% accuracy using unsupervised clustering. After processing, none of the sites could be distinctively clustered via MRQy measures in either cohort; suggesting that batch effects had been largely accounted for. Marked differences in specific MRQy measures were also able to identify outlier MRI datasets that needed to be corrected for common acquisition artifacts. Conclusions: MRQy is designed to be a standalone, unsupervised tool that can be efficiently run on a standard desktop computer. It has been made freely accessible and open‐source at http://github.com/ccipd/MRQy for community use and feedback.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101 · see citing works
Zwanenburg, A., Vallières, M., Abdalah, M. A., Aerts, H. J. W. L., Andrearczyk, V., Apte, A., Ashrafinia, S., Bakas, S., Beukinga, R. J., Boellaard, R., et al. (2020) · Radiology · doi:10.1148/radiol.2020191145 · Cited by 4,177 · see citing works
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J. S., Freymann, J. B., Farahani, K., & Davatzikos, C. (2017) · Scientific Data · doi:10.1038/sdata.2017.117 · Cited by 3,049 · see citing works
The Human Connectome Project: A data acquisition perspective
Essen, D. C. V., Uǧurbil, K., Auerbach, E. J., Barch, D. M., Behrens, T. E. J., Bucholz, R. D., Chang, A., Chen, L., Corbetta, M., Curtiss, S. W., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.018 · Cited by 2,605 · see citing works
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196 · see citing works
Monti, S., Tamayo, P., Mesirov, J., & Golub, T. (2003) · Machine Learning · doi:10.1023/a:1023949509487 · Cited by 2,116 · see citing works
MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites
Esteban, O., Birman, D., Schaer, M., Koyejo, O. O., Poldrack, R. A., & Gorgolewski, K. J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0184661 · Cited by 920 · see citing works
HistoQC: An Open-Source Quality Control Tool for Digital Pathology Slides
Janowczyk, A. R., Zuo, R., Gilmore, H. L., Feldman, M. D., & Madabhushi, A. (2019) · JCO Clinical Cancer Informatics · doi:10.1200/cci.18.00157 · Cited by 357 · see citing works
Self-citationuwot: The Uniform Manifold Approximation and Projection (UMAP) Method for Dimensionality Reduction
Melville, J. L. (2019) · doi:10.32614/cran.package.uwot · Cited by 212 · see citing works
Qoala-T: A supervised-learning tool for quality control of FreeSurfer segmented MRI data
Klapwijk, E. T., van de Kamp, F., van der Meulen, M., Peters, S., & Wierenga, L. M. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.01.014 · Cited by 192 · see citing works
Beig, N. G., Bera, K., Prasanna, P., Antunes, J. T., Correa, R., Singh, S., Bamashmos, A. S., Ismail, M., Braman, N. M., Verma, R., et al. (2020) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-19-2556 · Cited by 145 · see citing works
Self-citationThe parallel coordinate plot in action: design and use for geographic visualization
Edsall, R. M. (2003) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(02)00295-5 · Cited by 101 · see citing works
No-Reference Image Quality Metrics for Structural MRI
Woodard, J. P., & Carley-Spencer, M. P. (2006) · Neuroinformatics · doi:10.1385/ni:4:3:243 · Cited by 87 · see citing works
Crowdsourced MRI quality metrics and expert quality annotations for training of humans and machines
Estéban, O., Blair, R. W., Nielson, D. M., Varada, J. C., Marrett, S., Thomas, A. G., Poldrack, R. A., & Gorgolewski, K. J. (2019) · Scientific Data · doi:10.1038/s41597-019-0035-4 · Cited by 79 · see citing works
TCGA-TCIA Impact on Radiogenomics Cancer Research: A Systematic Review
Zanfardino, M., Pane, K., Mirabelli, P., Salvatore, M., & Franzese, M. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20236033 · Cited by 75 · see citing works
Antunes, J. T., Ofshteyn, A., Bera, K., Wang, E. Y., Brady, J. T., Willis, J. E., Friedman, K. A., Marderstein, E. L., Kalady, M. F., Stein, S. L., et al. (2020) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.27140 · Cited by 73 · see citing works
Self-citationMindcontrol: A web application for brain segmentation quality control
Keshavan, A., Datta, E., McDonough, I. M., Madan, C. R., Jordan, K., & Henry, R. G. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.055 · Cited by 63 · see citing works
Delis, H., Christaki, K. E., Healy, B., et al. (2017) · Physica Medica · doi:10.1016/j.ejmp.2017.04.007 · Cited by 43 · see citing works
Call for Data Standardization: Lessons Learned and Recommendations in an Imaging Study
Basu, A., Warzel, D., Eftekhari, A., Kirby, J., Freymann, J., Knable, J., Sharma, A., & Jacobs, P. M. (2019) · JCO Clinical Cancer Informatics · doi:10.1200/cci.19.00056 · Cited by 33 · see citing works
Fast algorithm for calculation of inhomogeneity gradient in magnetic resonance imaging data
Hui, C., Yu, Z., & Narayana, P. A. (2010) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.22344 · Cited by 27 · see citing works
LAB–QA2GO: A Free, Easy-to-Use Toolbox for the Quality Assessment of Magnetic Resonance Imaging Data
Vogelbacher, C., Bopp, M. H. A., Schuster, V., et al. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00688 · Cited by 16 · see citing works
Supervised machine learning quality control for magnetic resonance artifacts in neonatal data sets
Ding, Y., Suffren, S., Bellec, P., & Lodygensky, G. A. (2018) · Human Brain Mapping · doi:10.1002/hbm.24449 · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [5]Bushberg JT, Seibert JA, Leidholdt EM, and Boone JM, The essential physics of medical imaging, Lippincott Williams & Wilkins, 2011.
- [18]Maaten L. v. d. and Hinton G, Visualizing data using t-SNE, Journal of machine learning research 9, 2579– 2605 (2008).
- [20]Scarpace L, Mikkelsen T, Cha S, Rao S, Tekchandani S, Gutman D, Saltz J, Erickson B, Pedano N, Flanders A, Barnholtz-Sloan J, Ostrom Q, Barboriak D, and Pierce L, Radiology Data from The Cancer Genome Atlas Glioblastoma Multiforme [TCGA-GBM] collection, The Cancer Imaging Archive, 10.7937/K9/TCIA.2016.RNYFUYE9. · DOI
Cited by
Sorted by citations
Mitigating Bias in Radiology Machine Learning: 1. Data Handling
Rouzrokh, P., Khosravi, B., Faghani, S., Moassefi, M., Vera Garcia, D. V., Singh, Y., Zhang, K., Conte, G. M., & Erickson, B. J. (2022) · Radiology: Artificial Intelligence · doi:10.1148/ryai.210290 · Cited by 145 · see citing works
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58 · see citing works
Effects of MRI scanner manufacturers in classification tasks with deep learning models
Kushol, R., Parnianpour, P., Wilman, A. H., Kalra, S., & Yang, Y.-H. (2023) · Scientific Reports · doi:10.1038/s41598-023-43715-5 · Cited by 54 · see citing works
Prabhudesai, S., Wang, N. C., Ahluwalia, V., Huan, X., Bapuraj, J. R., Banovic, N., & Rao, A. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.740353 · Cited by 13 · see citing works
- Authors
- Sadri, A. R., Janowczyk, A., Zhou, R., Verma, R., Beig, N., Antunes, J., Madabhushi, A., Tiwari, P., & Viswanath, S. E.
- Affiliations
- Case Western Reserve University (US)
- Centre Hospitalier Universitaire Vaudois (CH)
- Louis Stokes Cleveland VA Medical Center (US)
- Corresponding author
- Viswanath, Satish E
- ORCID
- Viswanath, Satish E
- Venue
- Medical Physics
- Publisher
- Wiley
- Published
- 2020-11-27
- Volume
- 47
- Issue
- 12
- Pages
- 6029-6038
- ISSN
- 0094-2405 (ISSN-L)
- 2473-4209
- PMC
- PMC8176950
- OpenAlex
- W3015365640
- Cited by
- 63
- References
- 22
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 13 · 2023: 10 · 2022: 10 · 2021: 12 · 2020: 2
- FWCI
- 4.14
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Medical Physics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Veterans Affairs (IK6 BX006185, I01 BX004121)
- National Institutes of Health (1f31ca216935-01a1, UL1TR002548)
- Clinical and Translational Science Collaborative of Cleveland, School of Medicine, Case Western Reserve University (UL1TR0002548)
- National Heart, Lung, and Blood Institute (R01HL151277)
- National Cancer Institute (U01CA248226, F31 CA216935, U24CA199374, R01CA216579, U01CA239055, R01CA208236, R01CA202752, R01CA220581, 1F31CA216935-01A1)
- National Institute of Biomedical Imaging and Bioengineering (R43EB028736)
- National Center for Research Resources (C06RR012463, 1C06RR12463‐01)
- National Center for Advancing Translational Sciences
- DOD Prostate Cancer Research Program (W81XWH-20-1-0851, W81XWH-15-1-0558)
- DOD Peer Reviewed Cancer Research Program (W81XWH-16–1-0329, W81XWH-18-1-0404)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- MRI quality control, MRI artifacts, batch effects, site-specific variation, noise ratio, image resolution, image contrast, inhomogeneity correction, outlier detection, unsupervised clustering, The Cancer Imaging Archive, brain MRI, rectal MRI, open source software
- MeSH terms
- Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Quality Control, Cohort Studies, Artifacts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
