Computer-aided Diagnosis: How to Move from the Laboratory to the Clinic
Open accessAbstract
Computer-aided diagnosis (CAD), encompassing computer-aided detection and quantification, is an established and rapidly growing field of research. In daily practice, however, most radiologists do not yet use CAD routinely. This article discusses how to move CAD from the laboratory to the clinic. The authors review the principles of CAD for lesion detection and for quantification and illustrate the state-of-the-art with various examples. The requirements that radiologists have for CAD are discussed: sufficient performance, no increase in reading time, seamless workflow integration, regulatory approval, and cost efficiency. Performance is still the major bottleneck for many CAD systems. Novel ways of using CAD, extending the traditional paradigm of displaying markers for a second look, may be the key to using the technology effectively. The most promising strategy to improve CAD is the creation of publicly available databases for training and validation. This can identify the most fruitful new research directions, and provide a platform to combine multiple approaches for a single task to create superior algorithms.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Literature has mainly blamed the failure of early CAD systems on low technical performance, while other potential barriers such as organizational or social aspects have been largely ignored.
“Literature has mainly blamed this on low technical performance of these early applications [5, 14–16], while other potential barriers to successful implementation, such as organizational or social aspects, have been largely ignored [17].”
From Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors (2020) · cites this paper as [14]
Cites
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290 · see citing works
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,312 · see citing works
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9_3 · Cited by 5,053 · see citing works
Fundamentals of digital image processing
(1989) · Computer Vision Graphics and Image Processing · doi:10.1016/0734-189x(89)90041-8 · Cited by 4,703 · see citing works
The Singularity Is near: When Humans Transcend Biology
Mead, W. R., & Kurzweil, R. (2006) · Foreign Affairs · doi:10.2307/20031996 · Cited by 3,448 · see citing works
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1645 · Cited by 2,928 · see citing works
Radiographic Atlas of Skeletal Development of the Hand and Wrist
&NA; (1960) · Southern Medical Journal · doi:10.1097/00007611-196011000-00033 · Cited by 2,031 · see citing works
Computer-aided diagnosis in medical imaging: historical review, current status and future potential
Doi, K. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.02.002 · Cited by 1,855 · see citing works
Image processing, analysis and machine vision
Dijkstra, T. (1994) · Neurocomputing · doi:10.1016/0925-2312(94)90073-6 · Cited by 1,818 · see citing works
Comparison and evaluation of methods for liver segmentation from CT datasets
Heimann, T., van Ginneken, B., Styner, M. A., Arzhaeva, Y., Aurich, V., Bauer, C., Beck, A., Becker, C., Beichel, R., Bekes, G., et al. (2009) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2013851 · Cited by 1,135 · see citing works
Self-citationPattern Recognition and Machine Learning
Burr, T. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s236 · Cited by 1,112 · see citing works
Müller, N. L., Staples, C. A., Miller, R. R., et al. (1988) · CHEST Journal · doi:10.1378/chest.94.4.782 · Cited by 803 · see citing works
Lessons from the Netflix prize challenge
Bell, R. M., & Koren, Y. (2007) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1345448.1345465 · Cited by 727 · see citing works
Bennett, K. P., & Campbell, C. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/380995.380999 · Cited by 689 · see citing works
Computer analysis of computed tomography scans of the lung: a survey
Sluimer, I. C., Schilham, A. M. R., Prokop, M., & van Ginneken, B. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2005.862753 · Cited by 559 · see citing works
Self-citationvan Ginneken, B., Stegmann, M. B., & Loog, M. (2005) · Medical Image Analysis · doi:10.1016/j.media.2005.02.002 · Cited by 540 · see citing works
Computer-aided diagnosis in chest radiography: a survey
van Ginneken, B., Romeny, B. M. T. H., & Viergever, M. A. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.974918 · Cited by 517 · see citing works
Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights
Bell, R. M., & Koren, Y. (2007) · Proceedings · doi:10.1109/icdm.2007.90 · Cited by 505 · see citing works
The BoneXpert Method for Automated Determination of Skeletal Maturity
Thodberg, H. H., Kreiborg, S., Juul, A., & Pedersen, K. D. (2009) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2008.926067 · Cited by 438 · see citing works
Lung Image Database Consortium: Developing a Resource for the Medical Imaging Research Community
Armato, S. G., McLennan, G., McNitt‐Gray, M. F., Meyer, C. R., Yankelevitz, D. F., Aberle, D. R., Henschke, C. I., Hoffman, E. A., Kazerooni, E. A., MacMahon, H., et al. (2004) · Radiology · doi:10.1148/radiol.2323032035 · Cited by 381 · see citing works
Schaap, M., Metz, C., van Walsum, T., van der Giessen, A. G., Weustink, A. C., Mollet, N. R., Bauer, C., Bogunović, H., Castro‐González, C., Deng, X. B., et al. (2009) · Medical Image Analysis · doi:10.1016/j.media.2009.06.003 · Cited by 364 · see citing works
Overview of Advanced Computer Vision Systems for Skin Lesions Characterization
Maglogiannis, I. G., & Doukas, C. N. (2009) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2017529 · Cited by 342 · see citing works
Giger, M. L., Chan, H., & Boone, J. (2008) · Medical Physics · doi:10.1118/1.3013555 · Cited by 311 · see citing works
Detrano, R., Anderson, M. L., Nelson, J. C., Wong, N. D., Carr, J. J., McNitt‐Gray, M. F., & Bild, D. E. (2005) · Radiology · doi:10.1148/radiol.2362040513 · Cited by 298 · see citing works
Volumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibility
Wormanns, D., Kohl, G., Klotz, E., Marheine, A., Beyer, F., Heindel, W. L., & Diederich, S. (2004) · European Radiology · doi:10.1007/s00330-003-2132-0 · Cited by 285 · see citing works
Abràmoff, M. D., Niemeijer, M., Suttorp-Schulten, M. S. A., Viergever, M. A., Russell, S. R., & van Ginneken, B. (2008) · Diabetes Care · doi:10.2337/dc07-1312 · Cited by 277 · see citing works
MDCT-based 3-D texture classification of emphysema and early smoking related lung pathologies
Xu, Y., Sonka, M., McLennan, G., et al. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2006.870889 · Cited by 235 · see citing works
3D Segmentation in the Clinic: A Grand Challenge
van Ginneken, B., Heimann, T., & Styner, M. A. (2007) · Cited by 212 · see citing works
Data explosion: the challenge of multidetector-row CT
Rubin, G. D. (2000) · European Journal of Radiology · doi:10.1016/s0720-048x(00)00270-9 · Cited by 208 · see citing works
Emphysema: Effect of Reconstruction Algorithm on CT Imaging Measures
Boedeker, K., McNitt‐Gray, M. F., Rogers, S. R., et al. (2004) · Radiology · doi:10.1148/radiol.2321030383 · Cited by 191 · see citing works
Guest editorial computer-aided diagnosis in medical imaging
Giger, M. L., Karssemeijer, N., & Armato, S. G. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2001.974915 · Cited by 187 · see citing works
Current status and future directions of computer-aided diagnosis in mammography
Nishikawa, R. M. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.02.009 · Cited by 176 · see citing works
Dean, J. C., & Ilvento, C. C. (2006) · American Journal of Roentgenology · doi:10.2214/ajr.05.0111 · Cited by 163 · see citing works
de Hoop, B., Gietema, H. A., van Ginneken, B., Zanen, P., Groenewegen, G., & Prokop, M. (2008) · European Radiology · doi:10.1007/s00330-008-1229-x · Cited by 161 · see citing works
Self-citationLiver Surgery Planning Using Virtual Reality
Reitinger, B., Bornik, A., Beichel, R., & Schmalstieg, D. (2006) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2006.131 · Cited by 150 · see citing works
The Coding of Roentgen Images for Computer Analysis as Applied to Lung Cancer
Lodwick, G. S., Keats, T. E., & Dorst, J. P. (1963) · Radiology · doi:10.1148/81.2.185 · Cited by 146 · see citing works
Jiang, Y., Nishikawa, R. M., Schmidt, R. A., Toledano, A. Y., & Doi, K. (2001) · Radiology · doi:10.1148/radiol.220001257 · Cited by 137 · see citing works
Karssemeijer, N., Bluekens, A. M. J., Beijerinck, D., et al. (2009) · Radiology · doi:10.1148/radiol.2532090225 · Cited by 136 · see citing works
Halligan, S., Altman, D. G., Mallett, S. V., et al. (2006) · Gastroenterology · doi:10.1053/j.gastro.2006.09.051 · Cited by 131 · see citing works
Computer-aided Detection versus Independent Double Reading of Masses on Mammograms
Karssemeijer, N., Otten, J. D. M., Verbeek, A. L. M., Groenewoud, J. H., de Koning, H. J., Hendriks, J. H. C. L., & Holland, R. (2003) · Radiology · doi:10.1148/radiol.2271011962 · Cited by 127 · see citing works
Small Pulmonary Nodules: Effect of Two Computer-aided Detection Systems on Radiologist Performance
Das, M., Mühlenbruch, G., Mahnken, A. H., et al. (2006) · Radiology · doi:10.1148/radiol.2412051139 · Cited by 115 · see citing works
Išgum, I., Rutten, A., Prokop, M., & van Ginneken, B. (2007) · Medical Physics · doi:10.1118/1.2710548 · Cited by 106 · see citing works
Self-citationHorsch, K., Giger, M. L., Vyborny, C. J., et al. (2006) · Radiology · doi:10.1148/radiol.2401050208 · Cited by 101 · see citing works
Automatic determination of Greulich and Pyle bone age in healthy Dutch children
van Rijn, R. R., Lequin, M. H., & Thodberg, H. H. (2009) · Pediatric Radiology · doi:10.1007/s00247-008-1090-8 · Cited by 96 · see citing works
Computer-aided diagnosis in chest radiography: Beyond nodules
van Ginneken, B., Hogeweg, L., & Prokop, M. (2009) · European Journal of Radiology · doi:10.1016/j.ejrad.2009.05.061 · Cited by 92 · see citing works
Self-citationComputer-Aided Diagnosis of Lung Cancer and Pulmonary Embolism in Computed Tomography—A Review
Chan, H., Hadjiiski, L. M., Zhou, C., et al. (2008) · Academic Radiology · doi:10.1016/j.acra.2008.01.014 · Cited by 84 · see citing works
Martin, D. D., Deusch, D., Schweizer, R., et al. (2009) · Pediatric Radiology · doi:10.1007/s00247-008-1114-4 · Cited by 82 · see citing works
Local noise weighted filtering for emphysema scoring of low-dose CT images
Schilham, A. M. R., van Ginneken, B., Gietema, H. A., et al. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2006.871545 · Cited by 79 · see citing works
Road Maps for Advancement of Radiologic Computer-aided Detection in the 21st Century
Summers, R. M. (2003) · Radiology · doi:10.1148/radiol.2291030010 · Cited by 74 · see citing works
Benefits of the DICOM Structured Report
Noumeir, R. (2006) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-006-0631-7 · Cited by 74 · see citing works
Computer-aided Diagnosis in Radiology
Lodwick, G. S. (1966) · Investigative Radiology · doi:10.1097/00004424-196601000-00032 · Cited by 70 · see citing works
Gietema, H. A., Schilham, A. M. R., van Ginneken, B., van Klaveren, R. J., Lammers, J. J., & Prokop, M. (2007) · Radiology · doi:10.1148/radiol.2443061330 · Cited by 68 · see citing works
Self-citationSluimer, I. C., Prokop, M., Hartmann, I. J. C., et al. (2006) · Medical Physics · doi:10.1118/1.2207131 · Cited by 60 · see citing works
Self-citationNoble, M., Bruening, W., Uhl, S., & Schoelles, K. M. (2008) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-008-0841-y · Cited by 52 · see citing works
Computer‐aided detection, in its present form, is not an effective aid for screening mammography
Nishikawa, R. M., Kallergi, M., & Orton, C. G. (2006) · Medical Physics · doi:10.1118/1.2168063 · Cited by 49 · see citing works
Tanner, J. M., Oshman, D., Lindgren, G., Grunbaum, J. A., Elouki, R., & Labarthe, D. R. (2008) · Hormone Research · doi:10.1159/000184211 · Cited by 49 · see citing works
AI's greatest trends and controversies
Simon, H. A., Bibel, W., Bundy, A., Berliner, H., Feigenbaum, E. A., Buchanan, B. G., Selfridge, O. G., Michie, D., Nilsson, N. J., Sloman, A., et al. (2000) · IEEE Intelligent Systems and their Applications · doi:10.1109/5254.820322 · Cited by 47 · see citing works
The manufacturer's perspective
Roehrig, J. (2005) · British Journal of Radiology · doi:10.1259/bjr/25058162 · Cited by 45 · see citing works
Radiological interpretation 2020: Toward quantitative image assessment
Boone, J. M. (2007) · Medical Physics · doi:10.1118/1.2789501 · Cited by 41 · see citing works
Computer-aided detection (CAD) of lung nodules and small tumours on chest radiographs
Boo, D. W. D., Prokop, M., Uffmann, M., et al. (2009) · European Journal of Radiology · doi:10.1016/j.ejrad.2009.05.062 · Cited by 41 · see citing works
Self-citationBoyer, B., Balleyguier, C., Granat, O., et al. (2008) · European Journal of Radiology · doi:10.1016/j.ejrad.2008.07.042 · Cited by 32 · see citing works
Quantifizierung des Lungenemphysems in der Mehrschicht-CT mittels verschiedener Softwareverfahren
Heußel, C. P., Achenbach, T., Buschsieweke, C., et al. (2006) · RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren · doi:10.1055/s-2006-926823 · Cited by 31 · see citing works
Zhou, Z., Liu, B. J., & Lê, A. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.02.015 · Cited by 30 · see citing works
Potential of Computer-Aided Diagnosis to Improve CT Lung Cancer Screening
Lee, N., Laine, A. F., Mrquez, G., Levsky, J. M., & Gohagan, J. K. (2009) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2009.2034022 · Cited by 30 · see citing works
Computer aided detection of masses in mammograms as decision support
Karssemeijer, N., Otten, J. D. M., Rijken, H., et al. (2006) · British Journal of Radiology · doi:10.1259/bjr/37622515 · Cited by 27 · see citing works
Clinical Versus Research Approach to Breast Cancer Detection with CAD: Where Are We Now?
Brem, R. F. (2006) · American Journal of Roentgenology · doi:10.2214/ajr.06.1449 · Cited by 24 · see citing works
Superlearning and neural network magic
Duin, R. P. W. (1994) · Pattern Recognition Letters · doi:10.1016/0167-8655(94)90052-3 · Cited by 24 · see citing works
The nature and subtlety of abnormal findings in chest radiographs
MacMahon, H., Montner, S. M., Doi, K., et al. (1991) · Medical Physics · doi:10.1118/1.596708 · Cited by 19 · see citing works
Robinson, C., Halligan, S., Taylor, S. A., et al. (2008) · Radiology · doi:10.1148/radiol.2461070121 · Cited by 18 · see citing works
Rumberger, J. A. (2008) · Seminars in Ultrasound CT and MRI · doi:10.1053/j.sult.2008.02.008 · Cited by 16 · see citing works
The Use of a Computerized Method of Bone Age Assessment in Clinical Practice
Albanese, A., Hall, C. M., & Stanhope, R. G. (2008) · Hormone Research · doi:10.1159/000184665 · Cited by 14 · see citing works
Gur, D. (2008) · Radiology · doi:10.1148/radiol.2471070822 · Cited by 7 · see citing works
Li, F., Engelmann, R. M., Doi, K., et al. (2009) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-009-9201-0 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803 · see citing works
Artificial intelligence in cancer imaging: Clinical challenges and applications
Bi, W. L., Hosny, A., Schabath, M. B., Giger, M. L., Birkbak, N. J., Mehrtash, A., Allison, T., Arnaout, O., Abbosh, C., Dunn, I. F., et al. (2019) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21552 · Cited by 1,846 · see citing works
Large scale deep learning for computer aided detection of mammographic lesions
Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C. I., Mann, R., den Heeten, A., & Karssemeijer, N. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.07.007 · Cited by 965 · see citing works
Self-citationCheng, J.-Z., Ni, D., Chou, Y.-H., Qin, J., Tiu, C.-M., Chang, Y.-C., Huang, C.-S., Shen, D., & Chen, C.-M. (2016) · Scientific Reports · doi:10.1038/srep24454 · Cited by 755 · see citing works
A deep learning framework for supporting the classification of breast lesions in ultrasound images
Han, S., Kang, H.-K., Jeong, J.-Y., Park, M.-H., Kim, W., Bang, W.-C., & Seong, Y.-K. (2017) · Physics in Medicine & Biology · doi:10.1088/1361-6560/aa82ec · Cited by 358 · see citing works
Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329 · see citing works
A systematic survey of computer-aided diagnosis in medicine: Past and present developments
Yanase, J., & Triantaphyllou, E. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112821 · Cited by 303 · see citing works
van Ginneken, B., Setio, A. A. A., Jacobs, C., & Ciompi, F. (2015) · 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2015.7163869 · Cited by 278 · see citing works
Fehr, J., Konigorski, S., Olivier, S., Gunda, R., Surujdeen, A., Gareta, D., Smit, T., Baisley, K., Moodley, S., Moosa, Y., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00471-y · Cited by 74 · see citing works
Chae, K. J., Jin, G. Y., Ko, S. B., Wang, Y., Zhang, H., Choi, E. J., & Choi, H. (2020) · Academic Radiology · doi:10.1016/j.acra.2019.05.018 · Cited by 40 · see citing works
- Authors
- van Ginneken, B., Schaefer-Prokop, C. M., & Prokop, M.
- Affiliations
- Radboud University Nijmegen (NL)
- ORCID
- Prokop, Mathias
- Venue
- Radiology
- Published
- 2011-12
- Volume
- 261
- Issue
- 3
- Pages
- 719-732
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2167896953
- Cited by
- 306
- References
- 73
- Citations by year
- 2026: 15 · 2025: 29 · 2024: 25 · 2023: 20 · 2022: 36 · 2021: 30 · 2020: 17 · 2019: 29 · 2018: 24 · 2017: 25 · 2016: 18 · 2015: 8 · 2014: 12 · 2013: 11 · 2012: 7
- FWCI
- 1.66
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Pure Amsterdam UMC (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- computer-aided diagnosis, computer-aided detection, lesion quantification, workflow integration, regulatory approval, cost efficiency
- MeSH terms
- Algorithms, Diagnosis, Computer-Assisted, Diagnostic Imaging, Humans, Radiology Information Systems, Software, Biomarkers, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
