Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
The use of artificial intelligence in medicine is currently an issue of great interest, especially with regard to the diagnostic or predictive analysis of medical images. Adoption of an artificial intelligence tool in clinical practice requires careful confirmation of its clinical utility. Herein, the authors explain key methodology points involved in a clinical evaluation of artificial intelligence technology for use in medicine, especially high-dimensional or overparameterized diagnostic or predictive models in which artificial deep neural networks are used, mainly from the standpoints of clinical epidemiology and biostatistics. First, statistical methods for assessing the discrimination and calibration performances of a diagnostic or predictive model are summarized. Next, the effects of disease manifestation spectrum and disease prevalence on the performance results are explained, followed by a discussion of the difference between evaluating the performance with use of internal and external datasets, the importance of using an adequate external dataset obtained from a well-defined clinical cohort to avoid overestimating the clinical performance as a result of overfitting in high-dimensional or overparameterized classification model and spectrum bias, and the essentials for achieving a more robust clinical evaluation. Finally, the authors review the role of clinical trials and observational outcome studies for ultimate clinical verification of diagnostic or predictive artificial intelligence tools through patient outcomes, beyond performance metrics, and how to design such studies. © RSNA, 2018
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This common scenario means that estimations of AI performance based on internal test data may overestimate its real-world performance on external data [73–75].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [74]
In the citing paper
“Similar to internal validation of the AIPM, external AIPM model validation can be based on discrimination (area under the receiver operating characteristic curve, sensitivity, specificity, positive and negative predictive values), calibration (calibration plot)5,6,13,17,49,56,78, and Decision Curve Analysis5,6,17,49,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [56]
In the citing paper
“To ensure the generalizability of the AIPM to the intended healthcare setting, developers are advised to extensively validate the model for representative data from that setting6–8,10,13–15,24,27,56,64,66,68,72,77,79,87,88 (see phase 1, Representativeness).”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [56]
In the citing paper
“The literature recommends to assess generalizability on external data from a different time period, place, or healthcare setting3,7,8,10,16,17,27,56,68,72,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [56]
In the citing paper
“Although prospective validation is rare, it is preferred by the literature5,14,56, as it provides a better idea of the AIPM’s true applicability to medical practice and allows the healthcare professionals to identify and review errors in real time19,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [56]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
(2016) · Cited by 11,679
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · Radiology · doi:10.1148/radiol.2015151516 · Cited by 1,337
Deep Learning in Medical Imaging: General Overview
Lee, J.-G., Jun, S., Cho, Y.-W., Lee, H., Kim, G. B., Seo, J. B., & Kim, N. (2017) · Korean Journal of Radiology · doi:10.3348/kjr.2017.18.4.570 · Cited by 1,276
Research works printed but not linked: 3
- Steyerberg EW, 2010, Clinical prediction models: a practical approach to development, validation, and updating, 83
- Steyerberg EW, 2010, Clinical prediction models: a practical approach to development, validation, and updating, 299
- Newman TB, 2013, Designing clinical research, 117
Cited by
Items 501–600 of 1,000
Identification of Intracranial Hemorrhage using ResNeXt Model
Navaneet Bhat, K., Biradar, V. G., Mallya, A. K. S., Sabat, S. S., Murali.G, M. S. R., & Pareek, P. K. (2022) · IEEE Mysore Sub Section International Conference (MysuruCon) · doi:10.1109/mysurucon55714.2022.9972396 · Cited by 11
“The paper [18] provides guidelines for performance evaluation wherein assessment can be done on separate datasets that were unused for training.”
BackgroundNomura, Y., Hanaoka, S., Nakao, T., Hayashi, N., Yoshikawa, T., Miki, S., Watadani, T., & Abe, O. (2021) · Japanese Journal of Radiology · doi:10.1007/s11604-021-01153-1 · Cited by 11
Dreizin, D., Khatri, G. D., Staziaki, P. V., Buch, K. A., Unberath, M., Mohammed, M., Sodickson, A. D., Khurana, B., Agrawal, A., Spann, J. S., et al. (2024) · Emergency Radiology · doi:10.1007/s10140-024-02306-1 · Cited by 10
“…imaging require every effort to develop temporal and external validation datasets free of spectrum bias (e.g., by using consecutive selection rather than convenience sampling), so that results from validation datasets reflect, as closely as possible, performance in the clinical setting [35, 36].”
InfluentialOverview of artificial intelligence in hand surgery
Ryhänen, J., Wong, G. C., Anttila, T., & Chung, K. C. (2025) · Journal of Hand Surgery (European Volume) · doi:10.1177/17531934251322723 · Cited by 10
“Strawman bias manifests when experts lack the expected level of expertise for accurate model training and evaluation (Abu-Mostafa et al., 2012; Park and Han, 2018; Zech et al., 2018).”
Developing and Implementing a Web-Based educational platform for Children with Special Needs
Hayek, F. A. A., Khelaif, M. M., Shaikh, Z. A. A., Alshammari, H. K., Alshabanah, M., Alrajhi, D., Alsmadi, M. K., & Almarashdeh, I. (2020) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst207163 · Cited by 10
Jeong, S. Y., Suh, C. H., Park, H. Y., Heo, H., Shim, W. H., & Kim, S. J. (2022) · Journal of the Korean Society of Radiology · doi:10.3348/jksr.2022.0048 · Cited by 10
Jager, A., Postema, A. W., van der Linden, H. C., Nooijen, P. T. G. A., Bekers, E. M., Kweldam, C. F., Daures, G., Zwart, W., Mischi, M., Beerlage, H. P., et al. (2023) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-023-03589-4 · Cited by 10
“The succes of any AI algorithm relies heavily on the quality of the ground truth data used in its development and training [7].”
BackgroundHwang, S. H., Shin, H. J., Kim, E., Lee, E. H., & Lee, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-47194-6 · Cited by 10
Deng, L., Shuai, P., Liu, Y.-R., Yong, T., Liu, Y.-P., Li, H., & Zheng, X.-X. (2024) · Osteoporosis International · doi:10.1007/s00198-024-07136-y · Cited by 9
“Typically, internal validation for an initial evaluation or fine-tuning of a model under development often overestimates the performance of the model [32].”
InfluentialMutasa, S., Chang, P., Nemer, J. S., Sant, E. P. V., Sun, M., McIlvride, A., Siddique, M., & Ha, R. S. (2020) · Clinical Breast Cancer · doi:10.1016/j.clbc.2020.06.001 · Cited by 10
A dynamic lesion model for differentiation of malignant and benign pathologies
Cao, W., Liang, Z., Gao, Y., Pomeroy, M. J., Han, F., Abbasi, A., & Pickhardt, P. J. (2021) · Scientific Reports · doi:10.1038/s41598-021-83095-2 · Cited by 10
Matos, I. G., Villacampa, G. J., Hierro, C., Martín-Liberal, J., Berché, R., Pedrola, A., Braña, I., Azaro, A., Vieito, M., Saavedra, O., et al. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.05.040 · Cited by 10
“To validate the tool, we assessed the discrimination and calibration performance of the model [15].”
Nakamura, K., Zhou, X. M., Sahara, N., Toyoda, Y., Enomoto, Y., Hara, H., Noro, M., Sugi, K., Huang, M. S., Moroi, M., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12122947 · Cited by 10
“Calibration was evaluated at 1 and 2 years during follow-up using a calibration plot [25] and a Hosmer–Lemeshow test simultaneously.”
MethodYanhong, G., Dong, Y., Wei, X., & Dong, Y. (2023) · Sustainability · doi:10.3390/su151410760 · Cited by 10
Artificial Intelligence for Colorectal Polyp Detection and Characterization
Mori, Y., Kudo, S., Misawa, M., Takeda, K., Kudo, T., Itoh, H., Oda, M., & Mori, K. (2020) · Current Treatment Options in Gastroenterology · doi:10.1007/s11938-020-00287-x · Cited by 10
Editor’s Note: Publication of AI Research in Radiology
Bluemke, D. A. (2018) · Radiology · doi:10.1148/radiol.2018184021 · Cited by 10
Park, S. H., & Hwang, E. J. (2024) · Korean Journal of Radiology · doi:10.3348/kjr.2024.0144 · Cited by 10
“Calibration performance refers to the degree of similarity of the AI-predicted probabilities to the actual probabilities [2].”
BackgroundMethodSelf-citationRichard, T., de Naide, V. D. V., Nogues, V., Génisson, T., Narceau, K., He, K., Klaar, R., Durand, B., Boullé, T., Poirot, G., et al. (2025) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.29777 · Cited by 9
“The accuracy of the automated endocardial segmentation was assessed using the Dice coefficient [17] and the Jaccard index [18], calculated both on a per-slice basis (apical, middle, basal) and globally.”
Seheult, J. N., Otteson, G. E., Weybright, M. J., Timm, M., Han, W., Jevremović, D., Horna, P., Olteanu, H., & Shi, M. (2025) · Cancers · doi:10.3390/cancers17101688 · Cited by 9
“As highlighted in recent publications addressing AI implementation in healthcare, without consistent and rigorous performance monitoring, the benefits of AI/ML could be overshadowed by inaccuracies, inefficiencies, and potential patient harm [13,14].”
Larson, D. B., Poff, J. A., Krishnan, S., Avondo, J., Armstrong, B. A., Na, H. S., Chaudhari, A. S., & Kottler, N. E. (2026) · American Journal of Roentgenology · doi:10.2214/ajr.25.34340 · Cited by 9
Holm, C. E., Grazal, C. F., Rædkjær, M., Baad‐Hansen, T., Nandra, R. S., Grimer, R. J., Forsberg, J. A., Petersen, M. M., & Sørensen, M. S. (2022) · SAGE Open Medicine · doi:10.1177/20503121221076387 · Cited by 9
“0001a Head and neck 20 (1) 50 (7) 70 (2) Lower extremity 2118 (61) 355 (47) 2473 (58) Pelvic girdle 642 (18) 117 (16) 759 (18) Spine 0 32 (4) 32 (1) Upper extremity 471 (14) 103 (14) 574 (14) Upper trunk 230 (7) 93 (12) 323 (8) Missing 12 21 33 Metastasis at diagnosis 0.”
Xi, C., Du, R., Wang, R., Wang, Y., Hou, L., Luan, M., Zheng, X., Huang, H., Liang, Z., Ding, X., et al. (2023) · View · doi:10.1002/viw.20220057 · Cited by 9
Hybrid morphological-convolutional neural networks for computer-aided diagnosis
Canales-Fiscal, M. R., & Tamez‐Peña, J. G. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1253183 · Cited by 9
“The limited availability of large datasets for training CNNs poses significant challenges, particularly in medical settings where high-quality images and annotations are essential for supervised training, validation, and testing of AI algorithms (Park and Han, 2018).”
BackgroundNicolò, V. D., & Torre, D. L. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13671-9.00004-1 · Cited by 9
External Validation of a Machine Learning Model for Schizophrenia Classification
He, Y., Sakuma, K., Kishi, T., Li, Y., Matsunaga, M., Tanihara, S., Iwata, N., & Ota, A. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13102970 · Cited by 9
“However, this approach carries the risk of inflating the model performance due to overfitting [10].”
BackgroundKim, P. J., Kim, D., Lee, J., Kim, H. C., Seo, J. H., Lee, S. Y., Kwon, D. H., Park, H., Yoo, J., & Park, S.-H. (2025) · Scientific Reports · doi:10.1038/s41598-025-90214-w · Cited by 9
“To evaluate the accuracy and reliability of the predictive model, we calculated the Brier Score and generated Calibration plots 22,23 .”
Self-citationA Survey of Deep Learning in Histopathological Nuclear Segmentation
Qin, L., Yang, X., Xu, X., & Zhu, Z. (2025) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2025.3564274 · Cited by 9
“As the medical training and practices of different pathologists differ, inconsistent diagnoses for the same case may occur [2], [3].”
Kim, B., Moon, J. Y., Shin, J., Jeon, H. R., Oh, S. S., & Kim, S. Y. (2022) · Trials · doi:10.1186/s13063-022-06425-3 · Cited by 9
White, R., Demirer, M., Gupta, V., Sebro, R., Kusumoto, F. M., & Erdal, B. S. (2022) · Journal of medical imaging · doi:10.1117/1.jmi.9.5.054504 · Cited by 9
“Tier-2 assessment of the discrimination performance of the multiclass AI model for LLIED-type identification used the area under the receiver operating characteristic curve (AUC ROC) methodology.(36,73)”
BackgroundMethodHuman-Curated Validation of Machine Learning Algorithms for Health Data
Boman, M. (2023) · Digital Society · doi:10.1007/s44206-023-00076-w · Cited by 9
“A recent example involves software that always trains on the validation set, leading to higher quantitative scores on the validation set than on the test set, a case of spectrum bias in which the target population is inadequately represented (Park & Han, 2018).”
MethodData Engineering for Machine Learning in Women's Imaging and Beyond
Chen, C., Chou, S.-H. S., Brattain, L. J., Lehman, C. D., & Samir, A. E. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20464 · Cited by 9
“Overfitting, which refers to a learning model overly customized to its training data [71], should also be avoided.”
BackgroundGrowth of thymic epithelial tumors and thymic cysts: Differential radiological points
Kim, H., Yoon, S. H., Kim, J., Lee, K. W., Choi, Y. R., Cho, H., & Goo, J. M. (2019) · Thoracic Cancer · doi:10.1111/1759-7714.13016 · Cited by 9
Three simple steps to improve the interpretability of EEG-SVM studies
Joucla, C., Gabriel, D., Ortega, J., & Haffen, É. (2022) · Journal of Neurophysiology · doi:10.1152/jn.00221.2022 · Cited by 9
Joseph, M., & Baseer, K. K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-2322-9_34 · Cited by 9
Morley, J. (2023) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-jxprn9ry5 · Cited by 9
Papageorgiou, P. S., Christodoulou, R. C., Pitsillos, R., Petrou, V., Vamvouras, G., Kormentza, E. V., Papagelopoulos, P. J., & Georgiou, M. F. (2025) · Applied Sciences · doi:10.3390/app15169005 · Cited by 8
Gao, Q., Jin, Y., Sun, Y., Jin, M., Tang, L., Chen, Y., She, Y., & Li, M. (2026) · Diagnostics · doi:10.3390/diagnostics16050752 · Cited by 8
“As emphasized by Park and Han, demonstrating in silico diagnostic accuracy is merely the preliminary step; the definitive measure of success is the technology’s tangible impact on workflow efficiency and downstream patient outcomes [94].”
Ahmad, I. A., Ba‐Hattab, R., Al‐Maweri, S. A., & Tamimi, F. (2026) · International Endodontic Journal · doi:10.1111/iej.70207 · Cited by 8
Chang, I.-Y., & Huang, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-22506-4 · Cited by 8
Yun, S., Park, J. E., Kim, N., Park, S. Y., & Kim, H. S. (2023) · European Radiology · doi:10.1007/s00330-023-10318-7 · Cited by 8
“A diagnostic cohort study within a pre-defined clinical setting will present diagnostic difficulties and provide evidence of the real-world performance of a DL [20, 21].”
Self-citationDash, R., Kohinoor, M. S. R., & Chowdhury, P. G. (2024) · International Conference on Intelligent Technologies (CONIT) · doi:10.1109/conit61985.2024.10626618 · Cited by 8
“The landscape of healthcare has undergone a paradigm shift as a result of the recent integration of state-of-the-art technologies, most notably machine and deep learning into the realm of medical diagnosis [2].”
BackgroundLi, T., Gan, T., Wang, J., Long, Y., Zhang, K., & Liao, M. (2024) · Clinical Imaging · doi:10.1016/j.clinimag.2024.110275 · Cited by 8
Bedia, S. V., Shapurwala, M. A., Kharge, B. P., Bedia, A. S., & Patil, A. (2025) · Cureus · doi:10.7759/cureus.78718 · Cited by 8
“…with staff, followed by targeted training to ensure seamless integration into daily operations [21] Quarterly performance assessments, focusing on system efficiency, user satisfaction, and feedback, should be conducted to identify areas for improvement and optimize software utilization [22].”
Abdelhamed, W., & El‐Kassas, M. (2025) · Hepatoma Research · doi:10.20517/2394-5079.2024.138 · Cited by 8
A new AI assisted medical molecular image diagnostic model
Premananthan, G., Nagaraj, B., & Jaya, J. (2023) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-223354 · Cited by 8
“Park et al [20] insisted on making use of AI as an essential methodology to carry out clinical evaluation in the field of medicine through utilization of high-dimensional diagnostic models in like artificial DNN to determine life threatening conditions that demands immediate attention and requires…”
Zheng, S., He, K., Zhang, L., Li, M., Zhang, H., & Gao, P. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110912 · Cited by 8
Framework for Classifying Explainable Artificial Intelligence (XAI) Algorithms in Clinical Medicine
Gniadek, T. J., Kang, J., Theparee, T., & Krive, J. (2023) · Online Journal of Public Health Informatics · doi:10.2196/50934 · Cited by 8
“are technically correct, match a reference method, or agree with expectations [21].”
BackgroundHaase, R., Lehnen, N. C., Schmeel, F. C., Deike‐Hofmann, K., Rüber, T., Radbruch, A., & Paech, D. (2024) · Scientific Reports · doi:10.1038/s41598-024-59590-7 · Cited by 8
Ting, D. S. W., Tan, T., & Lim, C. C. T. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/ciy969 · Cited by 8
Park, S. H., Choi, J., & Byeon, J. (2020) · Journal of Korean Medical Association · doi:10.5124/jkma.2020.63.11.696 · Cited by 8
Self-citationA Study and Novel AI/ML-Based Framework to Detect COVID-19 Virus Using Smartphone Embedded Sensors
Sharma, M., Sharma, B., Gupta, A., Khosla, D., Goyal, S., & Pandey, D. (2021) · Springer eBooks · doi:10.1007/978-981-16-3227-3_4 · Cited by 8
Role of Artificial Intelligence in Mental Wellbeing: Opportunities and Challenges
Hanna, B. S. H., & Helmy, A. S. (2021) · Journal of Artificial Intelligence · doi:10.3923/jai.2022.1.8 · Cited by 8
Genomics and radiomics: Tools to see the unseen to personalize radiation therapy
Grass, G. D., Mills, M. N., Scott, J. G., Eschrich, S. A., & Torres–Roca, J. F. (2019) · Applied Radiation Oncology · doi:10.37549/aro1213 · Cited by 8
de Margerie‐Mellon, C., VanderLaan, P. A., & Bankier, A. A. (2018) · Radiology · doi:10.1148/radiol.2018182173 · Cited by 8
Deep Learning for Lung Cancer Nodal Staging and Real-World Clinical Practice
Park, C. M., & Lee, J. H. (2021) · Radiology · doi:10.1148/radiol.2021211981 · Cited by 8
Qiqi, K., & Ban, Y. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1745519 · Cited by 7
““ from my data to your data ” Many models fail to generalize due to “ dataset shift ” (115, 116).”
Zhang, Y., Kirchler, T., & Wang, A. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02376-3 · Cited by 7
“Some evidence indicates superior AI perfor - mance primarily under optimal conditions, including extensive, high-quality datasets and higher prevalence [18, 19].”
Applications of Artificial Intelligence in Mammography from a Development and Validation Perspective
Kim, K. H., & Lee, S. H. (2021) · Journal of the Korean Society of Radiology · doi:10.3348/jksr.2020.0205 · Cited by 7
“유방암 검출을 위한 딥러닝 알고리즘을 실제 의료현장에 사용하기 위해서는 적절한 임상자료를 이용한 성능 평가 과정을 거쳐야만 하며, 이 과정에서 의료인들의 역할이 강조되고 있다(30-32).”
Kline, T. L., Kitamura, F., Warren, D., Pan, I., Korchi, A. M., Tenenholtz, N., Moy, L. P., Gichoya, J. W., Santos, I. M. A., Moradi, K., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01548-w · Cited by 7
“Other high-level guides [3–6] have been published elsewhere and were considered when creating the current work.”
Jang, S., Kim, J., Lee, J. S., Jeong, Y., Nam, J. G., Kim, J., & Lee, K. W. (2025) · American Journal of Roentgenology · doi:10.2214/ajr.25.33063 · Cited by 7
Integrating statistical design and inference: A roadmap for robust and trustworthy medical AI
Wei, Q., Cui, M., Liu, Z., Liu, Z., Feng, G., Li, Y., Christiani, D. C., Li, L., Wang, J., Hao, Y., et al. (2025) · The Innovation Medicine · doi:10.59717/j.xinn-med.2025.100145 · Cited by 8
AI for Radiology: A Primer Part I. From Idea to Algorithm
Tejani, A. S., Rauschecker, A. M., Kohli, M. D., & Mongan, J. (2025) · Radiology · doi:10.1148/radiol.250692 · Cited by 7
Conspiracy Spoofing Orders Detection with Transformer-Based Deep Graph Learning
Kang, L., Mu, T., & Ning, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46664-9_33 · Cited by 7
Camelo-Filho, A. E., Lima, P. L. G. D. S. B., Cavalcante, F. L. H. B., Miyajima, O. R., Santos, C. F., da Rosa, R. F., Pessoa, A. L. S., Braga‐Neto, P., & Nóbrega, P. R. (2024) · Brain Sciences · doi:10.3390/brainsci14111159 · Cited by 7
“Notably, dementia improved in 10 out of 13 patients, pyramidal signs diminished or disappeared in 13 out of 17, cerebellar dysfunction cleared in 3 patients and improved significantly in 9, neuropathy improved in 6, electroencephalo-grams improved in 8, and an increase in white matter was observed…”
Almeida, A. L. M., Perger, E. L. P., Gomes, R. H. M., Sousa, G. D. S., Vasques, L. H., Rodokas, J. E. P., Neto, J. O., & Simões, R. P. (2020) · Journal of Immunological Methods · doi:10.1016/j.jim.2020.112870 · Cited by 7
Probing Accuracy-Speedup Tradeoff in Machine Learning Surrogates for Molecular Dynamics Simulations
Sun, F., Kadupitiya, J., & Jadhao, V. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c01282 · Cited by 8
Brioschi, G. C., Brioschi, M. L., Neto, C. D., & O'Young, B. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44511-8_1 · Cited by 7
Comparative Study of Machine Learning Models on Multiple Breast Cancer Datasets
Sujon, M. A. H., & Mustafa, H. A. (2023) · International Journal of Advanced Science Computing and Engineering · doi:10.62527/ijasce.5.1.105 · Cited by 7
“Elastography [20] is a recent technique which supports imaging technology that can be used when carcinoma tissue supports more substantial than the adjacent regular functional tissue.”
BackgroundLan, W., Tang, H., Wen, L.-B., Chen, Z., Zhou, Y.-L., Dai, W., Wang, M., Li, X., Wang, W., Tang, F., et al. (2024) · Eye & Contact Lens Science & Clinical Practice · doi:10.1097/icl.0000000000001091 · Cited by 7
Mosquera, C., Binder, F., Díaz, F. N., Seehaus, A., Ducrey, G., Ocantos, J., Aineseder, M., Rubin, L., Rabinovich, D., Quiroga, A. E., et al. (2021) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2021.100039 · Cited by 7
Medical Image Perception Research in the Emerging Age of Artificial Intelligence
Jacobson, F. L. (2019) · Radiology · doi:10.1148/radiol.2019192252 · Cited by 7
A deep-learning-enabled diagnosis of ovarian cancer – Authors' reply
Gao, Y., Li, H., Chen, L., Wu, Y. S., Ma, D., & Gao, Q. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00145-5 · Cited by 7
Clinical Applications of Artificial Intelligence in Uveal Melanoma
CHADWICK, W. F., Ganesh, S., Dadzie, A. K., Ebrahimi, B., Rahimi, M., Son, T., Alahmadi, R., Yao, X., & Heiferman, M. J. (2025) · Anticancer Research · doi:10.21873/anticanres.17817 · Cited by 6
“Key barriers to clinical integration include limited availability o f annotated datasets, lack o f standardized imaging protocols, and interoperability challenges across institutions (39 ‑ 40).”
Azhdarimoghaddam, A., Bigloo, A. M., Meigoli, M. S. S., Abdelbaset, M., Narimani, M., Tehrani, F. D., Anar, M. A., Abdolvand, F., Goudarzi, P., Ghazizadeh, Y., et al. (2025) · BioMedical Engineering OnLine · doi:10.1186/s12938-025-01487-1 · Cited by 6
“Standards such as TRIPOD-AI (for transparent model development and reporting), SPIRIT-AI (for clinical trial protocol design involving AI tools), and STARD-AI (for diagnostic accuracy studies) are increasingly being recognized as prerequisites for model credibility and regulatory approval [32, 33].”
Developing a Smart Nursery Application for Monitoring and Babies Care
Alqahtani, M. F., Bashunaym, R. S., Alotaibi, N. M., Alkhaldi, R. Z., Alshabanah, M., Alrajhi, D., Alsmadi, M. K., & Almarashdeh, I. (2020) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst207161 · Cited by 6
Wu, X., & Polychronis, A. (2023) · Journal of Cancer Metastasis and Treatment · doi:10.20517/2394-4722.2023.10 · Cited by 6
Борисов, А. А., Arzamasov, K. M., Семенов, С. С., Vladzimirsky, A. V., & Vasilev, Y. A. (2023) · Medical Visualization · doi:10.24835/1607-0763-1346 · Cited by 6
Yin, X., Wang, K., Wang, L., Yang, Z., Zhang, Y., Wu, P., Zhao, C., & Zhang, J. (2024) · Abdominal Radiology · doi:10.1007/s00261-024-04241-8 · Cited by 6
“AI model for prostate segmentation: A pretrained UNet-3D [20] deep learning model was used to segment the prostate region (Supplementary Material).”
Comparative Study of AI Modes in Ultrasound Diagnosis of Breast Lesions
Hong, Y.-T., Yu, Z., & Chou, C. (2025) · Diagnostics · doi:10.3390/diagnostics15050560 · Cited by 6
“Unlike internal cross-validation, which is primarily used for model optimization in controlled settings, external validation evaluates the model’s performance on an independent dataset that reflects real-world clinical scenarios [21].”
Chen, D. C., He, E., Pace, K., Chekay, M., & Raman, S. (2025) · The Oncologist · doi:10.1093/oncolo/oyaf112 · Cited by 6
“43 Given that only 5 out of 12 (41.67”
InfluentialMHealth: An Artificial Intelligence Oriented Mobile Application for Personal Healthcare Support
Afrah, I. A., & Köse, U. (2021) · Journal of Multidisciplinary Developments · doi:10.48550/arxiv.2108.09277 · Cited by 6
“In detail, as there are many examples of Artificial Intelligence based solutions for medical-healthcare [9-11] and web based mobile applications or modular applications are in common place generally [12-14], it is a necessary to build a mobile solution accordingly.”
BackgroundJeong, S. Y., Suh, C. H., Heo, H., Shim, W. H., & Kim, S. J. (2022) · Investigative Magnetic Resonance Imaging · doi:10.13104/imri.2022.26.4.237 · Cited by 6
Machine Learning Models for Histopathological Breast Cancer Image Classification
Haines, S., Eaton, E., & Ali, M. L. (2023) · IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot58121.2023.10174304 · Cited by 6
Prediction of Tuberculosis Disease Progression with AI Analysis of Clinical Data
Pandey, S. K., Singh, K. U., Dingankar, R. S., Jadhav, K., Gupta, K., & Yadav, R. K. (2023) · International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI) · doi:10.1109/icaiihi57871.2023.10489091 · Cited by 6
Lee, Y., Pyeon, J.-H., Han, S., Kim, N. J., Park, W.-J., & Park, J. (2024) · Applied Sciences · doi:10.3390/app14167342 · Cited by 6
Lin, Y., Cheng, M., Wu, C., Huang, Y., Zhu, T., Li, J., Gao, H., & Wang, K. (2024) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2024.101254 · Cited by 6
Application of artificial intelligence in gastrointestinal endoscopy in Vietnam: a narrative review
Dao, H. V., Nguyen, B. P., Tùng, N. T., Lam, H. N., Nguyen, T. T. K., Dang, T. T. P., Hoang, L. B., Le, H. T., & Dao, L. V. (2024) · Therapeutic Advances in Gastrointestinal Endoscopy · doi:10.1177/26317745241306562 · Cited by 6
Optimized SNR-based ECAP threshold determination is comparable to the judgement of human evaluators
Gärtner, L., Spitzer, P., Lauss, K., Takanen, M. O., Lenarz, T., & Hoth, S. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259347 · Cited by 6
Sheng, K., Offersen, C. M., Middleton, J., Carlsen, J. F., Truelsen, T. C., Pai, A., Johansen, J., & Nielsen, M. B. (2022) · Diagnostics · doi:10.3390/diagnostics12081878 · Cited by 6
“This also requires consecutive datasets not prone to spectrum biases and ground-truth labels unaffected by selection bias [13].”
BackgroundMethodInfluentialKim, D., Lee, J. H., Jang, M., Park, J., Hong, W., Lee, C. S., Yang, S. Y., & Park, C. M. (2023) · Bioengineering · doi:10.3390/bioengineering10091077 · Cited by 6
A COMPREHENSIVE REVIEW OF INTEGRATING AI AND MEDICAL IMAGING IN LUNG CANCER DIAGNOSIS
Mostafavi, M., & Shabani, M. (2023) · doi:10.53555//jptcp.v31i3.4929 · Cited by 6
Jhang, H., Park, S. J., Sul, A.-R., Jang, H. Y., & Park, S. H. (2024) · Korean Journal of Radiology · doi:10.3348/kjr.2023.1281 · Cited by 6
“Securing insurance coverage for the use of AI in healthcare hinges on demonstrating its value in improving ultimate patient clinical outcomes when compared to traditional care [21,23].”
Self-citationWolfs, C. J. A., Varfalvy, N., Canters, R. A. M., Nijsten, S. M. J. J. G., Hattu, D., Archambault, L., & Verhaegen, F. (2020) · Medical Physics · doi:10.1002/mp.14385 · Cited by 6
Trends in Software Engineering
(2002) · Advances in computers · doi:10.1016/s0065-2458(01)x8000-5 · Cited by 6
Qu, C., Zou, Y., Dai, Q., Ma, Y., He, J., Liu, Q., Kuang, W., Jia, Z., Chen, T., & Gong, Q. (2021) · Psychoradiology · doi:10.1093/psyrad/kkab017 · Cited by 6
“…to train and validate algorithms were from the same dataset; that is, only the internal validation method was used to evaluate the performance of the GAN model, which can be divided into cross-validation and split-sample validation (Park and Han, 2018; Kim et al. , 2020a; Kulkarni et al. , 2021).”
BackgroundMethodProspective Clinical Trials: 10-year Trends in Radiology
Alvin, M. D., Moshiri, M., & Bluemke, D. A. (2020) · Radiology · doi:10.1148/radiol.2020204003 · Cited by 6
Deep Learning to Detect Pancreatic Cancer at CT: Artificial Intelligence Living Up to Its Hype
Aisen, A. M., & Rodrigues, P. S. (2022) · Radiology · doi:10.1148/radiol.222126 · Cited by 6
Thijssen, A., Schreuder, R.-M., Dehghani, N., Schor, M., de With, P. H. N., van der Sommen, F., Boonstra, J. J., Moons, L. M. G., & Schoon, E. J. (2024) · Endoscopy International Open · doi:10.1055/a-2403-3103 · Cited by 5
Cui, Z., Yan, Y., Wang, H., Bai, Y., Zhang, L., Yu, M., Zhang, F., Yuan, X., Wang, S., Ouyang, B., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-087588 · Cited by 5
Ma, R., Wang, Y., Li, Y., Li, Z., Feng, Z., Kong, X., Yin, Y., Wu, Y., & He, Q. (2025) · Analytical Chemistry · doi:10.1021/acs.analchem.4c05881 · Cited by 5
Metadata
- Authors
- Park, S. H., & Han, K.
- Affiliations
- Yonsei University (KR)
- Venue
- Radiology
- Published
- 2018-03
- Volume
- 286
- Issue
- 3
- Pages
- 800-809
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2783687327
- Cited by
- 894
- References
- 49
- Citations by year
- 2026: 123 · 2025: 98 · 2024: 96 · 2023: 135 · 2022: 111 · 2021: 121 · 2020: 103 · 2019: 80 · 2018: 25
- FWCI
- 1.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Ministry of Trade, Industry and Energy
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- artificial intelligence in healthcare, deep neural network, medical image diagnosis, prediction model, discrimination and calibration, overfitting, spectrum bias, external validation, disease prevalence, outcomes research, high-dimensional models
- MeSH terms
- Artificial Intelligence, Data Interpretation, Statistical, Diagnostic Imaging, Humans, Image Interpretation, Computer-Assisted, Terminology as Topic, Predictive Value of Tests, Technology Assessment, Biomedical, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.