Hidden stratification causes clinically meaningful failures in machine learning for medical imaging
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Machine learning models for medical image analysis often suffer from poor performance on important subsets of a population that are not identified during training or testing. For example, overall performance of a cancer detection model may be high, but the model may still consistently miss a rare but aggressive cancer subtype. We refer to this problem as hidden stratification, and observe that it results from incompletely describing the meaningful variation in a dataset. While hidden stratification can substantially reduce the clinical efficacy of machine learning models, its effects remain difficult to measure. In this work, we assess the utility of several possible techniques for measuring hidden stratification effects, and characterize these effects both via synthetic experiments on the CIFAR-100 benchmark dataset and on multiple real-world medical imaging datasets. Using these measurement techniques, we find evidence that hidden stratification can occur in unidentified imaging subsets with low prevalence, low label quality, subtle distinguishing features, or spurious correlates, and that it can result in relative performance differences of over 20% on clinically important subsets. Finally, we discuss the clinical implications of our findings, and suggest that evaluation of hidden stratification should be a critical component of any machine learning deployment in medical imaging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In this example, reproduced from Oakden-Rayner and colleagues, 13 the artificial intelligence model learns to detect the artifacts of clinical care (chest drains) and fails to adequately learn the features of the pathology itself.”
From The medical algorithmic audit (2022) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI systems can make errors that may be hard to foresee but that, if allowed to be deployed at scale, could have catastrophic consequences53.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI systems can make errors that may be hard to foresee but that, if allowed to be deployed at scale, could have catastrophic consequences45.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [45]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Furthermore, this drop in performance might not be obvious at the aggregate level of typical artificial intelligence testing, but rather in subsets of the target cohort or specific strata within the input data, a concept that has been described as hidden stratification.13”
From The medical algorithmic audit (2022) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“It has been shown that AI systems may be systematically biased toward different outputs, which may lead to different or even unfair treatment, on the basis of extant features53,57–59.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
Rousseeuw, P. J. (1987) · Journal of Computational and Applied Mathematics · doi:10.1016/0377-0427(87)90125-7 · Cited by 21,392 · cited 1×
“While these results support other findings that demonstrate the utility of hidden-state clustering in model development [20], the relatively simple technique presented here should be considered only a first attempt at unsupervised identification of hidden stratification [5, 32].”
Result"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“Esteva et al. [12] is particularly notable, as this dataset is labelled for more than 2,000 diagnostic subclasses but the results presented only consider “top-level” diagnostic categories.”
BackgroundAggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · cited 1×
“Figure 1(a) presents the performance of a ResNeXt-29, 8x64d Convolutional Neural Network (CNN) trained on the 20 CIFAR-100 superclasses using the training schedule reported in [37] and the implementation provided by [38].”
MethodGulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · cited 1×
“Gulshan et al. [16], for instance, present variations in retinopathy detection performance on subsets with images obtained in different locations, with differing levels of disease severity, and with different degrees of pupil dilation.”
BackgroundA guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312 · cited 1×
“Deep learning systems have shown remarkable promise in medical image analysis, often claiming performance rivaling that of human experts [13].”
BackgroundWang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · cited 2×
“As a result, medical imagery analysts have commonly begun either to use semi-automated labelling techniques [34, 14, 18, 10, 14] or to apply human expertise to produce a narrow or incomplete set of visual labels [25] rather than exhaustively labelling all possible findings and variations.”
BackgroundA dendrite method for cluster analysis
Caliński, T., & Harabasz, J. (1974) · Communications in Statistics · doi:10.1080/03610927408827101 · Cited by 4,480
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 1×
“]. However, the cost of manual labelling is often prohibitive, appropriate augmentation transforms can be difficult to define, and oversampling an underperforming subset can cause degradation on others [14, 28, 4, 40]. As a result, medical imagery analysts have commonly begun either to use semi-automated labelling techn…”
BackgroundClinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799 · cited 1×
“observed [6, 34]; such analyses may be helpful in identifying error modes via human review, but do not characterize the full space of subset performance [33].”
Background
Research works printed but not linked: 1
- Irvin Jeremy, Rajpurkar Pranav, Ko Michael, Yu Yifan, Ciurea-Ilcus Silviana, Chute Chris, Marklund Henrik, Haghgoo Behzad, Ball Robyn, Shpanskaya Katie, and Others. 2019 Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. arXiv preprint arXiv:1901.07031 (2019). · arXiv
Cited by
Items 301–400 of 440
Contrastive Learning for Domain Adaptation Under Domain Shift: A Critical Survey and Taxonomy
Nabi, S. T. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10603-w · Cited by 4
“Concept shift is common in complex real-world scenarios where contextual factors influence label interpretation, such as changes in diagnostic criteria in medical imaging or variations in object definitions across datasets [ 50, 51].”
Cai, L., Meng, B., Huang, J., Ding, G., Ju, M., Wang, W.-W., Shi-jie, D., Lai, L., Wang, J., Yang, C.-X., et al. (2026) · LabMed discovery. · doi:10.1016/j.lmd.2026.100130 · Cited by 1
Rubak, J. A. B., Haghighat, S., Jain, S., Aldesoki, M., Chaurasia, A., Ehsani, S., Ghanatkaman, F. D., Ghazali, A. B., Issa, J., Khalil, B., et al. (2026) · Applied Sciences · doi:10.3390/app16126154 · Cited by 2
Giovanni, L., Rijsbosch, B., Meding, K., Spanakis, G., DIJCK, G. V., & Kollnig, K. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806436 · Cited by 4
“Failing to stratify on such properties could lead to clinically meaningful failures [36] All of these issues can be further complicated when we have multiple images coming from the same patient.”
There is no free benchmark: An institutional view of legal AI benchmarking
Guha, N., Zhang, A. K., Tsang, C., Manning, C. D., Nyarko, J., & Ho, D. E. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2509757122 · Cited by 4
Bias in AI: a framework for understanding the sources of bias in artificial intelligence
Robinig, W., & Wallner, J. P. (2026) · AI and Ethics · doi:10.1007/s43681-026-01283-0 · Cited by 3
“Oakden-Rayner et al. [ 23] show that a model can achieve high overall accuracy on a diagnostic task while systematically failing on clinically important subgroups that are not separately labeled—a problem they term hidden stratification .”
Echo Chambers: Bias and Representation in Cardiac Imaging Datasets for Artificial Intelligence
Gorijavolu, R., Jaiswal, N., Pratap, J. S., Madapati, K., López, M. D. P. A., Leite, M., Muwanguzi, D. P., Izath, N., Samsel, K., Shoyombo, I., et al. (2026) · Current Cardiology Reports · doi:10.1007/s11886-026-02406-z · Cited by 1
“Work on label noise and hidden stratification shows that inconsistent or low-quality annotations can substantially degrade generalization and lead to clinically meaningful failures, even when using state-of-the-art architectures [70–72 ].”
Soni, M. K., Jain, A. K., & Varma, S. (2026) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.6.10s.2026.1199-1124 · Cited by 1
A Medical Image Classification Model Based on Adversarial Lesion Enhancement
Zhang, B., & Xu, H. (2021) · Scientific Programming · doi:10.1155/2021/4265650 · Cited by 1
“[21] evaluated several possible techniques for measuring the effect of hidden stratification and found that hidden stratification can occur in unrecognized low prevalence, low label quality, subtle distinguishing features, and it can lead to more than 20% relative performance difference of the models.”
BackgroundDeep learning for reliable detection of epileptogenic lesions
Pedersen, M., Verspoor, K. M., Abbott, D. F., Jenkinson, M., Sinclair, B., Law, M., & Jackson, G. D. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90037-9.00013-8 · Cited by 1
Unlocking clinical insights from medical images using deep learning
Ushaa, E., & Vishal, E. (2023) · i-manager s Journal on Artificial Intelligence & Machine Learning · doi:10.26634/jaim.1.2.20044 · Cited by 1
GEP-NETs radiomics in action: a systematical review of applications and quality assessment
Wei, C., Jiang, T., Wang, K., Gao, X., Zhang, H., & Wang, X. (2024) · Clinical and Translational Imaging · doi:10.1007/s40336-024-00617-4 · Cited by 1
“Studies with small samples are more susceptible to overfitting, since information is not abundant enough for the model to learn a valid association and minimize meaningless noise [88].”
Pandey, R. K., & Shukla, P. K. (2024) · International Conference on Cybernation and Computation (CYBERCOM) · doi:10.1109/cybercom63683.2024.10803179 · Cited by 1
“In 2020, Oakden-Rayner et al. [14] outlined the challenges and limitations of AI in medical image processing and placed particular emphasis on the concept of "hidden stratification".”
Synthetic Data for Assessing Robustness of Deep Learning Models to Distributional Shifts
Rahul, V., Sathish, R., Pattanaik, M., & Babu, K. A. R. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10980769 · Cited by 1
“To overcome the limitations of aggregate measure as a performance of model performance, machine learning engineers have proposed evaluating additional measures like worst group error [6] and assessing performance under distribution shifts [7].”
Weighted Utility: A Utility Metric Based on the Case-Wise Raters’ Perceptions
Campagner, A., Conte, E., & Cabitza, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-84060-0_13 · Cited by 2
A Roadmap for Boosting Model Generalizability for Predicting Hospital Encounters for Asthma
Luo, G. (2022) · JMIR Medical Informatics · doi:10.2196/33044 · Cited by 1
“Many methods to improve fairness in machine learning exist [50-52].”
BackgroundMethodInfluentialCrafting an Undergraduate Course at the Intersection of Machine Learning and Software Engineering
Chenoweth, S., & Linos, P. K. (2022) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie56618.2022.9962523 · Cited by 1
“In addition, our references [14, 15, 16, 17, 18, 19] support the thesis that development processes for creating successful ML systems are significantly different from processes for normal business systems.”
BackgroundAnalysis of Potential Biases on Mammography Datasets for Deep Learning Model Development
Gerbolés, B. Z., López‐Linares, K., García, M. J., Rebescher, K. M., Lalaguna, I., Marco, E. A., Álvarez, M. B. C., Garcia, J., Ledesma‐Carbayo, M. J., & Macía, I. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17721-7_7 · Cited by 2
Mapping DNN Embedding Manifolds for Network Generalization Prediction
O’Brien, M., Wolfinger, B., Bukowski, J. V., Unberath, M., Pezeshk, A. X., & Hager, G. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00646 · Cited by 1
Beyond Distribution Shift: Spurious Features Through the Lens of Training Dynamics
Murali, N., Puli, A., Yu, K., Ranganath, R., & Batmanghelich, K. N. (2023) · PubMed · doi:10.48550/arxiv.2302.09344 · Cited by 3
“Therefore, the presence of a chest drain in the lung is positively correlated with the presence of Pneumothorax and can be used by the deep learning model (Oakden-Rayner et al., 2020).”
BackgroundMethodResultInfluentialSidulova, M., Sun, X., & Gossmann, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43993-3_64 · Cited by 2
Active Assessment of Prediction Services as Accuracy Surface Over Attribute CombinationsPreprint
Piratla, V., Chakrabarty, S., & Sarawagi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06514 · Cited by 2
“Sub-population performance: Several recent papers have focused on identifying sub-populations with significantly worse accuracy than aggregated accuracy [2, 3, 6, 8, 25, 26].”
BackgroundGlocker, B., & Winzeck, S. (2021) · arXiv (Cornell University) · Cited by 2
Kumar, S., & Cui, Y. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.18.658431 · Cited by 1
“Population stratification, the presence of distinct subgroups within a population that differ systematically in characteristics such as age, sex, genetic ancestry, or socioeconomic status, significantly impacts machine learning applications across various domains [1-3].”
Uzoechina, G., & Osajiuba, T. (2025) · Research Square · doi:10.21203/rs.3.rs-7876598/v1 · Cited by 1
Erravelli, B., Bandi, L., Srujith, C. H., Kodamala, P. R., Ravulakollu, K. K., Sambangi, S. S. S., & Kuthadi, D. (2026) · Research Square · doi:10.21203/rs.3.rs-9190460/v1 · Cited by 1
Liu, Y., Yang, L., Phene, S., & Peng, L. H. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-821259-2.00013-2 · Cited by 1
Luo, L., Chen, H., Xiao, Y., Zhou, Y., Wang, X., Vardhanabhuti, V., Wu, M., & Heng, P. (2021) · arXiv (Cornell University) · Cited by 1
A Topological-Framework to Improve Analysis of Machine Learning Model PerformancePreprint
Kvinge, H., Wight, C., Akers, S. M., Howland, S., Choi, W., Ma, X., Gosink, L., Jurrus, E. R., Kappagantula, K., & Emerson, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04714
“The present work is inspired by a line of research investigating how models can lack robustness when they systematically fail on certain subpopulations of a dataset (Oakden-Rayner et al., 2020).”
育てる支える(vol.17)教育と臨床の協働でつくるフィジカルアセスメント研修--知識と実践が結びつくシナリオシミュレーションの実際
寺田, 英., & 高橋, 恵. (2011) · 看護展望
Jesudasen, S. J., & Montesi, S. B. (2022) · CHEST Journal · doi:10.1016/j.chest.2022.04.152
Papineni, D. D., & Lind, M. L. (2023) · Journal of Strategic Innovation and Sustainability · doi:10.33423/jsis.v18i1.6058
Curriculum gDRO: Improving Lung Malignancy Classification through Robust Curriculum Task Learning
Sivakumar, A., Wang, Y., Tchoua, R., Ramaraj, T., Furst, J. D., & Raicu, D. S. (2023) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms58004.2023.00290 · Cited by 1
Osteoporosis Biomarker Identification and Use of Machine Learning in Osteoporosis Treatment
Malviya, R., Rajput, S., & Vaidya, M. (2024) · Wiley eBooks · doi:10.1002/9781394230914.ch7
Detecting Pulmonary Lesions in Low-Prevalence Real-World Settings Using Deep Learning
Kvak, D., Chromcová, A., Ovesná, P., Dandár, J., Biros, M., Hrubý, R., Dufek, D., & Pajdaković, M. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-1335-6_1
Czerniawski, T. (2024) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784485224.024
Cabitza, F., & Campagner, A. (2024) · Wiley eBooks · doi:10.1002/9781119846567.ch14
Fundamentals on transparency, reproducibility, and validation
Camarasu-Pop, S., Vila, G., Tristan Glatard, B., Bonnet, A., Frindel, C., & Ratiney, H. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00017-1 · Cited by 1
Effective Guidance for Model Attention with Simple Yes-no Annotations
Lee, S., Payani, A., & Chau, D. H. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825776
“Such model attention to less relevant areas results in reduced trustworthiness [5], poor generalization [6], [7], and biased performance [8], [9], [10], [11].”
DETECCIÓN AUTOMÁTICA DE HALLAZGOS EN RADIOGRAFÍAS DE TÓRAX CON REDES NEURONALES
García, R. C., & Jiménez, J. J. (2025) · doi:10.60096/fundacionavanza/5072025
Assessment of the role of emotions in audiovisual associations through an enactive approach
Cenerini, C., Vollero, L., Stefano, N. D., Santonico, M., Pennazza, G., & Keller, F. (2025) · PLoS ONE · doi:10.1371/journal.pone.0322449 · Cited by 2
“Firstly, the k-means clustering algorithm was applied to the form results to verify the absence of any hidden strat-ifications [42] in the subject population.”
Beyond the Surface: A Comprehensive Analysis of Implicit Bias in Vision-Language Models
Capitani, G., Lucarini, A., Bonicelli, L., Bolelli, F., Calderara, S., Vezzali, L., & Ficarra, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92648-8_3 · Cited by 2
“Unsupervised methods have gained traction [29,34,36] as they do not need protected-group labels. large, uncurated datasets.”
Kheok, S. W., Hu, G., Lee, M. H., Wong, S. C. P., Zheng, K., Htoon, H. M., Zhu, L., Tan, A. S. M., Chan, L. L., Ooi, B. C., et al. (2025) · Orbit · doi:10.1080/01676830.2025.2521868
Optimizing Medical Image Captioning with Conditional Prompt Encoding
Fernandes, R. F., Oliveira, H. S., Ribeiro, P., & Oliveira, H. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99568-2_16
Blunk, J., Bodesheim, P., & Denzler, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05060-1_9
“We consider the binary classification task of detecting pneumothorax (diagnosed vs. not diagnosed), where the presence of chest drains (a treatment artifact) acts as a spurious feature [23, 27].”
Predicting Patient Self-reported Race From Skin Histological Images with Deep Learning
Chen, S., Verma, R., Clare, K., Jegminat, J., Alleva, E., Huang, K., Veremis, B. M., Fuchs, T. J., & Campanella, G. S. V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05870-6_11 · Cited by 1
“While algorithmic fairness techniques have been explored to mitigate bias [4], several studies have also demonstrated that feature–confounder correlations, such as the presence of treatment artifacts or institution-specific markers, can undermine model generalizability and fairness [5, 6, 7].”
meval: A Statistical Toolbox for Fine-Grained Model Performance Analysis
Sutariya, D., & Petersen, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05870-6_19 · Cited by 1
“It is increasingly well-recognized that developers should assess the performance of their machine learning models not just in the aggregate, but also dis-aggregated – or stratified – by attributes characterizing model inputs and targets [25,7,27].”
Artificial intelligence in surgery: Redefining intelligence assessment for the Operating room
Huang, K. A., & Kuo, P. C. (2025) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2025.116676 · Cited by 1
Subclass-Aware Inclusive Classifier via Repulsive Hidden Strata
Bajpai, N., Paik, J. H., & Sarkar, S. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761150
Hinge, C., Rodell, A. B., Zuehlsdorff, S., Spottiswoode, B. S., Korsholm, K., Fischer, B. M., Ladefoged, C. N., & Andersen, F. L. (2025) · Scientific Reports · doi:10.1038/s41598-025-26827-y · Cited by 1
“This transparency contrasts with conventional classification and segmentation networks, which often lack interpretable outputs and provide little insight into their predictions 4,5 .”
MultiModal craniocerebral diagnose based on 3D CT and image reports
He, W., Chen, Q., Gao, W., Xu, H., & Wei, Z. X. (2025) · Scientific Reports · doi:10.1038/s41598-025-30249-1
FedSplit: A Federated Based Split Learning For Heartbeat Classification Using ECG Data
Ragam, P., Danaveni, P., Jyothi, S., Chunchu, L. P., Dhanvanthini, P., & Karthik, G. (2025) · IEEE India Council International Subsections Conference (INDISCON) · doi:10.1109/indiscon66021.2025.11251928
“However, training deep CNN models requires large, labelled datasets, which can be challenging to 979-8-3315-1504-1/25/$31.00 ©2025 IEEE obtain due to privacy concerns and data-sharing limitations [7].”
InfluentialAli, L. M., Yim, A. K., Gerbi, E., Nguyen, T., Tu, N., Ikede, F., Sampaleanu, R., Grigore, D., Waligorski, J. E., Kremitzki, C. L., et al. (2025) · npj Systems Biology and Applications · doi:10.1038/s41540-025-00627-6
Rahman, M. S., Humayara, F., Rabbi, S. M. E., & Rashid, M. M. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01770-6 · Cited by 1
“Numerous studies in medical image retrieval report performance metrics that are difficult to reconcile with the practical challenges of clinical deployment [16, 17].”
Empirical Bayesian Methods and BNNs for Medical OOD Detection
Raina, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06593-3_20
Zhang, G., & Wang, X. (2025) · International Conference on New Trends in Computational Intelligence (NTCI) · doi:10.1109/ntci67886.2025.11308540
“Missing cancerous regions in this context is especially dangerous [9, 10].”
Hunt, A., Schulze, H., Samuel, K., Fisher, R. B., & Bachmann, T. T. (2026) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2025.1725145
Explainable Quantum Artificial Intelligence for Enhanced Radiology Decision Support
Meenakshi, S. K., Saraswathi, V., Malathy, M., & Subbulakshmi, G. (2025) · IEEE International Conference on Emerging Trends in Engineering and Computing (ETECOM) · doi:10.1109/etecom66111.2025.11319091 · Cited by 1
“Some studies [11], [12] have proposed adapting feature attribution techniques to quantum classifiers, while others suggest inherently interpretable quantum circuits.”
AI for Radiographers: Industry Perspectives
Spencer, N. J. B., Sutherland, K., King, G. J. W., & Jones, D. F. (2026) · Springer eBooks · doi:10.1007/978-3-032-05080-9_13
Xu, M., Yang, T., Zhou, T., Chen, Q., Wang, S., & Zhang, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356968
Probabilistic Runtime Verification, Evaluation, and Risk Assessment of Visual Deep Learning Systems
Torpmann-Hagen, B., Halvorsen, P., Riegler, M. A., & Johansen, D. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3795523
“This has been studied extensively in several medical case studies, specifically for polyp segmentation [2], X-ray-based pneumonia detection [49], and in the context of hidden stratification in various medical tasks [8, 47], among others [33].”
Machine Unlearning for Pathological Image Classification Models
Kim, M. J., Cho, J. H., Yook, H. J., Kim, Y. S., Kim, S. Y., & Lee, Y. K. (2025) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc66702.2025.11388747
“There is a need to modify classes that are no longer clinically valid [3].”
On Demographic Group Fairness Guarantees in Deep Learning
Luo, Y., Wen, C., Shi, M., Huang, H., Fang, Y., & Wang, M. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3674484
“Through extensive experiments on diverse datasets across various modalities (image, tabular data, and text), including FairVision (eye disease detection), CheXpert (pleuraleffusiondetection), HAM10000 (skin lesion classification), FairFace (facial attribute recognition), ACS Income (income…”
Evolving and novel applications of artificial intelligence in interventional oncology
Almashni, S. Y., Fayek, F. B., Javens, D. C., Boulis, M. T., & Makary, M. S. (2026) · World Journal of Clinical Oncology · doi:10.5306/wjco.v17.i3.113226
Leveraging Neurosymbolic AI for Slice Discovery
Collevati, M., Eiter, T., & Higuera, N. (2026) · Neurosymbolic Artificial Intelligence · doi:10.1177/29498732261419315
Bayona, D., Ebner, D. K., Ekama, L., Talom, B. K., Shiraishi, S., Osorio, M. B., Mastroleo, F., Lozko, Y., Namazi, J., Waddle, M. R., et al. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00167
Lefebvre, O., Lemyre, F. C., Éthier, J., Chikouche, L. H., Amriou, L., Poenaru, D., & Vallières, M. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag034 · Cited by 1
Clustering and knowledge integration
Nghiem, N.-V.-D. (2021) · theses.fr (ABES) · doi:10.70675/788e8265z6f63z40e7z94efz6067e34e9106
Collateral-Free Learning of Deep Representations : From Natural Images to Biomedical Applications
Barbano, C. A. (2023) · theses.fr (ABES) · doi:10.70675/d0edb18bzae2bz48f8zba04z48d7240e8e0d
Gamboa, A., Araya, M., & Sotomayor, C. G. (2026) · Applied Sciences · doi:10.3390/app16094067
Aravinda, C. V., Shaffi, N., Viswan, V., Al-Rahbi, A., Choy, K. W., Bouchareb, Y., & Sirasanagandla, S. R. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1807189
Chavoshi, M., Trivedi, H., Newsome, J. M., Mansuri, A., Aitcheson, K., Li, F., Dapamede, T., & Gichoya, J. W. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01961-9
Smart hands: robotic systems as surgical collaborators in plastic and reconstructive surgery
Novotny, M., Fast, A., & Radtke, C. (2026) · Plastic and Aesthetic Research · doi:10.20517/2347-9264.2025.121
Kim, Y. K., Bouza, L., Millington, G., Eow, J., Shah, R., Wiedemann, T. G., & Li, R. (2026) · Applied Sciences · doi:10.3390/app16104997
Liu, Y., Bhatia, K. K., & Fetit, A. E. (2026) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi61048.2026.11515714
Label-Efficient Self-Supervised Representation Learning for Chest X-Ray Pathology Classification
Deepa, S., Sharmila, G., Sheetal, Sadhana, S. M., Alli, A., & Thiyagarajan, C. (2026) · International Conference on Current Trends in Advanced Computing (ICCTAC) · doi:10.1109/icctac68277.2026.11524516
Sehrawat, H., Singh, M., Rawat, P., Chandra, S., Singh, S., Meha, M., Singh, C., & Bhatt, A. (2026) · International Journal of Drug Delivery Technology · doi:10.25258/ijddt.16.49s.98
“Although AI systems have impressive pattern recognition, they exhibit fundamental failures in contextual reasoning, or being able to comprehend findings in the greater clinical context [28].”
Sources of Bias in Clinical Artificial Intelligence and Applications in Rheumatology
Creasman, M., García-Agúndez, A., Yazdany, J., & Schmajuk, G. (2026) · Rheumatic Disease Clinics of North America · doi:10.1016/j.rdc.2026.03.005
Sharma, M., Kant, R., Pichaimani, T., Sudharsanam, S. R., Tomar, M., & Sahu, A. K. (2026) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom70271.2026.11526490
“The problem of domain generalization is evident by the large drops in AUROC values in cross-dataset experiments, especially in the Indian healthcare setting, which is quite heterogeneous [12], [13].”
Kohzai, M., Yamamoto, S., Matsushita, M., Ueno, Y., & Tanigawa, N. (2026) · Diagnostics · doi:10.3390/diagnostics16111637
“The same was reported in previous studies that examined compressed images through visual evaluation [6–9].”
de Filippis, R., & Foysal, A. A. (2026) · OALib · doi:10.4236/oalib.1115138
Bhattacharya, S., Karunaker, A., Anderson, B., Wells, D., Gao, Y., Marks, L., Das, S., Rajasekar, A., & Mazur, L. M. (2026) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai68641.2026.11536379
“Hidden stratification in medical imaging datasets can cause clinically meaningful failures that may not be apparent during initial development phases [26].”
Almalki, A. J., Kadry, H., Ardah, H., Ismail, G. M., Alharbi, W. R., Eid, M. R., & Abdel‐Aty, A. (2026) · Journal of Radiation Research and Applied Sciences · doi:10.1016/j.jrras.2026.102473
Cheng, D., Liu, D., Dong, K., Zhang, Y., Cheng, S., Bao, Y., Li, H., Liang, Z., Jiang, N., & Jia, W. (2026) · The Visual Computer · doi:10.1007/s00371-026-04528-5
Gómez-Vela, F., López, A. L., Divina, F., & García-Torres, M. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07398-5
Artificial Intelligence Versus Physician Diagnostic Accuracy in Spine and Spinal Cord Conditions
Maki, S. (2026) · Springer eBooks · doi:10.1007/978-981-92-0619-3_59
Corvaglia, A., Montagna, M., Vignale, D., Palmisano, A., Pisu, F., Colombo, A., Liberotti, M., Soggetti, C., Denti, M., Rovere, P. Q., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9926339/v1
Ghadhban, H. Q. (2026) · AL-Yarmouk Journal · doi:10.65766/alyj.2026.24.01.13
Crilly, M. (2026) · Scottish Affairs · doi:10.3366/scot.2026.0598
Impact of Dataset Bias and Artifacts on Deep Learning Models for Chest X-Ray Image Analysis
Akhmetov, K., & Kaibassova, D. (2026) · IEEE International Conference on Smart Information Systems and Technologies (SIST) · doi:10.1109/sist61674.2026.11596038
“This observation aligns with previous studies emphasizing the risks of dataset bias and shortcut learning in medical computer vision [5], [6], [12].”
Cross-domain uncertainty-aware attention for trustworthy cardiomegaly detection in chest radiographs
Odeh, V. A., XiaoLing, Z., & Agaba, C. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116044
Artificial wisdom in gastrointestinal diagnostics: A new frontier in translational reasoning
Issa, I. A., & Issa, T. (2026) · Artificial Intelligence in Gastroenterology · doi:10.35712/aig.v7.i2.118884
TRÎNC, E.-C., Iacob, E. R., Arvinti, B., & Stolojescu-Crișan, C. (2026) · Applied Sciences · doi:10.3390/app16167964
Mohan, M., Lingisetty, S., Ahmad, U., Kumar, A., Harikrishnan, S., Chamarty, S., Nijjer, K., & Ahmed, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35393-1_1
Pathology Shortcuts: Detecting and Mitigating Reliance on Non-Target Pathologies in Chest X-ray AI
Shu, H., Chang, S.-T., Huang, M.-P., Lee, H., Tohyama, T., Han, A., & Kuo, P. (2026) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi69079.2026.00015
“Prior work has documented gaps associated with dataset composition, such as demographic imbalance in sex and age [11], [20], as well as hidden stratification tied to race and socio-demographic factors [13].”
Haupt, M., & Maurer, M. H. (2026) · Academic Radiology · doi:10.1016/j.acra.2026.07.076
Auditing Class-Conditional Acquisition Confounding Across Five Open Tuberculosis Chest X-ray CorporaPreprint
Bilal, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7279581
Helmy, A., & Mahmoud, M. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111322
Metadata
- Authors
- Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C.
- Affiliations
- Australian Institute for Machine Learning, University of Adelaide
- Stanford University (US)
- The University of Adelaide (AU)
- Department of Computer Science, Stanford University
- Venue
- Proceedings of the ACM Conference on Health, Inference, and Learning
- Publisher
- ACM
- Published
- 2020-04-02
- Volume
- 2020
- Pages
- 151-159
- PMC
- PMC7665161
- OpenAlex
- W2975035192
- Cited by
- 411
- References
- 33
- Citations by year
- 2026: 86 · 2025: 66 · 2024: 60 · 2023: 53 · 2022: 49 · 2021: 54 · 2020: 40
- FWCI
- 25.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Health, Inference, and Learning (CHIL) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Toshiba
- Teradata
- Ant Financial
- Intel Corporation
- NEC Corporation
- Taiwan Semiconductor Manufacturing Company
- Accenture
- Qualcomm
- Analog Devices
- VMWare
- Infosys
- National Institutes of Health (U54EB020405)
- NXP
- Xilinx
- LETI-CEA
- American Family Insurance
- Swiss Re
- Intelligence Community Postdoctoral Research Fellowship Program
- National Science Foundation (CCF1763315, CCF1563078, CC1937301)
- IBM
- Hitachi
- Okawa Foundation for Information and Telecommunications
- Google Cloud
- Office of Naval Research (N000141712266)
- Gordon and Betty Moore Foundation
- ARM
- Defense Advanced Research Projects Agency (FA87501720095, FA86501827865, FA86501827882)
- Microsoft
- BASF
- Ericsson
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- medical image analysis, rare diseases, spurious correlations, label quality, model evaluation, clinical implementation of AI, CIFAR-100
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.