Does your dermatology classifier know what it doesn’t know? Detecting the long-tail of unseen conditions
Open accessAbstract
Supervised deep learning models have proven to be highly effective in classification of dermatological conditions. These models rely on the availability of abundant labeled training examples. However, in the real-world, many dermatological conditions are individually too infrequent for per-condition classification with supervised learning. Although individually infrequent, these conditions may collectively be common and therefore are clinically significant in aggregate. To prevent models from generating erroneous outputs on such examples, there remains a considerable unmet need for deep learning systems that can better detect such infrequent conditions. These infrequent 'outlier' conditions are seen very rarely (or not at all) during training. In this paper, we frame this task as an out-of-distribution (OOD) detection problem. We set up a benchmark ensuring that outlier conditions are disjoint between the model training, validation, and test sets. Unlike traditional OOD detection benchmarks where the task is to detect dataset distribution shift, we aim at the more challenging task of detecting subtle differences resulting from a different pathology or condition. We propose a novel hierarchical outlier detection (HOD) loss, which assigns multiple abstention classes corresponding to each training outlier class and jointly performs a coarse classification of inliers vs. outliers, along with fine-grained classification of the individual classes. We demonstrate that the proposed HOD loss based approach outperforms leading methods that leverage outlier data during training. Further, performance is significantly boosted by using recent representation learning methods (BiT, SimCLR, MICLe). Further, we explore ensembling strategies for OOD detection and propose a diverse ensemble selection process for the best result. We also perform a subgroup analysis over conditions of varying risk levels and different skin types to investigate how OOD performance changes over each subgroup and demonstrate the gains of our framework in comparison to baseline. Furthermore, we go beyond traditional performance metrics and introduce a cost matrix for model trust analysis to approximate downstream clinical impact. We use this cost matrix to compare the proposed method against the baseline, thereby making a stronger case for its effectiveness in real-world scenarios.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For a disease in the long tail of the training distribution, the neural network has not seen enough data points to capture all of its possible manifestations and refine its decision boundary.
“For a disease in the long tail of the training distribution, the neural network has not seen enough data points to capture all of its possible manifestations and refine its decision boundary45.”
From Generating dermatopathology reports from gigapixel whole slide images with HistoGPT (2025) · cites this paper as [45]
The Introduction (pp. 1–2) says long-tail dermatological conditions have too few per-condition training examples for practical supervised classification. It does not say that the network fails to capture all possible manifestations or refine its decision boundary; those are additional explanations supplied by the citing sentence. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
Estimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168 · see citing works
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 4,834 · see citing works
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · see citing works
Self-citationWu, Y., & He, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_1 · Cited by 3,601 · see citing works
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · see citing works
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · see citing works
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968 · see citing works
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · see citing works
Ruff, L., Vandermeulen, R. A., Görnitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018) · International Conference on Machine Learning · Cited by 1,719 · see citing works
Zhou, S. K., Greenspan, H. K., Davatzikos, C. A., Duncan, J. S., van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,294 · see citing works
Weight Uncertainty in Neural Network
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,280 · see citing works
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00823 · Cited by 1,250 · see citing works
One-class svms for document classification
Manevitz, L. M., & Yousef, M. (2002) · doi:10.5555/944790.944808 · Cited by 1,219 · see citing works
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial AttacksPreprint
Lee, K., Lee, K., Lee, H., & Shin, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03888 · Cited by 1,166 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T. (2024) · TIB Data Manager · doi:10.57702/iuuvgtaz · Cited by 1,140 · see citing works
Enhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,117 · see citing works
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 1,080 · see citing works
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_29 · Cited by 1,039 · see citing works
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019) · arXiv · doi:10.48550/arxiv.1906.02530 · Cited by 989 · see citing works
Self-citationHD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition
Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Di, W., & Yu, Y. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.314 · Cited by 936 · cited 1× · see citing works
“Note that Yan et al. (2015) also incorporate the hierarchy among image classes into the model but in a different way: independent fine grained classification heads are constructed for each high level class.”
BackgroundDeep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932 · see citing works
AugMix: A Simple Data Processing Method to Improve Robustness and UncertaintyPreprint
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02781 · Cited by 893 · see citing works
Self-citationA deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853 · see citing works
Self-citationDeep Ensembles: A Loss Landscape PerspectivePreprint
Fort, S., Hu, H., & Lakshminarayanan, B. (2019) · arXiv.org · doi:10.48550/arxiv.1912.02757 · Cited by 820 · see citing works
Self-citationBig Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685 · see citing works
Self-citationMuehlematter, U. J., Daniore, P., & Vokinger, K. N. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30292-2 · Cited by 665 · see citing works
Using Pre-Training Can Improve Model Robustness and UncertaintyPreprint
Hendrycks, D., Lee, K., & Mazeika, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09960 · Cited by 531 · see citing works
Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation
Kamnitsas, K., Bai, W., Ferrante, E., McDonagh, S., Sinclair, M. D., Pawlowski, N., Rajchl, M., Lee, M., Kainz, B., Rueckert, D., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75238-9_38 · Cited by 485 · see citing works
Self-citationEnhancing the reliability of out-of-distribution image detection in neural networks
Liang, S. (2025) · TIB Data Manager · doi:10.57702/uwcm8hhf · Cited by 474 · see citing works
Likelihood Ratios for Out-of-Distribution DetectionPreprint
Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R. E., DePristo, M. A., Dillon, J. V., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02845 · Cited by 407 · see citing works
Self-citationPretrained Transformers Improve Out-of-Distribution Robustness
Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., & Song, D. X. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.244 · Cited by 312 · see citing works
Anomalous Example Detection in Deep Learning: A Survey
Bulusu, S., Kailkhura, B., Li, B., Varshney, P. K., & Song, D. X. (2020) · IEEE Access · doi:10.1109/access.2020.3010274 · Cited by 210 · see citing works
A baseline for detecting misclassified and out-of-distribution examples in neural networksPreprint
Hendrycks, D. (2025) · arXiv (Cornell University) · doi:10.57702/9gs720u1 · Cited by 196 · see citing works
(2020) · International Conference on Learning Representations · Cited by 168 · see citing works
Self-citationHybrid Models for Open Set Recognition
Zhang, H., Li, A., Guo, J. J., & Guo, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_7 · Cited by 151 · see citing works
Hyperparameter ensembles for robustness and uncertainty quantification
Wenzel, F., Snoek, J., Tran, D., & Jenatton, R. (2020) · arXiv · doi:10.48550/arxiv.2006.13570 · Cited by 122 · see citing works
Contrastive Training for Improved Out-of-Distribution DetectionPreprint
Winkens, J., Bunel, R., Roy, A. G., Stanforth, R., Natarajan, V., Ledsam, J. R. E., MacWilliams, P., Kohli, P., Karthikesalingam, A., Köhl, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05566 · Cited by 122 · see citing works
Self-citationNoise Contrastive Priors for Functional UncertaintyPreprint
Hafner, D., Tran, D., Lillicrap, T., Irpan, A., & Davidson, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09289 · Cited by 83 · see citing works
Uncertainty Estimation in Deep Neural Networks for Dermoscopic Image Classification
Combalia, M., Hueto, F., Puig, S., Malvehy, J., & Vilaplana, V. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00380 · Cited by 82 · see citing works
Supervised Transfer Learning at Scale for Medical ImagingPreprint
Mustafa, B., Loh, A., Freyberg, J., MacWilliams, P., Wilson, M., McKinney, S. M., Sieniek, M., Winkens, J., Liu, Y., Bui, P., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05913 · Cited by 71 · see citing works
Self-citationvan Amersfoort, J., Smith, L., Teh, Y., & Gal, Y. (2020) · International Conference on Machine Learning · doi:10.48550/arxiv.2003.02037 · Cited by 52 · see citing works
On Out-of-Distribution Detection Algorithms with Deep Neural Skin Cancer Classifiers
Pacheco, A. G. C., Sastry, C. S., Trappenberg, T. P., Oore, S., & Krohling, R. A. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00374 · Cited by 44 · see citing works
A Benchmark of Medical Out of Distribution DetectionPreprint
Cao, T., Huang, C.-W., Hui, D. Y.-T., & Cohen, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04250 · Cited by 39 · see citing works
Density of States Estimation for Out-of-Distribution DetectionPreprint
Morningstar, W. R., Ham, C., Gallagher, A., Lakshminarayanan, B., Alemi, A. A., & Dillon, J. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09273 · Cited by 29 · see citing works
Self-citationUniversum Prescription: Regularization Using Unlabeled Data
Zhang, X., & LeCun, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10768 · Cited by 27 · see citing works
Out-of-Distribution Detection for Skin Lesion Images with Deep Isolation Forest
Li, X., Lu, Y., Desrosiers, C., & Liu, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59861-7_10 · Cited by 23 · see citing works
An Effective Baseline for Robustness to Distributional Shift
Thulasidasan, S., Thapa, S., Dhaubhadel, S., Chennupati, G., Bhattacharya, T., & Bilmes, J. (2021) · International Conference on Machine Learning and Applications · doi:10.1109/icmla52953.2021.00050 · Cited by 21 · see citing works
Linmans, J., van der Laak, J., & Litjens, G. J. S. (2020) · Medical Imaging with Deep Learning · Cited by 16 · see citing works
Self-Supervised Out-of-Distribution Detection in Brain CT ScansPreprint
Venkatakrishnan, A. R., Kim, S. T., Eisawy, R., Pfister, F., & Navab, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05428 · Cited by 13 · see citing works
Calibrating Healthcare AI: Towards Reliable and Interpretable Deep Predictive ModelsPreprint
Thiagarajan, J. J., Sattigeri, P., Rajan, D., & Venkatesh, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14480 · Cited by 13 · see citing works
FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysisPreprint
Çallı, E., Murphy, K., Sogancioglu, E., & van Ginneken, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01253 · Cited by 10 · see citing works
Prototypical Clustering Networks for Dermatological Disease DiagnosisPreprint
Prabhu, V., Kannan, A., Ravuri, M., Chablani, M., Sontag, D. A., & Amatriain, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03066 · Cited by 9 · see citing works
Addressing the Real-world Class Imbalance Problem in DermatologyPreprint
Weng, W., Deaton, J., Natarajan, V., Elsayed, G. F., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04308 · Cited by 9 · see citing works
Self-citationOut-of-Distribution Detection for Skin Lesion Images with Deep Isolation ForestPreprint
Li, X., Lu, Y., Desrosiers, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09365 · Cited by 4 · see citing works
CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-raysPreprint
Shi, S., Malhi, I., Tran, K., Ng, A. Y., & Rajpurkar, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04590 · Cited by 3 · see citing works
Yasin, Y., Rumala, D. J., Purnomo, M. H., Ratna, A. A. P., Hidayati, A. N., Nurtanio, I., Rachmadi, R. F., & Purnama, I. K. E. (2020) · International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM) · doi:10.1109/cenim51130.2020.9297994 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., Song, D., 2020b. Pretrained transformers improve out-ofdistribution robustness. arXiv preprint arXiv:2004.06100 . · arXiv
- Qiao, S., Wang, H., Liu, C., Shen, W., Yuille, A., 2019. Weight standardization. arXiv preprint arXiv:1903.10520 . · arXiv
- Thulasidasan, S., Thapa, S., Dhaubhadel, S., Chennupati, G., Bhattacharya, T., Bilmes, J., 2021. A simple and effective baseline for out-of-distribution detection using abstention URL: https: //openreview.net/forum?id=q\_Q9MMGwSQu .
- Van Amersfoort, J., Smith, L., Teh, Y.W., Gal, Y., 2020. Uncertainty estimation using a single deep deterministic neural network. International Conference on Machine Learning , 9690-9700.
Cited by
Sorted by citations
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028 · see citing works
Evidential Deep Learning for Guided Molecular Property Prediction and Discovery
Soleimany, A. P., Amini, A., Goldman, S., Rus, D. L., Bhatia, S. N., & Coley, C. W. (2021) · ACS Central Science · doi:10.1021/acscentsci.1c00546 · Cited by 231 · see citing works
Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology
Daneshjou, R., Barata, C., Betz‐Stablein, B., Celebi, M. E., Codella, N., Combalia, M., Guitera, P., Gutman, D. A., Halpern, A. C., Helba, B., et al. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.4915 · Cited by 163 · see citing works
A vision transformer for decoding surgeon activity from surgical videos
Kiyasseh, D., Ma, R., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01010-8 · Cited by 151 · see citing works
Generalization—a key challenge for responsible AI in patient-facing clinical applications
Goetz, L., Seedat, N., Vandersluis, R., & van der Schaar, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01127-3 · Cited by 124 · see citing works
Mine yOur owN Anatomy: Revisiting Medical Image Segmentation With Extremely Limited Labels
You, C., Dai, W., Liu, F., Min, Y., Dvornek, N. C., Li, X., Clifton, D. A., Staib, L. H., & Duncan, J. S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3461321 · Cited by 106 · see citing works
Homeyer, A., Geißler, C., Schwen, L. O., Zakrzewski, F., Evans, T., Strohmenger, K., Westphal, M., Bülow, R. D., Kargl, M., Karjauv, A., et al. (2022) · Modern Pathology · doi:10.1038/s41379-022-01147-y · Cited by 83 · see citing works
Predictive uncertainty estimation for out-of-distribution detection in digital pathology
Linmans, J., Elfwing, S., van der Laak, J., & Litjens, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102655 · Cited by 75 · see citing works
Zhang, X., Wang, C., Zhou, W., Xu, J., & Han, T. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3387481 · Cited by 70 · see citing works
Estimating Example Difficulty using Variance of Gradients
Agarwal, C., D’souza, D., & Hooker, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01012 · Cited by 75 · see citing works
Calibrated geometric deep learning improves kinase–drug binding predictions
Luo, Y., Liu, Y., & Peng, J. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00751-0 · Cited by 67 · see citing works
Self-citationGroh, M., Harris, C., Daneshjou, R., Badri, O., & Koochek, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555634 · Cited by 66 · see citing works
Yue, G., Wei, P., Zhou, T., Jiang, Q., Yan, W., & Wang, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3204646 · Cited by 46 · see citing works
Confidence-Based Out-of-Distribution Detection: A Comparative Study and Analysis
Berger, C., Paschali, M., Glocker, B., & Kamnitsas, K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87735-4_12 · Cited by 42 · see citing works
Diffusion models for out-of-distribution detection in digital pathology
Linmans, J., Raya, G., van der Laak, J. A. W. M., & Litjens, G. J. S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103088 · Cited by 40 · see citing works
On the Impact of Spurious Correlation for Out-of-Distribution Detection
Ming, Y., Yin, H., & Li, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21244 · Cited by 37 · see citing works
Decoupling multi-task causality for improved skin lesion segmentation and classification
Song, L., Wang, H., & Wang, Z. J. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108995 · Cited by 34 · see citing works
Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
Tran, M., Schmidle, P., Guo, R. R., Wagner, S. J., Koch, V., Lupperger, V., Novotny, B., Murphree, D. H., Hardway, H. D., D’Amato, M., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-60014-x · Cited by 36 · see citing works
“Background images were detected and excluded using RGB thresh-olding and Canny edge detection.”
Yuan, M., Xia, Y., Dong, H., Chen, Z., Yao, J., Qiu, M., Yan, K., Yin, X., Shi, Y., Chen, X., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02287 · Cited by 32 · see citing works
Uncertainty-guided man–machine integrated patient-specific quality assurance
Yang, X., Li, S., Shao, Q., Cao, Y. S., Yang, Z., & Zhao, Y. (2022) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2022.05.016 · Cited by 24 · see citing works
Self-supervised multi-modality learning for multi-label skin lesion classification
Wang, H., Ahn, E., Bi, L., & Kim, J. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108729 · Cited by 25 · see citing works
ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Prompting
Jiang, Y., Huang, Z., Zhang, R., Zhang, X., & Zhang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01082 · Cited by 22 · see citing works
Shen, Y., Li, H., Sun, C., Ji, H., Zhang, D., Hu, K., Tang, Y., Chen, Y., Wei, Z., & Lv, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01014-x · Cited by 21 · see citing works
AI-Based Anomaly Detection for Clinical-Grade Histopathological Diagnostics
Dippel, J., Prenißl, N., Hense, J., Liznerski, P., Winterhoff, T., Schallenberg, S., Kloft, M., Buchstab, O., Horst, D. A., Alber, M., et al. (2024) · NEJM AI · doi:10.1056/aioa2400468 · Cited by 20 · see citing works
Out-of-Distribution Detection Algorithms for Robust Insect Classification
Saadati, M., Balu, A., Chiranjeevi, S., Jubery, T. Z., Singh, A. K., Sarkar, S., Singh, A., & Ganapathysubramanian, B. (2024) · Plant Phenomics · doi:10.34133/plantphenomics.0170 · Cited by 19 · see citing works
Latent Transformer Models for out-of-distribution detection
Graham, M., Tudosiu, P.-D., Wright, P., Pinaya, W. H. L., Teikari, P., Patel, A., U-King-im, J., Mah, Y., Teo, J., Jäger, H., et al. (2023) · Medical Image Anal. · doi:10.1016/j.media.2023.102967 · Cited by 16 · see citing works
Araújo, T., Aresta, G., Schmidt-Erfurth, U. M., & Bogunović, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-43018-9 · Cited by 18 · see citing works
LesionMix: A Lesion-Level Data Augmentation Method for Medical Image Segmentation
Basaran, B. D., Zhang, W., Qiao, M., Kainz, B., Matthews, P. M., & Bai, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-58171-7_8 · Cited by 18 · see citing works
Learning Optimal Conformal ClassifiersPreprint
Stutz, D., Krishnamurthy, Dvijotham, Cemgil, A. T., & Arnaud Doucet, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09192 · Cited by 18 · see citing works
Out-of-Distribution Detection for Long-Tailed and Fine-Grained Skin Lesion Images
Mehta, D., Gal, Y., Bowling, A., Bonnington, C. P., & Ge, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16431-6_69 · Cited by 17 · see citing works
Modality Cycles with Masked Conditional Diffusion for Unsupervised Anomaly Segmentation in MRI
Liang, Z., Anthony, H., Wagner, F., & Kamnitsas, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47425-5_16 · Cited by 17 · see citing works
From Coarse to Fine-Grained Open-Set Recognition
Lang, N., Snæbjarnarson, V., Cole, E., Aodha, O. M., Igel, C., & Belongie, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01686 · Cited by 18 · see citing works
Dangers of Bayesian Model Averaging under Covariate ShiftPreprint
Izmailov, P., Nicholson, P., Lotfi, S., & Wilson, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11905 · Cited by 16 · see citing works
Anthony, H., & Kamnitsas, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44336-7_14 · Cited by 15 · see citing works
EAT: Towards Long-Tailed Out-of-Distribution Detection
Wei, T., Wang, B.-L., & Zhang, M.-L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29508 · Cited by 15 · see citing works
Steele, L., Tan, X. L., Olabi, B., Gao, J., Tanaka, R. J., & Williams, H. C. (2022) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.18814 · Cited by 15 · see citing works
Reliable and Trustworthy Machine Learning for Health Using Dataset Shift DetectionPreprint
Park, C., Awadalla, A., Kohno, T., & Patel, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14019 · Cited by 15 · see citing works
Evidence Reconciled Neural Network for Out-of-Distribution Detection in Medical Images
Fu, W., Chen, Y., Liu, W., Yue, X., & Ma, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43898-1_30 · Cited by 12 · see citing works
Artificial Intelligence in the Non-Invasive Detection of Melanoma
Mendi, B. İ., Köse, K., Fleshner, L., Adam, R., Safai, B., Farabi, B., & Atak, M. F. (2024) · Life · doi:10.3390/life14121602 · Cited by 14 · see citing works
Wang, Y., Ni, H., Zhou, J., Liu, L., Lin, J., Yin, M., Gao, J., Zhu, S., Yin, Q., Zhu, J., et al. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01123-9 · Cited by 12 · see citing works
Feng, S., & Wang, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106485 · Cited by 10 · see citing works
Self-supervised out-of-distribution detection in wireless capsule endoscopy images
Quindós, A., Laiz, P., Vitria, J. M., & Seguí, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102606 · Cited by 10 · see citing works
Exploring Inlier and Outlier Specification for Improved Medical OOD Detection
Narayanaswamy, V., Mubarka, Y., Anirudh, R., Rajan, D., & Thiagarajan, J. J. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00493 · Cited by 10 · see citing works
Utti, V., Bikia, V., Agarwal, A. A., & Daneshjou, R. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094521 · Cited by 7 · see citing works
Mehta, D. S., Primiero, C., Betz‐Stablein, B., Nguyen, T. D., Gal, Y., Bowling, A., Haskett, M., Sashindranath, M., Bonnington, C. P., Mar, V., et al. (2025) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.20551 · Cited by 7 · see citing works
Uncertainties in outcome modelling in radiation oncology
Dünger, L., Mäusel, E., Zwanenburg, A., & Löck, S. (2025) · Physics and Imaging in Radiation Oncology · doi:10.1016/j.phro.2025.100774 · Cited by 7 · see citing works
Beyond AUROC & co. for evaluating out-of-distribution detection performance
Humblot-Renaux, G., Escalera, S., & Moeslund, T. B. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00402 · Cited by 6 · see citing works
Custom CNN architectures for skin disease classification: binary and multi-class performance
Gupta, P. K., Nirmal, J. H., & Mehendale, N. D. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20503-5 · Cited by 5 · see citing works
Vahdani, A. M., Shariatnia, M., Rajpurkar, P., & Pareek, A. (2025) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1002/ksa.12737 · Cited by 5 · see citing works
Araojo, T., Aresta, G., & Bogunović, H. (2022) · doi:10.1109/isbic56247.2022.9854763 · Cited by 5 · see citing works
Raising the Bar on the Evaluation of Out-of-Distribution Detection
Mukhoti, J., Lin, T., Chen, B.-C., Shah, A., Torr, P. H. S., Dokania, P. K., & Lim, S.-N. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00471 · Cited by 5 · see citing works
Semi-Supervised and Class-Imbalanced Open Set Medical Image Recognition
Xu, Y., Wang, R., Zhao, R.-W., Xiao, X., & Feng, R. (2024) · IEEE Access · doi:10.1109/access.2024.3442569 · Cited by 4 · see citing works
Evaluating medical AI systems in dermatology under uncertain ground truth
Stutz, D., Cemgil, A. T., Roy, A. G., Matejovicova, T., Barsbey, M., Strachan, P., Schaekermann, M., Freyberg, J., Rikhye, R. V., Freeman, B., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103556 · Cited by 6 · see citing works
Self-citationArtificial intelligence in healthcare: a perspective from Google
Lehmann, L. S., Natarajan, V., & Peng, L. H. (2023) · Elsevier eBooks · doi:10.1016/b978-0-443-15688-5.00037-1 · Cited by 5 · see citing works
Self-citationOut-of-Distribution Detection for Open-Set Semi-Supervised Medical Image Classification
Zhang, J., Cui, Y., Wu, Z., Xi, H., & Zhu, J. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733412 · Cited by 3 · see citing works
Mascagni, P., Alapatt, D., Sestini, L., Yu, T., Alfieri, S., Morales‐Conde, S., Padoy, N., & Perretta, S. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.04.006 · Cited by 3 · see citing works
Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution DataPreprint
Tian, J., Hsu, Y.-C., Shen, Y., Jin, H., & Kira, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15231 · Cited by 3 · see citing works
HiTrace: Hierarchical Class Tracing Approach for Open-Set Recognition on Skin Lesions
Hsu, B. W., & Tseng, V. S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3560555 · Cited by 2 · see citing works
Sapaico-Alberto, A. F., Olaya-Cotera, S., & Flores-Castañeda, R. O. (2024) · Archives of Dermatological Research · doi:10.1007/s00403-024-03650-5 · Cited by 2 · see citing works
On the Effectiveness of Sparsification for Detecting the Deep Unknowns.Preprint
Sun, Y., & Li, Y. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
MoMo Strategy: Learn More from More Mistakes
Chulif, S., Lee, S. H., Chang, Y. L., Tsun, M. T. K., Chai, K. C., & Then, Y. L. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317346 · Cited by 1 · see citing works
Mascagni, P., Alapatt, D., Sestini, L., Yu, T., Alfieri, S., Morales‐Conde, S., Padoy, N., & Perretta, S. (2024) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2024.04.009 · Cited by 1 · see citing works
FlowCon: Out-of-Distribution Detection Using Flow-Based Contrastive Learning
Aathreya, S., & Canavan, S. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_11 · Cited by 1 · see citing works
Efficient spatio-temporal out-of-distribution detection for learning-enabled autonomous systems
Ren, J., Li, S., Li, Y., Yang, X. P. O., Liu, L., & Li, S. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103525 · Cited by 1 · see citing works
Self-citationThopalli, K., Narayanaswamy, V., & Thiagarajan, J. J. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00162 · Cited by 1 · see citing works
A Multi-scale Framework for Out-of-Distribution Detection in Dermoscopic Images
Huang, Z., Wang, T., Cai, Y., & Liang, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-20096-0_12 · Cited by 1 · see citing works
A Feature-Based Out-of-Distribution Detection Approach in Skin Lesion Classification
Carvalho, T. M., Vellasco, M. M. B. R., Amaral, J. F., & Figueiredo, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45389-2_23 · Cited by 1 · see citing works
Unsupervised Approaches for Out-Of-Distribution Dermoscopic Lesion DetectionPreprint
Torop, M., Ghimire, S., Liu, W., Brooks, D. H., Camps, O., Rajadhyaksha, M. M., Dy, J., & Köse, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04807 · Cited by 1 · see citing works
Evidential Open Set Recognition for Imbalanced Medical Images via Multi-level Data Augmentation
Zhao, R.-W., Xu, Y., Zhang, Y., & Feng, R. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822558
CRAFT: Class Ranking Aware Fine-Tuning for Enhanced Out-of-Distribution Detection
Karunanayake, N., Seneviratne, S., & Chawla, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00405
Sayres, R., Devon-Sand, A., Schaekermann, M., Smith, M. A., Strachan, P., Hong, G., Nguyen, T., Wong, D., Hammel, N., Siegel, D. H., et al. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720531
Self-citationDynamic dual mining framework for long-tailed out-of-distribution detection
Sheng, B., Pan, D., & Li, X. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06679-9
Sáez-Maldonado, F. J., García, L., Cooper, L., Goldstein, J. A., Molina, R., & Katsaggelos, A. K. (2025) · IEEE Access · doi:10.1109/access.2025.3593420 · Cited by 2 · see citing works
Learning from limited representation
Boland, C. T., Goatman, K. A., Tsaftaris, S. A., & Dahdouh, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-32-395674-1.00019-6
High-sensitivity pan-cancer AI assessment of lymph node metastasis via uncertainty quantification
Wang, X., Chen, Y., Liu, X., Qiu, C., Tang, H., Huang, T., Guo, S., Ma, S., Cai, M., Sun, Q., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02564-y
Popescu, M. H., Roitero, K., & Mea, V. D. (2026) · JMIR AI · doi:10.2196/78764
Automatic Diagnostic Tools in Dermatology: Current State and Future Directions
Han, S. S., & Cho, S. I. (2026) · Springer eBooks · doi:10.1007/978-3-032-37598-8_11
- Authors
- Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., Vo, N., Bui, P., Winter, S., MacWilliams, P., Corrado, G. S., Telang, U., Liu, Y., Cemgil, T., Karthikesalingam, A., Lakshminarayanan, B., & Winkens, J.
- Affiliations
- Google (United States) (US)
- ORCID
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2022-01
- Volume
- 75
- Pages
- 102274
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W3142659855
- Cited by
- 77
- References
- 59
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 21 · 2023: 17 · 2022: 11 · 2021: 9
- FWCI
- 5.08
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Medical Image Analysis (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- out-of-distribution detection, abstention, representation learning, ensemble selection, subgroup analysis, cost matrix, clinical impact
- MeSH terms
- Dermatology, Humans, Benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
