Differences between human and machine perception in medical diagnosis
Open accessAbstract
Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since they can fail for reasons unrelated to underlying pathology. Humans are less likely to make such superficial mistakes, since they use features that are grounded on medical science. It is therefore important to know whether DNNs use different features than humans. Towards this end, we propose a framework for comparing human and machine perception in medical diagnosis. We frame the comparison in terms of perturbation robustness, and mitigate Simpson’s paradox by performing a subgroup analysis. The framework is demonstrated with a case study in breast cancer screening, where we separately analyze microcalcifications and soft tissue lesions. While it is inconclusive whether humans and DNNs use different features to detect microcalcifications, we find that for soft tissue lesions, DNNs rely on high frequency components ignored by radiologists. Moreover, these features are located outside of the region of the images found most suspicious by radiologists. This difference between humans and machines was only visible through subgroup analysis, which highlights the importance of incorporating medical domain knowledge into the comparison.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An ML model based on convolutional neural networks has demonstrated near-physician accuracy in diagnosing atopic dermatitis.
“An ML model based on convolutional neural networks (CNN) has demonstrated near-physician accuracy in diagnosing atopic dermatitis.”
From Artificial Intelligence for Drug Discovery: Are We There Yet? (2024) · cites this paper as [43]
The Abstract says deep neural networks have achieved human-level performance in various image-based medical diagnosis tasks and gives a dermatologist-level skin cancer classifier as an example. It does not report near-physician accuracy for diagnosing atopic dermatitis; searches for atopic dermatitis and eczema found no such passage. The paper’s own case study concerns breast cancer screening. (LLM, conf=0.85)
Experimental — accuracy not yet measuredThis CNN model performs well in differentiating atopic dermatitis from other skin conditions such as impetigo, mycosis fungoides, herpes simplex, and Kaposi varicelliform eruption.
“Furthermore, this CNN model performs well in differentiating atopic dermatitis from other skin conditions such as impetigo, mycosis fungoides, herpes simplex, and Kaposi varicelliform eruption (43).”
From Artificial Intelligence for Drug Discovery: Are We There Yet? (2024) · cites this paper as [43]
The Abstract and Results describe a breast-cancer screening case study, not a CNN differentiating atopic dermatitis from the listed skin conditions. A skin-disease differential-diagnosis study appears in the References, but our copy does not report the claimed performance; searches for the named conditions found no supporting passage. No page numbers are available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
<i>Stan</i> : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457 · see citing works
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · see citing works
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930 · see citing works
Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0447-x · Cited by 2,167 · see citing works
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · see citing works
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · see citing works
A deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853 · see citing works
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Wu, N., Phang, J., Park, J., Shen, Y., Huang, Z., Zorin, M., Jastrzȩbski, S., Févry, T., Katsnelson, J., Kim, E. S., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2945514 · Cited by 736 · see citing works
Self-citationRodriguez-Ruiz, A., Lång, K., Gubern-Merida, A., Broeders, M., Gennaro, G., Clauser, P., Helbich, T. H., Chevalier, M., Tan, T., Mertelmeier, T., et al. (2019) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djy222 · Cited by 692 · see citing works
Deep neural network improves fracture detection by clinicians
Lindsey, R., Daluiski, A., Chopra, S., Lachapelle, A. S., Mozer, M. C., Sicular, S., Hanel, D. P., Gardner, M. J., Gupta, A., Hotchkiss, R., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806905115 · Cited by 667 · see citing works
Kim, H.-E., Kim, H. H., Han, B., Kim, K., Han, K., Nam, H., Lee, E. H., & Kim, E. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30003-0 · Cited by 538 · see citing works
Self-citationGeneralisation in humans and deep neural networksPreprint
Geirhos, R., Temme, C. R. M., Rauber, J., Schütt, H. H., Bethge, M., & Wichmann, F. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08750 · Cited by 522 · see citing works
A Study and Comparison of Human and Deep Learning Recognition Performance under Visual Distortions
Dodge, S., & Karam, L. J. (2017) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn.2017.8038465 · Cited by 434 · see citing works
Hidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411 · see citing works
Schaffter, T., Buist, D. S. M., Lee, C. I., Nikulin, Y., Ribli, D., Guan, Y., Lotter, W., Jie, Z., Du, H., Wang, S., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.0265 · Cited by 409 · see citing works
Self-citationWinkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1735 · Cited by 365 · see citing works
Shen, Y., Wu, N., Phang, J., Park, J., Liu, K., Tyagi, S., Heacock, L. C., Kim, S., Moy, L. P., Cho, K., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101908 · Cited by 222 · see citing works
Self-citationComment: understanding Simpson’s paradox
Pearl, J. (2014) · The American Statistician · doi:10.1080/00031305.2014.876829 · Cited by 206 · see citing works
Measuring the tendency of CNNs to Learn Surface Statistical RegularitiesPreprint
Jo, J., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11561 · Cited by 201 · see citing works
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural NetworksPreprint
Geras, K. J., Wolfson, S., Shen, Y., Wu, N., Kim, S.-G., Kim, E. S., Heacock, L. C., Parikh, U., Moy, L. P., & Cho, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07047 · Cited by 189 · see citing works
Self-citationPerformance vs. competence in human–machine comparisons
Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905334117 · Cited by 150 · see citing works
Five points to check when comparing visual perception in humans and machines
Funke, C. M., Borowski, J., Stosio, K., Brendel, W., Wallis, T. S. A., & Bethge, M. (2021) · Journal of Vision · doi:10.1167/jov.21.3.16 · Cited by 98 · see citing works
Model Patching: Closing the Subgroup Performance Gap with Data AugmentationPreprint
Goel, K., Gu, A., Li, Y., & Ré, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.06775 · Cited by 69 · see citing works
Globally-Aware Multiple Instance Classifier for Breast Cancer Screening
Shen, Y., Wu, N., Phang, J., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32692-0_3 · Cited by 33 · see citing works
Emergent Properties of Foveated Perceptual SystemsPreprint
Deza, A., & Konkle, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07991 · Cited by 29 · see citing works
Mammographic Artifacts on Full-Field Digital Mammography
Choi, J. J., Kim, S. H., Kang, B. J., et al. (2013) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-013-9641-4 · Cited by 12 · see citing works
Abdullah, A. K., Kelly, J., Thompson, J. D., et al. (2017) · British Journal of Radiology · doi:10.1259/bjr.20160871 · Cited by 12 · see citing works
Wu, N., Huang, Z., Shen, Y., Park, J., Phang, J., Makino, T., Kim, S., Cho, K., Heacock, L. C., Moy, L. P., et al. (2021) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-021-00530-6 · Cited by 12 · see citing works
Self-citationImproving localization-based approaches for breast cancer screening exam classificationPreprint
Févry, T., Phang, J., Wu, N., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00615 · Cited by 8 · see citing works
Wu, N., Jastrzȩbski, S., Park, J., et al. (2020) · PubMed · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 9
- [1]Krizhevsky, A., Sutskever, I., & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In NIPS 1106–1114 (2012).
- [2]Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In ICLR (2015).
- [23]Szegedy, C. et al. Intriguing properties of neural networks. In ICLR (2014).
- [27]Hendrycks, D., & Dietterich, T. G. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR (2019).
- [28]Yin D Lopes RG Shlens J Cubuk ED Gilmer J A fourier perspective on model robustness in computer vision NeurIPS. 2019 32 13255 13265
- [30]Sagawa, S., Koh, P. W., Hashimoto, T. B., & Liang, P. Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case generalization. In ICLR (2020).
- [36]Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In ICLR. (2019).
- [44]Kull, M., Perelló-Nieto, M., Kängsepp, M., Menezes e Silva Filho, T. de, Song, H., & Flach, P. A. Beyond temperature scaling: obtaining well-calibrated multi-class probabilities with Dirichlet calibration. In NeurIPS. 12295–12305. (2019)
- [45]Kucukelbir, A., Ranganath, R., Gelman, A., & Blei, D. Automatic variational inference in Stan. In NIPS. 568–576. (2015).
Cited by
Sorted by citations
Artificial Intelligence for Drug Discovery: Are We There Yet?
Hasselgren, C., & Oprea, T. I. (2024) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-040323-040828 · Cited by 236 · see citing works
“framework may be needed for comparing human and AI perception in medical diagnosis (42).”
BackgroundExplainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01657-x · Cited by 227 · see citing works
Product liability for defective AI
Buiten, M. C. (2024) · European Journal of Law and Economics · doi:10.1007/s10657-024-09794-z · Cited by 41 · see citing works
- Authors
- Makino, T., Jastrzębski, S., Oleszkiewicz, W., Chacko, C., Ehrenpreis, R., Samreen, N., Chhor, C., Kim, E., Lee, J., Pysarenko, K., Reig, B., Toth, H., Awal, D., Du, L., Kim, A., Park, J., Sodickson, D. K., Heacock, L., Moy, L., Cho, K., & Geras, K. J.
- Affiliations
- NYU Langone Health (US)
- Warsaw University of Technology (PL)
- New York University (US)
- Corresponding author
- Makino, Taro
- Geras, Krzysztof J.
- ORCID
- Venue
- Scientific Reports
- Published
- 2022-04-27
- Volume
- 12
- Issue
- 1
- Pages
- 6877-6877
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC9046399
- OpenAlex
- W3106683985
- Cited by
- 30
- References
- 38
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 5 · 2023: 4 · 2022: 7 · 2021: 1 · 2020: 2
- FWCI
- 2.06
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (HDR-1922658)
- National Institutes of Health (P41EB017183, R21CA225175)
- Gordon and Betty Moore Foundation (9683)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- deep neural network, medical image diagnosis, breast cancer screening, microcalcifications, soft tissue lesions, robustness to disturbances, Simpson's paradox, subgroup analysis
- MeSH terms
- Radiologists, Breast Neoplasms, Calcinosis, Female, Humans, Perception, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
