Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review
Open accessAbstract
IMPORTANCE: Clinical artificial intelligence (AI) algorithms have the potential to improve clinical care, but fair, generalizable algorithms depend on the clinical data on which they are trained and tested. OBJECTIVE: To assess whether data sets used for training diagnostic AI algorithms addressing skin disease are adequately described and to identify potential sources of bias in these data sets. DATA SOURCES: In this scoping review, PubMed was used to search for peer-reviewed research articles published between January 1, 2015, and November 1, 2020, with the following paired search terms: deep learning and dermatology, artificial intelligence and dermatology, deep learning and dermatologist, and artificial intelligence and dermatologist. STUDY SELECTION: Studies that developed or tested an existing deep learning algorithm for triage, diagnosis, or monitoring using clinical or dermoscopic images of skin disease were selected, and the articles were independently reviewed by 2 investigators to verify that they met selection criteria. CONSENSUS PROCESS: Data set audit criteria were determined by consensus of all authors after reviewing existing literature to highlight data set transparency and sources of bias. RESULTS: A total of 70 unique studies were included. Among these studies, 1 065 291 images were used to develop or test AI algorithms, of which only 257 372 (24.2%) were publicly available. Only 14 studies (20.0%) included descriptions of patient ethnicity or race in at least 1 data set used. Only 7 studies (10.0%) included any information about skin tone in at least 1 data set used. Thirty-six of the 56 studies developing new AI algorithms for cutaneous malignant neoplasms (64.3%) met the gold standard criteria for disease labeling. Public data sets were cited more often than private data sets, suggesting that public data sets contribute more to new development and benchmarks. CONCLUSIONS AND RELEVANCE: This scoping review identified 3 issues in data sets that are used to develop and test clinical AI algorithms for skin disease that should be addressed before clinical translation: (1) sparsity of data set characterization and lack of transparency, (2) nonstandard and unverified disease labels, and (3) inability to fully assess patient diversity used for algorithm development and testing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fewer studies have concentrated specifically on the implications and consequences of biased AI in healthcare.
“Fewer studies have concentrated specifically on the implications and consequences of biased AI in healthcare [19–22].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [21]
Existing reviews differ in scope, depth, and focus, leaving a gap in understanding how bias permeates the development of AI for clinical decisions.
“While these reviews have made valuable contributions to our understanding of bias in medical AI, they differ in scope, depth, and focus, leaving a gap in understanding how bias permeates the development of AI for clinical decisions.”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [21]
Bias in AI algorithms can manifest as selection or information bias, potentially leading to unfair and inaccurate outcomes.
“This bias in AI algorithms can manifest in various ways such as selection or information bias, potentially leading to unfair and inaccurate outcomes [10].”
From Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education (2024) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Tschandl, P., Rosendahl, C., & Kittler, H. (2018) · Scientific Data · doi:10.1038/sdata.2018.161 · Cited by 3,628 · see citing works
Scoping reviews: Time for clarity in definition, methods, and reporting
Colquhoun, H. L., Levac, D., O'Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., Kastner, M., & Moher, D. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.03.013 · Cited by 2,783 · see citing works
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy166 · Cited by 1,571 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024 · see citing works
Codella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.03368 · Cited by 984 · see citing works
Self-citationA deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853 · see citing works
Hellström (2019) · doi:10.21037/atm · Cited by 798 · see citing works
Han, S. S., Kim, M. S., Lim, W., Park, G., Park, I., & Chang, S. E. (2018) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2018.01.028 · Cited by 715 · see citing works
Machine Learning and Health Care Disparities in Dermatology
Adamson, A. S., & Smith, A. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.2348 · Cited by 639 · see citing works
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Holland-Letz, T., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.04.001 · Cited by 605 · see citing works
Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30333-x · Cited by 555 · see citing works
Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D. E., & Zou, J. (2021) · Nature Medicine · doi:10.1038/s41591-021-01312-x · Cited by 494 · see citing works
Self-citationClinical applications of machine learning algorithms: beyond the black box
Watson, D. S., Krutzinna, J., Bruce, I. N., Griffiths, C. E., McInnes, I. B., Barnes, M. R., & Floridi, L. (2019) · BMJ · doi:10.1136/bmj.l886 · Cited by 450 · see citing works
Deep neural networks are superior to dermatologists in melanoma image classification
Brinker, T. J., Hekler, A., Enk, A. H., Berking, C., Haferkamp, S., Hauschild, A., Weichenthal, M., Klode, J., Schadendorf, D., Holland-Letz, T., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.05.023 · Cited by 415 · see citing works
Melanoma detection by analysis of clinical images using convolutional neural network
Nasr-Esfahani, E., Samavi, S., Karimi, N., Soroushmehr, S. M. R., Jafari, M. H., Ward, K., & Najarian, K. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7590963 · Cited by 387 · see citing works
Al‐masni, M. A., Kim, D., & Kim, T. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105351 · Cited by 385 · see citing works
MED-NODE: A computer-assisted melanoma diagnosis system using non-dermoscopic images
Giotis, I., Molders, N., Land, S., Biehl, M. L., Jonkman, M. F., & Petkov, N. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.04.034 · Cited by 365 · see citing works
Winkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1735 · Cited by 365 · see citing works
Superior skin cancer classification by the combination of human and artificial intelligence
Hekler, A., Utikal, J. S., Enk, A. H., Hauschild, A., Weichenthal, M., Maron, R. C., Berking, C., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.07.019 · Cited by 357 · see citing works
Bluemke, D. A., Moy, L., Bredella, M. A., Ertl-Wagner, B. B., Fowler, K. J., Goh, V. J., Halpern, E. F., Hess, C. P., Schiebler, M. L., & Weiss, C. R. (2020) · Radiology · doi:10.1148/radiol.2019192515 · Cited by 343 · see citing works
Marchetti, M. A., Codella, N., Dusza, S. W., Gutman, D. A., Helba, B., Kalloo, A., Mishra, N. K., Carrera, C., Celebi, M. E., DeFazio, J., et al. (2017) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2017.08.016 · Cited by 329 · see citing works
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Tschandl, P., Rosendahl, C. O., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J. F., Lallas, A., et al. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.4378 · Cited by 327 · see citing works
Multimodal skin lesion classification using deep learning
Yap, J., Yolland, W., & Tschandl, P. (2018) · Experimental Dermatology · doi:10.1111/exd.13777 · Cited by 326 · see citing works
Fujisawa, Y., Otomo, Y., Ogata, Y., Nakamura, Y., Fujita, R., Ishitsuka, Y., Watanabe, R., Okiyama, N., Ohara, K., & Fujimoto, M. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16924 · Cited by 316 · see citing works
Han, S. S., Park, G. H., Lim, W., Kim, M. S., Na, J. I., Park, I., & Chang, S. E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191493 · Cited by 305 · see citing works
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Fröhling, S., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.02.005 · Cited by 304 · see citing works
A GAN-based image synthesis method for skin lesion classification
Qin, Z., Liu, Z., Zhu, P., & Xue, Y. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105568 · Cited by 300 · see citing works
Ballerini, L., Fisher, R. B., Aldridge, B., & Rees, J. (2012) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-94-007-5389-1_4 · Cited by 289 · see citing works
Skin typing: Fitzpatrick grading and others
Gupta, V., & Sharma, V. (2019) · Clinics in Dermatology · doi:10.1016/j.clindermatol.2019.07.010 · Cited by 289 · see citing works
Jinnai, S., Yamazaki, N., Hirano, Y., Sugawara, Y., Ohe, Y., & Hamamoto, R. (2020) · Biomolecules · doi:10.3390/biom10081123 · Cited by 276 · see citing works
Freeman, K., Dinnes, J., Chuchu, N., Takwoingi, Y., Bayliss, S. E., Matin, R. N., Jain, A., Walter, F. M., Williams, H. C., & Deeks, J. J. (2020) · BMJ · doi:10.1136/bmj.m127 · Cited by 260 · see citing works
Saba, T., Khan, M. A., Rehman, A. U., & Larabi-Marie-Sainte, S. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1413-3 · Cited by 250 · see citing works
Han, S. S., Park, I., Eun Chang, S., Lim, W., Kim, M. S., Park, G. H., Chae, J. B., Huh, C. H., & Na, J.-I. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.01.019 · Cited by 248 · see citing works
Haenssle, H. A., Fink, C., Toberer, F., Winkler, J. K., Stolz, W., Deinlein, T., Hofmann‐Wellenhof, R., Lallas, A., Emmert, S., Buhl, T., et al. (2020) · Annals of Oncology · doi:10.1016/j.annonc.2019.10.013 · Cited by 237 · see citing works
Phillips, M. R., Marsden, H., Jaffe, W., Matin, R., Wali, G. N., Greenhalgh, J., McGrath, E., James, R., Ladoyanni, E., Bewley, A., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.13436 · Cited by 231 · see citing works
Maron, R. C., Weichenthal, M., Utikal, J. S., Hekler, A., Berking, C., Hauschild, A., Enk, A. H., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.06.013 · Cited by 219 · see citing works
Premaladha, J., & Ravichandran, K. S. (2016) · Journal of Medical Systems · doi:10.1007/s10916-016-0460-2 · Cited by 208 · see citing works
Keratinocytic Skin Cancer Detection on the Face Using Region-Based Convolutional Neural Network
Han, S. S., Moon, I. J., Lim, W., Suh, I. S., Lee, S. Y., Na, J., Kim, S. H., & Chang, S. E. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.3807 · Cited by 147 · see citing works
Huang, H.-Y., Hsu, B. W., Lee, C., & Tseng, V. S. M. (2020) · The Journal of Dermatology · doi:10.1111/1346-8138.15683 · Cited by 131 · see citing works
Acral melanoma detection using a convolutional neural network for dermoscopy images
Yu, C., Yang, S., Kim, W. O., Jung, J. W., Chung, K. Y., Lee, S. W., & Oh, B. H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193321 · Cited by 130 · see citing works
A deep learning, image based approach for automated diagnosis for inflammatory skin diseases
Wu, H., Yin, H., Chen, H., Sun, M., Liu, X., Yu, Y., Tang, Y., Long, H., Zhang, B., Zhang, J., et al. (2020) · Annals of Translational Medicine · doi:10.21037/atm.2020.04.39 · Cited by 129 · see citing works
Self-citationTowards improving diagnosis of skin diseases by combining deep neural network and human knowledge
Zhang, X., Wang, S., Liu, J., & Tao, C. (2018) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-018-0631-9 · Cited by 126 · see citing works
The effects of skin lesion segmentation on the performance of dermatoscopic image classification
Mahbod, A., Tschandl, P., Langs, G., Ecker, R., & Ellinger, I. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105725 · Cited by 118 · see citing works
Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
Yun, S., Oh, S. J., Heo, B., Han, D., Choe, J., & Chun, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00237 · Cited by 115 · see citing works
Winkler, J. K., Sies, K., Fink, C., Toberer, F., Enk, A. H., Deinlein, T., Hofmann‐Wellenhof, R., Thomas, L., Lallas, A., Blum, A., et al. (2020) · European Journal of Cancer · doi:10.1016/j.ejca.2019.11.020 · Cited by 112 · see citing works
Marchetti, M. A., Liopyris, K., Dusza, S. W., Codella, N., Gutman, D. A., Helba, B., Kalloo, A., Halpern, A. C., Soyer, H. P., Curiel‐Lewandrowski, C. N., et al. (2019) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2019.07.016 · Cited by 106 · see citing works
Melanoma detection using adversarial training and deep transfer learning
Zunair, H., & Hamza, A. B. (2020) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ab86d3 · Cited by 104 · see citing works
Deep learning based classification of facial dermatological disorders
Göçeri, E. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104118 · Cited by 97 · see citing works
Maron, R. C., Utikal, J. S., Hekler, A., Hauschild, A., Sattler, E., Sondermann, W., Haferkamp, S., Schilling, B., Heppt, M. V., Jansen, P., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/18091 · Cited by 87 · see citing works
Pangti, R., Mathur, J., Chouhan, V., Kumar, S. M., Rajput, L., Shah, S., Gupta, A., Dixit, A., Dholakia, D., Gupta, S., et al. (2020) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16967 · Cited by 87 · see citing works
Celebi, M., & Schaefer, G. (2012) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-94-007-5389-1 · Cited by 85 · see citing works
Diagnostic accuracy of content‐based dermatoscopic image retrieval with deep classification features
Tschandl, P., Argenziano, G., Razmara, M., & Yap, J. (2018) · British Journal of Dermatology · doi:10.1111/bjd.17189 · Cited by 85 · see citing works
Seité, S., Khammari, A., Benzaquen, M., Moyal, D. D., & Dréno, B. (2019) · Experimental Dermatology · doi:10.1111/exd.14022 · Cited by 82 · see citing works
Data augmentation in dermatology image recognition using machine learning
Aggarwal, S. L. P. (2019) · Skin Research and Technology · doi:10.1111/srt.12726 · Cited by 80 · see citing works
Automated detection of erythema migrans and other confounding skin lesions via deep learning
Burlina, P., Joshi, N., Ng, E., Billings, S., Rebman, A. W., & Aucott, J. N. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.12.007 · Cited by 74 · see citing works
Winters, N., Langer, L., & Geniets, A. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019827 · Cited by 71 · see citing works
Zhao, S., Xie, B., Li, Y., Zhao, X., Kuang, Y., Su, J., He, X., Wu, X., Fan, W., Huang, K., et al. (2019) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.15965 · Cited by 71 · see citing works
Fink, C., Blum, A., Buhl, T., Mitteldorf, C., Hofmann‐Wellenhof, R., Deinlein, T., Stolz, W., Trennheuser, L., Cussigh, C., Deltgen, D., et al. (2020) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16165 · Cited by 71 · see citing works
Phillips, M. R., Greenhalgh, J., Marsden, H., & Palamaras, I. (2019) · Dermatology Practical & Conceptual · doi:10.5826/dpc.1001a11 · Cited by 71 · see citing works
Muñoz‐López, C., Ramírez‐Cornejo, C., Marchetti, M. A., Han, S. S., Del Barrio‐Díaz, P., Jaque, A., Uribe, P., Majerson, D., Curi, M., Del Puerto, C., et al. (2021) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16979 · Cited by 70 · see citing works
Evaluation of the Number-Needed-to-Biopsy Metric for the Diagnosis of Cutaneous Melanoma
Nelson, K. C., Swetter, S. M., Saboda, K., Chen, S. C., & Curiel‐Lewandrowski, C. N. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1514 · Cited by 69 · see citing works
Deep Learning for Diagnostic Binary Classification of Multiple-Lesion Skin Diseases
Thomsen, K. G., Christensen, A. L., Iversen, L., Lomholt, H. B., & Winther, O. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.574329 · Cited by 68 · see citing works
Effects of Label Noise on Deep Learning-Based Skin Cancer Classification
Hekler, A., Kather, J. N., Krieghoff-Henning, E., Utikal, J. S., Meier, F., Gellrich, F. F., Upmeier zu Belzen, J., French, L., Schlager, J. G., Ghoreschi, K., et al. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00177 · Cited by 58 · see citing works
Ros‐NET: A deep convolutional neural network for automatic identification of rosacea lesions
Binol, H., Plotner, A., Sopkovich, J., Kaffenberger, B., Niazi, M. K. K., & Gurcan, M. N. (2020) · Skin Research and Technology · doi:10.1111/srt.12817 · Cited by 56 · see citing works
A superpixel-driven deep learning approach for the analysis of dermatological wounds
Blanco, G., Traina, A. J. M., Traina, C., de Azevedo‐Marques, P. M., Jorge, A. E. S., de Oliveira, D., & Bedo, M. V. N. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105079 · Cited by 54 · see citing works
Yang, Y., Ge, Y., Guo, L., Wu, Q., Peng, L., Zhang, E., Xie, J., Li, Y., & Lin, T. (2020) · Skin Research and Technology · doi:10.1111/srt.12911 · Cited by 53 · see citing works
Assessing the effectiveness of artificial intelligence methods for melanoma: A retrospective review
Cui, X., Wei, R., Gong, L., Qi, R., Zhao, Z., Chen, H., Song, K., Abdulrahman, A. A., Wang, Y., Chen, J. Z. S., et al. (2019) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2019.06.042 · Cited by 51 · see citing works
Kim, Y. J., Han, S. S., Yang, H. J., & Chang, S. E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234334 · Cited by 47 · see citing works
Real-time burn depth assessment using artificial networks: a large-scale, multicentre study
Wang, Y., Zuo, K., He, Z., et al. (2020) · Burns · doi:10.1016/j.burns.2020.07.010 · Cited by 47 · see citing works
Sies, K., Winkler, J. K., Fink, C., Bardehle, F., Toberer, F., Buhl, T., Enk, A., Blum, A., Rosenberger, A., & Haenssle, H. A. (2020) · European Journal of Cancer · doi:10.1016/j.ejca.2020.04.043 · Cited by 46 · see citing works
Lucius, M., All, J. D., All, J. A. D., Belvisi, M., Radizza, L., Lanfranconi, M., Lorenzatti, V., & Galmarini, C. M. (2020) · Diagnostics · doi:10.3390/diagnostics10110969 · Cited by 42 · see citing works
(1988) · Annals of Internal Medicine · doi:10.7326/0003-4819-109-3-260_1 · Cited by 40 · see citing works
Lee, S., Lee, J. W., Choe, S. J., Yang, S., Koh, S. B., Ahn, Y. S., & Lee, W.-S. (2020) · JAMA Dermatology · doi:10.1001/jamadermatol.2020.2188 · Cited by 35 · see citing works
Clinical photography in skin of colour: tips and best practices
Lester, J., Clark, L., Linos, E., & Daneshjou, R. (2021) · British Journal of Dermatology · doi:10.1111/bjd.19811 · Cited by 35 · see citing works
Self-citationDulmage, B., Tegtmeyer, K., Zhang, M. Z., Colavincenzo, M. L., & Xu, S. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.08.027 · Cited by 35 · see citing works
From Deep Learning Towards Finding Skin Lesion Biomarkers
Li, X., Wu, J., Chen, E. Z., & Jiang, H. (2019) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857334 · Cited by 34 · see citing works
Brinker, T. J., Hekler, A., Enk, A. H., & Kalle, C. V. (2019) · PLoS ONE · doi:10.1371/journal.pone.0218713 · Cited by 34 · see citing works
How to evaluate deep learning for cancer diagnostics – factors and recommendations
Daneshjou, R., He, B. D., Ouyang, D., & Zou, J. (2021) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2021.188515 · Cited by 34 · see citing works
Self-citationZhao, X., Wu, X. W., Li, F., et al. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1414-2 · Cited by 32 · see citing works
Clinically relevant vulnerabilities of deep machine learning systems for skin cancer diagnosis
Du-Harpur, X., Arthurs, C., Ganier, C., Woolf, R., Laftah, Z., Lakhan, M., Salam, A., Wan, B., Watt, F. M., Luscombe, N. M., et al. (2021) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.07.034 · Cited by 31 · see citing works
Diagnostic performance of the MelaFind device in a real‐life clinical setting
Fink, C., Jaeger, C., Jaeger, K., et al. (2017) · JDDG Journal der Deutschen Dermatologischen Gesellschaft · doi:10.1111/ddg.13220 · Cited by 30 · see citing works
Wang, S., Zhang, X.-Y., Liu, J., Tao, C., Zhu, C., Shu, C., Xu, T., & Jin, H. (2020) · Chinese Medical Journal · doi:10.1097/cm9.0000000000001023 · Cited by 30 · see citing works
Multiclass Artificial Intelligence in Dermatology: Progress but Still Room for Improvement
Navarrete‐Dechent, C., Liopyris, K., & Marchetti, M. A. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.06.040 · Cited by 29 · see citing works
Taylor, M., Liu, X., Denniston, A. K., Esteva, A., Ko, J., Daneshjou, R., & Chan, A. (2021) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2021.02.744 · Cited by 28 · see citing works
Self-citationTschandl, P., Kittler, H., & Argenziano, G. (2017) · British Journal of Dermatology · doi:10.1111/bjd.15695 · Cited by 25 · see citing works
Li, C., Fei, W., Shen, C., Wang, Z., Jing, Y., Meng, R., & Cui, Y. (2020) · Chinese Medical Journal · doi:10.1097/cm9.0000000000001002 · Cited by 25 · see citing works
Minagawa, A., Koga, H., Sano, T., et al. (2020) · The Journal of Dermatology · doi:10.1111/1346-8138.15640 · Cited by 25 · see citing works
Keiser, E., Linos, E., Kanzler, M., Lee, W., Sainani, K. L., & Tang, J. Y. (2012) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2011.02.044 · Cited by 20 · see citing works
Chin, Y. P., Hou, Z. Y., Lee, M.-Y., Chu, H. M., Wang, H. H., Lin, Y. T., Gittin, A., Chien, S.-C., Nguyễn, P. A., Li, L. C., et al. (2020) · British Journal of Dermatology · doi:10.1111/bjd.18859 · Cited by 17 · see citing works
An online platform for interactive feedback in biomedical machine learning
Abid, A., Ali, A., Abid, A., Khan, D., Alfozan, A., & Zou, J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0147-8 · Cited by 14 · see citing works
Self-citationCarvalho, N. D., Welzel, J., Schuh, S., et al. (2018) · Experimental Dermatology · doi:10.1111/exd.2018.27.issue-11 · Cited by 9 · see citing works
Pangti, R., Chouhan, V., Mathur, J., et al. (2020) · International Journal of Dermatology · doi:10.1111/ijd.15242 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [3]Holland S, Hosny A, Newman S, Joseph J, Chmielinski K. The dataset nutrition label: a framework to drive higher data quality standards. arXiv. Preprint posted online May 9, 2018. 1805.03677.
- [4]Gebru T, Morgenstern J, Vecchione B, Datasheets for datasets. arXiv. Preprint posted online March 19, 2020. 1803.09010.
- [5]Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009: 248– 255.
- [85]Kinyanjui NM, Odonga T, Cintas C, Estimating skin tone and effects on classification performance in dermatology datasets. Paper presented at: NeurIPS 2019 Workshop on Fair ML for Health; Vancouver, Canada; 2019.
Cited by
Sorted by citations
Bias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607 · see citing works
Disparities in dermatology AI performance on a diverse, curated clinical image set
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. (2022) · Science Advances · doi:10.1126/sciadv.abq6147 · Cited by 544 · see citing works
Self-citationResnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228 · see citing works
“As will be discussed below, AI systems can make systemic and random errors [70, 174].”
Deep learning-aided decision support for diagnosis of skin disease across skin tones
Groh, M., Badri, O., Daneshjou, R., Koochek, A., Harris, C., Soenksen, L. R., Doraiswamy, P. M., & Picard, R. (2024) · Nature Medicine · doi:10.1038/s41591-023-02728-3 · Cited by 160 · see citing works
Self-citationFranco D’Souza, R., Mathew, M., Mishra, V., & Surapaneni, K. M. (2024) · Medical Education Online · doi:10.1080/10872981.2024.2330250 · Cited by 147 · see citing works
Abdelwanis, M., Alarafati, H. K., Tammam, M. M. S., & Simsekler, M. C. E. (2024) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2024.06.001 · Cited by 137 · see citing works
Transparent medical image AI via an image–text foundation model grounded in medical literature
Kim, C., Gadgil, S. U., DeGrave, A. J., Omiye, J. A., Cai, Z. R., Daneshjou, R., & Lee, S.-I. (2024) · Nature Medicine · doi:10.1038/s41591-024-02887-x · Cited by 125 · see citing works
Self-citationTasci, E., Zhuge, Y., Camphausen, K., & Krauze, A. V. (2022) · Cancers · doi:10.3390/cancers14122897 · Cited by 117 · see citing works
Systematic Review of Deep Learning Image Analyses for the Diagnosis and Monitoring of Skin Disease
Choy, S. P., Kim, B. J., Paolino, A., Tan, W. R., Lim, S. M. L., Seo, J., Tan, S. P., Francis, L., Tsakok, T., Simpson, M., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00914-8 · Cited by 105 · see citing works
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76 · see citing works
The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective
Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P. (2024) · Life · doi:10.3390/life14060652 · Cited by 76 · see citing works
Weidener, L., & Fischer, M. (2023) · JMIR Medical Education · doi:10.2196/46428 · Cited by 70 · see citing works
Review on Federated Learning for digital transformation in healthcare through big data analytics
Babar, M., Qureshi, B., & Koubaa, A. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.05.046 · Cited by 55 · see citing works
Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000390 · Cited by 61 · see citing works
Sangers, T. E., Wakkee, M., Moolenburgh, F. J., Nijsten, T., & Lugtenberg, M. (2022) · Archives of Dermatological Research · doi:10.1007/s00403-022-02492-3 · Cited by 38 · see citing works
Vodrahalli, K., Ko, J., Chiou, A. S., Novoa, R., Abid, A., Phung, M., Yekrang, K., Petrone, P., Zou, J., & Daneshjou, R. (2023) · JAMA Dermatology · doi:10.1001/jamadermatol.2023.0091 · Cited by 30 · see citing works
Self-citationIgbinovia, M. O., & Mensah Danquah, M. (2025) · Technical Services Quarterly · doi:10.1080/07317131.2025.2512282 · Cited by 22 · see citing works
The complex ethics of applying ChatGPT and language model artificial intelligence in dermatology
Ferreira, A. L., & Lipoff, J. B. (2023) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2023.05.054 · Cited by 18 · see citing works
Slavych, B. K., Atcherson, S. R., & Zraick, R. (2024) · Perspectives of the ASHA Special Interest Groups · doi:10.1044/2024_persp-23-00167 · Cited by 16 · see citing works
- Authors
- Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J.
- Affiliations
- Stanford Medicine (US)
- Memorial Sloan Kettering Cancer Center (US)
- Icahn School of Medicine at Mount Sinai (US)
- Chan Zuckerberg Biohub San Francisco (US)
- Venue
- JAMA Dermatology
- Publisher
- American Medical Association (AMA)
- Published
- 2021-11-01
- Volume
- 157
- Issue
- 11
- Pages
- 1362
- ISSN
- 2168-6068 (ISSN-L)
- PMC
- PMC9379852
- OpenAlex
- W3199500316
- Cited by
- 403
- References
- 94
- Citations by year
- 2026: 97 · 2025: 132 · 2024: 89 · 2023: 61 · 2022: 20 · 2021: 4
- FWCI
- 3.59
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- JAMA Dermatology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (U01MH098953)
- National Cancer Institute (P30CA008748)
- National Institute of Arthritis and Musculoskeletal and Skin Diseases (T32AR007422)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- clinical AI, skin diseases, dermoscopic images, deep learning, training data transparency, algorithmic bias, skin cancer diagnosis, patient diversity, skin tone, algorithmic fairness
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Neoplasms, Triage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review