The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions
Open accessAbstract
Importance: A high proportion of suspicious pigmented skin lesions referred for investigation are benign. Techniques to improve the accuracy of melanoma diagnoses throughout the patient pathway are needed to reduce the pressure on secondary care and pathology services. Objective: To determine the accuracy of an artificial intelligence algorithm in identifying melanoma in dermoscopic images of lesions taken with smartphone and digital single-lens reflex (DSLR) cameras. Design, Setting, and Participants: This prospective, multicenter, single-arm, masked diagnostic trial took place in dermatology and plastic surgery clinics in 7 UK hospitals. Dermoscopic images of suspicious and control skin lesions from 514 patients with at least 1 suspicious pigmented skin lesion scheduled for biopsy were captured on 3 different cameras. Data were collected from January 2017 to July 2018. Clinicians and the Deep Ensemble for Recognition of Malignancy, a deterministic artificial intelligence algorithm trained to identify melanoma in dermoscopic images of pigmented skin lesions using deep learning techniques, assessed the likelihood of melanoma. Initial data analysis was conducted in September 2018; further analysis was conducted from February 2019 to August 2019. Interventions: Clinician and algorithmic assessment of melanoma. Main Outcomes and Measures: Area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity of the algorithmic and specialist assessment, determined using histopathology diagnosis as the criterion standard. Results: The study population of 514 patients included 279 women (55.7%) and 484 white patients (96.8%), with a mean (SD) age of 52.1 (18.6) years. A total of 1550 images of skin lesions were included in the analysis (551 [35.6%] biopsied lesions; 999 [64.4%] control lesions); 286 images (18.6%) were used to train the algorithm, and a further 849 (54.8%) images were missing or unsuitable for analysis. Of the biopsied lesions that were assessed by the algorithm and specialists, 125 (22.7%) were diagnosed as melanoma. Of these, 77 (16.7%) were used for the primary analysis. The algorithm achieved an AUROC of 90.1% (95% CI, 86.3%-94.0%) for biopsied lesions and 95.8% (95% CI, 94.1%-97.6%) for all lesions using iPhone 6s images; an AUROC of 85.8% (95% CI, 81.0%-90.7%) for biopsied lesions and 93.8% (95% CI, 91.4%-96.2%) for all lesions using Galaxy S6 images; and an AUROC of 86.9% (95% CI, 80.8%-93.0%) for biopsied lesions and 91.8% (95% CI, 87.5%-96.1%) for all lesions using DSLR camera images. At 100% sensitivity, the algorithm achieved a specificity of 64.8% with iPhone 6s images. Specialists achieved an AUROC of 77.8% (95% CI, 72.5%-81.9%) and a specificity of 69.9%. Conclusions and Relevance: In this study, the algorithm demonstrated an ability to identify melanoma from dermoscopic images of selected lesions with an accuracy similar to that of specialists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
A test of missing completely at random for multivariate data with missing values
Little, R. J. A. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478722 · Cited by 8,963
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652
Diagnostic accuracy of dermoscopy
Kittler, H., Pehamberger, H., Wolff, K., & Binder, M. (2002) · The Lancet Oncology · doi:10.1016/s1470-2045(02)00679-4 · Cited by 1,349
Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection
Wolf, J. A., Moreau, J. F., Akilov, O., Patton, T., English, J. C., Ho, J., & Ferris, L. K. (2013) · JAMA Dermatology · doi:10.1001/jamadermatol.2013.2382 · Cited by 393
Teledermatology for diagnosing skin cancer in adults.
Chuchu, N., Dinnes, J., Takwoingi, Y., Matin, R. N., Bayliss, S. E., Davenport, C., Moreau, J. F., Bassett, O., Godfrey, K., O'Sullivan, C., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013193 · Cited by 175
Self-citationKassianos, A. P., Emery, J. D., Murchie, P., & Walter, F. M. (2015) · British Journal of Dermatology · doi:10.1111/bjd.13665 · Cited by 151
Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma
Chuchu, N., Takwoingi, Y., Dinnes, J., Matin, R. N., Bassett, O., Moreau, J. F., Bayliss, S. E., Davenport, C., Godfrey, K., O'Connell, S., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013192 · Cited by 144
Self-citationDiagnosing and managing cutaneous pigmented lesions: primary care physicians versus dermatologists.
Chen, S. C., Pennie, M. L., Kolm, P., Warshaw, E. M., Weisberg, E. L., Brown, K. M., Ming, M. E., & Weintraub, W. S. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00462.x · Cited by 122
Cited by
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Chatbots in Education and Research: A Critical Examination of Ethical Implications and Solutions
Kooli, C. (2023) · Sustainability · doi:10.3390/su15075614 · Cited by 666
Manickam, P., Mariappan, S. A., Murugesan, S. M., Hansda, S., Kaushik, A., Shinde, R., & Thipperudraswamy, S. P. (2022) · Biosensors · doi:10.3390/bios12080562 · Cited by 637
Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
Jones, O. T., Matin, R. N., van der Schaar, M., Prathivadi Bhayankaram, K., Ranmuthu, C. K. I., Islam, M. S., Behiyat, D., Boscott, R., Calanzani, N., Emery, J., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00023-1 · Cited by 272
Self-citation
Metadata
- Authors
- Phillips, M., Marsden, H., Jaffe, W., Matin, R. N., Wali, G. N., Greenhalgh, J., McGrath, E., James, R., Ladoyanni, E., Bewley, A., Argenziano, G., & Palamaras, I.
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2019-10-16
- Volume
- 2
- Issue
- 10
- Pages
- e1913436
- ISSN
- 2574-3805 (ISSN-L)
- OpenAlex
- W2980349812
- Cited by
- 223
- References
- 9
- FWCI
- 7.07
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions. https://app.cochat.ai/papers/10.1001/jamanetworkopen.2019.13436
- The paper
- Phillips, M., Marsden, H., Jaffe, W., Matin, R. N., Wali, G. N., Greenhalgh, J., McGrath, E., James, R., Ladoyanni, E., Bewley, A., Argenziano, G., & Palamaras, I. (2019). Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions. JAMA Network Open. https://doi.org/10.1001/jamanetworkopen.2019.13436
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.