Cited by 144Cutaneous Melanoma Detection and Managementmelanin and skin pigmentationAI in cancer detection

The DOI matches its Crossref record; 82 of the 88 references it cites were found in a registry or an index; none of the 8 sources its claims cite is checked for support yet.

Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma

Open access

Chuchu, Naomi, Takwoingi, Yemisi, Dinnes, Jacqueline, Matin, Rubeta N, Bassett, Oliver, Moreau, Jacqueline F, Bayliss, Susan E, Davenport, Clare, Godfrey, Kathie, O'Connell, Susan, Jain, Abhilash, Walter, Fiona M, Deeks, Jonathan J, Williams, Hywel C, Cochrane Skin Cancer Diagnostic Test Accuracy Group·2018·Cochrane Database of Systematic Reviews

DOI 10.1002/14651858.cd013192PMC PMC6517294

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Melanoma accounts for a small proportion of all skin cancer cases but is responsible for most skin cancer-related deaths. Early detection and treatment can improve survival. Smartphone applications are readily accessible and potentially offer an instant risk assessment of the likelihood of malignancy so that the right people seek further medical attention from a clinician for more detailed assessment of the lesion. There is, however, a risk that melanomas will be missed and treatment delayed if the application reassures the user that their lesion is low risk. OBJECTIVES: To assess the diagnostic accuracy of smartphone applications to rule out cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults with concerns about suspicious skin lesions. SEARCH METHODS: We undertook a comprehensive search of the following databases from inception to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA: Studies of any design evaluating smartphone applications intended for use by individuals in a community setting who have lesions that might be suspicious for melanoma or atypical intraepidermal melanocytic variants versus a reference standard of histological confirmation or clinical follow-up and expert opinion. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Due to scarcity of data and poor quality of studies, we did not perform a meta-analysis for this review. For illustrative purposes, we plotted estimates of sensitivity and specificity on coupled forest plots for each application under consideration. MAIN RESULTS: This review reports on two cohorts of lesions published in two studies. Both studies were at high risk of bias from selective participant recruitment and high rates of non-evaluable images. Concerns about applicability of findings were high due to inclusion only of lesions already selected for excision in a dermatology clinic setting, and image acquisition by clinicians rather than by smartphone app users.We report data for five mobile phone applications and 332 suspicious skin lesions with 86 melanomas across the two studies. Across the four artificial intelligence-based applications that classified lesion images (photographs) as melanomas (one application) or as high risk or 'problematic' lesions (three applications) using a pre-programmed algorithm, sensitivities ranged from 7% (95% CI 2% to 16%) to 73% (95% CI 52% to 88%) and specificities from 37% (95% CI 29% to 46%) to 94% (95% CI 87% to 97%). The single application using store-and-forward review of lesion images by a dermatologist had a sensitivity of 98% (95% CI 90% to 100%) and specificity of 30% (95% CI 22% to 40%).The number of test failures (lesion images analysed by the applications but classed as 'unevaluable' and excluded by the study authors) ranged from 3 to 31 (or 2% to 18% of lesions analysed). The store-and-forward application had one of the highest rates of test failure (15%). At least one melanoma was classed as unevaluable in three of the four application evaluations. AUTHORS' CONCLUSIONS: Smartphone applications using artificial intelligence-based analysis have not yet demonstrated sufficient promise in terms of accuracy, and they are associated with a high likelihood of missing melanomas. Applications based on store-and-forward images could have a potential role in the timely presentation of people with potentially malignant lesions by facilitating active self-management health practices and early engagement of those with suspicious skin lesions; however, they may incur a significant increase in resource and workload. Given the paucity of evidence and low methodological quality of existing studies, it is not possible to draw any implications for practice. Nevertheless, this is a rapidly advancing field, and new and better applications with robust reporting of studies could change these conclusions substantially.

AI snapshot

Written from the paper’s full text.

TL;DR

This Cochrane review found only two eligible studies (332 lesions, 86 melanomas) and could not produce summary estimates, so the accuracy of smartphone apps for melanoma triage remains unreliable. Automated apps missed a substantial proportion of melanomas, while a store-and-forward dermatologist-review app had high sensitivity but low specificity and frequent test failures.

Key findings

Across four artificial intelligence-based apps, sensitivity ranged from 7% (95% CI 2% to 16%) to 73% (95% CI 52% to 88%) and specificity from 37% (95% CI 29% to 46%) to 94% (95% CI 87% to 97%); with melanoma prevalence of 18% to 37%, the number of melanomas missed was 7 to 55.
The single store-and-forward app had sensitivity of 98% (95% CI 90% to 100%) and specificity of 30% (95% CI 22% to 40%); the dermatologist missed one melanoma, but 6/60 melanomas assessed were classed as unevaluable.
Test failures/unevaluable lesions ranged from 3 to 31 (2% to 18% of lesions analysed); the store-and-forward app had one of the highest test-failure rates at 15%.
In Maier 2015, face-to-face dermatologist diagnosis of the same lesions had sensitivity of 85% (95% CI 65% to 96%) and specificity of 97% (95% CI 93% to 99%), though the review notes these data are not directly comparable to the app’s output.

Methods

Systematic review of diagnostic test accuracy studies; searched MEDLINE, Embase, CENTRAL, CINAHL and other sources to August 2016, including adults with pigmented or suspicious skin lesions, smartphone apps intended for public use, and histology or acceptable reference standards.
Included two studies (Wolf 2013, USA; Maier 2015, Germany) providing six datasets; 332 lesions were successfully analysed, including 86 melanomas; the unit of analysis was the lesion.
Due to scarcity and poor quality of data, no meta-analysis was performed; results were plotted on coupled forest plots, and risk of bias/applicability was assessed with a tailored QUADAS-2 tool.

Limitations

The paper states the small number and poor quality of included studies reduce reliability, with high risk of bias for patient selection (selective inclusion; case-control design in Wolf 2013) and high applicability concerns because participants were secondary-care lesions scheduled for excision and photographs were taken by doctors rather than smartphone users.
The paper states test failures/unevaluable images were excluded (2% to 18% of lesions), likely overestimating sensitivity; studies did not report key patient information such as age and gender, and melanoma prevalence was 18% and 35%, much higher than in community settings.
Evident from the text: no meta-analysis, heterogeneity investigation, sensitivity analyses, or publication-bias assessment was possible, and only two studies were included, so conclusions could change substantially with new higher-quality studies.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/14651858.cd013192

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 8 cited sources checked · 0 supported · 0 flagged

Show all 63 claims

It may state two separate claims as one. 62
Probably not a substantial claim. 81
The quote only partly states this claim. 1

In the paper
“It is notable that Kassianos 2015 identified 39 applications, and we were only able to identify test accuracy evaluations for five.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Strengths and weaknesses of the review
What the sources say
- Not checked yetcites Kassianos 2015 source
Strength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The photographs used were taken by doctors rather than by smartphone users, which seriously impacts the applicability of results.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Who do the results of this review apply to?
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The prevalence of melanoma in the included studies is much higher than that observed in community settings.
In the paper
“We assessed risk of bias and applicability of included studies using the QUADAS-2 checklist (Whiting 2011), tailored to the topic of skin cancer diagnosis (see Appendix 5).”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Assessment of methodological quality
What the sources say
- Not checked yetcites Whiting 2011 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
In the paper
“Due to uncertainty about the determinants of publication bias for diagnostic accuracy studies and the inadequacy of tests for detecting funnel plot asymmetry (Deeks 2005), we did not perform any tests to detect publication bias.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Assessment of reporting bias
What the sources say
- Not checked yetcites Deeks 2005 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“used cross-validation approaches such as 'leave-one-out' cross-validation (Efron 1983);”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Additional information
What the sources say
- Not checked yetcites Efron 1983 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
In the paper
“In terms of exclusions, we considered studies that excluded particular lesion types (e.g.lentigo maligna), particular lesion sites, or other lesions on the basis of image quality or lack of observer agreement (e.g. on histopathology) to be at high risk of bias.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Additional information
What the sources say
- Not checked yetcites Whiting 2011 source
Strength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Available data have limited applicability in practice due to selective participant recruitment from secondary referral settings and the use of images not acquired by the intended users of the smartphone applications (i.e.members of the public).”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > A U T H O R S ' C O N C L U S I O N S Implications for practice
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: Smartphone applications for skin lesion triage are intended to be used by members of the public in a community setting.
In the paper
“In judging the applicability of patient populations to the review question, we considered restriction to particular lesion populations, such as melanocytic, nodular, high risk or restrictions by size to be of high concern for applicability.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Additional information
What the sources say
- Not checked yetcites Whiting 2011 source
Strength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We planned to supplement the database searches by searching the annual meetings of appropriate organisations (e.g.British Association of Dermatologists Annual Meeting, American Academy of Dermatology Annual Meeting, European Academy of Dermatology and Venereology Meeting, Society for Melanoma Research Congress, World Congress of Dermatology, European Association of Dermato Oncology)”
- Also stated in: § Acknowledgements · plain
  “however, due to the volume of evidence retrieved from database searches and time restrictions we were unable to do this.”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Acknowledgements
Strength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This interval was chosen based on a study showing that most false-negative melanomas will be diagnosed within three months of the initial negative index test, although a small number will be diagnosed up to 12 months subsequently (Altamura 2008).”
§ Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma (Review) > Flow and timing domain (4)
What the sources say
- Not checked yetcites Altamura 2008 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.

126 of 126

The paper relies on this without support.
The prevalence of melanoma in the included studies is much higher than that observed in community settings.
“The percentage of people with a final diagnosis of melanoma was 18% and 35%, much higher than that observed in community settings.”
The paper relies on this without support.
Smartphone applications for skin lesion triage are intended to be used by members of the public in a community setting.
“rather than the community setting where these applications are intended to be used”
“the intended users of the smartphone applications (i.e.members of the public)”
The authors state this as an assumption.
The review assumes that the 'two-point differences cases' excluded in Maier 2015 are lesions whose results fall in non-consecutive risk classes.
“we assume this to mean lesions with results in non-consecutive risk classes, e.g. 1 high risk and 2 low risk”
The authors state this as an assumption.
The review assumes that the interval between the index test and the reference standard in Maier 2015 was less than one month.
“assume it is < 1 month as images of the lesions were taken prior to excision”
The paper relies on this without support.
In routine practice, the clinical diagnosis is included on pathology request forms and histopathologists use it to help interpret the pathology.
“It is normal practice for the clinical diagnosis (usually by visual inspection or dermoscopy) to be included on pathology request forms and for the histopathologist to use this diagnosis to help with the pathology interpretation.”
The paper relies on this without support.
Blinded reading of the histopathology reference standard is not standard practice.
“blinded interpretation of the histopathology reference standard is not normal practice”
“histopathology interpretation is usually conducted with knowledge of the clinical diagnosis (from visual inspection or dermoscopy or both).”

6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

Figure 8 gives the dermatologist test the wrong decision threshold
Minor
The Figure 8 caption labels the face-to-face diagnosis as high risk versus medium/low risk, but the data table identifies its decision as melanoma versus not melanoma.
Maier melanoma count changes from 40 to 60
Minor
The table footnote says the original Maier sample of 195 lesions contained 60 melanomas, whereas the study characteristics give 40 melanomas for that same sample.
Wolf exclusions are described as both common and uncommon lesions
Minor
The methodological-quality discussion says Wolf excluded common lesions, whereas the results section says it excluded uncommon lesions. Both passages also mention equivocal diagnoses, making the difference in lesion frequency explicit.
Six Wolf lesions are unaccounted for in App 3 results
Minor
The table gives 188 Wolf lesions, but App 3 has 170 successfully assessed lesions and 12 unevaluable lesions, totaling only 182. The other six lesions are not accounted for in the reported App 3 flow.
Reported 188 · recomputed 182

4 of 4

Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012
Ferlay, J., Soerjomataram, I., Dikshit, R., Eser, S., Mathers, C., Rebelo, M., Parkin, D. M., Forman, D., & Bray, F. (2015) · International Journal of Cancer · doi:10.1002/ijc.29210 · Cited by 28,766
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation
Cancer statistics, 2015
Siegel, R. L., Miller, K. D., & Jemal, A. (2015) · CA A Cancer Journal for Clinicians · doi:10.3322/caac.21254 · Cited by 11,854
Final Version of 2009 AJCC Melanoma Staging and Classification
Balch, C. M., Gershenwald, J. E., Soong, S.-J., Thompson, J. F., Atkins, M. B., Byrd, D. R., Buzaid, A. C., Cochran, A. J., Coit, D. G., Ding, S., et al. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.23.4799 · Cited by 4,653
STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660
The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed
Deeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Self-citation
Prognostic Factors Analysis of 17,600 Melanoma Patients: Validation of the American Joint Committee on Cancer Melanoma Staging System
Balch, C. M., Soong, S., Gershenwald, J. E., Thompson, J. F., Reintgen, D. S., Cascinelli, N., Urist, M. M., McMasters, K. M., Ross, M. I., Kirkwood, J. M., et al. (2001) · Journal of Clinical Oncology · doi:10.1200/jco.2001.19.16.3622 · Cited by 2,383
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Cancer statistics, 1994
Boring, C. C., Squires, T. S., Tong, T., & Montgomery, S. (1994) · CA A Cancer Journal for Clinicians · doi:10.3322/canjclin.44.1.7 · Cited by 1,313
Meta-analysis of risk factors for cutaneous melanoma: II. Sun exposure
Gandini, S., Sera, F., Cattaruzza, M. S., Pasquini, P., Picconi, O., Boyle, P., & Melchi, C. F. (2005) · European Journal of Cancer · doi:10.1016/j.ejca.2004.10.016 · Cited by 1,266
Meta-analysis of risk factors for cutaneous melanoma: I. Common and atypical naevi
Gandini, S., Sera, F., Cattaruzza, M. S., Pasquini, P., ABENI, D. D., Boyle, P., & Melchi, C. F. (2004) · European Journal of Cancer · doi:10.1016/j.ejca.2004.10.015 · Cited by 879
Early Detection of Malignant Melanoma: The Role of Physician Examination and Self-Examination of the Skin
Friedman, R. J., Rigel, D. S., & Kopf, A. (1985) · CA A Cancer Journal for Clinicians · doi:10.3322/canjclin.35.3.130 · Cited by 859
Increasing Burden of Melanoma in the United States
Linos, E., Swetter, S. M., Cockburn, M., Colditz, G. A., & Clarke, C. A. (2009) · Journal of Investigative Dermatology · doi:10.1038/jid.2008.423 · Cited by 747
Incidence of noncutaneous melanomas in the U.S.
McLaughlin, C. C., Wu, X., Jemal, A., Martin, H. J., Roche, L. M., & Chen, V. W. (2005) · Cancer · doi:10.1002/cncr.20866 · Cited by 741
Meta-Analysis of Phase II Cooperative Group Trials in Metastatic Stage IV Melanoma to Determine Progression-Free and Overall Survival Benchmarks for Future Phase II Trials
Korn, E. L., Liu, P.-Y., Lee, S. J., Chapman, J. W., Niedzwiecki, D., Suman, V. J., Moon, J., Sondak, V. K., Atkins, M. B., Eisenhauer, E. A., et al. (2008) · Journal of Clinical Oncology · doi:10.1200/jco.2007.12.7837 · Cited by 704
Cutaneous melanoma attributable to sunbed use: systematic review and meta-analysis
Boniol, M., Autier, P., Boyle, P., & Gandini, S. (2012) · BMJ · doi:10.1136/bmj.e4757 · Cited by 670
International trends in the incidence of malignant melanoma 1953–2008—are recent generations at higher or lower risk?
Erdmann, F., Lortet‐Tieulent, J., Schüz, J., Zeeb, H., Greinert, R., Breitbart, E. W., & Bray, F. (2012) · International Journal of Cancer · doi:10.1002/ijc.27616 · Cited by 647
Evidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W., Reitsma, J. B., Nisio, M. D., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 623
Diagnosis and treatment of melanoma. European consensus-based interdisciplinary guideline – Update 2016
Garbe, C., Peris, K., Hauschild, A., Saïag, P., Middleton, M. R., Bastholt, L., Grob, J., Malvehy, J., Newton‐Bishop, J., Stratigos, A., et al. (2016) · European Journal of Cancer · doi:10.1016/j.ejca.2016.05.005 · Cited by 617
Epidemiological trends in skin cancer
Apalla, Z., Lallas, A., Sotiriou, E., Lazaridou, E., & Ioannides, D. (2017) · Dermatology Practical & Conceptual · doi:10.5826/dpc.0702a01 · Cited by 583
Case–Control and Two-Gate Designs in Diagnostic Accuracy Studies
Rutjes, A. W., Reitsma, J. B., Vandenbroucke, J. P., Glas, A. S., & Bossuyt, P. M. (2005) · Clinical Chemistry · doi:10.1373/clinchem.2005.048595 · Cited by 516
Revised U.K. guidelines for the management of cutaneous melanoma 2010
Marsden, J., Newton‐Bishop, J., Burrows, L., Cook, M. G., Corrie, P. G., Cox, N. H., Gore, M. E., Lorigan, P. C., Mackie, R. M., Nathan, P., et al. (2010) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2010.09883.x · Cited by 395
Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection
Wolf, J. A., Moreau, J. F., Akilov, O., Patton, T., English, J. C., Ho, J., & Ferris, L. K. (2013) · JAMA Dermatology · doi:10.1001/jamadermatol.2013.2382 · Cited by 393
Self-citation
Xeroderma pigmentosum
Lehmann, A. R., McGibbon, D., & Stefanini, M. (2011) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-6-70 · Cited by 343
Cancer incidence in the United Kingdom: projections to the year 2030
Mistry, M., Parkin, D. M., Ahmad, A. S., & Sasieni, P. D. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.430 · Cited by 277
Systemic treatments for metastatic cutaneous melanoma
Pasquali, S., Hadjinicolaou, A. V., Chiarion‐Sileni, V., Rossi, C. R., & D, S. M. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011123.pub2 · Cited by 258
Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults
Dinnes, J., Deeks, J. J., Chuchu, N., di Ruffano, L. F., Matin, R., Thomson, D. R., Wong, K. Y., Aldridge, R. B., Abbott, R. A., Fawzy, M., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011902.pub2 · Cited by 255
Self-citation
The risk of melanoma in patients with congenital nevi: A cohort study
Swerdlow, A. J., English, J. S. C., & Qiao, Z. (1995) · Journal of the American Academy of Dermatology · doi:10.1016/0190-9622(95)90343-7 · Cited by 248
Trends in incidence and predictions of cutaneous melanoma across Europe up to 2015
Arnold, M., Holterhues, C., Hollestein, L., Coebergh, J., Nijsten, T., Pukkala, E., Holleczek, B., Tryggvadottir, L., Comber, H., Bento, M., et al. (2014) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.12236 · Cited by 227
Surgical excision margins for primary cutaneous melanoma
Sladden, M. J., Balch, C. M., BARZILAI, D. A., Berg, D., Freiman, A., Handiside, T., Hollis, S., Lens, M. B., & Thompson, J. F. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004835.pub2 · Cited by 224
Melanoma Incidence and Mortality Among US Whites, 1969-1999
Geller, A. C. (2002) · JAMA · doi:10.1001/jama.288.14.1719 · Cited by 193
Fractal Analysis of Contours of Breast Masses in Mammograms
Rangayyan, R. M., & Nguyen, T. M. (2006) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-006-0860-9 · Cited by 185
Teledermatology for diagnosing skin cancer in adults.
Chuchu, N., Dinnes, J., Takwoingi, Y., Matin, R. N., Bayliss, S. E., Davenport, C., Moreau, J. F., Bassett, O., Godfrey, K., O'Sullivan, C., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013193 · Cited by 175
Self-citation
When Do Changes in Cancer Survival Mean Progress? The Insight From Population Incidence and Mortality
Cho, H., Mariotto, A. B., Schwartz, L. M., Luo, J. R., & Woloshin, S. (2014) · JNCI Monographs · doi:10.1093/jncimonographs/lgu014 · Cited by 174
Smartphone applications for melanoma detection by community, patient and generalist clinician users: a review
Kassianos, A. P., Emery, J. D., Murchie, P., & Walter, F. M. (2015) · British Journal of Dermatology · doi:10.1111/bjd.13665 · Cited by 151
Self-citation
Computer-assisted diagnosis techniques (dermoscopy and spectroscopy-based) for diagnosing skin cancer in adults
di Ruffano, L. F., Takwoingi, Y., Dinnes, J., Chuchu, N., Bayliss, S. E., Davenport, C. F., Matin, R., Godfrey, K., O'Sullivan, C., Gulati, A., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013186 · Cited by 151
Self-citation
Clinical recognition of early invasive malignant melanoma.
Mackie, R. M. (1990) · British medical journal · doi:10.1136/bmj.301.6759.1005 · Cited by 150
Assessment of the optimal interval for and sensitivity of short-term sequential digital dermoscopy monitoring for the diagnosis of melanoma
Altamura, D., Avramidis, M., & Menzies, S. W. (2008) · Archives of Dermatology · doi:10.1001/archderm.144.4.502 · Cited by 148
Optical coherence tomography for diagnosing skin cancer in adults
di Ruffano, L. F., Dinnes, J., Deeks, J. J., Chuchu, N., Bayliss, S. E., Davenport, C. F., Takwoingi, Y., Godfrey, K., O'Sullivan, C., Matin, R., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013189 · Cited by 145
Self-citation
Melanoma Screening with Cellular Phones
Massone, C., Hofmann-Wellenhof, R., Ahlgrimm-Siess, V., Gabler, G., Ebner, C., & Peter Soyer, H. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000483 · Cited by 136
Sun exposure and skin cancer, and the puzzle of cutaneous melanoma
Armstrong, B. K., & Cust, A. E. (2017) · Cancer Epidemiology · doi:10.1016/j.canep.2017.04.004 · Cited by 131
Second cancer following cutaneous melanoma and cancers of the brain, thyroid, connective tissue, bone, and eye in Connecticut, 1935-82.
Tucker, M. A., Jd, B., & Hoffman, D. A. (1985) · PubMed · Cited by 127
A Mobile Automated Skin Lesion Classification System
Ramlakhan, K., & Shang, Y. (2011) · Proceedings - International Conference on Tools with Artificial Intelligence, TAI · doi:10.1109/ictai.2011.29 · Cited by 124
Are My Symptoms Serious Dr Google? A Resource-Based Typology of Value Co-Destruction in Online Self-Diagnosis
Robertson, N. L., Polonsky, M. J., & McQuilken, L. (2014) · Australasian Marketing Journal (AMJ) · doi:10.1016/j.ausmj.2014.08.009 · Cited by 113
Non-genetic risk factors for cutaneous melanoma and keratinocyte skin cancers: An umbrella review of meta-analyses
Belbasis, L., Stefanaki, I., Stratigos, A., & Εvangelou, Ε. (2016) · Journal of Dermatological Science · doi:10.1016/j.jdermsci.2016.09.003 · Cited by 106
The Contribution of Nodular Subtype to Melanoma Mortality in the United States, 1978 to 2007
Shaikh, W. R. (2011) · Archives of Dermatology · doi:10.1001/archdermatol.2011.264 · Cited by 104
Accuracy of a smartphone application using fractal image analysis of pigmented moles compared to clinical diagnosis and histological result
Maier, T., Kulichova, D., Schotten, K., Astrid, R., Ruzicka, T., Berking, C., & Udrea, A. (2015) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.12648 · Cited by 103
Reflectance confocal microscopy for diagnosing cutaneous melanoma in adults
Dinnes, J., Deeks, J. J., Saleh, D. B., Chuchu, N., Bayliss, S. E., Patel, L., Davenport, C. F., Takwoingi, Y., Godfrey, K., Matin, R., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013190 · Cited by 102
Self-citation
High-frequency ultrasound for diagnosing skin cancer in adults
Dinnes, J., Bamber, J. C., Chuchu, N., Bayliss, S. E., Takwoingi, Y., Davenport, C. F., Godfrey, K., O'Sullivan, C., Matin, R., Deeks, J. J., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013188 · Cited by 102
Self-citation
Socioeconomic Status and Survival in Older Patients with Melanoma
Ortíz, C., Goodwin, J. S., Freeman, J. L., & Kuo, Y. (2006) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2006.00943.x · Cited by 94
The Empirical Foundations of Teledermatology: A Review of the Research Evidence
Bashshur, R. L., Shannon, G. W., Tejasvi, T., Kvedar, J. C., & Gates, M. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2015.0146 · Cited by 90
Gizmo Idolatry
Leff, B. A. (2008) · JAMA · doi:10.1001/jama.299.15.1830 · Cited by 84
Surgical excision margins in primary cutaneous melanoma: A meta-analysis and Bayesian probability evaluation
Wheatley, K., Wilson, J. S., Gaunt, P., & Marsden, J. (2015) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2015.10.013 · Cited by 82
Visual inspection for diagnosing cutaneous melanoma in adults
Dinnes, J., Deeks, J. J., Grainge, M. J., Chuchu, N., di Ruffano, L. F., Matin, R., Thomson, D. R., Wong, K. Y., Aldridge, R. B., Abbott, R. A., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013194 · Cited by 81
Self-citation
Diagnosis and management of lentigo maligna: a review
Kasprzak, J., & Xu, Y. G. (2015) · Drugs in Context · doi:10.7573/dic.212281 · Cited by 79
Fractals in microscopy
Landini, G. (2010) · Journal of Microscopy · doi:10.1111/j.1365-2818.2010.03454.x · Cited by 77
Implementation of the 7-point checklist for melanoma detection on smart handheld devices
Wadhawan, T., Situ, N., Rui, H., Lancaster, K., Yuan, X., & Zouridakis, G. (2011) · 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/iembs.2011.6090866 · Cited by 72
Growth of mobile applications in dermatology - 2017 update
Flaten, H. K., Claire, C. S., Schlager, E., Dunnick, C. A., & Dellavalle, R. P. (2018) · Dermatology Online Journal · doi:10.5070/d3242038180 · Cited by 70
Characteristics and survival of patients with invasive amelanotic melanoma in the USA
Moreau, J. F., Weissfeld, J. L., & Ferris, L. K. (2013) · Melanoma Research · doi:10.1097/cmr.0b013e32836410fe · Cited by 57
Self-citation
Iatrogenic immunosuppression and cutaneous malignancy
DePry, J. L., Reed, K. B., Cook‐Norris, R. H., & Brewer, J. D. (2011) · Clinics in Dermatology · doi:10.1016/j.clindermatol.2011.08.009 · Cited by 51
Critical Assessment of Clinical Prognostic Tools in Melanoma
Mahar, A. L., Compton, C. C., Halabi, S., Hess, K. R., Gershenwald, J. E., Scolyer, R. A., & Groome, P. A. (2016) · Annals of Surgical Oncology · doi:10.1245/s10434-016-5212-5 · Cited by 49
Cutaneous fibrosis induced by docetaxel
Cleveland, M. G., Ajaikumar, B. S., & Reganti, R. (2000) · Cancer · doi:10.1002/(sici)1097-0142(20000301)88:5 · Cited by 48
DERMA/Care: An Advanced image-Processing Mobile Application for Monitoring Skin Cancer
Karargyris, A., Karargyris, O., & Pantelopoulos, A. (2012) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2012.180 · Cited by 48
Caution in melanoma risk analysis with smartphone application technology: Correspondence
Robson, Y., Blackford, S., & Roberts, D. (2012) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2012.11046.x · Cited by 39
Accuracy of teledermatology/teledermoscopy and clinic-based dermatology for specific categories of skin neoplasms
Warshaw, E. M., Gravely, A. A., & Nelson, D. B. (2010) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2009.10.037 · Cited by 37
Tests to assist in the diagnosis of cutaneous melanoma in adults: a generic protocol
Dinnes, J., Matin, R., Moreau, J. F., Patel, L., Chan, S. A., Chuchu, N., Bayliss, S. E., Grainge, M. J., Takwoingi, Y., Davenport, C. F., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011902 · Cited by 28
Self-citation
e-Health Tools for Targeting and Improving Melanoma Screening: A Review
Tyagi, A., Miller, K., & Cockburn, M. (2012) · Journal of Skin Cancer · doi:10.1155/2012/437502 · Cited by 26
Malignant melanoma in Chile: different site distribution between private and state patients
Zemelman, V., Valenzuela, C. Y., Ivo Sazunic, Y., & Irene Araya, B. (2014) · Biological Research · doi:10.1186/0717-6287-47-34 · Cited by 26
A Comparative Study Between Smartphone-Based Microscopy and Conventional Light Microscopy in 1021 Dermatopathology Specimens
Jahan-Tigh, R. R., Chinn, G. M., & Rapini, R. P. (2015) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2014-0593-oa · Cited by 22
Melanoma and Other Skin Lesion Detection Using Smart Handheld Devices
Zouridakis, G., Wadhawan, T., Situ, N., Hu, R., Yuan, X., Lancaster, K., & Queen, C. M. (2014) · Methods in molecular biology · doi:10.1007/978-1-4939-2172-0_30 · Cited by 21
Dermatological cancer screening: Evaluation of a new community pharmacy service
Kjome, R. L. S., Wright, D. J., Bjaaen, A.-K. B., Garstad, K. W., & Valeur, M. (2016) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2016.12.001 · Cited by 20
Application of fractal analysis to mammography
Raguso, G., Ancona, A., Chieppa, L., L’Abbate, S., Pepe, M. L., Mangieri, F. F., Palo, M. D., & Rangayyan, R. M. (2010) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2010.5627180 · Cited by 17
High-Dynamic-Range Dermoscopy Imaging and Diagnosis of Hypopigmented Skin Cancers
Braun, R. P., & Marghoob, A. A. (2014) · JAMA Dermatology · doi:10.1001/jamadermatol.2014.4714 · Cited by 17
Is initial excision of cutaneous melanoma by General Practitioners (GPs) dangerous? Comparing patient outcomes following excision of melanoma by GPs or in hospital using national datasets and meta-analysis
Murchie, P., Raja, E. A., Brewster, D., Iversen, L., & Lee, A. J. (2017) · European Journal of Cancer · doi:10.1016/j.ejca.2017.09.034 · Cited by 17
Tests to assist in the diagnosis of keratinocyte skin cancers in adults: a generic protocol
Dinnes, J., Wong, K. Y., Gulati, A., Chuchu, N., Leonardi‐Bee, J., Bayliss, S. E., Takwoingi, Y., Davenport, C. F., Matin, R., Bath-Hextall, F. J., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011901 · Cited by 14
Self-citation
Mobile teledermatology for skin tumour screening
Varma, S. (2011) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2011.10356.x · Cited by 11
A Comparative Study Between Smartphone-Based Microscopy and Conventional Light Microscopy in 1021 Dermatopathology Specimens
Jahan-Tigh, R. R., Chinn, G. M., & Rapini, R. P. (2015) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2014-0593-oa.s1 · Cited by 8
Textbook of Melanoma: Pathology, Diagnosis and Management
Riley, P. A. (2004) · Melanoma Research · doi:10.1097/00008390-200402000-00013 · Cited by 6
Diagnostic accuracy of smartphone applications
Burki, T. K. (2013) · The Lancet Oncology · doi:10.1016/s1470-2045(13)70003-2 · Cited by 5
Polarization-sensitive digital dermoscopy for image processing-assisted evaluation of atypical nevi: towards step-wise detection of melanoma
Yu, L. S., Joseph, A. O., Lindsley, E. H., & Farkas, D. L. (2011) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.891083 · Cited by 4
DermLens: Device for mobile teledermatology
(2015) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2015.02.364 · Cited by 2
Melanoma detection using a mobile phone app
Diniz, L. E., & Ennser, K. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2212446 · Cited by 1

Flagged first, then by citations

6 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

Braunmühl, 2015, Smartphone apps for skin cancer diagnosis? The Munich study, Kosmetische Medizin, 36, 152
Australian Cancer Database, 2017, Melanoma of the skin for Australia (ICD10 C43)
Ndegwa, 2010, Teledermatology services: rapid review of diagnostic, clinical management, and economic outcomes

Sorted by citations

Artificial Intelligence in Medicine: Where Are We Now?
Kulkarni, S., Seneviratne, N., Baig, M. S., & Khan, A. H. A. (2019) · Academic Radiology · doi:10.1016/j.acra.2019.10.001 · Cited by 301
Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies
Freeman, K., Dinnes, J., Chuchu, N., Takwoingi, Y., Bayliss, S. E., Matin, R. N., Jain, A., Walter, F. M., Williams, H. C., & Deeks, J. J. (2020) · BMJ · doi:10.1136/bmj.m127 · Cited by 260
Self-citation
Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults
Dinnes, J., Deeks, J. J., Chuchu, N., di Ruffano, L. F., Matin, R., Thomson, D. R., Wong, K. Y., Aldridge, R. B., Abbott, R. A., Fawzy, M., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011902.pub2 · Cited by 255
Self-citation
Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions
Phillips, M. R., Marsden, H., Jaffe, W., Matin, R., Wali, G. N., Greenhalgh, J., McGrath, E., James, R., Ladoyanni, E., Bewley, A., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.13436 · Cited by 236
Self-citation
Artificial Intelligence in Dermatology—Where We Are and the Way to the Future: A Review
Hogarty, D. T., Su, J. C., Phan, K. H., Attia, M., Hossny, M., Nahavandi, S., Lenane, P., Moloney, F. J., & Yazdabadi, A. (2019) · American Journal of Clinical Dermatology · doi:10.1007/s40257-019-00462-6 · Cited by 195

See all 144

Authors: Chuchu, N., Takwoingi, Y., Dinnes, J., Matin, R. N., Bassett, O., Moreau, J. F., Bayliss, S. E., Davenport, C., Godfrey, K., O'Connell, S., Jain, A., Walter, F. M., Deeks, J. J., Williams, H. C., & Cochrane Skin Cancer Diagnostic Test Accuracy Group
Affiliations: University of Birmingham (GB)
University Hospitals Birmingham NHS Foundation Trust (GB)
Churchill Hospital (GB)
Addenbrooke's Hospital (GB)
University of Pittsburgh Medical Center (US)
University of Nottingham (GB)
Cardiff and Vale University Health Board (GB)
Imperial College Healthcare NHS Trust (GB)
University of Cambridge (GB)
Corresponding author: Dinnes, Jacqueline
ORCID: Takwoingi, Yemisi
Dinnes, Jacqueline
Matin, Rubeta N
Moreau, Jacqueline F
Bayliss, Susan E
Davenport, Clare
O'Connell, Susan
Jain, Abhilash
Walter, Fiona M
Deeks, Jonathan J
Williams, Hywel C
Venue: Cochrane Database of Systematic Reviews
Publisher: Wiley
Published: 2018-12-04
Volume: 2018
Issue: 12
Pages: CD013192-CD013192
ISSN: 1361-6137 (ISSN-L)
1465-1858
DOI: 10.1002/14651858.cd013192
PMC: PMC6517294
OpenAlex: W2886914906
Semantic Scholar: c82c23917430d434cb08deb9f9339bc64340303d
Cited by: 144
References: 88
Citations by year: 2026: 4 · 2025: 10 · 2024: 16 · 2023: 18 · 2022: 18 · 2021: 29 · 2020: 26 · 2019: 15 · 2018: 7
FWCI: 2.02
Type: journal article
Language: en
Open access: bronze
doi.org
Locations: Cochrane Database of Systematic Reviews (publishedVersion)
Europe PMC (PubMed Central) (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Nottingham ePrints (University of Nottingham) (submittedVersion, cc-by)
Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
Funders: Melanoma Research Alliance
National Institute for Health and Care Research (13/89/15, NIHR-CS-012-030)
British Association of Dermatologists
Topics: Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
Life Sciences › Biochemistry, Genetics and Molecular Biology › Cell Biology › melanin and skin pigmentation
Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Keywords: melanoma, mobile application, mobile health, skin lesions, artificial intelligence, diagnostic accuracy, sensitivity and specificity, early detection of skin cancer, image classification, histopathological confirmation, delayed diagnosis, quality of evidence, risk of bias, applicability, cost-effectiveness
MeSH terms: Smartphone, Cutaneous Malignant Melanoma, Adult, Algorithms, Diagnostic Errors, Humans, Melanoma, Sensitivity and Specificity, Skin Neoplasms, Triage, Early Detection of Cancer, Mobile Applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma

Abstract

Overview

AI snapshot

TL;DR

Key findings

Methods

Limitations

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks