CoChat

The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index; 15 of the 22 sources its claims cite were checked for support.

Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis

Open access

Abstract

Background: Artificial intelligence (AI) applications in dermatology have expanded beyond diagnosis and have shifted towards assessing disease severity. Objectives: To qualitatively and quantitatively evaluate the performance of image-based AI models in severity assessment for various skin diseases. Methods: In this systematic review and meta-analysis, we collected studies using four electronic databases, including PubMed, Embase, Institute of Electrical and Electronics Engineers Xplore and Web of Science, published from 1 January 2017 to 6 April 2023, and updated the search in November 2023. Studies assessing the performance of deep learning AI models on the severity of skin diseases were included. We excluded studies that utilized a nonvalidated severity index, lacked clinical images, and assessed wounds, ulcers or burns. Two independent reviewers extracted prespecified study characteristics for the summary table. For the meta-analysis, contingency tables were extracted, when possible, and reconstructed for each severity measure. Accuracy was calculated using a bivariate model in Metandi package, and meta-regression was performed by disease type and scoring system. This study was registered with PROSPERO (CRD42023487228). Results: Our initial search identified 7737 records. After duplicate removal and abstract screening, we reviewed the full text of 192 articles and included 45 studies for systematic review and 19 for meta-analysis. The pooled sensitivity and specificity of AI models were 80.5% [95% confidence interval (CI) 76.2–84.2] and 96.2% (95% CI 94.9–97.2), respectively. Moreover, pooled sensitivity differed by disease (atopic dermatitis 91.8% vs. acne 80.7%, P = 0.005; acne 80.4% vs. psoriasis 71.1%, P = 0.044) and scoring system [Eczema Area and Severity Index 97.3% vs. Investigator’s Global Assessment (IGA) for atopic dermatitis 78.9%, P < 0.001; Hayashi Grading 89.7% vs. IGA for acne 69.8%, P < 0.001]. Conclusions: Our findings show that current AI models exhibit a high level of capacity in disease severity assessment. Nevertheless, efforts are urgently needed to improve transparency in data reporting and conduct high-quality prospective studies using objective reference standards in clinical settings to generate reliable evidence.

Overview

CoChat Summary

TL;DR

This systematic review and meta-analysis found that image-based AI models assess inflammatory skin disease severity with high pooled specificity and moderate-to-high sensitivity, though performance varied by disease and scoring system. The authors emphasize limited transparency, retrospective designs, and poor skin tone reporting as barriers to reliable clinical use.

Key findings

  • From 45 included studies and 19 in meta-analysis, pooled AI sensitivity was 80.5% (95% CI 76.2–84.2) and pooled specificity was 96.2% (95% CI 94.9–97.2); pooled reliability was 78.0% (95% CI 72.2–83.8).
  • Sensitivity differed by disease: atopic dermatitis 91.8% vs. acne 80.7% (P = 0.005), and acne 80.4% vs. psoriasis 71.1% (P = 0.044). Specificity was highest in atopic dermatitis (98.1% vs. acne 94.3%, P = 0.014).
  • Scoring system mattered: for atopic dermatitis, EASI sensitivity was 97.3% vs. IGA 78.9% (P < 0.001); for acne, Hayashi Grading/New Acne Severity sensitivity was 89.7% vs. IGA 69.8% (P < 0.001).
  • Most studies were from Asia (64%, 29/45) and Europe (29%, 13/45); psoriasis (31%, 14/45), acne (27%, 12/45), and atopic dermatitis (9%, 4/45) were most studied.

Methods

  • Systematic review and meta-analysis registered with PROSPERO (CRD42023487228), searching PubMed, Embase, IEEE Xplore, and Web of Science from 1 January 2017 to 6 April 2023, updated November 2023; 7,737 records initially identified, 45 studies included in systematic review and 19 in meta-analysis.
  • Included studies assessed deep learning/CNN AI models for severity of skin, hair, or nail diseases using clinical images and validated severity indices; excluded nonvalidated indices, segmentation-only studies, wounds, ulcers, burns, tumours, and photoageing.
  • Pooled sensitivity/specificity used a bivariate model in Stata 18 Metandi; meta-regression by disease type and scoring system used Meqrlogit; risk of bias assessed with QUADAS-2, with Deeks’ funnel plot asymmetry test for publication bias.

Limitations

  • The paper states that analysis was limited to studies providing contingency tables or ICCs, multiple contingency tables were treated independently, acne scoring systems were combined for subgroup analysis, and high heterogeneity limits generalizability; a small-study effect was noted for reliability.
  • The paper states that more than 75% of studies (78%, 35/45) did not report skin tone data, preventing assessment across diverse populations; most studies used private datasets, few reported demographic or skin tone metadata, and only one used a prospective design.
  • Evident from the text: one-third of studies had high risk of bias (n = 14), QUADAS-2 interrater agreement was low (κ = 0.27), and most studies were retrospective or proof-of-concept rather than clinical prospective validations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/bjd/ljaf250

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Study selection. AI, artificial intelligence; IEE, Institute of Electrical and Electronics Engineers.
Figure 1 Study selection. AI, artificial intelligence; IEE, Institute of Electrical and Electronics Engineers.CC BY · PubMed Central
Figure 2 Summary receiver operating characteristic (SROC) plot (overall). HSROC, hierarchical SROC.
Figure 2 Summary receiver operating characteristic (SROC) plot (overall). HSROC, hierarchical SROC.CC BY · PubMed Central
Figure 3 Summary receiver operating characteristic (SROC) plots by disease. (a) Eczema (32 contingency tables). (b) Acne (30 contingency tables). (c) Psoriasis (50 contingency tables). (d) Radiation dermatitis (7 contingency tables). HSROC, hierarchical SROC.
Figure 3 Summary receiver operating characteristic (SROC) plots by disease. (a) Eczema (32 contingency tables). (b) Acne (30 contingency tables). (c) Psoriasis (50 contingency tables). (d) Radiation dermatitis (7 contingency tables). HSROC, hierarchical SROC.CC BY · PubMed Central

Verification

Claims the paper makes

10 Valid · 0 Not valid · 6 Not checked · 76 Own results and interpretations

42 more claims · 1 with their cited sources checked

Create a free account to see all 92 claims and to verify the 1 reference still open.

See all claims — free account
  • It may state two separate claims as one. 31
  • Probably not a substantial claim. 22
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 65 references

Cited by

See all 8

Metadata

Authors
Cai, Z. R., Kim, J., Rezaei, S. J., Chen, M. L., Touma, F., Zhu, C., Onyeka, S., Fonjungo, F., Omiye, J. A., Krakowski, I., Nwandu, L., Biossonnette, R., Ko, J. M., & Linos, E.
Affiliations
  • Stanford Medicine (US)
  • McGill University (CA)
  • Meharry Medical College (US)
  • Karolinska Institutet (SE)
  • University of Illinois Urbana-Champaign (US)
  • Innovaderm (Canada) (CA)
Corresponding author
Kim, Jiyeong
Venue
British Journal of Dermatology
Published
2025-06-26
Volume
193
Issue
5
Pages
847-855
ISSN
  • 0007-0963 (ISSN-L)
  • 1365-2133
OpenAlex
W4411687990
Cited by
8
References
65
Citations by year
2026: 7 · 2025: 1
FWCI
0.47
Type
journal article
Language
en
Open access
Locations
  • British Journal of Dermatology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • National Institutes of Health (K01MH137386)
  • Midcareer Investigator Award in Patient-Oriented Research (K24AR075060)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Dermatology › Dermatology and Skin Diseases
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
Keywords
artificial intelligence, disease severity assessment, inflammatory skin diseases, deep learning, atopic dermatitis, acne, psoriasis, Eczema Area and Severity Index, Investigator's Global Assessment, diagnostic accuracy, sensitivity and specificity, dermatology, model transparency, reporting quality, generalizability, external validation, clinical scoring systems
MeSH terms
Deep Learning, Acne Vulgaris, Artificial Intelligence, Dermatology, Humans, Severity of Illness Index

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis. https://app.cochat.ai/papers/10.1093/bjd/ljaf250
The paper
Cai, Z. R., Kim, J., Rezaei, S. J., Chen, M. L., Touma, F., Zhu, C., Onyeka, S., Fonjungo, F., Omiye, J. A., Krakowski, I., Nwandu, L., Biossonnette, R., Ko, J. M., & Linos, E. (2025). Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis. British Journal of Dermatology. https://doi.org/10.1093/bjd/ljaf250

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.