Revamping AI models in dermatology: overcoming critical challenges for enhanced skin lesion diagnosis

“Resnet34 [45] was employed as our CNN architecture, while the Transformer Encoder-Decoder block of End-to-End Detection Transformer (DETR) [46] was adopted for encoding and decoding the features learned by the CNN.”

“Resnet34 [45] was employed as our CNN architecture, while the Transformer Encoder-Decoder block of End-to-End Detection Transformer (DETR) [46] was adopted for encoding and decoding the features learned by the CNN.”

“Our hierarchical framework provides a three-level prediction for skin lesion images, which is more informative than the traditional single-output prediction [8].”

“Mixup technique [37] combines two different images and their corresponding labels, which can help to create better decision boundaries between the two corresponding categories.”

“AI techniques, including deep learning and machine learning models, have demonstrated promising results [8, 9] and even outperformed dermatologists in certain tasks of skin cancer classification [10, 11].”

“Each encoder layer in the Transformer includes a standard architecture comprising a multi-head self-attention module and a feed-forward network (FFN), further supplemented with fixed positional encoding [47, 48] that is added to the input of each attention layer.”

“Each encoder layer in the Transformer includes a standard architecture comprising a multi-head self-attention module and a feed-forward network (FFN), further supplemented with fixed positional encoding [47, 48] that is added to the input of each attention layer.”

“If an OOD alert is raised, the input image should be treated as an outlier and its original prediction should not be trusted, which is vital for medical AI applications [30] to mitigate potential misleading of clinicians [31].”

“Recent advancements in skin imaging technology and Artificial Intelligence (AI) may assist in increasing the accessibility of dermatology expertise [5].”

“The hierarchical connection extends this split to level 1 and level 2 samples 4 .”

“AI techniques, including deep learning and machine learning models, have demonstrated promising results [8, 9] and even outperformed dermatologists in certain tasks of skin cancer classification [10, 11].”

“To do so we build upon the hierarchical module by combining mixup strategy [37] and prototype learning [38].”

“It is necessary to learn better decision boundaries to achieve OOD detection and clinical triage recommendation capabilities [39, 40].”

“Despite these AI advancements [12–19], challenges remain in the adoption of AI for clinical decision-making [20].”

“Providing high-quality care is crucial for alleviating the global burden of skin diseases [1].”

“…an AI model offering a more nuanced diagnostic capability, such as hierarchical akin to a differential diagnosis in addition to identifying an OOD image, or providing a triage recommendation, would better align with the clinical thought process, thereby enhancing confidence in the AI model [28].”

“AI models trained on images from a specific device can encounter domain shift [52, 53] issues when tested on external validation image sets from a different camera device.”

“Despite these AI advancements [12–19], challenges remain in the adoption of AI for clinical decision-making [20].”

“• AI models primarily use dermoscopic images [24], despite clinical skin photographs often being sufficient or even superior for diagnosing many common skin lesions [25].”

“…technique will suffer a trade-off between achieving a high OOD detection performance and retaining the In-Distribution classification performance [41] Thus, we also test our “Hierarchical+MPL” strategy for In-Distribution classification performance across the three levels and three image…”

“However, the shortage of dermatologists [2], especially evident during the COVID-19 crisis, has led to their escalating consultation costs [3, 4].”

“Our dataset lacks specific skin tone type information (as featured in the Fitzpatrick 17k dataset [51]), preventing us from evaluating our model’s generalizability across different skin tones.”

“Hierarchical models have previously proven to capture complex relationships and dependencies among categories [35], and more importantly leverage prior knowledge and domain expertise from the hierarchical structure of the dataset [36].”

“• The evaluation of AI models has been limited to datasets [13, 21, 22] that do not accurately represent real-world scenarios [23], lacking diversity and range in skin diseases.”

“AI techniques, including deep learning and machine learning models, have demonstrated promising results [8, 9] and even outperformed dermatologists in certain tasks of skin cancer classification [10, 11].”

“However, the shortage of dermatologists [2], especially evident during the COVID-19 crisis, has led to their escalating consultation costs [3, 4].”

“• AI models primarily use dermoscopic images [24], despite clinical skin photographs often being sufficient or even superior for diagnosing many common skin lesions [25].”

“Despite these AI advancements [12–19], challenges remain in the adoption of AI for clinical decision-making [20].”

“Secondly, patient metadata was not available alongside the images for training our framework, and therefore we cannot quantify important clinical aspects of the data such as sex, age, skin tone and ethnicity [42].”

“We determined the OOD threshold t ood based on the values of FPR and TPR [49].”

“It is necessary to learn better decision boundaries to achieve OOD detection and clinical triage recommendation capabilities [39, 40].”

“Hierarchical models have previously proven to capture complex relationships and dependencies among categories [35], and more importantly leverage prior knowledge and domain expertise from the hierarchical structure of the dataset [36].”

“The evolution of skin imaging technology such as total body photographs [6] and dermoscopy [7], combined with AI methodologies, has led to rapid progression in image-based skin lesion diagnosis.”

“AI models trained on images from a specific device can encounter domain shift [52, 53] issues when tested on external validation image sets from a different camera device.”

“That being said, histopathology diagnosis is also prone to considerable variability [43], and prospective testing of similar models using dermatologists diagnosis as ground truth have shown to be successful [44].”

“However, even considering a histopathological diagnosis, there is current debate as to whether melanoma in situ should in fact be classified as atypical or dysplastic as there is little to no evidence of melanoma in situ causing metastasis or death [50].”

“The evolution of skin imaging technology such as total body photographs [6] and dermoscopy [7], combined with AI methodologies, has led to rapid progression in image-based skin lesion diagnosis.”

“If an OOD alert is raised, the input image should be treated as an outlier and its original prediction should not be trusted, which is vital for medical AI applications [30] to mitigate potential misleading of clinicians [31].”

“In summary, our HOT model offers a comprehensive framework for diagnosing skin lesions and its integration has the potential to enhance end-user confidence and aid in making well-informed clinical diagnoses [33].”

“We utilize the Molemap [34] dataset, which is a highly representative skin lesion dataset exhibiting skewed and long-tailed 2 distribution, and is fine-grained in nature.”

“This inconsistency raises a question of whether to rely solely on an AI model’s diagnosis using a clinical lesion image or to obtain a dermoscopic image of the same lesion, leaving potential triage benefits unexplored [26, 27].”

“This provides a more nuanced understanding compared to a single output diagnosis, particularly for challenging borderline cases such as differentiating benign naevi from melanoma based on the dermoscopic image [29].”

“This inconsistency raises a question of whether to rely solely on an AI model’s diagnosis using a clinical lesion image or to obtain a dermoscopic image of the same lesion, leaving potential triage benefits unexplored [26, 27].”

“To tackle the OOD images and provide hierarchical classification in dermato-pathology, Mehta et al. (35) introduced the Hierarchical-Out Distribution-Clinical Triage (HOT) model.”