Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens

May combine two claims.

ResNet 101 achieved 98.39% accuracy, 98% precision, 98% recall, and 98% F1-Score on Dataset 2 (CT scan COVID-19 detection).

“| ResNet 101 | 98.39% | 98% | 98% | 98% |”

May combine two claims.

Ten participants expressed no clear opinion on the coloring scheme, and only three disagreed that it had a negative impact.

“Ten participants expressed no clear opinion, and only three disagreed.”

May combine two claims.

The DL diagnostic systems achieved accuracy rates of 90% for pneumonia and 98% for COVID-19 in the clinical scenarios.

“The achieved accuracy rates were 90% for pneumonia and 98% for COVID-19.”

May combine two claims.

Participants expressed concerns about misclassification of specific regions of interest and apprehension that greyed-out areas in LIME visualizations might lead to omission of essential details.

“They expressed concerns about misclassification of specific regions of interest and apprehension that the greyed-out areas in the visualizations might lead to the omission of essential details in the input image.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Most participants possess extensive experience in their medical fields and radiology image analysis.

“While most participants possess extensive experience in their medical fields and radiology image analysis”

Probably not a substantial claim: may not pass S3 (load-bearing).

13 participants have more than 10 years of experience analyzing radiology images.

“The histogram indicates that 13 participants have more than 10 years of experience”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The loss plot for Dataset 1 shows steady decrease while the accuracy plot shows steady increase over time before early stopping terminates the training to avoid overfitting.

“The loss plot for Dataset 1 (Pneumonia) shows steady decrease while the accuracy plot shows steady increase over time before early stopping terminates the training to avoid overfitting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

14 participants reported being "little familiar" with AI.

“The figure shows that 14 participants reported being “little familiar” with AI”

May combine two claims.

Participants who gave high scores found the heatmaps beneficial for focusing on critical areas, confirming medical suspicions, and improving precision.

“These participants found the heatmaps beneficial for focusing on critical areas, confirming medical suspicions, and improving precision.”

Classified as probably not a substantial claim.

Inclusive design was identified as a crucial need to ensure better alignment of XAI systems with user needs.

“Inclusive design was also identified as a crucial need to ensure better alignment of these systems with user needs.”

• AssumptionRests on an assumption: The user study participants are representative of medical practitioners whose perceptions matter for XAI acceptance.
• AssumptionRests on an assumption: The online survey responses accurately reflect participants' perceptions and are not substantially biased.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The loss plot for Dataset 2 shows steady decrease while the accuracy plot shows steady increase over time, however all epochs run until completion.

“Similarly the loss plot for Dataset 2 (COVID-19) shows steady decrease while the accuracy plot shows steady increase over time, however all epochs run until completion.”

Probably not a substantial claim: may not pass S3 (load-bearing).

18 participants have more than 10 years of total medical experience.

“The figure indicates that 18 participants have more than 10 years of experience”

May combine two claims.

Six participants rated the Grad-CAM heatmap visualizations below three for usefulness, expressing concerns about potential cognitive bias and inaccurate explanation.

“On the other hand, six participants rated the heatmap visualizations below three for usefulness, expressing concerns about potential cognitive bias and inaccurate explanation.”

May combine two claims.

Participants' responses primarily emphasised human supervision, highly accurate results, transparency in training/testing, clear understanding of ethical aspects, transparency in decision-making, and enhancement of their own AI knowledge as factors that would enhance acceptance of AI-based diagnostic systems.

“The responses primarily emphasised the importance of human supervision of the diagnostic system, the need for highly accurate results, transparency in the training and testing processes, clear understanding of the ethical aspects of AI-based diagnosis, transparency in the decision-making process, and the enhancement of their own AI knowledge.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A total of 26 clinicians participated in the survey, with 8 identifying as Radiologists and 18 having various medical specialties.

“A total of 26 clinicians participated in the survey, with 8 identifying as Radiologists, and the remaining 18 having various medical specialties”

May combine two claims.

Nine participants scored LIME less than 2 for the usefulness criteria, while 6 participants rated Grad-CAM below three.

“while 6 participants rated Grad-CAM below three, 9 participants scored LIME less than 2 for the usefulness criteria.”

• Also stated in: § 5.2 Part 3 Result: Evaluation and comparison of XAI tools · plain

  “The figure shows that nine participants scored LIME less than 2 for the usefulness criteria.”

May combine two claims.

Only 6 participants assigned a score of 4 or 5 for the comprehensibility criteria of LIME visualizations, compared to 9 participants for Grad-CAM.

“Regarding LIME visualizations, unlike Grad-CAM, where 9 participants provided rating of 4 or 5, only 6 participants assigned a score of 4 or 5 for the comprehensibility criteria”

• Also stated in: § 5.2 Part 3 Result: Evaluation and comparison of XAI tools · plain

  “The figure shows that only six participants assigned a score of 4 or 5 for the comprehensibility criteria.”

May combine two claims.

Five participants rated Grad-CAM heatmap visualisations below three for comprehensibility, primarily due to incorrect categorization, loss of focus on original image, and difficulty comprehending due to unfamiliarity.

“On the other hand, five participants rated the heatmap visualisations below three, primarily due to incorrect categorization in the heatmap, a loss of focus on the original image, and difficulty comprehending the heatmaps due to their unfamiliarity with them.”

May combine two claims.

Most participants preferred Grad-CAM heatmap visualizations over LIME, provided there is enhanced accuracy and additional details accompanying the visualizations.

“However, most participants preferred Grad-CAM heatmap visualizations over LIME, provided there is enhanced accuracy and additional details accompanying the visualizations to shed light on why the AI system identified certain areas as significant.”

• Also stated in: § 7 Conclusion and future work · plain

  “Interestingly, most participants favored Grad-CAM heatmaps over LIME, but they emphasized the importance of improved accuracy and more details to explain why the AI system identified certain areas as significant.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The participant who disagreed indicated that the models were ineffective and the XAI visualisations did not provide any help.

“In contrast, the participant who disagreed indicated that the models were ineffective, and the XAI visualisations did not provide any help.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Four participants neither agreed nor disagreed regarding involvement of medical practitioners in design and development of XAI systems for medical imaging.

“Regarding the involvement of medical practitioners in the design and development of XAI systems for medical imaging, four participants neither agreed nor disagreed.”

May combine two claims.

Twenty-one participants either agreed or strongly agreed with involving medical practitioners in XAI design and development, emphasising that clinicians are best suited to define their needs from medical imaging output.

“On the other hand, twenty-one participants either agreed or strongly agreed, emphasising that clinicians are best suited to define their needs from medical imaging output and should prioritise the development of AI systems to meet those needs.”

May combine two claims.

Nine participants expressed confidence in the accuracy of the XAI visualisations, while seven participants lacked confidence in the results.

“In terms of confidence in the output from both XAI tools, nine participants expressed confidence in the accuracy of the visualisations, while seven participants lacked confidence in the results.”

• Also stated in: § 5.2 Part 3 Result: Evaluation and comparison of XAI tools · plain

  “The figure shows that nine participants expressed confidence in the accuracy of the XAI visualisations, while seven participants lacked confidence in the results.”

May combine two claims.

Some participants found Grad-CAM visualisations to have ample detail in explanations and were easy to understand.

“Some found the visualisations to have ample detail in explanations and were easy to understand.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Those four participants noted that factors such as time and schedules of clinicians could affect their involvement in such projects.

“They noted that factors such as time and the schedules of clinicians could affect their involvement in such projects.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Four participants neither agreed nor disagreed, and one participant strongly disagreed that XAI visualisations have the potential to enhance radiology practices.

“Meanwhile, four participants neither agreed nor disagreed, and one participant strongly disagreed.”

May combine two claims.

One participant responded 'No', nine participants were uncertain, and fifteen participants responded 'Yes' to adding descriptive sentences (textual explanations) alongside visual explanations to improve clarity.

“With regards to adding descriptive sentences (textual explanations) alongside visual explanations to improve clarity, one participant responded ‘No’, nine participants were uncertain, and fifteen participants responded ‘Yes’.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The main rationale for participants in favour of textual explanations was that descriptions would assist in verifying the system's AI decision using a more coherent human language.

“The main rationale for those in favour was that descriptions would assist in verifying the system’s AI decision using a human language that is more coherent”

May combine two claims.

Six participants rated LIME visualisations below three for comprehensibility, primarily because some unimportant areas were inaccurately highlighted and greyed-out areas introduced cognitive bias.

“On the other hand, six participants rated the visualisations below three, primarily because some unimportant areas were inaccurately highlighted, and the greyed-out areas introduced cognitive bias.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some participants expressed that involvement of clinicians would boost their confidence in the output of AI systems.

“As illustrated in Fig 30, some participants expressed that the involvement of clinicians would boost their confidence in the output of AI systems”

May combine two claims.

Nineteen participants favoured Grad-CAM heatmap over LIME visualisations, while six participants preferred LIME.

“In terms of the understandability and preference between the two XAI methods, nineteen participants favoured Grad-CAM (heatmap) over LIME visualisations, which were preferred by six participants”

• Also stated in: § 5.2 Part 3 Result: Evaluation and comparison of XAI tools · plain

  “The figure shows that nineteen participants favoured Grad-CAM (heatmap) over LIME visualisations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants who agreed that XAI visualisations can enhance radiology practices believed they could alleviate workload, aid in triaging patients for urgent care, improve efficiency, and provide valuable assistance to clinicians.

“Those who expressed agreement believed that enhanced XAI visualisations could alleviate the workload on radiology services, aid in triaging patients for urgent care, improve the efficiency of medical radiology practices, and provide valuable assistance to clinicians.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The selected evaluation measures (comprehensibility, confidence, and clinical relevance) have been recently suggested by [24] for evaluating clinical XAI in medical image analysis.

“The selected evaluation measures has been recently suggested by [24] for evaluating clinical XAI in medical image analysis.”

• cites [24] W Jin, 2023, Guidelines and evaluation of clinical explainable AI in medical image analysis, Medical Image Analysis sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Another objective evaluation was conducted in [30] by comparing the same three visual XAI methods in the context of COVID-19 detection from CT images.

“Another objective evaluation was conducted in [30] by comparing the same three visual XAI methods in the context of COVID-19 detection from CT images.”

• cites [30] Ye Q, Xia J, Yang G. Explainable AI for COVID-19 CT classifiers: an initial comparison study. In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2021. p. 521–526. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The explainability information can be utilized by clinicians during clinical consultation sessions to inform patients and involve them in the decision-making process.

“Moreover, the explainability information can be utilized by clinicians during clinical consultation sessions to inform patients and involve them in the decision-making process.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In [17], Grad-CAM, LIME, and SHAP were compared and evaluated based on criteria such as consistency, fidelity, sensitivity, and relevance, within the context of a skin lesion image classification task.

“For instance, in [17], Grad-CAM, LIME, and Shapley Additive exPlanation (SHAP) [15]—XAI algorithms were compared and evaluated based on criteria such as consistency, fidelity, sensitivity, and relevance, within the context of a skin lesion image classification task.”

• cites [17] Sun J, Chakraborti T, Noble JA. A Comparative Study of Explainer Modules Applied to Automated Skin Lesion Classification. In: XI-ML@ KI; 2020.Not checked yet

  Not checked in the free run.

• cites [15] Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Advances in neural information processing systems. 2017;30.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The latter study [30] lacked using a clear evaluation measure and did not offer distinct findings on the strengths or weaknesses of the compared methods.

“However, the latter study lacked using a clear evaluation measure and did not offer distinct findings on the strengths or weaknesses of the compared methods.”

• cites [30] Ye Q, Xia J, Yang G. Explainable AI for COVID-19 CT classifiers: an initial comparison study. In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2021. p. 521–526. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Grad-CAM, LIME, and SHAP XAI algorithms have gained great popularity in the area of intelligent medical investigation and imaging.

“In recent years, Grad-CAM (Gradient-weighted Class Activation Mapping) [13], LIME (Local Interpretable Model Agnostic Explanations) [14], and SHAP (SHapley Additive exPlanations) [15] XAI algorithms have gained great popularity in the area of intelligent medical investigation and imaging (e.g. [8, 16, 17]).”

• cites [13] Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision; 2017. p. 618–626. sourceNot checked yet

  Not checked in the free run.

• cites [14] Ribeiro MT, Singh S, Guestrin C. “Why should i trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016. p. 1135–1144. sourceNot checked yet

  Not checked in the free run.

• cites [15] Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Advances in neural information processing systems. 2017;30.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The explainability information from these methods is valuable for radiologists in second reporting, providing a means to validate and/or assess AI decisions.

“This information is valuable for radiologists in second reporting, providing a means to validate and/or assess AI decisions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

According to recent findings, most XAI methods used in medical image analysis apply local post hoc explanations rather than global model-based ones.

“According to recent findings in [8, 9], most XAI methods used in medical image analysis apply local post hoc explanations rather than global model-based ones.”

• cites [8] BH Van der Velden, 2022, Explainable artificial intelligence (XAI) in deep learning-based medical image analysis, Medical Image Analysis sourceNot checked yet

  Not checked in the free run.

• cites [9] W Saeed, 2023, Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities, Knowledge-Based Systems sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is a growing demand, especially among the medical community, to enhance the interpretability of DL techniques applied to medical images, increasingly fueled by emerging regulations concerning AI privacy, fairness, and awareness such as the European GDPR.

“There is a growing demand, especially among the medical community, to enhance the interpretability of DL techniques applied to medical images, increasingly fueled by emerging regulations concerning AI privacy, fairness, and awareness (e.g. European general data protection regulation(GDPR) [7]).”

• cites [7] P Voigt, 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Another user study in [32] involving clinicians from two distinct acute care specialties identified specific elements of explainability that could enhance trust in AI models and the types of explanations clinicians deemed most relevant and crucial for effective translation into clinical practice.

“Another user study was conducted in [32], involving clinicians from two distinct acute care specialties, aiming at identifying specific elements of explainability that could enhance trust in AI models. The study helped to identify the types of explanations that clinicians deemed most relevant and crucial for effective translation into clinical practice.”

• cites [32] Tonekaboni S, Joshi S, McCradden MD, Goldenberg A. What clinicians want: contextualizing explainable machine learning for clinical end use. In: Machine learning for healthcare conference. PMLR; 2019. p. 359–380.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This classification system was also suggested in [29], however they referred to the two classes as heuristic-based and user-based metrics.

“This classification system was also suggested in [29], however, they referred to the two classes as heuristic-based and user-based metrics.”

• cites [29] Bibal A, Frénay B. Interpretability of machine learning models and representations: an introduction. In: 24th european symposium on artificial neural networks, computational intelligence and machine learning. CIACO; 2016. p. 77–82.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

[24] proposed five clinical guidelines to support the design and evaluation of clinically-oriented XAI systems: Understandability, Clinical Relevance, Truthfulness, Informative Plausibility, and Computational Efficiency.

“For instance, [24] proposed five clinical guidelines to support the design and evaluation of clinically-oriented XAI systems. These guidelines encompass Understandability, Clinical Relevance, Truthfulness, Informative Plausibility, and Computational Efficiency.”

• cites [24] W Jin, 2023, Guidelines and evaluation of clinical explainable AI in medical image analysis, Medical Image Analysis sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

These methods can be easily integrated with various DL architectures, making them applicable across a broad range of DL models.

“These methods can be easily integrated with various DL architectures, making them applicable across a broad range of DL models [18].”

• cites [18] A Holzinger, 2022 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deep learning methods, particularly Convolutional Neural Networks, have become the foundation of various Computer Assisted Diagnostic systems due to their ability to detect medical image abnormalities with high accuracy, including COVID-19 and pneumonia.

“In recent years, deep learning (DL) methodologies, particularly Convolutional Neural Networks (CNNs), have emerged as powerful tools for automatically detecting chest diseases using medical images like CT scans and X-rays [2–4].”

• Also stated in: § 1 Introduction · plain

  “These methods have become the foundation of various Computer Assisted Diagnostic systems due to their remarkable ability to detect medical image abnormalities with high accuracy, including COVID-19 and pneumonia diseases (e.g. [5, 6]).”

• cites [2] Song TH, Landini G, Fouad S, Mehanna H. Epithelial segmentation from in situ hybridisation histological samples using a deep central attention learning approach. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE; 2019. p. 1527–1531. sourceNot checked yet

  Not checked in the free run.

• cites [3] D Shen, 2017, Deep learning in medical image analysis, Annual review of biomedical engineering sourceNot checked yet

  Not checked in the free run.

• cites [4] S Fouad, 2021, Human papilloma virus detection in oropharyngeal carcinomas with in situ hybridisation using hand crafted morphological features and deep central attention residual networks, Computerized Medical Imaging and Graphics sourceNot checked yet

  Not checked in the free run.

• cites [5] SS Yadav, 2019, Deep convolutional neural network based medical image classification for disease diagnosis, Journal of Big data sourceNot checked yet

  Not checked in the free run.

• cites [6] Mporas I, Naronglerdrit P. COVID-19 identification from chest X-Rays. In: 2020 International conference on biomedical innovations and applications (BIA). IEEE; 2020. p. 69–72. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A research in [21] presented a case study illustrating the application of a human-centered design approach to AI-generated explanations, involving domain analysis, requirements elicitation and assessment, and multi-modal interaction design and evaluation for a Clinical Decision Support System focused on child health.

“In medical diagnosis problems, a research in [21] presented a case study illustrating the application of a human-centered design approach to AI-generated explanations. The case study involved domain analysis, requirements elicitation and assessment, and multi-modal interaction design and evaluation. The study designed explanations for a Clinical Decision Support System (DSS) focused on child”

• cites [21] TA Schoonderwoerd, 2021, Human-centered XAI: Developing design patterns for explanations of clinical decision support systems, International Journal of Human-Computer Studies sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

These methods mainly explain AI decisions by emphasizing image regions or data features that significantly contribute to model predictions.

“They mainly explain AI decisions by emphasizing image regions or data features that significantly contribute to model predictions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A user study in [31] revealed that clinicians expressed higher satisfaction with DSS outputs when provided with the rationale behind the AI decisions.

“A user study was conducted which relieved that clinicians expressed higher satisfaction with DSS outputs when provided with the rationale behind the AI decisions.”

• cites [31] Bussone A, Stumpf S, O’Sullivan D. The role of explanations on trust and reliance in clinical decision support systems. In: 2015 international conference on healthcare informatics. IEEE; 2015. p. 160–169. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

These guidelines were developed through a dual perspective, integrating insights from a physician user study on clinical requirements for explanations and the technical expertise of the authors.

“They were developed through a dual perspective, integrating insights from a physician user study on clinical requirements for explanations and the technical expertise of the authors.”

• cites [24] W Jin, 2023, Guidelines and evaluation of clinical explainable AI in medical image analysis, Medical Image Analysis sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A recent study suggested two main ways for evaluating AI explainability: objective evaluations using objective metrics and human-centered evaluations involving human judgment and feedback.

“A recent study in [28] has suggested two main ways for evaluating AI explainability, (a) objective evaluations, which involve research studies that utilize objective metrics, including data driven automated approaches to assess explainability methods, and (b) human-centered evaluations, which involve human judgment and feedback into the assessment of AI models’ explanations.”

• cites [28] G Vilone, 2021, Notions of explainability and evaluation approaches for explainable artificial intelligence, Information Fusion sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The majority of current XAI DL-based approaches for computer vision tasks provide visual interpretations, highlighting image regions that significantly influence a model's decision.

“The majority of current XAI DL-based approaches for computer vision tasks provide visual interpretations, highlighting image regions that significantly influence a model’s decision.”

Probably not a substantial claim: may not pass S4 (based).

As far as the authors are aware, human-centered evaluations for visual XAI methods have not yet been explored in the context of chest radiology imaging research.

“As far as we are aware, human-centered evaluations for visual XAI methods have not yet been explored in the context of chest radiology imaging research.”

May combine two claims.

Visual explainability holds great promise in AI-based tools for medical imaging, but several usability issues may arise that can hinder their effective adoption in clinical settings.

“While visual explainability holds great promise in AI-based tools for medical imaging, several usability issues may arise that can hinder their effective adoption in clinical settings [9, 12].”

• cites [9] W Saeed, 2023, Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities, Knowledge-Based Systems sourceNot checked yet

  Not checked in the free run.

• cites [12] AM Antoniadi, 2021, Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: a systematic review, Applied Sciences sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The explainability information can be incorporated into radiology reports to enhance quality assurance.

“It can also be incorporated into radiology reports to enhance quality assurance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This study [24] highlights the need for XAI systems to not only provide explanations but to ensure their usability and relevance in supporting informed decision-making in medical practices.

“This study highlights the need for XAI systems to not only provide explanations but to ensure their usability and relevance in supporting informed decision-making in medical practices [21–23].”

• cites [21] TA Schoonderwoerd, 2021, Human-centered XAI: Developing design patterns for explanations of clinical decision support systems, International Journal of Human-Computer Studies sourceNot checked yet

  Not checked in the free run.

• cites [22] Liao QV, Varshney KR. Human-Centered Explainable AI (XAI): From Algorithms to User Experiences. CoRR. 2021;abs/2110.10790.Not checked yet

  Not checked in the free run.

• cites [23] H Chen, 2022, Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review, NPJ digital medicine sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

User studies with experienced paediatricians uncovered necessary explanations during requirements elicitation and assessment, and the resulting interaction design patterns were tested in a second user study to evaluate the effectiveness of the explainable DSS in medical diagnosis.

“User studies with experienced paediatricians uncovered necessary explanations during requirements elicitation and assessment. The resulting interaction design patterns were tested in a second user study to evaluate the effectiveness of the explainable DSS in medical diagnosis.”

• cites [21] TA Schoonderwoerd, 2021, Human-centered XAI: Developing design patterns for explanations of clinical decision support systems, International Journal of Human-Computer Studies sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Adjustments were implemented in specific sections of the questionnaire based on feedback from the experienced radiologist.

“Following the pilot study, adjustments were implemented in specific sections of the questionnaire based on feedback received from the experienced radiologist.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Before designing the user study, the authors interviewed radiology and clinical specialists to aid in designing relevant survey questions.

“To this end, before designing our user study we interviewed radiology and clinical specialists to aid in designing relevant survey questions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The need for consent was waived by Aston University ethics committee.

“The need for consent was waived by Aston University ethics committee.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study utilised early stopping in the models to stop training after the optimal number of iterations has been reached.

“We also utilised early stopping in the models to stop training the model after its optimal number of iterations has been reached.”

May combine two claims.

Dataset 1 contains 5,863 X-ray images from Guangzhou Women and Children's Medical Center, split across pneumonia/normal categories, retrospectively selected from paediatric patients aged one to five years.

“Dataset 1: Chest X-ray Images for pneumonia detection. This dataset contains 5,863 X-ray images (JPEG) from Guangzhou Women and Children’s Medical Center, split across two categories—pneumonia/normal [33]. The chest X-ray images (anterior-posterior) were retrospectively selected from cohorts of paediatric patients (aged one to five years) from Guangzhou Women and Children’s Medical Center.”

• cites [33] D Kermany, 2018, Labeled optical coherence tomography (oct) and chest x-ray images for classification, Mendeley dataNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

CNN-based models were employed for automatic diagnosis of chest diseases from X-ray and CT scans due to their effectiveness in capturing spatial hierarchies and patterns within medical imaging data, enabling robust feature extraction for accurate diagnostic predictions.

“CNN-based models were employed for the automatic diagnosis of chest diseases from X-ray and CT scans due to their effectiveness in capturing spatial hierarchies and patterns within medical imaging data, enabling robust feature extraction for accurate diagnostic predictions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A pilot study involving an experienced radiologist (>15 years) was conducted to assess the questionnaire's appropriateness for clinicians.

“Following the completion of the questionnaire, a pilot study involving an experienced radiologist (> 15 years) was conducted to assess its appropriateness for clinicians.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A study in remote sensing image classification compared the performance of ten different XAI methods and found that Grad-CAM and LIME were the most interpretable and reliable.

“For instance, a study in remote sensing image classification [36] compared the performance of ten different XAI methods and found that Grad-CAM and LIME were the most interpretable and reliable.”

• cites [36] I Kakogeorgiou, 2021, Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing, International Journal of Applied Earth Observation and Geoinformation sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The architectures VGG-19, ResNet-50, ResNet-101, DenseNet169, and MobileNetV2 have shown promising results in similar automatic diagnostic tasks in medical imaging.

“To this end, we trained several CNN models, including VGG-19, ResNet-50, ResNet-101, DenseNet169 and MobileNetV2 on both datasets. These architectures have shown promising results in similar automatic diagnostic tasks in medical imaging [4, 5].”

• cites [4] S Fouad, 2021, Human papilloma virus detection in oropharyngeal carcinomas with in situ hybridisation using hand crafted morphological features and deep central attention residual networks, Computerized Medical Imaging and Graphics sourceNot checked yet

  Not checked in the free run.

• cites [5] SS Yadav, 2019, Deep convolutional neural network based medical image classification for disease diagnosis, Journal of Big data sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The pilot study validated the questions in alignment with the study's medical objectives, evaluated the coherence of the question flow, and optimized the use of medical terminology for clearer communication.

“The pilot study focused on validating the questions in alignment with the study’s medical objectives, evaluating the coherence of the question flow, and optimizing the use of medical terminology for clearer communication within the medical field.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The evaluation criteria emerged through collaborative effort and expertise from both clinicians and medical image analysis experts.

“The above criteria emerged through collaborative effort and expertise from both clinicians and medical image analysis experts.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants received a detailed email outlining the research project objectives and their role, and consent was obtained before they accessed the online survey.

“Participants received a detailed email outlining the research project objectives and their role in the study. Consent was obtained before they accessed the online survey, which was open for nearly a month to facilitate participation.”

• Also stated in: § 4.2 User study ethical statement · plain

  “Participants received an initial detailed email communication outlining the research project objectives and their role. After receiving positive responses, participants were provided with a link to the online survey, accompanied by a participant information sheet explaining their involvement, data handling procedures, and survey result utilization. Online consent (collected through online Microso”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both training and validation loss curves were continuously monitored to ensure that no significant divergence between these curves occurred, which is often a good indicator of overfitting.

“Furthermore, both the training and validation loss curves were continuously monitored to ensure that no significant divergence between these curves occurred, which is often a good indicator of overfitting.”

Classified as probably not a substantial claim. May combine two claims.

The findings highlight key user-driven explainability requirements, emphasizing the importance of multi-modal explainability and the necessity to increase awareness of XAI systems among medical practitioners.

“Our findings highlight key user-driven explainability requirements, emphasizing the importance of multi-modal explainability and the necessity to increase awareness of XAI systems among medical practitioners.”

• AssumptionRests on an assumption: The user study participants are representative of medical practitioners whose perceptions matter for XAI acceptance.
• AssumptionRests on an assumption: The online survey responses accurately reflect participants' perceptions and are not substantially biased.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior to utilizing the datasets and commencing the user study, ethical approval was obtained from Aston University (Ethics Approval Number: 234700-06).

“Prior to utilizing these datasets and commencing our user study, ethical approval was obtained from Aston University (Ethics Approval Number: 234700–06).”

• Also stated in: § 4.2 User study ethical statement · plain

  “Before commencing the user study, ethical approval was obtained from Aston University (Ethics Approval Number—234700–06).”

May combine two claims.

A research in [17] comparing Grad-CAM, SHAP, and LIME in the context of medical imaging concluded that Grad-CAM and LIME were more reliable, whereas SHAP was not the best for local accuracy in this application.

“Similarly, a research in [17] comparing Grad-CAM, SHAP, and LIME in the context of medical imaging concluded that Grad-CAM and LIME were more reliable, whereas SHAP was not the best for local accuracy in this application.”

• cites [17] Sun J, Chakraborti T, Noble JA. A Comparative Study of Explainer Modules Applied to Automated Skin Lesion Classification. In: XI-ML@ KI; 2020.Not checked yet

  Not checked in the free run.

May combine two claims.

Dataset 2 comprises 2,482 CT scans from patients in Sao Paulo, with 1,252 CT scans positive for COVID-19 infection and 1,230 CT scans negative.

“Dataset 2: CT Scans for SARS-CoV-2 (COVID-19) detection: This dataset comprises 2,482 CT scans from patients in Sao Paulo, with 1,252 CT scans positive for COVID-19 infection and 1,230 CT scans are negative [35].”

• cites [35] E Soares, 2020, SARS-CoV-2 CT-scan dataset: A large dataset of real patients CT scans for SARS-CoV-2 identification, MedRxivNot checked yet

  Not checked in the free run.

May combine two claims.

Findings in [15] highlight that while SHAP provides comprehensive feature importance in non-imaging datasets, it may produce less stable explanations in complex image classification tasks, leading to potential inconsistencies.

“This is consistent with another findings in [15], which highlights that while SHAP provides comprehensive feature importance in non-imaging datasets, it may produce less stable explanations in complex image classification tasks, leading to potential inconsistencies.”

• cites [15] Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Advances in neural information processing systems. 2017;30.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used L2 regularization and dropout to penalize model complexity and minimize the risk of overfitting.

“This includes applying regularization techniques, specifically L2 regularization and dropout to penalize model complexity and minimizing the risk of overfitting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous work has shown that effective image classification performance can be achieved through pre-trained models fine-tuned on specific tasks.

“Additionally, previous work has shown that effective image classification performance can be achieved through pre-trained models fine-tuned on specific tasks [6, 16].”

• cites [6] Mporas I, Naronglerdrit P. COVID-19 identification from chest X-Rays. In: 2020 International conference on biomedical innovations and applications (BIA). IEEE; 2020. p. 69–72. sourceNot checked yet

  Not checked in the free run.

• cites [16] Barzas K, Fouad S, Jasa G, Landini G. An Explainable Deep Learning Framework for Mandibular Canal Segmentation from Cone Beam Computed Tomography volumes. In: The 12th International Conference on Computational Advances in Bio and Medical Sciences. Springer; 2023.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both datasets are ethically approved for research and are licensed under the Creative Commons Attribution 4.0 International license.

“Both datasets are ethically approved for research and are licensed under the Creative Commons Attribution 4.0 International license, which allows for sharing, copying, and modifying the datasets for research purposes.”

May combine two claims.

The main limitation of this study is the few number of participants due to the scarcity of clinicians' availability and the low adoption of AI technology in the current radiology workflow.

“The main limitation of this study is the few number of participants due to the scarcity of clinicians’ availability and the low adoption of AI technology in the current radiology workflow.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The above points from this study are transferable to other medical imaging modalities such as pathology and MRI images.

“Even though this study was carried out in the field of chest radiology imaging, the above points are transferable to other medical imaging modality (such as pathology and MRI images).”

May combine two claims.

Some attempts to combine visual relevance and textual explanations (e.g [38]) were not designed with the user in mind and hence don't address the user's preferences and needs.

“While there are some attempts to combine visual relevance and textual explanations (e.g [38]), these techniques were not designed with the user in mind and hence don’t address the user’s preferences and needs.”

• cites [38] Wang X, Peng Y, Lu L, Lu Z, Summers RM. Tienet: Text-image embedding network for common thorax disease classification and reporting in chest x-rays. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2018. p. 9049–9058. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The lack of familiarity with AI and XAI tools can be attributed to little exposure to such tools in clinical settings and in current medical education.

“This lack of familiarity can be attributed to little exposure to such tools in clinical settings and in current medical education.”

• AssumptionRests on an assumption: XAI applications are not yet widely implemented in clinical settings nor in medical education.

Probably not a substantial claim: may not pass S3 (load-bearing).

The concept of human-centered explainability has recently received significant attention in the literature and among medical imaging researchers.

“The concept of human-centered explainability has recently received significant attention in the literature and among medical imaging researchers [21–23].”

• cites [21] TA Schoonderwoerd, 2021, Human-centered XAI: Developing design patterns for explanations of clinical decision support systems, International Journal of Human-Computer Studies sourceNot checked yet

  Not checked in the free run.

• cites [22] Liao QV, Varshney KR. Human-Centered Explainable AI (XAI): From Algorithms to User Experiences. CoRR. 2021;abs/2110.10790.Not checked yet

  Not checked in the free run.

• cites [23] H Chen, 2022, Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review, NPJ digital medicine sourceNot checked yet

  Not checked in the free run.