Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 50 of the 68 sources its claims cite were checked for support.
Abstract
BACKGROUND: Agentic artificial intelligence (AI) systems, characterized by autonomous goal-directed behavior, multi-step reasoning, task decomposition, and tool use, are increasingly proposed for healthcare applications. However, their autonomy raises concerns regarding transparency, accountability, and human oversight. While explainable AI (XAI) has been widely studied in traditional predictive models, less is known about how explainability is implemented within agentic architectures. OBJECTIVE: To map the emerging literature on explainable agentic AI (XAAI) in healthcare and characterize the types, scope, and forms of explainability used in these systems. METHODS: A scoping review was conducted following PRISMA-ScR guidelines. PubMed, Embase, IEEE Xplore, and ACM Digital Library were searched through November 2025. Eligible studies described healthcare-related agentic AI systems incorporating explicit explainability mechanisms. Data were extracted on system architecture, explainability type (intrinsic, post hoc, hybrid), explanation scope (local, global), explanation form, and reported clinical outcomes. RESULTS: Nine studies met the inclusion criteria. All systems demonstrated core agentic features, including autonomy, task decomposition, and tool integration, often within multi-agent frameworks. Explainability was predominantly intrinsic and workflow-native, typically delivered through textual reasoning traces and example-based grounding in retrieved clinical evidence. Feature-based and global explanations were comparatively rare and largely confined to hybrid architectures. Across domains including radiology, neurology, psychiatry, and biomedical research, XAAI systems were reported to improve performance and interpretability relative to baseline models in the included studies. However, these findings were derived from heterogeneous, predominantly experimental or retrospective studies, and structured human-in-the-loop oversight was infrequently described. CONCLUSIONS: Current XAAI systems appear to emphasize process transparency and evidence grounding rather than mechanistic model-level attribution. The available evidence remains limited and heterogeneous, and findings should be interpreted as early trends rather than established characteristics. Further progress will require standardized evaluation frameworks, clearer reporting of oversight mechanisms, and validation in real-world clinical settings to support safe and trustworthy integration of agentic AI into healthcare practice.
Overview
CoChat Summary
TL;DR
This PRISMA-ScR scoping review maps nine studies of explainable agentic AI (XAAI) in healthcare, finding that explainability is predominantly intrinsic, workflow-native, and delivered through textual reasoning traces plus evidence grounding. The evidence base is early, heterogeneous, and largely experimental or retrospective, with little operationalized human-in-the-loop oversight.
Key findings
- Nine studies met inclusion criteria; all systems showed core agentic features—autonomy, task decomposition, and tool integration—often in multi-agent frameworks. Explainability was predominantly intrinsic and workflow-native, typically via textual reasoning traces and example-based grounding in retrieved clinical evidence; feature-based and global explanations were comparatively rare and largely confined to hybrid architectures.
- Reported outcomes were generally favorable versus single-agent LLM, traditional ML, or non-reasoning baselines across radiology, neurology, psychiatry, and biomedical tasks. Specific reported numbers include DSM5AgentFlow F1 scores up to 77%, CASSIA annotation accuracy improvement of 12–41% across large-scale cell populations, and ADAM-1 higher and more stable F1 scores than traditional machine learning models.
- No included study reported prospective clinical deployment. Structured human-in-the-loop oversight was infrequently described, and none clearly operationalized how clinicians or domain experts would supervise, validate, or intervene in autonomous agentic workflows.
- The authors characterize current XAAI as emphasizing process transparency and evidence traceability over mechanistic model-level attribution, with findings to be interpreted as early trends rather than established characteristics.
Methods
- Scoping review conducted following PRISMA-ScR guidelines, with protocol registered in the Open Science Framework. PubMed, Embase, IEEE Xplore, and ACM Digital Library were searched on 24 November 2025; no restriction on publication date or language.
- Eligibility required healthcare-related agentic AI systems with autonomous, goal-directed execution, explicit multi-step reasoning or task decomposition, and explicit explainability/transparency features. Empirical research, simulations, prototypes, architecture papers, algorithmic evaluations, and qualitative studies were included; non-health, non-agentic, non-explainable, or insufficiently detailed studies were excluded.
- Two independent reviewers extracted data on system architecture, explainability type (intrinsic, post hoc, hybrid), scope (local, global), form (feature-based, textual, example-based), specificity, and reported outcomes. Synthesis was descriptive and narrative; no formal meta-analysis was performed due to substantial heterogeneity.
Limitations
- The paper states that the number of included studies is modest, most systems were evaluated experimentally or as proof-of-concept rather than in prospective clinical deployments, and substantial heterogeneity in tasks, architectures, datasets, and reporting limits direct comparison.
- The paper states that reported outcomes were extracted as described by study authors and were not independently verified or standardized; inclusion of preprints and non-peer-reviewed sources may introduce variability in rigor; the adapted XAAI explainability taxonomy has not undergone formal validation; and requiring explicit explainability may have introduced selection bias toward systems already framed as explainable.
- Evident from the text: no meta-analysis was possible, no included study reported prospective clinical deployment, and some trust/usability claims—such as DSM5AgentFlow’s transparent and auditable diagnostic assessments—were not formally evaluated in user-centered studies.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
17 Valid · 2 Not valid · 7 Not checked · 64 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Retrieval-augmented generation (RAG) was frequently incorporated to ground reasoning in external evidence, particularly in systems aiming to mitigate hallucinations or enhance reliability, such as CBM-RAG, DrugAgent, DSM5AgentFlow, CASSIA, and BRAD [19,20,24,25,26].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.3. Overview of Agentic Systems
What the sources say
- Disputedcites [19] Inoue, Y., Song, T., Wang, X., Luna, A., and Fu, T. (2025). DruGagent: Multi-Agent Large Language Model-Based Reasoning for Drug-Target Interaction Prediction. arXiv. sourceExperimental — accuracy not yet measured
The Discussion says a RAG agent “could be added,” presenting it as a possible extension rather than part of DrugAgent. DrugAgent does use search results and knowledge graphs as external evidence for its reasoning, but the cited paper does not describe RAG as incorporated into the system. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
- Supportedcites [20] Alam, H.M.T., Srivastav, D., Selim, A.M., Kadir, M.A., Shuvo, M.M.H., and Sonntag, D. (2025). CBM-RAG: Demonstrating Enhanced Interpretability in Radiology Report Generation with Multi-Agent RAG and Concept Bottleneck Models. Companion Proceedings of the 17th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Association for Computing Machinery. source
The Introduction describes CBM-RAG as using retrieval-augmented generation to incorporate external knowledge, including patient history, prior studies, and current research, into evidence-based radiology reports. It also identifies mitigating hallucinations and improving reliability as aims of the system. This citation supports the CBM-RAG example; it does not need to establish the claims about the other cited systems. (LLM, conf=0.93)
- Supportedcites [24] Ozgun, M.C., Pei, J., Hindriks, K., Donatelli, L., Liu, Q., and Wang, J. (2025). Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis. Proceedings of the 34th ACM International Conference on Information and Knowledge Management, Seoul, Republic of Korea, 10–14 November 2025, Association for Computing Machinery. source
The DSM5AgentFlow paper identifies its diagnostician agent as using RAG (Section 2.3, p. 4). It retrieves relevant DSM-5 passages to ground diagnostic reasoning in authoritative material rather than model priors, supporting the claim as applied to DSM5AgentFlow. (LLM, conf=0.93)
- Supportedcites [25] Pickard, 2025, Automatic biomarker discovery and enrichment with BRAD, Bioinformatics source
The Software architecture section (2.2, Document chat tool) says BRAD retrieves information from documents using a RAG workflow and incorporates it into the LLM prompt. It says this anchors responses in reliable, verifiable sources, supporting the claim as it applies to BRAD. (LLM, conf=0.94)
- Supportedcites [26] Xie, E., Cheng, L., Shireman, J., Cai, Y., Liu, J., Mohanty, C., Dey, M., and Kendziorski, C. (2024). CASSIA: A multi-agent large language model for reference free, interpretable, and automated cell annotation of single-cell RNA-sequencing data. bioRxiv. source
The “Retrieval-Augmented Generation (RAG) pipeline” section describes an optional CASSIA pipeline that queries cell-marker and ontology databases and integrates the resulting information into cell annotation. This supports citing CASSIA as an example of a system using RAG to ground its analysis in external evidence; the paper does not say RAG is used in every CASSIA run. (LLM, conf=0.91)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Consistent with this distribution, post hoc explainability was not used as a standalone strategy but appeared exclusively in hybrid systems that combined intrinsic agent reasoning with attribution-based methods [20,22].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.4. Explainable Methods Used
What the sources say
- Partly supportedcites [20] Alam, H.M.T., Srivastav, D., Selim, A.M., Kadir, M.A., Shuvo, M.M.H., and Sonntag, D. (2025). CBM-RAG: Demonstrating Enhanced Interpretability in Radiology Report Generation with Multi-Agent RAG and Concept Bottleneck Models. Companion Proceedings of the 17th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Association for Computing Machinery. sourceExperimental — accuracy not yet measured
The Methodology (page unavailable) describes a system combining concept contribution scores and saliency heatmaps with multi-agent RAG, including ReAct agents and an optional display of their sequential reasoning. This supports citing the paper as an example of a hybrid approach, but it does not establish that post hoc explainability appeared exclusively in hybrid systems or was never used alone. (LLM, conf=0.84)
- Partly supportedcites [22] Huang, 2025, ADAM-1: An AI Reasoning and Bioinformatics Model for Alzheimer’s Disease Detection and Microbiome-Clinical Data Integration, IEEE Access sourceExperimental — accuracy not yet measured
The Architecture of ADAM > Agentic System section describes a classification agent that combines chain-of-thought reasoning with SHAP feature importance, supporting ADAM as an example of a hybrid approach. The paper does not establish that post hoc explainability appeared exclusively in hybrid systems or was never used as a standalone strategy across the distribution described by the citing paper. No page number is available in our copy. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Multi-agent systems were predominantly role-based, with agents assigned to cognitively separable subtasks such as drafting, verification, retrieval, synthesis, supervision, or reporting [22,26,27].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.3. Overview of Agentic Systems
What the sources say
- Partly supportedcites [22] Huang, 2025, ADAM-1: An AI Reasoning and Bioinformatics Model for Alzheimer’s Disease Detection and Microbiome-Clinical Data Integration, IEEE Access sourceExperimental — accuracy not yet measured
The Architecture of ADAM > Agentic System section describes three agents with distinct computational, summarization, and classification roles, supporting the role-based part of the claim. It does not establish that multi-agent systems were predominantly role-based in general; retrieval and reporting are described as separate components, not assigned agent roles. (LLM, conf=0.82)
- Partly supportedcites [26] Xie, E., Cheng, L., Shireman, J., Cai, Y., Liu, J., Mohanty, C., Dey, M., and Kendziorski, C. (2024). CASSIA: A multi-agent large language model for reference free, interpretable, and automated cell annotation of single-cell RNA-sequencing data. bioRxiv. sourceExperimental — accuracy not yet measured
The Online Methods and Figure 1 description assign CASSIA’s agents distinct annotation, validation, formatting, scoring, and reporting tasks; the RAG pipeline adds retrieval-related tasks. This supports CASSIA as an example of role-based task separation, but the paper does not establish that multi-agent systems were predominantly organized this way. (LLM, conf=0.85)
- Supportedcites [27] Chan, T.K., and Dinh, N.D. (2025). ENTAgents: AI Agents for Complex Knowledge Otolaryngology. medRxiv. source
The Overview (§2.1) describes distinct Drafter, Revisor, Supervisor, and search-agent roles. The Multi-Agent System section (§2.4) assigns search agents to different retrieval tools and has them summarize findings for the Revisor, supporting this paper as an example of role-based task separation. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“By contrast, feature- or concept-based explanations appeared only in systems incorporating conventional predictive components amenable to attribution, such as saliency maps, concept bottleneck representations, or SHAP analyses over structured variables [31,32,33].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 4. Discussion
What the sources say
- Partly supportedcites [31] Singh, Y., Hathaway, Q.A., Keishing, V., Salehi, S., Wei, Y., Horvat, N., Vera-Garcia, D.V., Choudhary, A., Mula Kh, A., and Quaia, E. (2025). Beyond post hoc explanations: A comprehensive framework for accountable AI in medical imaging through transparency, interpretability, and explainability. Bioengineering, 12. sourceExperimental — accuracy not yet measured
Section 4.1, “Dominant XAI Approaches” (page unavailable), describes feature attribution with SHAP and visual explanations such as Grad-CAM for medical-imaging predictions. It does not say these explanations appeared only in systems with conventional predictive components, nor does it address concept bottleneño representations or SHAP analyses over structured variables. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [32] Dakhli, R., and Barhoumi, W. (2024). Toward a quantitative trustworthy evaluation of post-hoc XAI feature importance maps using saliency-based occlusion. Proceedings of the 2024 IEEE/ACS 21st International Conference on Computer Systems and Applications (AICCSA), IEEE. source
- Not checked — no full text of the sourcecites [33] Bhati, 2025, A survey of post-hoc xai methods from a visualization perspective: Challenges and opportunities, IEEE Access source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These dimensions were categorized according to established and previously validated XAI taxonomies [12,17,18], which were adapted to the context of agentic AI systems as a provisional analytic schema to enable consistent classification across heterogeneous implementations.”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 2. Materials and Methods > 2.3. Data Extraction and Synthesis
What the sources say
- Not checked — no full text of the sourcecites [12] Habib, A., Abdulmahmod, O.F., Raza, M., Gu, Y.H., Aydoğan, M., and Al-antari, M.A. (2025). Towards Explainable AI in Agentic Retrieval-Augmented Generation: A Systematic Review. Proceedings of the 2025 9th International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkiye, 6–7 September 2025, IEEE. source
- Partly supportedcites [17] Ali, 2023, Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence, Inf. Fusion sourceExperimental — accuracy not yet measured
Section 1.1 distinguishes post-hoc explanation methods from interpretable-by-design approaches, supporting the use of XAI categories. Our copy does not establish that its taxonomy was previously validated. The adaptation to agentic AI is described as the citing authors’ own analytic step, not a finding attributed to this source. (LLM, conf=0.78)
- Partly supportedcites [18] Chaddad, A., Peng, J., Xu, J., and Bouridane, A. (2023). Survey of explainable AI techniques in healthcare. Sensors, 23. sourceExperimental — accuracy not yet measured
Section 3, “Explainable Artificial Intelligence Techniques,” presents literature-based criteria for classifying XAI methods, including interpretation types and explanation scopes. It does not say these classifications were previously validated. The adaptation to agentic AI is described as the citing authors’ own analytic schema, not as something this source did. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
48 more claims · 8 with their cited sources checked
Create a free account to see all 90 claims and every source verdict.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 18
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.5. Main Clinical Findings
“ArgMed-Agents improved performance relative to direct generation and CoT baselines while enhancing interpretability by making reasoning processes more explicit, particularly in clarifying why specific clinical options were selected or rejected [27].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > References
“27. Chan T.K. Dinh N.D. ENTAgents: AI Agents for Complex Knowledge Otolaryngology medRxiv 2025 10.1101/2025.01.01.25319863”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.5. Main Clinical Findings
“multi-agent workflows such as ENTAgents demonstrated performance gains over baseline LLM approaches, particularly through self-correction mechanisms. Interpretability-related benefits were reflected in clearer and more comprehensive reasoning narratives [21].”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > References
“21. Hong S. Xiao L. Zhang X. Chen J. ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.4. Explainable Methods Used
“all systems relied on multi-agent workflows in which intermediate reasoning states, role-specific outputs, and verification steps were naturally surfaced as user-facing rationales.”
§ Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review > 3. Results > 3.3. Overview of Agentic Systems
“These architectures contrasted with the single-agent, tool-augmented design observed in BRAD [25], where specialization was achieved via modular tools rather than separate agents.”
Citations
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
McGuinness, L. A., & Higgins, J. P. T. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1411 · Cited by 5,579 · cited 1×
“The Robvis tool was used to generate the PRISMA flow chart [16].”
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · cited 3×
“…of explanation was defined as local when explanations pertained to individual decisions, instances, or queries; global when explanations aimed to characterize overall system behavior, decision logic, or generalizable rules; and both when systems supported multiple levels of explanation [17,18].”
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · cited 1×
“In this regard, the apparent predominance of intrinsic, workflow-native explainabil-ity aligns with international regulatory and governance frameworks (e.g., the EU AI Act and FDA framework) that emphasize transparency, traceability, documentation, and human oversight in healthcare AI [44–46].”
Agentic AI: Autonomous intelligence for complex goals–a comprehensive survey
Acharya, D. B., Kuppan, K., & Divya, B. (2025) · IEEE Access · doi:10.1109/access.2025.3532853 · Cited by 717 · cited 2×
“The rapid evolution of artificial intelligence (AI) has given rise to a new class of systems commonly referred to as agentic AI, characterized by autonomous, goal-directed behavior, multi-step reasoning, tool use, and, increasingly, multi-agent coordination [1,2].”
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591 · cited 4×
“…procedures), post hoc when explanations were produced after inference without influencing the underlying decision process (e.g., feature attribution, saliency maps, or SHAP/LIME-style methods), and hybrid when both intrinsic and post hoc mechanisms were combined within the same system [12,18].”
InfluentialAI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
Explainable AI for clinical and remote health applications: A survey on tabular and time series data
Di Martino, F., & Delmastro, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10304-3 · Cited by 216 · cited 1×
“This imbalance is noteworthy because global explainability, understanding how a system behaves across cases, is critical for governance, calibration, bias detection, and safe deployment [34,35].”
Mienye, I. D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I. D., & Ogbuokiri, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101587 · Cited by 212 · cited 1×
“This imbalance is noteworthy because global explainability, understanding how a system behaves across cases, is critical for governance, calibration, bias detection, and safe deployment [34,35].”
Research works printed but not linked: 7
- Blake, H. (2026, April 23). Transparency and Explainability in AI: Ensuring Trust and Understanding in Autonomous Decision-Making Systems. Available online: https://www.researchgate.net/publication/387419097_Transparency_and_Explainability_in_AI_Ensuring_Trust_and_Understanding_in_Autonomous_Decision-Making_Systems.
- Inoue, Y., Song, T., Wang, X., Luna, A., and Fu, T. (2025). DruGagent: Multi-Agent Large Language Model-Based Reasoning for Drug-Target Interaction Prediction. arXiv.
- Ebrahimi, S., and Asudeh, A. (2026, April 23). A Survey on Reliability, Transparency, Accountability, and Fairness in LLM-based Multi-Agent Systems through the Responsibility Lens. submitted. Available online: https://www.researchgate.net/profile/Abolfazl-Asudeh/publication/397650899_A_Survey_on_Reliability_Transparency_Accountability_and_Fairness_in_LLM-based_Multi-Agent_Systems_through_the_Responsibility_Lens/links/691933f9de8143098271909a/A-Survey-on-Reliability-Transparency-Accountability-and-Fairness-in-LLM-based-Multi-Agent-Systems-through-the-Responsibility-Lens.pdf.
- Afolabi, H., Afolabi, Z., Friel, E., Roberts, J., Ji-Xu, A., Chen, L., Ogbomo, E., Imevbore, E., Eneje, P., and El Ouahidi, W. (2025, January 6–7). Faithful or Just Plausible? Evaluating Faithfulness for Medical Reasoning in Closed-Source LLMs. Proceedings of the Second Workshop on GenAI for Health: Potential, Trust, and Policy Compliance, San Diego, CA, USA.
- Edwards, 2021, The EU AI Act: A summary of its significance and scope, Artif. Intell. (EU AI Act), 1, 25
- Garg, 2025, Explainable AI & Model Interpretability in Healthcare: Challenges & Future Directions, Eksplorium-Bul. Pus. Teknol. Bahan Galian Nukl., 46, 104
- Bauer, S.J. (2025). Responsible Artificial Intelligence in Healthcare: Impact on Patient Trust and Satisfaction with AI-Enabled Clinical Decision Support Systems. [Ph.D. Thesis, South College].
Cited by
Dhar, M., Singh, R., & Manikonda, S. C. K. (2026) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/26jul830 · Cited by 2
“Emerging Agentic AI approaches, multi-agent systems where specialized agents collaborate to plan, reason, and act [22][23], are particularly efficient and fit for hospitals because they mirror the multi-role nature of clinical and administrative teams.”
Metadata
- Authors
- Collaco, B. G., Prabha, S., Gomez-Cabello, C. A., Haider, S. A., Genovese, A., Wood, N. G., Gopala, N., Raman, R., Hester, E. O., & Forte, A. J.
- Affiliations
- Division of Plastic Surgery, Mayo Clinic, 4500 San Pablo Rd S, Jacksonville, FL 32224, USA
- Mayo Clinic (US)
- Corresponding author
- Forte, Antonio Jorge
- ORCID
- Venue
- Bioengineering
- Publisher
- MDPI AG
- Published
- 2026-04-28
- Volume
- 13
- Issue
- 5
- Pages
- 513
- ISSN
- 2306-5354 (ISSN-L)
- PMC
- PMC13203805
- OpenAlex
- W7159842253
- Cited by
- 3
- References
- 40
- Citations by year
- 2026: 3
- FWCI
- 2.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioengineering (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Schmidt Sciences
- Richard M. Schulze Family Foundation
- Gerstner Philanthropies
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- agentic AI, explainable AI, artificial intelligence in healthcare, multi-agent systems, post-hoc explanation, task decomposition, tool use, human-in-the-loop oversight, clinical decision support, radiology, neurology, biomedical research, transparency and accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.