A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement
Open accessAbstract
Virtual Mental Health Assistants (VMHAs) continuously evolve to support the overloaded global healthcare system, which receives approximately 60 million primary care visits and 6 million emergency room visits annually. These systems, developed by clinical psychologists, psychiatrists, and AI researchers, are designed to aid in Cognitive Behavioral Therapy (CBT). The main focus of VMHAs is to provide relevant information to mental health professionals (MHPs) and engage in meaningful conversations to support individuals with mental health conditions. However, certain gaps prevent VMHAs from fully delivering on their promise during active communications. One of the gaps is their inability to explain their decisions to patients and MHPs, making conversations less trustworthy. Additionally, VMHAs can be vulnerable in providing unsafe responses to patient queries, further undermining their reliability. In this review, we assess the current state of VMHAs on the grounds of user-level explainability and safety, a set of desired properties for the broader adoption of VMHAs. This includes the examination of ChatGPT, a conversation agent developed on AI-driven models: GPT3.5 and GPT-4, that has been proposed for use in providing mental health services. By harnessing the collaborative and impactful contributions of AI, natural language processing, and the mental health professionals (MHPs) community, the review identifies opportunities for technological progress in VMHAs to ensure their capabilities include explainable and safe behaviors. It also emphasizes the importance of measures to guarantee that these advancements align with the promise of fostering trustworthy conversations.
AI snapshot
Written from the paper’s full text.
TL;DR
This review assesses explainability and safety in virtual mental health assistants (VMHAs), including ChatGPT, Woebot, Wysa, Limbic, and ALLEVIATE, and argues that current systems often lack user-level explainability and clinically grounded safe responses. It proposes knowledge-infused learning with clinical guidelines such as PHQ-9 and GAD-7, plus new qualitative and knowledge-infusion evaluation metrics, to improve trustworthiness.
Key findings
- The review reports that existing VMHAs such as Woebot, Wysa, and ChatGPT generated assumptive questions or responses with no association to a clinical reference or clinical support in a real-world unemployment-related query.
- It cites evidence that safety lexicons from PHQ-9 and GAD-7 produced an 85% improvement in safety across sequence-to-sequence and attention-based language models, with explainability showing a 23% uptake in terms of safety across the same models.
- Approximately 70% of attendees at the inaugural safety for conversational AI workshop doubted the ability of present-day conversational systems relying on language models to produce safe responses.
- Table 2 contrasts existing VMHAs with a desired system: Woebot, Wysa, Limbic, and ALLEVIATE mostly lack user-level explainability, while the desired system is marked for screening, triaging, motivational interviewing, safety, user-level explainability, and qualitative metrics of harmlessness, adherence, and transparency.
Methods
- Narrative review organized around five research questions on defining explainability and safety in VMHAs, assessing current capabilities and limitations, analyzing AI challenges, exploring desired functionalities, and identifying evaluation changes.
- Evidence includes examination of ChatGPT/GPT-3.5/GPT-4, Woebot, Wysa, Limbic, and ALLEVIATE, conversational datasets, and a high-level analysis of publicly available user reviews on forums such as WebMD, AskaPatient, MedicineNet, and Reddit; Table 2 notes a survey of 40 participants at Prisma Health for Woebot, Wysa, and Alleviate.
- Main analysis is qualitative synthesis and taxonomy development, proposing qualitative metrics (adherence, harmlessness, transparency) and knowledge-infusion metrics (safety, logical coherence, semantic relations).
Limitations
- The paper states that there currently exists no mental health-specific dataset or established method rooted in clinical principles for refining LLMs to ensure safety, and that rule-based and data-driven safety methods are limited by size and context.
- The paper states that VMHAs may not fully understand behavioral and emotional instability, self-harm tendencies, and the user’s underlying psychological state.
- Evident from the text: as a narrative review, no systematic search strategy, inclusion/exclusion criteria, or risk-of-bias assessment is reported, and the proposed evaluation metrics are not empirically validated in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 33 cited sources checked · 0 supported · 0 flagged
- Findingown data
GPT 3.5 provides user-level explainability when prompted with clinically-relevant words and keyphrases such as pregnancy, morning sickness, vomiting, nausea, and anxiety caused by tranquilizers during pregnancy.
“Figure 4 GPT 3.5 provides user-level explainability when prompted with clinically-relevant words and keyphrases such as pregnancy, morning sickness, vomiting, nausea, and anxiety caused by tranquilizers during pregnancy.”
Quote in the paperNot a substantial claim- Rests on an assumption: The clinical knowledge bases UMLS and RxNorm contain accurate and sufficient relationships between pregnancy, symptoms, and causes for generating symbolic questions.
- Findingnone given
Current datasets typically have a foundation in psychology but are crowd-sourced rather than explicitly derived from clinically grounded guidelines of psychiatrists.
“The current datasets typically have a foundation in psychology but are crowd-sourced rather than explicitly derived from clinically grounded guidelines of psychiatrists.”
Quote in the paperNot a substantial claim - Findingown data
Without specific keyphrases, GPT 3.5 may produce incorrect inferences.
“Without these specific keyphrases, GPT 3.5 may produce incorrect inferences [shown in (b)].”
Quote in the paperSubstantial claim - Findingnone given
VMHAs generate incoherent and unsafe responses when a user tries to seek a response for clinically relevant questions or vice-versa.
“VMHAs (as exemplified in Figures 2, 3) generate incoherent and unsafe responses when a user tries to seek a response for clinically relevant questions or vice-versa.”
Quote in the paperSubstantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
- Findingnone given
VMHAs may not fully understand issues related to behavioral and emotional instability, self-harm tendencies, and the user's underlying psychological state.
“our observations suggest that VMHAs may not fully understand issues related to behavioral and emotional instability, self-harm tendencies, and the user's underlying psychological state.”
Quote in the paperSubstantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
- Findingown data
When keyphrases are used as prompts, the explanation provided by GPT 3.5 becomes more concise compared with the explanation generated without such prompting.
“When these keyphrases are used as prompts, the explanation provided by GPT 3.5 in (a) becomes more concise compared with the explanation in (b) generated without such prompting.”
Quote in the paperNot a substantial claim - Premisecitation
ChatGPT use in mental health for personalized advice without clinical explanation can hurt user safety and trust.
“A recent surge in the use of ChatGPT, in particular for mental health, is emergent for providing crucial personalized advice without clinical explanation, which can hurt user's safety, and thus trust (Sallam, 2023).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sallam, 2023 source
Not checked in the free run.
- Premisecitation
Bommasani et al. (2021) reports hallucination and harmful question generations as unexpected behaviors shown by large language models.
“To this end, Bommasani et al. (2021) reports hallucination and harmful question generations as unexpected behaviors shown by such LLMs and are referred to as black box models by other authors (Rai, 2020).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Bommasani et al. (2021)
Not checked in the free run.
- Premisecitation
The study by Roy et al. (2022a) showed an 85% improvement in safety across sequence-to-sequence and attention-based language models.
“The study showed an 85% improvement in safety across sequence-to-sequence and attention-based language models.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Roy et al., 2022a
Not checked in the free run.
- Premisecitation
Replika, a VMHA augmented with GPT-3, provides meditative suggestions to a user expressing self-harm tendencies.
“Recently, Replika, a VMHA, augmented with a GPT-3, provides meditative suggestions to a user expressing self-harm tendencies (Ineqe, 2022).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ineqe, 2022
Not checked in the free run.
- Premisecitation
The study by Roy et al. (2022a) showed an uptake of 23% in explainability in terms of safety across the same language models.
“In addition, explainability saw an uptake of 23% in terms of safety across the same language models.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Roy et al., 2022a
Not checked in the free run.
- Premisecitation
Roy et al. (2022a) developed safety lexicons from PHQ-9 and GAD-7 for safe and explainable functioning of language models.
“Roy et al. (2022a) develop safety lexicons from PHQ-9 and GAD-7 for safe and explainable functioning of language models.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Roy et al., 2022a
Not checked in the free run.
- Premisecitation
ALLEVIATE implements safety through the use of clinical knowledge.
“On the other hand, ALLEVIATE implements safety through the use of clinical knowledge.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Roy et al., 2023
Not checked in the free run.
- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
- Interpretationreasoning
VMHAs can efficiently integrate with clinical practice guidelines such as PHQ-9 and GAD-7, utilizing reinforcement learning.
“Given these circumstances, VMHAs can efficiently integrate with clinical practice guidelines such as PHQ-9 and GAD-7, utilizing reinforcement learning.”
Quote in the paperNot a substantial claim- Rests on an assumption: Clinical specification questions are safe and medically adapted.
- Interpretationreasoning
Techniques such as policy gradient-based learning can enhance the capability of chat systems in ensuring safe message generation.
“Techniques such as policy gradient-based learning can enhance the capability of chat systems in ensuring safe message generation.”
Quote in the paperSubstantial claim- Rests on an assumption: Clinical specification questions are safe and medically adapted.
- Interpretationcitation
Using DiG, a VMHA can rephrase its questions to obtain a meaningful response from the user while maintaining engagement.
“Using DiG, a VMHA can rephrase its questions to obtain a meaningful response from the user while maintaining engagement.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Roy et al., 2022a
Not checked in the free run.
- Premisecitation
Similar results were noticed when PHQ-9 was used in explainable training of language models.
“Similar results were noticed when PHQ-9 was used in explainable training of language models (Zirikly and Dredze, 2022).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zirikly and Dredze, 2022
Not checked in the free run.
- Interpretationcitation
Semantic enhancements in VMHA with clinical knowledge and associated guidelines, if they remain under-explored, may miss the hidden mental states in a given narrative, which is an essential component of question generation.
“We argue that semantic enhancements in VMHA with clinical knowledge and associated guidelines, if they remain under-explored, may miss the hidden mental states in a given narrative which is an essential component of question generation (Gaur et al., 2022a; Gupta et al., 2022).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Gaur et al., 2022a
Not checked in the free run.
- cites Gupta et al., 2022 source
Not checked in the free run.
- Premisecitation
The absence of ethical considerations in WoeBot and Wysa becomes evident in their responses to emergencies such as immediate harm or suicidal ideation, where they lack clinical grounding and contextual awareness.
“The absence of ethical considerations in WoeBot and Wysa becomes evident in their responses to emergencies such as immediate harm or suicidal ideation, where they lack clinical grounding and contextual awareness (Koutsouleris et al., 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Koutsouleris et al., 2022 source
Not checked in the free run.
- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
- May combine two claims. 23
- Classified as probably not a substantial claim. 31
- The quote only partly states this claim. 1
May combine two claims.
FindingcitationExisting VMHAs Woebot, Wysa, and ChatGPT initiated responsive conversation without focusing on context, producing assumptive questions and responses with no association with a clinical reference or clinical support.
“As a result, we found assumptive questions (e.g., anxiety) and responses from Wysa, Woebot, and ChatGPT with no association with a clinical reference or clinical support.”
Quote in the paperSubstantial claim0 of 3 sources checkedMay combine two claims.
FindingreasoningBoth Limbic and ALLEVIATE incorporate safety measures, but they do so with a nuanced distinction in their implementation approaches.
“Table 2 illustrates that both Limbic and ALLEVIATE incorporate safety measures, but they do so with a nuanced distinction in their implementation approaches.”
Quote in the paperNot a substantial claim- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingown dataThe next word prediction strategy of the generative AI within the VMHA suggesting that "the fact that you are using tranquilizer medication is a step in the right direction, but it is essential to address the root cause of your anxiety as well" is a harmful statement, because tranquilizers cause anxiety during pregnancy.
“The next word prediction strategy of the generative AI within the VMHA suggests that “the fact that you are using tranquilizer medication is a step in the right direction, but it is essential to address the root cause of your anxiety as well”. is a harmful statement, because tranquilizers cause anxiety during pregnancy (as shown Figure 4).”
Quote in the paperNot a substantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
- Rests on an assumption: The clinical knowledge bases UMLS and RxNorm contain accurate and sufficient relationships between pregnancy, symptoms, and causes for generating symbolic questions.
Classified as probably not a substantial claim.
FindingreasoningThere was not a pursuit of grounding datasets with clinical knowledge and clinical practice guidelines and use in training VMHAs.
“As such, there was not a pursuit of grounding datasets with clinical knowledge and clinical practice guidelines and use in training VMHAs.”
Quote in the paperSubstantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
May combine two claims.
Findingnone givenCurrently, there exists no mental health-specific datasets or established method rooted in clinical principles for refining LLMs to ensure their safety.
“Nonetheless, currently, there exists no mental health-specific datasets or established method rooted in clinical principles for refining LLMs to ensure their safety.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Findingnone givenVMHAs remain distant from translational research.
“However, VMHAs remain distant from such translational research.”
Quote in the paperNot a substantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone givenVMHAs' inability to explain their decisions to patients and MHPs makes conversations less trustworthy.
“One of the gaps is their inability to explain their decisions to patients and MHPs, making conversations less trustworthy.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenVMHAs can be vulnerable to providing unsafe responses to patient queries, which undermines their reliability.
“Additionally, VMHAs can be vulnerable in providing unsafe responses to patient queries, further undermining their reliability.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningLexicons in mental health resolve ambiguities in human language.
“Lexicons in mental health resolve ambiguities in human language.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningClinical practice guidelines such as PHQ-9 enable VMHAs to exercise control over content generation, preventing the generation of false or unsafe information.
“Such guidelines act as standards that enable VMHAs to exercise control over content generation, preventing generating false or unsafe information.”
Quote in the paperNot a substantial claim- Rests on an assumption: Clinical specification questions are safe and medically adapted.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenKIL involves injecting external knowledge into the learning process to enhance the AI model's performance and achieve user-level explainability and safety.
“The concept of KIL involves injecting external knowledge, such as domain-specific rules, ontologies, or expert knowledge, into the learning process to enhance the AI model's performance and achieve USEx and safety.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningEnsuring diversity in generation poses a natural challenge in open-domain conversations, but it becomes an unavoidable aspect in domain-specific conversations for VMHAs.
“Ensuring diversity in generation poses a natural challenge in open-domain conversations, but it becomes an unavoidable aspect in domain-specific conversations for VMHAs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationClinical specifications include questionnaires such as PHQ-9, C-SSRS, and GAD-7, and provide a sequence of questions clinicians follow to interview individuals with mental health conditions.
“Clinical specifications1 include questionnaires such as PHQ-9 (depression), Columbia Suicide Severity Rating Scale [C-SSRS; suicide (Posner et al., 2008)], Generalized Anxiety Disorder (GAD-7) (Coda-Forno et al., 2023). It provides a sequence of questions clinicians follow to interview individuals with mental health conditions.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Posner et al., 2008
Not checked in the free run.
- cites Coda-Forno et al., 2023
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningTo ensure that VMHAs are both safe and understandable, datasets need to be semantically enhanced with clinically grounded knowledge or clinical practice guidelines.
“To ensure that VMHAs are both safe and understandable, these datasets need to be semantically enhanced with clinically grounded knowledge [e.g., MedChatbot (Kazi et al., 2012)] or clinical practice guidelines [e.g., Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001)].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Kazi et al., 2012
Not checked in the free run.
- cites Kroenke et al., 2001 source
Not checked in the free run.
- Rests on an assumption: Clinical specification questions are safe and medically adapted.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationBy incorporating such knowledge, the decision-making ability of AI can be enhanced and better equipped to generate explanations that are more comprehensible to humans.
“By incorporating such knowledge, the decision-making ability of AI can be enhanced and better equipped to generate explanations that are more comprehensible to humans (Joyce et al., 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Joyce et al., 2023
Not checked in the free run.
- Rests on an assumption: Clinical specification questions are safe and medically adapted.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationWoebot is an Automated Coach designed to provide a coach-like experience without human intervention, promoting good thinking hygiene through lessons, exercises, and videos rooted in CBT.
“Woebot is an Automated Coach designed to provide a coach-like experience without human intervention, promoting good thinking hygiene through lessons, exercises, and videos rooted in Cognitive Behavioral Therapy (CBT) (Fitzpatrick et al., 2017; Grigoruta, 2018).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Fitzpatrick et al., 2017 source
Not checked in the free run.
- cites Grigoruta, 2018
Not checked in the free run.
- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThe above analysis supports the critical need for an explainable approach to the decision-making mechanism of VMHAs.
“The above analysis supports the critical need for an explainable approach to the decision-making mechanism of VMHAs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationApproximately 70% of workshop attendees doubted the ability of present-day conversational systems that rely on language models to produce safe responses.
“Approximately 70% of workshop attendees doubted the ability of present-day conversational systems that rely on language models to produce safe responses (Dinan, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Dinan, 2020
Not checked in the free run.
Classified as probably not a substantial claim.
InterpretationreasoningThe risk of user disengagement arises if the chatbot asks redundant questions or provides repetitive responses.
“The risk of user disengagement arises if the chatbot asks redundant questions or provides repetitive responses.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationThe absence of any link to a factual knowledge source that can help LLMs reason on their generation introduces the black box effect.
“The absence of any link to a factual knowledge source that can help LLMs reason on their generation introduce what is known as the “black box” effect (Rudin, 2019).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Rudin, 2019 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationWysa uses a CBT conversational agent to engage in empathetic and therapeutic conversations and activities, aiming to help users with various mental health problems.
“Wysa uses a CBT conversational agent to engage in empathetic and therapeutic conversations and activities, aiming to help users with various mental health problems (Inkster et al., 2018).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Inkster et al., 2018 source
Not checked in the free run.
- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
Classified as probably not a substantial claim. May combine two claims. The quote only partly states this claim.
InterpretationreasoningAs a conversation unfolds, it becomes necessary to continually update the knowledge graph to ensure safety, because patients may experience varying levels of mental health conditions due to comorbidities and evolving severity, and contextual dynamics may shift.
“As the conversation unfolds, it becomes necessary to continually update the KG to ensure safety, which holds particular significance in VMHA.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn Limbic, patient safety is considered to be a spontaneous assessment of the severity of the mental health condition of the user (a classification problem), and it prioritizes patients seeking in-person clinical care.
“In Limbic, patient safety is considered to be a spontaneous assessment of the severity of the mental health condition of the user (a classification problem). It prioritizes patients seeking in-person clinical care (Sohail, 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sohail, 2023
Not checked in the free run.
May combine two claims.
PremisecitationPeople are particularly forthcoming about their mental health problems while interacting with conversational agents, which may increase the danger of agreeing with user utterances that imply self-harm.
“People have been proven to be particularly forthcoming about their mental health problems while interacting with conversational agents, which may increase the danger of “agreeing with those user utterances that imply self-harm”.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Dinan et al., 2022 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningIn mental health, clinical specifications can serve as a substitute for rules to confirm that the AI model is functioning within safe limits.
“In mental health, clinical specifications can serve as a substitute for rules to confirm that the AI model is functioning within safe limits.”
Quote in the paperNot a substantial claim- Rests on an assumption: Clinical specification questions are safe and medically adapted.
May combine two claims.
PremisecitationThe component can generate false information or inference upon personally identifiable information, thus sacrificing user privacy.
“The component can generate false information or inference upon personally identifiable information, thus sacrificing user privacy (Coghlan et al., 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Coghlan et al., 2023
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenAdherence to guidelines helps VMHA maintain context and ensure safe conversation.
“Adherence to guidelines helps VMHA maintain context and ensure safe conversation.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationMost traditional quantitative metrics are reference-based, which are limited in availability and make it very difficult to ensure the quality of the human-written references.
“In addition, most of these traditional quantitative metrics are reference-based, which are limited in availability and make it very difficult to ensure the quality of the human-written references (Bao et al., 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Bao et al., 2022
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
InterpretationreasoningThe question generation and response generation in ALLEVIATE are bound by the subgraph and information in the backend knowledge bases, thus ensuring accountable, transparent, and safe conversation.
“The question generation and response generation in ALLEVIATE are bound by the subgraph and information in the backend knowledge bases, thus ensuring accountable, transparent, and safe conversation.”
Quote in the paperNot a substantial claim- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
May combine two claims.
PremisecitationThe conversational agents generate harmful, unsafe, and sometimes incoherent information, which are the negative effects of generative AI.
“The conversational agents generate harmful, unsafe, and sometimes incoherent information, which are the negative effects of generative AI (Welbl et al., 2021).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Welbl et al., 2021
Not checked in the free run.
- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationPeople perceive the quality of a service based on the price paid for it and the word of mouth buzz around it.
“A well-understood result in marketing is that people perceive the quality of a service based on the price paid for it and the word of mouth buzz around it (Liu and Lee, 2016).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu and Lee, 2016
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenObtaining training data from real conversations can be challenging due to privacy concerns.
“While obtaining training data from real conversations can be challenging due to privacy concerns, some approaches can help address these issues and still provide valuable and useful outputs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningTransparency can be achieved by either augmenting or incorporating external knowledge.
“Transparency can be achieved by either augmenting or incorporating external knowledge.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
InterpretationreasoningIn contrast to the in-person sessions, the conversational agents miss the personalized and contextual information from non-verbal communication hindering the efficacy of VMHAs.
“In contrast to the in-person sessions (leveraged on both verbal and non-verbal communication), the conversational agents miss the personalized and contextual information from non-verbal communication hindering the efficacy of VMHAs.”
Quote in the paperNot a substantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone givenCreating a realistic conversational experience for VMHAs is important for user acceptance.
“Creating a realistic conversational experience for VMHAs is important for user acceptance.”
Quote in the paperNot a substantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
May combine two claims.
InterpretationreasoningDespite the efforts, VMHA is still susceptible to generating harmful and untrustworthy content, as these methods are limited by size and context.
“Despite the efforts, VMHA is still susceptible to generating harmful and untrustworthy content, as these methods are limited by size and context.”
Quote in the paperNot a substantial claim- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningTraditional metrics such as accuracy, precision, and recall may not be sufficient to capture the nuances of these complex requirements.
“Traditional metrics such as accuracy, precision, and recall may not be sufficient to capture the nuances of these complex requirements.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationThe lack of safe responses, DiG, and UsEx within VMHAs has drawn criticism and raised concerns about the impact on adherence.
“Despite the AI community's considerable interest in evaluating health assistants' adherence to user needs (Davis et al., 2020), the lack of safe responses, DiG, and UsEx within VMHAs has drawn criticism and raised concerns about the impact on adherence.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Davis et al., 2020
Not checked in the free run.
- Rests on an assumption: The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationALLEVIATE creates a subgraph from the user's utterances and chatbot questions during the conversation, constructed by actively querying UMLS for disorders and symptoms and Rx-NORM for medicine.
“ALLEVIATE creates a subgraph from the user's utterances and chatbot questions during the conversation. This subgraph is constructed by actively querying two knowledge bases: UMLS, for disorders and symptoms and Rx-NORM for medicine (Liu et al., 2005).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Liu et al., 2005
Not checked in the free run.
- Rests on an assumption: The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenKnowledge in various human-curated knowledge bases (both online and offline) is more exhaustive in terms of context.
“In contrast, knowledge in various human-curated knowledge bases (both online and offline) is more exhaustive in terms of context.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Fabrication and hallucination problems already exist in smaller language models and are even more pronounced in larger ones.
“While these problems already exist in smaller language models, they are even more pronounced in larger ones.”
The paper relies on this without support.
Clinical specification questions are safe and medically adapted.
“Such questions are safe and medically adapted.”
The paper relies on this without support.
The examples in Figures 2, 3, and 4 are representative of how VMHAs generally behave.
“VMHAs (as exemplified in Figures 2, 3) generate incoherent and unsafe responses when a user tries to seek a response for clinically relevant questions or vice-versa.”
“Without these specific keyphrases, GPT 3.5 may produce incorrect inferences [shown in (b)].”
The paper relies on this without support.
The survey of 40 participants at Prisma Health provides a valid basis for the high-level analysis of Woebot, Wysa, and Alleviate.
“For Woebot, Wysa, and Alleviate, a survey of 40 participants was carried out at Prisma Health.”
The paper relies on this without support.
The clinical knowledge bases UMLS and RxNorm contain accurate and sufficient relationships between pregnancy, symptoms, and causes for generating symbolic questions.
“In Figure 4(a), the process generates two symbolic questions based on the relationship between pregnancy, symptoms, and causes found in clinical knowledge sources UMLS and RxNorm.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI systems introduce novel stressors that are continuous, personally salient, and socially immersive, thereby functioning as 24-hour contextual stimuli capable of modulating arousal, perception, and belief formation.
“These systems introduce novel stressors that are continuous, personally salient, and socially immersive, thereby functioning as 24-hour contextual stimuli capable of modulating arousal, perception, and belief formation [19].”
From Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis” (2025) · cites this paper as [19]
Section 5.1 (page unavailable) says Woebot and Wysa track mood changes throughout the day and warns that discussing certain topics could exacerbate a user's mental health condition. It does not characterize these systems as continuous, socially immersive stressors or 24-hour stimuli, or say they modulate arousal, perception, and belief formation. (LLM, conf=0.77)
Experimental — accuracy not yet measuredSystematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits.
“Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits33,76,77.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [77]
Section 2.1 says the consequences of LLMs’ black-box behavior are more concerning than their utility, particularly in mental health. Sections 5 and 6.1 describe incoherent, hallucinated, and unsafe responses from virtual mental health assistants. The source supports the risk–benefit concern in that setting, but it does not identify itself as a systematic or scoping review or establish the claim for LLM chatbots generally. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Gioia, D. A., Mehra, A., & Weick, K. E. (1996) · Academy of Management Review · doi:10.2307/259169 · Cited by 15,604 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
(2004) · Cited by 9,718 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,367 · see citing works
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1× · see citing works
“A recent surge in the use of ChatGPT, in particular for mental health, is emergent for providing crucial personalized advice without clinical explanation, which can hurt user’s safety , and thus trust (Sallam, 2023).”
BackgroundFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · see citing works
Czeisler, M. É., Lane, R. I., Petrosky, E., Wiley, J. F., Christensen, A., Njai, R., Weaver, M. D., Robbins, R., Facer-Childs, E. R., Barger, L. K., et al. (2020) · MMWR. Morbidity and Mortality Weekly Report · doi:10.15585/mmwr.mm6932a1 · Cited by 2,686 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Knowledge Graph Embedding: A Survey of Approaches and Applications
Wang, Q., Mao, Z., Wang, B., & Guo, L. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2754499 · Cited by 2,654 · see citing works
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · see citing works
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312 · see citing works
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · see citing works
The Medical Dictionary for Regulatory Activities (MedDRA)
Brown, E. G., Wood, L., & Wood, S. (1999) · Drug Safety · doi:10.2165/00002018-199920020-00002 · Cited by 1,396 · see citing works
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317 · see citing works
The DSM-5: Classification and criteria changes
Regier, D. A., Kuhl, E. A., & Kupfer, D. J. (2013) · World Psychiatry · doi:10.1002/wps.20050 · Cited by 1,138 · see citing works
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063 · see citing works
Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset
Rashkin, H., Smith, E. M., Li, M., & Boureau, Y.-L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1534 · Cited by 883 · see citing works
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715 · see citing works
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · see citing works
Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare
Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0288-5 · Cited by 553 · see citing works
SNOMED-CT: The advanced terminology and coding system for eHealth.
Donnelly, K. (2006) · PubMed · Cited by 541 · see citing works
Structured Clinical Interview for the <i> <scp>DSM</scp> </i> ( <scp>SCID</scp> )
First, M. B. (2015) · The Encyclopedia of Clinical Psychology · doi:10.1002/9781118625392.wbecp351 · Cited by 532 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500 · see citing works
Constituency Parsing with a Self-Attentive Encoder
Kitaev, N., & Klein, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1249 · Cited by 483 · see citing works
Natural language processing applied to mental illness detection: a narrative review
Zhang, T., Schoene, A. M., Ji, S., & Ananiadou, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00589-7 · Cited by 476 · see citing works
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442 · see citing works
Improved Image Captioning via Policy Gradient optimization of SPIDEr
Liu, S., Zhu, Z., Ye, N., Guadarrama, S., & Murphy, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.100 · Cited by 430 · see citing works
Columbia-Suicide Severity Rating Scale (C-SSRS)
Shahid, A., Wilkinson, K., Marcu, S., & Shapiro, C. M. (2011) · Springer eBooks · doi:10.1007/978-1-4419-9893-4_24 · Cited by 384 · see citing works
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works
DSM-IV und DSM-5: Was hat sich tatsächlich verändert?
Ehret, A. M., & Berking, M. (2013) · Verhaltenstherapie · doi:10.1159/000356537 · Cited by 345 · see citing works
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326 · see citing works
Althoff, T., Clark, K., & Leskovec, J. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00111 · Cited by 322 · see citing works
Perceptions and opinions of patients about mental health chatbots: scoping review
Abd-Alrazaq, A. A., Alajlani, M., Ali, N., Denecke, K., Bewick, B. M., & Househ, M. (2021) · Journal of Medical Internet Research · doi:10.2196/17828 · Cited by 301 · see citing works
Service quality and price perception of service: Influence on word-of-mouth and revisit intention
Liu, C.-H. S., & Lee, T. (2016) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2015.12.007 · Cited by 280 · see citing works
RxNorm: prescription for electronic drug information exchange
Liu, S., Ma, W., Moore, R., Ganesan, V., & Nelson, S. (2005) · IT Professional · doi:10.1109/mitp.2005.122 · Cited by 274 · see citing works
Joyce, D. W., Kormilitzin, A., Smith, K., & Cipriani, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00751-9 · Cited by 259 · see citing works
From promise to practice: towards the realisation of AI-informed mental health care
Koutsouleris, N., Hauser, T. U., Skvortsova, V., & De Choudhury, M. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00153-4 · Cited by 251 · see citing works
To chat or bot to chat: Ethical issues with using chatbots in mental health
Coghlan, S., Leins, K., Sheldrick, S., Cheong, M., Gooding, P., & D’Alfonso, S. (2023) · Digital Health · doi:10.1177/20552076231183542 · Cited by 247 · see citing works
A Mental Health Chatbot for Regulating Emotions (SERMO) - Concept and Usability Test
Denecke, K., Vaaheesan, S., & Arulnathan, A. (2020) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.2974478 · Cited by 240 · see citing works
Lessons from Natural Language Inference in the Clinical Domain
Romanov, A., & Shivade, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1187 · Cited by 226 · see citing works
Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions
Longo, L., Goebel, R. G., Lécué, F., Kieseberg, P., & Holzinger, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_1 · Cited by 182 · see citing works
PARADISE: A Framework for Evaluating Spoken Dialogue AgentsPreprint
Walker, M. A., Litman, D. J., Kamm, C., & Abella, A. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9704004 · Cited by 175 · see citing works
Why we should have seen that coming
Wolf, M. J., Miller, K., & Grodzinsky, F. S. (2017) · ACM SIGCAS Computers and Society · doi:10.1145/3144592.3144598 · Cited by 174 · see citing works
Sweeney, C., Potts, C., Ennis, E., Bond, R., Mulvenna, M. D., O’Neill, S. M., Malcolm, M., Kuosmanen, L., Kostenius, C., Vakaloudis, A., et al. (2021) · ACM Transactions on Computing for Healthcare · doi:10.1145/3453175 · Cited by 173 · see citing works
Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media
Yazdavar, A. H., Al-Olimat, H. S., Ebrahimi, M., Bajaj, G., Banerjee, T., Thirunarayan, K., Pathak, J., & Sheth, A. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3123028 · Cited by 169 · see citing works
Sharma, A. K., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450097 · Cited by 168 · see citing works
Ethical Challenges in Data-Driven Dialogue Systems
Henderson, P., Sinha, K., Gontier, N., Ke, N. R., Fried, G., Lowe, R., & Pineau, J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278777 · Cited by 166 · see citing works
An emotion and cognitive based analysis of mental health disorders from social media data
Uban, A. S., Chulvi, B., & Rosso, P. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.05.032 · Cited by 146 · see citing works
Assigning Personality/Profile to a Chatting Machine for Coherent Conversation Generation
Qian, Q., Huang, M., Zhao, H., Xu, J., & Zhu, X. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/595 · Cited by 142 · see citing works
Normalising Medical Concepts in Social Media Texts by Learning Semantic Representation
Limsopatham, N., & Collier, N. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1096 · Cited by 134 · see citing works
Designing Personality-Adaptive Conversational Agents for Mental Health Care
Ahmad, R., Siemon, D., Gnewuch, U., & Robra-Bissantz, S. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10254-9 · Cited by 134 · see citing works
Recipes for Safety in Open-domain ChatbotsPreprint
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07079 · Cited by 130 · see citing works
Chatbots to Support Young Adults’ Mental Health: An Exploratory Study of Acceptability
Koulouri, T., Macredie, R. D., & Olakitan, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3485874 · Cited by 127 · see citing works
EMMA: an emotion-aware wellbeing chatbot
Ghandeharioun, A., McDuff, D., Czerwinski, M., & Rowan, K. (2019) · 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2019.8925455 · Cited by 126 · see citing works
Mertes, S., Huber, T., Weitz, K., Heimerl, A., & André, E. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.825565 · Cited by 119 · see citing works
Seitz, L., Bekmeier‐Feuerhahn, S., & Gohil, K. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102848 · Cited by 117 · see citing works
Cheng, Y., & Jiang, H. (2020) · Journal of Contingencies and Crisis Management · doi:10.1111/1468-5973.12319 · Cited by 116 · see citing works
Trust me, i'm a chatbot: how artificial intelligence in health care fails the Turing Test
Powell, J. (2019) · Journal of Medical Internet Research · doi:10.2196/16222 · Cited by 115 · see citing works
Chatgpt: a meta-analysis after 2.5 months
Leiter, C., Zhang, R., Chen, Y., Belouadi, J., Larionov, D., Fresen, V., & Eger, S. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100541 · Cited by 115 · see citing works
Challenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112 · see citing works
Building and Using Personal Knowledge Graph to Improve Suicidal Ideation Detection on Social Media
Cao, L., Zhang, H., & Feng, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3046867 · Cited by 104 · see citing works
Chung, K., Cho, H. Y., & Park, J. Y. (2020) · JMIR Medical Informatics · doi:10.2196/18607 · Cited by 94 · see citing works
Westra, H. A., Aviram, A., & Doell, F. K. (2011) · The Canadian Journal of Psychiatry · doi:10.1177/070674371105601102 · Cited by 81 · see citing works
ProsocialDialog: A prosocial backbone for conversational agents
Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., & Sap, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.267 · Cited by 80 · see citing works
An Overview of Catastrophic AI RisksPreprint
Hendrycks, D., Mazeika, M., & Woodside, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12001 · Cited by 79 · see citing works
Learning Rich Representation of Keyphrases from Text
Kulkarni, M., Mahata, D., Arora, R., & Bhowmik, R. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.67 · Cited by 71 · see citing works
Psychoanalyzing artificial intelligence: the case of Replika
Possati, L. M. (2023) · AI & SOCIETY · doi:10.1007/s00146-021-01379-7 · Cited by 65 · see citing works
Davis, C. R., Murphy, K. J., Curtis, R., & Maher, C. A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17239137 · Cited by 61 · see citing works
Thuraisingham, B. M. (2022) · IEEE Intelligent Systems · doi:10.1109/mis.2022.3152946 · Cited by 60 · see citing works
MedChatBot: An UMLS based Chatbot for Medical Students
Kazi, H., Chowdhry, B. S., & Memon, Z. (2012) · International Journal of Computer Applications · doi:10.5120/8844-2886 · Cited by 59 · see citing works
Gaur, M., Gunaratna, K., Srinivasan, V., & Jin, H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21312 · Cited by 57 · see citing works
Self-citationConversational Agents and Mental Health
Miner, A. S., Chow, A., Adler, S., Zaitsev, I., Tero, P., Darcy, A. M., & Paepcke, A. (2016) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/2974804.2974820 · Cited by 56 · see citing works
Noble, J. M., Zamani, A., Gharaat, M., Merrick, D., Maeda, N., Lambe Foster, A., Nikolaidis, I., Goud, R., Stroulia, E., Agyapong, V. I. O., et al. (2022) · JMIR Research Protocols · doi:10.2196/33717 · Cited by 53 · see citing works
Rollwage, M., Juechems, K., Habicht, J., Carrington, B., Hauser, T. U., & Harper, R. J. (2024) · BMJ Innovations · doi:10.1136/bmjinnov-2023-001110 · Cited by 48 · see citing works
Diversifying Dialogue Generation with Non-Conversational Text
Su, H., Shen, X., Zhao, S., Zhou, X., Hu, P., Zhong, R., Niu, C., & Zhou, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.634 · Cited by 45 · see citing works
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44 · see citing works
Process Knowledge-Infused AI: Toward User-Level Explainability, Interpretability, and Safety
Sheth, A., Gaur, M., Roy, K., Venkataramanan, R., & Khandelwal, V. (2022) · IEEE Internet Computing · doi:10.1109/mic.2022.3182349 · Cited by 43 · see citing works
Self-citationAccurate Clinical and Biomedical Named Entity Recognition at Scale
Kocaman, V., & Talby, D. (2022) · Software Impacts · doi:10.1016/j.simpa.2022.100373 · Cited by 39 · see citing works
Inducing anxiety in large language models increases exploration and biasPreprint
Coda-Forno, J., Witte, K., Jagadish, A. K., Binz, M., Akata, Z., & Schulz, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11111 · Cited by 39 · see citing works
Sheth, A., Yip, H. Y., & Shekarpour, S. (2019) · IEEE Intelligent Systems · doi:10.1109/mis.2019.2905748 · Cited by 38 · see citing works
Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines
Trella, A. L., Zhang, K. W., Nahum-Shani, I. B., Shetty, V., Doshi‐Velez, F., & Murphy, S. A. (2022) · Algorithms · doi:10.3390/a15080255 · Cited by 38 · see citing works
Knowledge-Intensive Language Understanding for Explainable AI
Sheth, A., Gaur, M., Roy, K., & Faldu, K. (2021) · IEEE Internet Computing · doi:10.1109/mic.2021.3101919 · Cited by 35 · see citing works
Self-citationKnowledge-Infused Learning: A Sweet Spot in Neuro-Symbolic AI
Gaur, M., Gunaratna, K., Bhatt, S., & Sheth, A. (2022) · IEEE Internet Computing · doi:10.1109/mic.2022.3179759 · Cited by 34 · see citing works
Self-citationNUBIA: NeUral Based Interchangeability Assessor for Text Generation
Kane, H., Kocyigit, M. Y., Abdalla, A., Ajanoh, P., & Coulibali, M. (2020) · arXiv · doi:10.48550/arxiv.2004.14667 · Cited by 32 · see citing works
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
Dinan, E., Abercrombie, G., Bergman, A., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.284 · Cited by 32 · see citing works
Hennemann, S., Kuhn, S., Witthöft, M., & Jungmann, S. M. (2022) · JMIR Mental Health · doi:10.2196/32832 · Cited by 31 · see citing works
Raza, S., Schwartz, B., & Rosella, L. C. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04751-6 · Cited by 31 · see citing works
Hartmann, R., Sander, C., Lorenz, N., Böttger, D., & Hegerl, U. (2019) · JMIR Mental Health · doi:10.2196/11671 · Cited by 28 · see citing works
Deep Learning Mental Health Dialogue System
Brocki, L., Dyer, G. C., Gładka, A., & Chung, N. C. (2023) · 2023 IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp57234.2023.00097 · Cited by 28 · see citing works
Gupta, S., Agarwal, A., Gaur, M., Roy, K., Narayanan, V., Kumaraguru, P., & Sheth, A. (2022) · Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology · doi:10.18653/v1/2022.clpsych-1.12 · Cited by 27 · see citing works
Self-citationInterdisciplinary IoT and Emotion Knowledge Graph-Based Recommendation System to Boost Mental Health
Gyrard, A., & Boudaoud, K. (2022) · Applied Sciences · doi:10.3390/app12199712 · Cited by 24 · see citing works
Adapting and Implementing Apps for Mental Healthcare
Zielasek, J., Reinhardt, I., Schmidt, L., & Gouzoulis‐Mayfrank, E. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01350-3 · Cited by 23 · see citing works
Fadhil, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09844 · Cited by 21 · see citing works
Rollwage, M., Juechems, K., Habicht, J., et al. (2022) · medRxiv · doi:10.1101/2022.11.03.22281887 · Cited by 20 · see citing works
Towards augmenting crisis counselor training by improving message retrieval
DeMasi, O., Hearst, M. A., & Recht, B. (2019) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w19-3001 · Cited by 18 · see citing works
Srivastava, B. (2021) · Patterns · doi:10.1016/j.patter.2021.100308 · Cited by 18 · see citing works
Self-citationHEAL: A Knowledge Graph for Distress Management Conversations
Welivita, A., & Pu, P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21398 · Cited by 18 · see citing works
Roy, K., Gaur, M., Soltani, M., Rawte, V., Kalyan, A., & Sheth, A. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1056728 · Cited by 18 · see citing works
Self-citationSemantic Diversity in Dialogue with Natural Language Inference
Stasaski, K., & Hearst, M. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.6 · Cited by 17 · see citing works
Explaining Models of Mental Health via Clinically Grounded Auxiliary Tasks
Zirikly, A., & Dredze, M. H. (2022) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2022.clpsych-1.3 · Cited by 16 · see citing works
Evaluating Superhuman Models with Consistency Checks
Fluri, L., Paleka, D., & Tramèr, F. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00017 · Cited by 14 · see citing works
Ghosh, S. K., Ekbal, A., & Bhattacharyya, P. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/704 · Cited by 13 · see citing works
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
Clinciu, M., Eshghi, A., & Hastie, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.202 · Cited by 13 · see citing works
Evaluation of In-Person Counseling Strategies To Develop Physical Activity Chatbot for Women
Liang, K.-H., Lange, P. L., Oh, Y. J., Zhang, J., Fukuoka, Y., & Yu, Z. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.5 · Cited by 12 · see citing works
Towards Detecting Need for Empathetic Response in Motivational Interviewing
Wu, Z., Helaoui, R., Kumar, V., Recupero, D. R., & Riboni, D. (2020) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3395035.3425228 · Cited by 12 · see citing works
AttViz: Online exploration of self-attention for transparent neural language modelingPreprint
Škrlj, B., Eržen, N., Sheehan, S., Luz, S. F., Robnik‐Šikonja, M., & Pollak, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05716 · Cited by 9 · see citing works
Functional Transparency for Structured Data: a Game-Theoretic ApproachPreprint
Lee, G.-H., Jin, W., Alvarez-Melis, D., & Jaakkola, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09737 · Cited by 9 · see citing works
Process Knowledge-infused Learning for Suicidality Assessment on Social MediaPreprint
Roy, K., Gaur, M., Zhang, Q., & Sheth, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.12560 · Cited by 7 · see citing works
Self-citationChatGPT and Medicine: Fears, Fantasy, and the Future of Physicians
Peterson, C. (2023) · The Southwest Respiratory and Critical Care Chronicles · doi:10.12746/swrccc.v11i48.1193 · Cited by 4 · see citing works
Using In-Context Learning to Improve Dialogue SafetyPreprint
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tür, D. Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.00871 · Cited by 1 · see citing works
Evaluating Superhuman Models with Consistency ChecksPreprint
Fluri, L., Paleka, D., & Tramèr, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.09983 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 11
- Bao, 2022, Docasref: A pilot empirical study on repurposing reference-based summary quality metrics reference-freely, arXiv [Preprint]. arXiv:2212.10013, 10.48550/arxiv.2212.1001 · DOI
- 2015, Mental Health Answers from Counselors
- DinanR. 1st Safety for Conversational AI Workshop | ACL Member Portal2020
- 2022, Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed to Build Safer Machine Learning Systems
- Huang, 2015, Language Use in Teenage Crisis Intervention and the Immediate Outcome: A Machine Automated Analysis of Large Scale Text Data
- Hyman, 2008, Self-Disclosure and its Impact on Individuals Who Receive Mental Health Services (hhs pub. no. sma-08-4337)
- Kruzan, 2019, Self-Injury Support Online: Exploring Use of the Mobile Peer Support Application TalkLife
- Lee, 2021, Restatement and question generation for counsellor chatbot. In 1st Workshop on Natural Language Processing for Programming (NLP4Prog), 1
- Roy, 2023, Alleviate ChatBot
- Welivita, “Curating a large-scale motivational interviewing dataset using peer support forums,”, 3315
- Zhang, 2019, Conversation generation with concept
Cited by
Sorted by citations
Exploring the Ethical Challenges of Conversational AI in Mental Health Care: Scoping Review
Rahsepar Meadi, M., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., & Batelaan, N. (2025) · JMIR Mental Health · doi:10.2196/60432 · Cited by 211 · see citing works
Applications of large language models in psychiatry: a systematic review
Omar, M., Soffer, S., Charney, A. W., Landi, I., Nadkarni, G. N., & Klang, E. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1422807 · Cited by 85 · see citing works
Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis”
Hudon, A., & Stip, E. (2025) · JMIR Mental Health · doi:10.2196/85799 · Cited by 88 · see citing works
“These systems introduce novel stressors that are continuous, personally salient, and socially immersive, thereby functioning as 24-hour contextual stimuli capable of modulating arousal, perception, and belief formation [19].”
Safety of Large Language Models in Addressing Depression
Heston, T. F. (2023) · Cureus · doi:10.7759/cureus.50729 · Cited by 61 · see citing works
Abbas, S. R., Seol, H., Abbas, Z., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13141642 · Cited by 55 · see citing works
From robots to chatbots: unveiling the dynamics of human-AI interaction
Łukasik, A., & Gut, A. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1569277 · Cited by 50 · see citing works
AI Chatbots for Psychological Health for Health Professionals: Scoping Review
Baek, G., Cha, C., & Han, J.-H. (2025) · JMIR Human Factors · doi:10.2196/67682 · Cited by 42 · see citing works
Conversational AI in Pediatric Mental Health: A Narrative Review
Mansoor, M. A., Hamide, A., & Tran, T. (2025) · Children · doi:10.3390/children12030359 · Cited by 41 · see citing works
Mental Health Applications of Generative AI and Large Language Modeling in the United States
Banerjee, S. K., Dunn, P. J., Conard, S. E., & Ali, A. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21070910 · Cited by 27 · see citing works
Understanding Attitudes and Trust of Generative AI Chatbots for Social Anxiety Support
Wang, Y., Wang, Y., Crace, K., & Zhang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714286 · Cited by 25 · see citing works
Chen, K., Xu, W., & Li, X. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.08.052 · Cited by 22 · see citing works
Tilton, A. K., Caplan, B. E., & Cole, B. J. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1616488 · Cited by 21 · see citing works
Building trustworthy NeuroSymbolic AI Systems: Consistency, reliability, explainability, and safety
Gaur, M., & Sheth, A. (2024) · AI Magazine · doi:10.1002/aaai.12149 · Cited by 27 · see citing works
Self-citationBoit, S., & Patil, R. (2025) · JMIR Mental Health · doi:10.2196/75078 · Cited by 24 · see citing works
Dalal, S., Tilwani, D., Gaur, M., Jain, S., Shalin, V. L., & Sheth, A. P. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3483577 · Cited by 19 · see citing works
Self-citationEsmaeilzadeh, P., Hassanein, K., & Head, M. M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2477754 · Cited by 17 · see citing works
Elyoseph, Z., Levkovich, I., Rabin, E., Shemo, G., Szpiler, T., Shoval, D. H., & Levi‐Belz, Y. (2025) · npj Mental Health Research · doi:10.1038/s44184-025-00139-5 · Cited by 12 · see citing works
Mental Healthcare Chatbot Based on Custom Diagnosis Documents Using a Quantized Large Language Model
Kumar, A., Sharma, S., Gupta, S., & Kumar, D. (2024) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito61523.2024.10522371 · Cited by 10 · see citing works
Neary, M., Fulton, E. A., Rogers, V., Wilson, J., Griffiths, Z., Chuttani, R., & Sacher, P. M. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1460236 · Cited by 9 · see citing works
Vairamani, A. D. (2024) · The Springer series in applied machine learning · doi:10.1007/978-3-031-63821-3_12 · Cited by 6 · see citing works
The dangers of AI friendship: Adolescent mental health and ethical oversight
Katoch, H., & Sandhu, D. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-41660-6.00016-8 · Cited by 5 · see citing works
Predicting Digital Addiction Patterns with Machine Learning for Personalised Mental Health Support
Ibitoye, A. O., Ravindran, S., Afe, O. F., & Abiodun, A. O. (2025) · Journal of Social Computing · doi:10.23919/jsc.2025.0020 · Cited by 5 · see citing works
A scoping review on the mental health harms of LLM-based chatbots
Diel, A., Torous, J., Cuijpers, P., Kleesiek, J., Nensa, F., Weber, N., Faust, F., Lalgi, T. J., Mellis, F. S., Teufel, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03054-x · Cited by 4 · see citing works
“Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential bene fi ts 33,76,77 .”
Generative AI Health Assistants in Modern Healthcare: Drivers and Barriers to Adoption
Al-Lataifeh, Z., Harris, M. A., Smith, J., & Chin, A. G. (2025) · Information Systems Frontiers · doi:10.1007/s10796-025-10681-4 · Cited by 3 · see citing works
How explanatory features of AI and time frame reshape adolescents’ decision-making
Shen, Z., Chen, Y., Zhang, J., & Chen, H. (2026) · Computers & Education · doi:10.1016/j.compedu.2026.105563 · Cited by 4 · see citing works
Haider, B. A., Gorti, A., Chadha, A. R., & Gaur, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-20447-9_5 · Cited by 3 · see citing works
Self-citationThe psychiatry of sycophancy: When agreeable artificial intelligence becomes a mental health risk
Hikmat, R., Nilmanat, K., & Damkliang, J. (2026) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2026.105064 · Cited by 3 · see citing works
Digital Psychiatry: Opportunities, Challenges, and Future Directions
Sidani, L., Nadar, S. M., Tfaili, J., Rayes, S. E., Sharara, F., Elhage, J. C., & Fakhoury, M. (2024) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000819 · Cited by 5 · see citing works
Recommendation endpoints and safety of an online self-triage for depression symptoms
Miller, N. E., North, F., Curry, E. N., Thompson, M. C., & Pecina, J. L. (2024) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x241245161 · Cited by 3 · see citing works
The Role of Affective Computing in Social Justice: Harnessing Equity and Inclusion
Chaube, N., Kerketta, R., Sharma, S., & Shinde, A. (2024) · The Springer series in applied machine learning · doi:10.1007/978-3-031-63821-3_4 · Cited by 3 · see citing works
A bibliometric analysis of large language models in mental health research
Hussiny, M. A., Trust Saidi, Pikkarainen, M. A., & Gazerani, P. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1838199 · Cited by 1 · see citing works
A Bibliometric Analysis of Large Language Models in Mental Health Research (Preprint)Preprint
Hussiny, M. A., Trust Saidi, Pikkarainen, M., & Gazerani, P. (2025) · doi:10.2196/preprints.87333
Using Explainable AI for Assessment of Depression: A Systematic Literature Review
Macaulay, A., Din, F. U., Nguyen, P. A., Chiong, R., & Tully, P. J. (2026) · Springer eBooks · doi:10.1007/978-3-032-07267-2_15
Dalal, S., Tilwani, D., Gaur, M., Jain, S., Shalin, V. L., & Sheth, A. P. (2024) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2e7io-a2k6
Self-citationIkegami, M., Kuribayashi, M., Kato, T., Aoyagi, S., & Hirai, T. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798615
Erdem, G., Kafesçioğlu, N., & Başer, G. (2026) · Journal of Marital and Family Therapy · doi:10.1111/jmft.70168
Trends and Challenges of Explainable AI in Medical Diagnostics
Khan, A. H., Jan, S., Shah, I. A., Farooq, U., Ali, A., Jamil, M., Bakhtiar, A., & Bahadar, R. (2026) · Information systems engineering and management · doi:10.1007/978-3-032-22426-2_12
Salinas-Layana, A., Jiménez-Molina, Á., Lira, D., Montoya, F. V., Venegas, A., Muñoz, N., Chandía, M., Rojas, R., & Martínez, V. (2026) · JMIR Research Protocols · doi:10.2196/91677
- Authors
- Sarkar, S., Gaur, M., Chen, L. K., Garg, M., & Srivastava, B.
- Affiliations
- University of Maryland, Baltimore County (US)
- Mayo Clinic (US)
- University of South Carolina (US)
- Corresponding author
- Sarkar, Surjodeep
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2023-10-12
- Volume
- 6
- Pages
- 1229805-1229805
- ISSN
- 2624-8212 (ISSN-L)
- PMC
- PMC10601652
- OpenAlex
- W4387578765
- Semantic Scholar
- 6f542f60a5d4f540d056bb49d47f89f1035ad0cf
- Cited by
- 38
- References
- 121
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 11 · 2023: 1
- FWCI
- 0.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Artificial Intelligence (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Digital Repository at the University of Maryland (University of Maryland College Park) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- chatbots, explainability, AI safety, ChatGPT, GPT-4, cognitive behavioral therapy, mental health professionals, trust in AI, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
