CoChat
Sign Up
Cited by 213Artificial Intelligence in Healthcare and EducationArtificial Intelligence in HealthcareAI in Service Interactions

Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI

Open access
Abbasian, Mahyar, Khatibi, Elahe, Azimi, Iman, Oniani, David, Shakeri Hossein Abad, Zahra, Thieme, Alexander, Sriram, Ram, Yang, Zhongqi +6 more·2024·npj Digital Medicine
DOI 10.1038/s41746-024-01074-zPMC PMC10980701
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative Artificial Intelligence is set to revolutionize healthcare delivery by transforming traditional patient care into a more personalized, efficient, and proactive process. Chatbots, serving as interactive conversational models, will probably drive this patient-centered transformation in healthcare. Through the provision of various services, including diagnosis, personalized lifestyle recommendations, dynamic scheduling of follow-ups, and mental health support, the objective is to substantially augment patient health outcomes, all the while mitigating the workload burden on healthcare providers. The life-critical nature of healthcare applications necessitates establishing a unified and comprehensive set of evaluation metrics for conversational models. Existing evaluation metrics proposed for various generic large language models (LLMs) demonstrate a lack of comprehension regarding medical and health concepts and their significance in promoting patients’ well-being. Moreover, these metrics neglect pivotal user-centered aspects, including trust-building, ethics, personalization, empathy, user comprehension, and emotional support. The purpose of this paper is to explore state-of-the-art LLM-based evaluation metrics that are specifically applicable to the assessment of interactive conversational models in healthcare. Subsequently, we present a comprehensive set of evaluation metrics designed to thoroughly assess the performance of healthcare chatbots from an end-user perspective. These metrics encompass an evaluation of language processing abilities, impact on real-world clinical tasks, and effectiveness in user-interactive conversations. Finally, we engage in a discussion concerning the challenges associated with defining and implementing these metrics, with particular emphasis on confounding factors such as the target audience, evaluation methods, and prompt techniques involved in the evaluation process.

AI snapshot

Written from the paper’s full text.

TL;DR

This perspective paper argues that existing LLM evaluation metrics are inadequate for healthcare chatbots because they miss medical semantics and user-centered concerns such as trust, empathy, ethics, personalization, and hallucination. It proposes a comprehensive set of foundation metrics grouped into accuracy, trustworthiness, empathy, and performance, along with an evaluation framework and discussion of implementation challenges.

Key findings

  • Existing intrinsic metrics such as BLEU and ROUGE rely on surface-form similarity and fail to capture medical concepts, semantics, context, and human-centered aspects; the paper illustrates this with two semantically similar health sentences that scored only 0.39 BLEU and 0.13 ROUGE on a 0–1 scale.
  • Existing extrinsic and multi-metric evaluations remain incomplete for healthcare chatbots, often neglecting empathy, reasoning, up-to-dateness, hallucinations, personalization, relevance, and latency.
  • The paper proposes four user-centered metric groups—accuracy, trustworthiness, empathy, and performance—with submetrics such as groundedness, privacy, bias, emotional support, health literacy, fairness, personalization, memory efficiency, FLOP, token limit, and number of parameters.
  • It identifies three confounding variables for evaluation—user type, domain type, and task type—and presents a framework with models, environment, interface, interacting users, and leaderboard components.

Methods

  • Narrative review and position paper: reviews intrinsic and extrinsic evaluation metrics for LLMs, including general-purpose, health-specific, and multi-metric approaches.
  • Synthesizes a proposed set of healthcare chatbot metrics from the literature, categorizes them into four groups, and maps each metric to healthcare problems and existing benchmarks in Table 2.
  • No empirical data collection, model training, or experimental evaluation is reported; the paper proposes an evaluation framework and discusses challenges and future implementation.

Limitations

  • The paper states that future work is needed to implement the proposed evaluation framework, conduct extensive metric assessment using benchmarks and case studies, and establish unified benchmarks tailored to healthcare chatbots.
  • It discusses challenges including metric associations and correlations, selection of evaluation methods, subjectivity and need for diverse expert annotators, and effects of prompt techniques and model parameters.
  • Evident from the text: the proposed metrics and framework are not empirically validated in this paper, and no sample, dataset, or model evaluation results are reported beyond the illustrative BLEU/ROUGE example.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-024-01074-z

Cites

130 of 130 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 58 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    For two contextually similar example sentences, the obtained BLEU and ROUGE scores are 0.39 and 0.13, respectively, on a scale of 0 to 1.

    “Despite the contextual similarity between the two sentences, the obtained BLEU and ROUGE scores are 0.39 and 0.13, respectively, on a scale of 0 to 1, reflecting low alignment.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites HuggingFace24

      Not checked in the free run.

  • Premisecitation

    Laing et al. presented a multi-metric approach as part of the HELM benchmark to scrutinize LLMs concerning accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency.

    “Notably, Laing et al.5 have presented a multi-metric approach, as part of the HELM benchmark, to scrutinize LLMs concerning their accuracy, calibration (proficiency in assigning meaningful probabilities for generated text), robustness, fairness, bias, toxicity, and efficiency.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Liang, P. et al. Holistic evaluation of language models. Trans. Machine Learn. Res. https://openreview.net/forum?id=iO4LZibEqW (2023).

      Not checked in the free run.

  • Premisecitation

    Intrinsic evaluation metrics solely rely on surface-form similarity and language-specific perspectives, rendering them inadequate for healthcare chatbots.

    “However, they solely rely on surface-form similarity and language-specific perspectives, rendering them inadequate for healthcare chatbots.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [19] Sai, A. B., Mohankumar, A. K. & Khapra, M. M. A survey of evaluation metrics used for NLG systems. ACM Comput. Surv. (CSUR) 55, 1–39 (2022). source

      Not checked in the free run.

    • cites [20] Zhang*, T., Kishore*, V., Wu*, F., Weinberger, K. Q. & Artzi, Y. Bertscore: evaluating text generation with bert. In International Conference on Learning Representations. (International Conference on Learning Representations (ICLR), 2020).

      Not checked in the free run.

    • cites [21] Khurana, D., Koli, A., Khatter, K. & Singh, S. Natural language processing: state of the art, current trends and challenges. Multimedia Tools Appl. 82, 3713–3744 (2023). source

      Not checked in the free run.

  • Interpretationreasoning

    If BLEU and ROUGE metrics are solely used to evaluate a healthcare chatbot, an inaccurate answer may receive a high score compared with the reference answer.

    “Therefore, if we solely use these metrics to evaluate a healthcare chatbot, an inaccurate answer may receive a high score compared with the reference answer.”
    Quote in the paperSubstantial claim
  • Premisecitation

    Existing extrinsic context-aware evaluation methods have concentrated only on specific aspects, such as the robustness of generated answers within a particular medical domain.

    “However, these methods have merely concentrated on specific aspects, such as the robustness of the generated answers within a particular medical domain.”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [7] Thoppilan, R. et al. LaMDA: language models for dialog applications. Preprint at https://arxiv.org/abs/2201.08239 (2022).

      Not checked in the free run.

    • cites [9] Adiwardana, D. et al. Towards a human-like open-domain chatbot. Preprint at https://arxiv.org/abs/2001.09977 (2020).

      Not checked in the free run.

    • cites [12] Peng, B. et al. GODEL: large-scale pre-training for goal-directed dialog. Preprint at https://arxiv.org/abs/2206.11309 (2022).

      Not checked in the free run.

    • cites [13] Touvron, H. et al. LLaMA: open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023).

      Not checked in the free run.

    • cites [14] Wang, P. et al. Large language models are not fair evaluators. Preprint at https://arxiv.org/abs/2305.17926 (2023).

      Not checked in the free run.

    • cites [15] Resnik, P. et al. Using intrinsic and extrinsic metrics to evaluate accuracy and facilitation in computer-assisted coding. In Perspectives in Health Information Management Computer Assisted Coding Conference Proceedings (The American Health Information Management Association (AHIMA), 2006).

      Not checked in the free run.

    • cites [16] Liu, P. et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv. 55, 1–35 (2023).

      Not checked in the free run.

  • Classified as probably not a substantial claim. 41
  • May combine two claims. 11

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The evaluation of a healthcare chatbot model's performance encompasses diverse factors such as safety and privacy, which are contingent upon the specific users or audience involved.

    “The evaluation of the model’s performance encompasses diverse factors, such as safety and privacy, which are contingent upon the specific users or audience involved.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Existing intrinsic metrics such as BLEU and ROUGE inadequately capture semantic nuances, contextual relevance, long-range dependencies, changes in critical semantic ordering, and human-centric perspectives, limiting their effectiveness in evaluating healthcare chatbots.

    “For this reason, they inadequately capture vital aspects like semantic nuances, contextual relevance, long-range dependencies, changes in critical semantic ordering, and human-centric perspectives11, thereby limiting their effectiveness in evaluating healthcare chatbots.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Novikova, J., Dušek, O., Cercas Curry, A. & Rieser, V. Why we need new evaluation metrics for NLG. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2241–2252 (Association for Computational Linguistics, Copenhagen, Denmark, 2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Groundedness leverages relevant factual information, promoting sound reasoning and staying up-to-date, ensuring validity.

    “To address this, groundedness leverages relevant factual information, promoting sound reasoning and staying up-to-date ensuring validity.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In the healthcare domain, safety and currentness of information are paramount.

  • Classified as probably not a substantial claim.

    Premisecitation

    The definition of accuracy metrics is contingent upon the domain and task types involved.

    “The definition of these accuracy metrics is contingent upon the domain and task types involved5,25.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [5] Liang, P. et al. Holistic evaluation of language models. Trans. Machine Learn. Res. https://openreview.net/forum?id=iO4LZibEqW (2023).

      Not checked in the free run.

    • cites [25] AI Risk Management Framework—nist.gov. https://www.nist.gov/itl/ai-risk-management-framework (2023).

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The evaluation of a healthcare chatbot model and metric scoring may differ depending on the specific task at hand.

    “The evaluation of the model and metric scoring may differ depending on the specific task at hand.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chang et al. presented organized evaluation methodologies for LLMs through three dimensions: what to evaluate, where to evaluate, and how to evaluate.

    “In addition, Chang et al.8 have presented organized evaluation methodologies for LLMs through three essential dimensions: “what to evaluate,” “where to evaluate,” and “how to evaluate.””
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] Chang, Y. et al. A survey on evaluation of large language models. ACM ransactions on Intelligent Systems and Technology. https://doi.org/10.1145/3641289 (2024). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Robustness plays a critical role in ensuring the validity of a healthcare chatbot's responses.

    “Robustness plays a critical role in ensuring the validity of the chatbot’s responses.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Human inquiries may not precisely align with their underlying issues or intent.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The evaluation metrics required for assessing healthcare chatbots can be influenced by the healthcare domain they cater to.

    “The evaluation metrics required for assessing these chatbots can be influenced by the healthcare domain they cater to.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Each existing study of extrinsic metrics has been confined to a distinct set of metrics, neglecting the comprehensive and all-encompassing aspect concerning healthcare language models and chatbots.

    “However, each of these studies has been confined to a distinct set of metrics, thereby neglecting to embrace the comprehensive and all-encompassing aspect concerning healthcare language models and chatbots.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    The proposed healthcare chatbot evaluation metrics are categorized into four groups: accuracy, trustworthiness, empathy, and performance, based on their dependencies on the confounding variables.

    “As outlined below, the metrics are categorized into four distinct groups: accuracy, trustworthiness, empathy, and performance, based on their dependencies on the confounding variables.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The evaluation process for healthcare chatbot models takes into account three confounding variables: user type, domain type, and task type.

    “In this evaluation process, three confounding variables are taken into account: user type, domain type, and task type.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Wang et al. assessed the trustworthiness of GPT-3.5 and GPT-4 from eight aspects including toxicity, bias, robustness, privacy, machine ethics, and fairness.

    “Likewise, Wang et al.6 have assessed the trustworthiness of GPT-3.5 and GPT-4 from eight discerning aspects encompassing toxicity, bias, robustness, privacy, machine ethics, and fairness.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Wang, B. et al. Decodingtrust: a comprehensive assessment of trustworthiness in GPT models. Preprint at https://arxiv.org/abs/2306.11698 (2023).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Intrinsic metrics lack the capability to capture semantics, context, distant dependencies, semantically critical ordering change, and human perspectives, particularly in real-world scenarios.

    “These metrics lack the capability to capture essential elements such as semantics19,20, context19,21, distant dependencies22,23, semantically critical ordering change21, and human perspectives, particularly in real-world scenarios.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [19] Sai, A. B., Mohankumar, A. K. & Khapra, M. M. A survey of evaluation metrics used for NLG systems. ACM Comput. Surv. (CSUR) 55, 1–39 (2022). source

      Not checked in the free run.

    • cites [20] Zhang*, T., Kishore*, V., Wu*, F., Weinberger, K. Q. & Artzi, Y. Bertscore: evaluating text generation with bert. In International Conference on Learning Representations. (International Conference on Learning Representations (ICLR), 2020).

      Not checked in the free run.

    • cites [21] Khurana, D., Koli, A., Khatter, K. & Singh, S. Natural language processing: state of the art, current trends and challenges. Multimedia Tools Appl. 82, 3713–3744 (2023). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Existing evaluations overlook performance aspects of models, such as computational efficiency and model size, which are crucial for practical implementation.

    “Moreover, existing evaluations overlook performance aspects of models, such as computational efficiency and model size, which are crucial for practical implementation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Existing evaluation metrics overlook a wide range of crucial user-centered aspects that indicate the extent to which a chatbot establishes a connection and conveys support and emotion to the patient.

    “Second, it is evident that the existing evaluation metrics overlook a wide range of crucial user-centered aspects that indicate the extent to which a chatbot establishes a connection and conveys support and emotion to the patient.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Emotional bonds play a vital role in physician–patient communications.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A chatbot's ability to generalize enhances its validity in effectively addressing a wide range of medical scenarios.

    “A chatbot’s ability to generalize enhances its validity in effectively addressing a wide range of medical scenarios.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: There is scarcity of data and information across various medical domains and categories.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Up-to-dateness significantly enhances the validity of a chatbot by ensuring that its information aligns with the latest evidence and guidelines.

    “Up-to-dateness significantly enhances the validity of a chatbot by ensuring that its information aligns with the latest evidence and guidelines.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In the healthcare domain, safety and currentness of information are paramount.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    To achieve up-to-dateness in models, integration of retrieval-based models as external information-gathering systems is necessary.

    “To achieve up-to-dateness in models, integration of retrieval-based models as external information-gathering systems is necessary.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Current methods fail to address the issue of hallucination, wherein chatbots generate misleading or inaccurate information.

    “Furthermore, the current methods fail to address the issue of hallucination, wherein chatbots generate misleading or inaccurate information.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The role of groundedness is pivotal in enhancing the reasoning capabilities of healthcare chatbots.

    “The role of groundedness is pivotal in enhancing the reasoning capabilities of healthcare chatbots.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In the healthcare domain, safety and currentness of information are paramount.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    In medical report generation, ensuring the reliability and factuality of generated text is of utmost importance, a requirement that might not be as critical when the task involves acting as an assistant.

    “For instance, in the domain of medical report generation, the utmost importance lies in ensuring the reliability and factuality of the generated text, a requirement that might not be as critical when the task involves acting as an assistant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Accuracy metrics might remain invariant with regard to the user's type, as the ultimate objective of generated text is to achieve the highest level of accuracy irrespective of the intended recipient.

    “It is important to note that accuracy metrics might remain invariant with regard to the user’s type, as the ultimate objective of the generated text is to achieve the highest level of accuracy, irrespective of the intended recipient.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Metrics in trustworthiness and empathy may be correlated.

    “For instance, metrics in trustworthiness and empathy may be correlated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    A significant relationship exists between performance metrics and the other three categories.

    “A significant relationship exists between performance metrics and the other three categories.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The integration of the aforementioned requirements should result in the desired scores, treating the evaluation component as a black box.

    “The integration of the aforementioned requirements should result in the desired scores, treating the evaluation component as a black box.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Existing benchmarks may lack comprehensive assessments of the chatbot model's robustness concerning confounding variables specific to the target user type, domain type, and task type.

    “However, a notable concern arises when employing existing benchmarks (see Table 2) to automatically evaluate relevant metrics. These benchmarks may lack comprehensive assessments of the chatbot model’s robustness concerning confounding variables specific to the target user type, domain type, and task type.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Human-based evaluation methods present two main challenges: subjectivity and the need for a variety of domain expert annotators.

    “Human-based methods involve providing questions or guidelines to human annotators who score the chatbot’s generated answers based on given criteria. This approach presents two main challenges: subjectivity and the need for a variety of domain expert annotators.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Enhancing fairness within a healthcare chatbot's responses contributes to increased reliability by ensuring equitable and unbiased answers.

    “Enhancing fairness within a healthcare chatbot’s responses contributes to increased reliability by ensuring that the chatbot consistently provides equitable and unbiased answers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    An increase in parameters may introduce complexity, potentially affecting accuracy, trustworthiness, and empathy metrics positively or negatively.

    “An increase in parameters may introduce complexity, potentially affecting these metrics positively or negatively.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Empathy often necessitates personalization, which can potentially compromise privacy and lead to biased responses.

    “Empathy often necessitates personalization, which can potentially compromise privacy and lead to biased responses.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    A model trained on an imbalanced dataset with dominant samples from white males and limited samples from Hispanic females might exhibit bias and provide unfair responses to Hispanic females.

    “For instance, a model trained on an imbalanced dataset, with dominant samples from white males and limited samples from Hispanic females, might exhibit bias due to the imbalanced training dataset. Consequently, it may provide unfair responses to Hispanic females, as their patterns were not accurately learned during the training process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Temperature can influence specific metric scores.

    “temperature67, which can influence specific metric scores.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [67] Chung, J., Kamar, E. & Amershi, S. Increasing diversity while maintaining accuracy: Text data generation with large language models and human interventions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 575–593 (Association for Computational Linguistics, Toronto, 2023). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The ability to adjust confounding variables in the evaluation framework ensures alignment among all stakeholders evaluating the target healthcare chatbot model, fostering a consistent and uniform evaluation perspective.

    “The ability to adjust these variables in the evaluation framework ensures alignment among all stakeholders evaluating the target healthcare chatbot model, fostering a consistent and uniform evaluation perspective.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Reducing the number of parameters, which often leads to decreased memory usage and FLOPs, is likely to improve usability and latency.

    “Reducing the number of parameters, which often leads to decreased memory usage and FLOPs, is likely to improve usability and latency, making the model more efficient and effective in practical applications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Modifying model parameters during inference can influence the chatbot's behavior when responding to queries.

    “Modifying these parameters can influence the chatbot’s behavior when responding to queries.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Personalization enhances the reliability of a chatbot by tailoring its interactions and healthcare recommendations to individual users.

    “Personalization enhances the reliability of a chatbot by tailoring its interactions and healthcare recommendations to individual users, ensuring that responses align closely with their preferences and health-related data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Popular LLMs, such as GPT-4, Llama, and BERT, often require large memory capacity, making it challenging to run them on devices with limited memory.

    “Popular LLMs, such as GPT-4, Llama, and BERT, often require large memory capacity13,58–61, making it challenging to run them on devices with limited memory, such as embedded systems, laptops, and mobile phones62.”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [13] Touvron, H. et al. LLaMA: open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023).

      Not checked in the free run.

    • cites [58] Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).

      Not checked in the free run.

    • cites [59] Achiam, J. et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023).

      Not checked in the free run.

    • cites [60] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186 (Association for Computational Linguistics, Minneapolis, 2019).

      Not checked in the free run.

    • cites [61] Lee, J. et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 1234–1240 (2019). source

      Not checked in the free run.

    • cites [62] Zhuang, B. et al. A survey on efficient training of transformers. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23 (ed. Elkind, E.) 6823–6831 (International Joint Conferences on Artificial Intelligence Organization, 2023). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Study design and conductcitation

    Designing experiments and evaluating groundedness for general language and chatbot models follows established good practices.

    “Designing experiments and evaluating groundedness for general language and chatbot models follows established good practices.7,30,34–37”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [7] Thoppilan, R. et al. LaMDA: language models for dialog applications. Preprint at https://arxiv.org/abs/2201.08239 (2022).

      Not checked in the free run.

    • cites [30] Jin, Z. et al. Can large language models infer causation from correlation? Preprint at https://arxiv.org/abs/2306.05836 (2023).

      Not checked in the free run.

    • cites [34] Glaese, A. et al. Improving alignment of dialogue agents via targeted human judgements. Preprint at https://arxiv.org/abs/2209.14375 (2022).

      Not checked in the free run.

    • cites [35] Gekhman, Z., Herzig, J., Aharoni, R., Elkind, C. & Szpektor, I. Trueteacher: Learning factual consistency evaluation with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2053–2070 (Association for Computational Linguistics, 2023). source

      Not checked in the free run.

    • cites [36] Manakul, P., Liusie, A. & Gales, M. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 9004–9017 (Association for Computational Linguistics, 2023). source

      Not checked in the free run.

    • cites [37] Laban, P. et al. LLMs as factual reasoners: insights from existing benchmarks and beyond. Preprint at https://arxiv.org/abs/2305.14540 (2023).

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Factuality assessment requires examining the presence of true-causal relations among generated words, which must be supported by evidence from reliable reference sources.

    “This assessment requires examining the presence of true-causal relations among generated words30, which must be supported by evidence from reliable reference sources7,12.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [30] Jin, Z. et al. Can large language models infer causation from correlation? Preprint at https://arxiv.org/abs/2306.05836 (2023).

      Not checked in the free run.

    • cites [7] Thoppilan, R. et al. LaMDA: language models for dialog applications. Preprint at https://arxiv.org/abs/2201.08239 (2022).

      Not checked in the free run.

    • cites [12] Peng, B. et al. GODEL: large-scale pre-training for goal-directed dialog. Preprint at https://arxiv.org/abs/2206.11309 (2022).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    The proposed metrics demonstrate both within-category and between-category associations, with the potential for negative or positive correlations among them.

    “The proposed metrics demonstrate both within-category and between-category associations, with the potential for negative or positive correlations among them.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The number of parameters in a language model can impact accuracy, trustworthiness, and empathy metrics.

    “For instance, the number of parameters in a language model can impact accuracy, trustworthiness, and empathy metrics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    As the number of tokens increases, the memory and computation needed also increase, leading to higher latency and reduced usability.

    “As the number of tokens increases, the memory and computation needed also increase63, leading to higher latency and reduced usability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Hoffmann, J. et al. An empirical analysis of compute-optimal large language model training. In Advances in Neural Information Processing Systems (eds Oh, A. H. et al.) (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2022).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Within the accuracy metrics category, up-to-dateness and groundedness show a positive correlation.

    “For instance, within the accuracy metrics category, up-to-dateness and groundedness show a positive correlation, as ensuring the chatbot utilizes the most recent and valid information enhances the factual accuracy of answers, thereby increasing groundedness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    A low parameter count can limit the model's knowledge acquisition and influence the values of accuracy, trustworthiness, and empathy metrics.

    “Conversely, a low parameter count can limit the model’s knowledge acquisition and influence the values of these metrics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The selection of evaluation methods significantly impacts metric scores.

    “Various automatic and human-based evaluation methods can quantify each metric, and the selection of evaluation methods significantly impacts metric scores.”
    Quote in the paperNot a substantial claim
45 of 45

  • The paper relies on this without support.

    Healthcare applications are life-critical.

    “Due to the life-critical nature of healthcare applications, using conversational models necessitates establishing a unified and comprehensive set of foundation metrics1 that enable a meticulous evaluation of the models’ performance, capabilities, identification of potential errors, and implementation of effective feedback mechanisms.”

  • The paper relies on this without support.

    Emotional bonds play a vital role in physician–patient communications.

    “Emotional bonds play a vital role in physician–patient communications, but they are often ignored during the development and evaluation of chatbots.”

  • The paper relies on this without support.

    In the healthcare domain, safety and currentness of information are paramount.

    “In particular, in the healthcare domain, where safety and currentness of information are paramount, hallucinations pose a significant concern.”

  • The paper relies on this without support.

    There is scarcity of data and information across various medical domains and categories.

    “In the context of healthcare, the significance of the generalization metric becomes pronounced due to the scarcity of data and information across various medical domains and categories.”

  • The paper relies on this without support.

    Human inquiries may not precisely align with their underlying issues or intent.

    “In healthcare chatbots, where human inquiries may not precisely align with their underlying issues or intent, robustness assumes paramount importance.”

  • The paper relies on this without support.

    The desired level of interpretability for a generated text may vary between a patient and a nurse.

    “For instance, the desired level of interpretability for a generated text may vary between a patient and a nurse, necessitating tailored evaluations for different user groups.”

  • The paper relies on this without support.

    Empathy plays a crucial role in building trust between users and chatbots.

    “Empathy also plays a crucial role in building trust between users and chatbots.”

  • The paper relies on this without support.

    Unempathetic responses can erode trust and credibility in the system.

    “Unempathetic responses can erode trust and credibility in the system, as users may feel unheard, misunderstood, or invalidated.”

  • The paper relies on this without support.

    Low latency ensures prompt and efficient communication, enabling users to obtain timely responses.

    “Low latency ensures prompt and efficient communication, enabling users to obtain timely responses.”

  • The paper relies on this without support.

    Performance metrics may remain invariant concerning the three confounding variables (user type, domain type, and task type).

    “It is important to note that performance metrics may remain invariant concerning the three confounding variables (user type, domain type, and task type).”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The application gap in traditional metrics is attributed to surface-form similarity and lexicographic analytic restrictions.

    “The application gap is attributed to surface-form similarity and lexicographic analytic restrictions41.”

    From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [41]

  • Medical image reports assist in differential diagnosis and are characterized by complex paraphrasing, high token counts (>100), and numerous negative descriptions, which conflict with common metric evaluation contexts.

    “Medical image reports assist in differential diagnosis and thus are characterized by complex paraphrasing, high token counts (>100), and numerous negative descriptions, which conflict with common metric evaluation contexts24,41.”

    From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [41]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • OpenAI Evals points to an unrelated reference

    Minor

    Table 2 cites reference 108 for OpenAI Evals, but reference 108 is titled as a paper about the cost of large language models.

  • Performance metrics' dependence on confounding variables is inconsistent

    Minor

    The figure says performance metrics are evaluated based on all three confounding variables, while the Performance section says they may remain invariant with respect to those same variables.

  • WinoGender citation points to LLaMA

    Minor

    Table 2 attributes WinoGender to reference 13, but that reference is the LLaMA paper.

  • BigBench citation identifies a big-data benchmark

    Minor

    Table 2 lists BigBench as a chatbot-evaluation benchmark and cites reference 112, whose title instead identifies a benchmark for big-data analytics.

4 of 4

Cites

Flagged first, then by citations

13 of the 143 references couldn’t be matched to a known paper.

Research works printed but not linked: 19

  1. Resnik, P. et al. Using intrinsic and extrinsic metrics to evaluate accuracy and facilitation in computer-assisted coding. In Perspectives in Health Information Management Computer Assisted Coding Conference Proceedings (The American Health Information Management Association (AHIMA), 2006).
  2. Zhang*, T., Kishore*, V., Wu*, F., Weinberger, K. Q. & Artzi, Y. Bertscore: evaluating text generation with bert. In International Conference on Learning Representations. (International Conference on Learning Representations (ICLR), 2020).
  3. Napoles, C., Van Durme, B. & Callison-Burch, C. Evaluating sentence compression: Pitfalls and suggested remedies. In Proceedings of the Workshop on Monolingual Text-To-Text Generation, 91–97 (Association for Computational Linguistics, 2011). https://aclanthology.org/volumes/W11-16/.
  4. Zhou, C. et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems 36 (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024).
  5. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).
  6. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186 (Association for Computational Linguistics, Minneapolis, 2019).
  7. Hoffmann, J. et al. An empirical analysis of compute-optimal large language model training. In Advances in Neural Information Processing Systems (eds Oh, A. H. et al.) (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2022).
  8. Text REtrieval Conference (TREC) Home Page — trec.nist.gov. https://trec.nist.gov/. [Accessed 28-07-2023]
  9. Ge, Y. et al. Openagi: When llm meets domain experts. Advances in Neural Information Processing Systems 36 (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024).
  10. Wei, J. et al. Finetuned language models are zero-shot learners. In International Conference on Learning Representations (ICLR, 2022).
  11. Radford, A. et al. Language models are unsupervised multitask learners. OpenAI Blog 1, 9 (2019).
  12. Zhang, K. et al. BiomedGPT: a unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks. Preprint at https://arxiv.org/abs/2305.17100 (2023). · arXiv
  13. Banerjee, S. & Lavie, A. Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65–72 (Association for Computational Linguistics, 2005).
  14. Peng, Y.-S., Tang, K.-F., Lin, H.-T. & Chang, E. Refuel: exploring sparse features in deep reinforcement learning for fast disease diagnosis. Adv. Neural Inf. Process. Syst. 31, (2018).
  15. Hermann, K. M. et al. Teaching machines to read and comprehend. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, 1693–1701 (MIT Press, Cambridge, 2015).
  16. Aryan, A., Nain, A. K., McMahon, A., Meyer, L. A. & Sahota, H. S. The costly dilemma: are large language models the pay-day loans of machine learning? https://abiaryan.com/assets/EMNLP%20Submission_Non-Anon.pdf. (2023).
  17. Yuan, L. et al. Revisiting out-of-distribution robustness in nlp: Benchmarks, analysis, and LLMs evaluations. Advances in Neural Information Processing Systems 36 (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024).
  18. Hendrycks, D. et al. Measuring coding challenge competence with APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2021).
  19. Maas, A. et al. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 142–150 (Association for Computational Linguistics, 2011).

Cited by

Sorted by citations

Related

Authors
Abbasian, M., Khatibi, E., Azimi, I., Oniani, D., Shakeri Hossein Abad, Z., Thieme, A., Sriram, R., Yang, Z., Wang, Y., Lin, B., Gevaert, O., Li, L.-J., Jain, R., & Rahmani, A. M.
Affiliations
  • University of California, Irvine (US)
  • University of Pittsburgh (US)
  • Stanford University (US)
  • National Institute of Standards and Technology (US)
Corresponding author
  • Abbasian, Mahyar
  • Khatibi, Elahe
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2024-03-29
Volume
7
Issue
1
Pages
82-82
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-024-01074-z
PMC
PMC10980701
OpenAlex
W4393335480
Semantic Scholar
8d1211cbbdf161feaae2c87832ede063346e76dd
Cited by
213
References
143
Citations by year
2026: 92 · 2025: 83 · 2024: 38
FWCI
4.81
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Institute of Standards and Technology
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
generative AI in healthcare, healthcare chatbots, evaluation metrics, large language models, chatbots, patient-centered care, user trust, empathy, emotional support, personalized health recommendations, confounding factors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review