CoChat
Sign Up
Cited by 147Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIEthics and bioethics in healthcare

Chatbot breakthrough in the 2020s? An ethical reflection on the trend of automated consultations in health care

Open access
Parviainen, Jaana, Rantala, Juho·2022·Medicine, Health Care and Philosophy
DOI 10.1007/s11019-021-10049-wPMC PMC8416570
Open this paper in CoChat — free accountOpen PDF

Abstract

Many experts have emphasised that chatbots are not sufficiently mature to be able to technically diagnose patient conditions or replace the judgements of health professionals. The COVID-19 pandemic, however, has significantly increased the utilisation of health-oriented chatbots, for instance, as a conversational interface to answer questions, recommend care options, check symptoms and complete tasks such as booking appointments. In this paper, we take a proactive approach and consider how the emergence of task-oriented chatbots as partially automated consulting systems can influence clinical practices and expert–client relationships. We suggest the need for new approaches in professional ethics as the large-scale deployment of artificial intelligence may revolutionise professional decision-making and client–expert interaction in healthcare organisations. We argue that the implementation of chatbots amplifies the project of rationality and automation in clinical practice and alters traditional decision-making practices based on epistemic probability and prudence. This article contributes to the discussion on the ethical challenges posed by chatbots from the perspective of healthcare professional ethics.

AI snapshot

Written from the paper’s full text.

TL;DR

This conceptual paper argues that the COVID-19-driven uptake of task-oriented health chatbots amplifies the project of rationality and automation in clinical practice, altering decision-making that traditionally rests on epistemic probability and prudence (phronesis). The authors call for new ethico-political approaches in professional ethics, framing chatbots as a matter of systemic change rather than individual or organisational challenge.

Key findings

  • The authors argue that chatbots are partially automated, user-facing "gateways" between patient and clinician, and that their rigid, formal models capture only a slice of medical knowledge; expertise instead requires intersubjective circulation of knowledge and criticism.
  • They list ten risks/concerns from the HCP perspective, including erosion of practical wisdom in diagnosis, market logic driving care, incompleteness of chatbot assessment (especially in emergencies), increased patient self-care responsibility, trust decay, more patient–physician conflict, new gateways/bottlenecks, increased workload and costs, reduced employee well-being (shift from operator to supervisor), and a "domino effect" pushing patients into automated systems.
  • Cited empirical work: a small survey (N = 11) of nursing students on HCP use of Omaolo found it seemed to save work time and that interviewees predicted increased use; Judson et al. estimated the Conversa chatbot "saved employees over 15 000 h of time waiting to be screened" (March–May 2020); Palanica et al. found most physicians believed chatbots could not effectively care for all patient needs or understand/display human emotion; Dennis et al. found perceived agent ability was the primary driver of patient response, with chatbots viewed more positively than human agents.
  • Omaolo is described as administering 17 separate symptom assessments in Finnish, Swedish and English, built on Duodecim's EBMEDS logical reasoning; Pasquale's cited Australian study of 82 mobile apps for bipolar disorder found they were generally not in line with practice guidelines or established self-management principles.

Methods

  • Design: proactive, theoretical and philosophical ethical reflection; the authors state the paper's main contributions are theoretical and philosophical and that they do not intend to provide empirical evidence for or against chatbots.
  • Evidence base: findings from recent empirical studies of health chatbots (e.g., Dennis et al. 2020; Palanica et al. 2019; Judson et al. 2020; Pynnönen et al. 2020; Nordheim et al. 2019) plus ethical literature drawing on Aristotle's phronesis.
  • Analysis: conceptual discussion in three parts—health chatbots and their technical capabilities; chatbots, rationality/automation and epistemic probability in professional decision-making; and ethical/social issues for professional ethics, concluding with a list of HCP-perspective risks.

Limitations

  • Stated: the authors acknowledge "the difficulty in identifying the nature of systemic change and looking at its complex network-like structure in the functioning of health organisations"; they note their concluding points "are not yet grounded in evidence-based knowledge"; and they state there are currently no established ethical principles to evaluate healthcare chatbots.
  • Evident from the text: the paper offers no primary data or systematic review, relying on secondary empirical findings and philosophical argument, so its claims about long-run effects on clinical practice and patient–clinician relationships are prospective rather than demonstrated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11019-021-10049-w

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    The COVID-19 pandemic has significantly increased the utilisation of health-oriented chatbots in health care.

    “The COVID-19 pandemic, however, has significantly increased the utilisation of health-oriented chatbots, for instance, as a conversational interface to answer questions, recommend care options, check symptoms and complete tasks such as booking appointments.”
    • Also stated in: § Introduction · plain
      “The emergence of COVID-19 as a global pandemic has significantly advanced the development of telehealth and the utilisation of health-oriented chatbots in the diagnosis and treatment of coronavirus infection (AlgorithmWatch 2020; McGreevey et al. 2020).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites AlgorithmWatch 2020

      Not checked in the free run.

    • cites McGreevey et al. 2020 source

      Not checked in the free run.

  • Premisecitation

    The design principles of most health technologies are based on the idea that technologies should mimic human decision-making capacity.

    “The design principles of most health technologies are based on the idea that technologies should mimic human decision-making capacity.”
    • Also stated in: § Introduction · plain
      “These systems are computer programmes that are ‘programmed to try and mimic a human expert’s decision-making ability’ (Fischer and Lam 2016, p. 23).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fischer and Lam 2016

      Not checked in the free run.

  • Premisecitation

    COVID-19 screening is an ideal application for chatbots because it is a well-structured process involving clearly defined questions and a risk score.

    “COVID-19 screening is considered an ideal application for chatbots because it is a well-structured process that involves asking patients a series of clearly defined questions and determining a risk score (Dennis et al. 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dennis et al. 2020 source

      Not checked in the free run.

  • Premisecitation

    So far, there has been scant discussion on how digitalisation, including chatbots, transforms medical practices, especially in the context of human capabilities in exercising practical wisdom.

    “So far, there has been scant discussion on how digitalisation, including chatbots, transform medical practices, especially in the context of human capabilities in exercising practical wisdom (Bontemps-Hommen et al. 2019).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bontemps-Hommen et al. 2019 source

      Not checked in the free run.

  • Interpretationcitation

    One consequence of chatbots can be the shift from operator to supervisor, that is, expert work becomes more about monitoring and surveillance than before.

    “One of the consequences can be the shift from operator to supervisor, that is, expert work becomes more about monitoring and surveillance than before (Zerilli et al. 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zerilli et al. 2019 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 71
  • May combine two claims. 29

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    If primary healthcare services are increasingly built on chatbots and other digital solutions, the tech industry will increasingly gain in health care and contribute to the corporate privatization of public functions.

    “For instance, if primary healthcare services are increasingly built on chatbots and other digital solutions, the tech industry will increasingly gain in health care and contribute to the ‘corporate privatization of public functions’ (Suarez-Villa 2012, p. 188).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Suarez-Villa 2012

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbots have been trying to mimic human behaviour in text-based conversation.

    “From the emergence of the first chatbot, ELIZA, developed by Joseph Weizenbaum (1966), chatbots have been trying to ‘mimic human behaviour in a text-based conversation’ (Shum et al. 2018, p. 10; Abd-Alrazaq et al. 2020).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Shum et al. 2018 source

      Not checked in the free run.

    • cites Abd-Alrazaq et al. 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    To limit face-to-face meetings during the pandemic, chatbots have been used as a conversational interface to answer questions, recommend care options, check symptoms and complete tasks such as booking appointments.

    “To limit face-to-face meetings in health care during the pandemic, chatbots have being used as a conversational interface to answer questions, recommend care options, check symptoms and complete tasks such as booking appointments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Chatbot platforms seek to automate some aspects of professional decision-making by systematising traditional analytics of decision-making techniques.

    “Thus, chatbot platforms seek to automate some aspects of professional decision-making by systematising the traditional analytics of decision-making techniques (Snow 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Snow 2019

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Systems such as chatbots seem to complement and sometimes even substitute HCP patient consultations.

    “The latter, that is, systems such as chatbots, seem to complement and sometimes even substitute HCP patient consultations (p. 55).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Pasquale 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The key feature of chatbots is language and speech recognition, i.e., natural language processing, which enables them to understand the user's language to a certain extent.

    “Thus, their key feature is language and speech recognition, that is, natural language processing (NLP), which enables them to understand, to a certain extent, the language of the user (Gentner et al. 2020, p. 2).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gentner et al. 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    A complete algorithmic health system requires a back-up system or practices that imply increased costs and the emergence of new problems.

    “A complete system also requires a ‘back-up system’ or practices that imply increased costs and the emergence of new problems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    A pandemic can accelerate the digitalisation of health care, but not all consequences are necessarily predictable or positive from the perspectives of patients and professionals.

    “A pandemic can accelerate the digitalisation of health care, but not all consequences are necessarily predictable or positive from the perspectives of patients and professionals.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning

    Health consultation chatbots are partially automated proactive decision-making agents that guide the actions of healthcare personnel.

    “Conversely, health consultation chatbots are partially automated proactive decision-making agents that guide the actions of healthcare personnel.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The change brought by partial ADM is systemic, pervading all parts of a system and taking into account interrelationships and interdependencies among these parts.

    “Thus, instead of only re-organising work, we are talking about systemic change (e.g. Simondon 2017), that is, change that pervades all parts of a system, taking into account the interrelationships and interdependencies among these parts.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Simondon 2017

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Relieving pressure on contact centres by chatbots is especially important in the present COVID-19 situation, thus making chatbots cost-effective.

    “This relieving of pressure on contact centres is especially important in the present COVID-19 situation (Dennis et al. 2020, p. 1727), thus making chatbots cost-effective.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dennis et al. 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    New technologies may form new gatekeepers of access to specialty care or entirely usurp human doctors in many patient cases.

    “New technologies may form new gatekeepers of access to specialty care or entirely usurp human doctors in many patient cases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Decision-making is a carefully articulated mode of reasoning, with its own specific standards, procedures and problems.

    “Therefore, decision-making is a carefully articulated mode of reasoning, with its own specific standards, procedures and problems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The rigid and formal systems of chatbots, even with the ML bend, are locked in certain a priori models of calculation.

    “The ‘rigid’ and formal systems of chatbots, even with the ML bend, are locked in certain a priori models of calculation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    The dominos fall when chatbots push patients from traditional clinical face-to-face practice to more complicated automated systems.

    “The dominos fall when chatbots push patients from traditional clinical face-to-face practice to more complicated automated systems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    In the long run, clinical practices can become monotonous: the shift from operator to supervisor, which also decreases the embodied and emotional relationship between HCPs and patients.

    “In the long run, clinical practices can become monotonous: the shift from operator to supervisor, which also decreases the embodied and emotional relationship between HCPs and patients.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The implementation of chatbots without the consent of HCPs may reduce HCP motivation and capabilities to control their own work.

    “The implementation of chatbots without the consent of HCPs may reduce HCP motivation and capabilities to control their own work.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    More simple solutions can lead to new costs and workload when the usage of new technology creates unexpected problems in practice.

    “More simple solutions can lead to new costs and workload when the usage of new technology creates unexpected problems in practice.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Mathematical or statistical probability in medical diagnosis has become one of the principal targets, with the consequence that AI is expected to improve diagnostics in the long run.

    “Mathematical or statistical probability in medical diagnosis has become one of the principal targets, with the consequence that AI is expected to improve diagnostics in the long run.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Due to partly automated systems, patient frustration can reach boiling point when patients feel that they must first communicate with chatbots before they can schedule an appointment.

    “Due to partly automated systems, patient frustration can reach boiling point when patients feel that they must first communicate with chatbots before they can schedule an appointment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbots can be classified based on their purpose (assistant or conversation) and mode of communication (text or voice).

    “As Lee et al. (2020, p. 339) have pointed out, chatbots ‘can be classified based on their purpose (i.e., assistant or conversation) and mode of communication (i.e., text or voice)’.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lee et al. 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    As computerised chatbots are characterised by a lack of human presence, which is the reverse of traditional face-to-face interactions with HCPs, they may increase distrust in healthcare services.

    “As computerised chatbots are characterised by a lack of human presence, which is the reverse of traditional face-to-face interactions with HCPs, they may increase distrust in healthcare services.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Being regarded as devoid of human features enables people to see chatbots as more trustworthy and non-judgmental.

    “This enables people to see them as more trustworthy and non-judgmental.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    More conversational bots, such as those that strive to help with mental illnesses and conditions, cannot be constructed-at least not easily-using formal thought models.

    “However, more conversational bots, for example, those that strive to help with mental illnesses and conditions, cannot be constructed—at least not easily—using these thought models.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Omaolo may have potentially prevented unnecessary visits to healthcare facilities.

    “The COVID-19 pandemic has arguably accelerated the development of digital health services within the Finnish healthcare system, and Omaolo ‘may have…potentially prevented unnecessary visits to healthcare facilities’ (Tiirinki et al. 2020, p. 660).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tiirinki et al. 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Phronesis, prudence or practical wisdom refers to the flexible, interpretive capacity that enables the physician to determine the best course of action when knowledge depends on circumstance.

    “Phronesis, prudence or practical wisdom refers to the flexible, interpretive capacity that enables the physician to determine the best course of action when knowledge depends on circumstance (Montgomery 2006).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Montgomery 2006

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    When physicians observe a patient presenting with specific signs and symptoms, they assess the subjective probability of the diagnosis, called diagnostic probabilities, a form of epistemic probability.

    “When physicians observe a patient presenting with specific signs and symptoms, they assess the subjective probability of the diagnosis. Such probabilities have been called diagnostic probabilities (Wulff et al. 1986), a form of epistemic probability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wulff et al. 1986

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    In practice, chatbot expertise has to do with giving a correct answer, i.e., provision of accurate and relevant information.

    “In practice, ‘chatbot expertise’ has to do with, for example, giving a correct answer (provision of accurate and relevant information).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Complex algorithmic systems represent a growing resource of interactive, autonomous and often self-learning agency, potentially transforming cooperation between machines and professionals by emphasising the agency of machines.

    “Complex algorithmic systems represent a growing resource of interactive, autonomous and often self-learning (in the ML sense) agency, potentially transforming cooperation between machines and professionals by emphasising the agency of machines (Morley et al. 2019).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Morley et al. 2019

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    Bots do not get sick or tired and can be up and running 24 hours per day.

    “Bots also do not get sick or tired, and they can be up and running 24 h per day.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    HCPs and patients lack trust in the ability of chatbots, which may lead to concerns about their clinical care risks, accountability and an increase in the clinical workload rather than a reduction.

    “HCPs and patients lack trust in the ability of chatbots, which may lead to concerns about their clinical care risks, accountability and an increase in the clinical workload rather than a reduction.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Partial ADM presumably transforms healthcare professionals' work conditions, the skills required for jobs, the education system, and embodied and emotional relationships between professionals and customers.

    “As a profound socio-digital change, partial ADM presumably transforms healthcare professionals’ work conditions, the skills required for jobs and, therefore, in the long run, the education system and re-arranges embodied and emotional relationships between professionals and customers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Sophisticated AI-based chatbots require a great deal of human resources, for instance, experts of data analytics, whose work also needs to be publicly funded.

    “Sophisticated AI-based chatbots require a great deal of human resources, for instance, experts of data analytics, whose work also needs to be publicly funded.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Medical decision-making has three constituent elements: showing off experience and knowledge, bending the rules, and taking the middle course.

    “Following Heinrichs’ (2007, p. 289) account of phronesis, we emphasise three constituent elements in medical decision-making: (1) showing off experience and knowledge, (2) bending the rules (i.e. adjusting to changing circumstances rather than following strict rules) and (3) taking the middle course (conciliatory view between contradictions and conflicts).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Heinrichs 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Chatbots are regarded as devoid of human features (in the usual sense).

    “One key aspect is that chatbots are regarded as devoid of human features (in the usual sense).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    If experts lean on the false ideals of chatbot capability, this can lead to patient overconfidence and ethical problems.

    “If experts lean on the false ideals of chatbot capability, this can also lead to patient overconfidence and, furthermore, ethical problems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Chatbots could become a surrogate for nonmedical caregivers.

    “In particular, chatbots could become a surrogate for nonmedical caregivers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Chatbots are able to quickly provide patients with information about their health problems based on the symptoms reported on respective platforms.

    “In general, chatbots are able to quickly provide patients with information about their health problems based on the symptoms reported on respective platforms.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    There are risks involved when patients are expected to self-diagnose, such as a misdiagnosis provided by the chatbot or patients potentially lacking an understanding of the diagnosis.

    “There are risks involved when patients are expected to self-diagnose, such as a misdiagnosis provided by the chatbot or patients potentially lacking an understanding of the diagnosis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    When chatbots are developed by private healthcare companies, they usually follow market logic such as profit maximisation, or at least this dimension is dominant.

    “When chatbots are developed by private healthcare companies, they usually follow the market logic, such as profit maximisation, or at the very least, this dimension is dominant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbot purpose can be divided into task-oriented (goal-oriented) and social.

    “Their purpose can be divided into two categories: task-oriented (or goal-oriented) and social (Galitsky 2019, p. 3).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Galitsky 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    AI technologies can perform some narrow tasks or functions better than humans, and their calculation power is faster and memory more reliable.

    “AI technologies can perform some narrow tasks or functions better than humans, and their calculation power is faster and memory more reliable.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    AI technologies are occasionally presented, more or less implicitly, as replacements of the human actor on a task, suggesting that they or their abilities are identifiable with human beings.

    “However, occasionally, these technologies are presented, more or less implicitly, as replacements of the human actor on a task, suggesting that they—or their abilities/capabilities—are identifiable with human beings (or their abilities/capabilities).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Persuasion and negotiation may increase the workload of professionals and create new tensions between patients and physicians.

    “This persuasion and negotiation may increase the workload of professionals and create new tensions between patients and physicians.”
    • Also stated in: § Chatbots and the pressure on professional ethics · plain
      “This level of persuasion and negotiation increases the workload of professionals and creates new tensions between patients and physicians.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Patients' overconfidence in the ability of chatbots is a downside.

    “One of the downsides is patients’ overconfidence in the ability of chatbots.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Formal models, such as game-theoretical models, do not completely describe reality or the phenomenon in question and its processes; they grasp only a slice of the phenomenon.

    “However, it is worth noting that formal models, such as game-theoretical models, do not completely describe reality or the phenomenon in question and its processes; they grasp only a slice of the phenomenon.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    One of the key elements of expertise and its recognition is that patients and others can trust the opinions and decisions offered by the expert/professional.

    “One of the key elements of expertise and its recognition is that patients and others can trust the opinions and decisions offered by the expert/professional.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The details of experiencing chatbots and their expertise as trustworthy are a complex matter.

    “However, the details of experiencing chatbots and their expertise as trustworthy are a complex matter.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Medical diagnosis and decision-making require prudence, a mode of reasoning about contingent matters to select the best course of action.

    “Thus, medical diagnosis and decision-making require ‘prudence’, that is, ‘a mode of reasoning about contingent matters in order to select the best course of action’ (Hariman 2003, p. 5).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hariman 2003

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Most healthcare chatbots are simple, user-faced and task-oriented.

    “Most of them are simple, user-faced and task-oriented (similar to ELIZA).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisecitation

    Based on previous research on health chatbots, consultation chatbots have been found to play a beneficial role in supporting, motivating and coaching patients and streamlining organisational tasks.

    “Based on the previous research on health chatbots, consultation chatbots have been found to play a beneficial role in supporting, motivating and coaching patients and streamlining organisational tasks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    People can trust chatbots if they are seen as experts or as possessing expertise of some kind, while expertise itself requires maintaining this trust or trustworthiness.

    “People can trust chatbots if they are seen as ‘experts’ (or as possessing expertise of some kind), while expertise itself requires maintaining this trust or trustworthiness.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Task-oriented chatbots employ a question-answer style or step-by-step conversation and depend on supervised learning, reinforcement learning and extensive domain-specific knowledge.

    “The former employs a question–answer style or step-by-step conversation. These chatbots depend on ‘supervised learning, reinforcement learning and an extensive domain-specific knowledge’ (p. 3).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Galitsky 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Like falling dominoes, the large-scale deployment of chatbots can push HCPs and patients into novel forms of healthcare delivery, which can affect patients' access to care and drive some to new provider options.

    “Like falling dominoes, the large-scale deployment of chatbots can push HCPs and patients into novel forms of healthcare delivery, which can affect patients’ access to care and drive some to new provider options.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Expertise generally requires the intersubjective circulation of knowledge, that is, a pool of dynamic knowledge and intersubjective criticism of data, knowledge and processes.

    “Expertise generally requires the intersubjective circulation of knowledge, that is, a pool of dynamic knowledge and intersubjective criticism of data, knowledge and processes (e.g. Prior 2003; Collins and Evans 2007).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Prior 2003 source

      Not checked in the free run.

    • cites Collins and Evans 2007 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Social chatbots are not usually task-oriented unless sociality itself is the task or goal.

    “The latter, namely social chatbots, are not usually task-oriented, unless the ‘sociality’ itself is the task or goal.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Task-oriented chatbots follow formal models of thought in a precise manner; their functions are easily derived from prior expert processes performed by humans.

    “Task-oriented chatbots follow these models of thought in a precise manner; their functions are easily derived from prior expert processes performed by humans.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Experienced doctors, as prudential actors, are capable of working in a challenging triad of messy practice, ethical standards and context-sensitive scientific knowledge.

    “Experienced doctors, as prudential actors, are capable of working in a challenging triad of messy practice, ethical standards and context-sensitive scientific knowledge.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Chatbots can connect patients to HCPs.

    “In addition, they can connect patients to HCPs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Chatbot users need to see and experience the bots as providing answers reflecting knowledge, competence, and experience.

    “Chatbot users (patients) need to see and experience the bots as ‘providing answers reflecting knowledge, competence, and experience’ (p. 24)—all of which are important to trust.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Nordheim et al. 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    Encountering the unexpected is an occupational hazard in clinical practice.

    “Encountering the unexpected is an occupational hazard in clinical practice.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    So far, there has been poor adoption of chatbot technologies among physicians, and patients have shown weak adherence to them.

    “So far, there has been poor adoption of chatbot technologies among physicians, and patients have shown weak adherence to them (Palanica et al. 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Palanica et al. 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Ethical issues within chatbots in health professional work do not simply fall in the domain of individual or organisational challenges; they require policy decisions on how the new tools can be implemented in order to automate decisions based on human assessment.

    “Ethical issues within chatbots in health professional work do not simply fall in the domain of individual or organisational challenges; they require policy decisions on how the new tools can be implemented in order to automate decisions based on human assessment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Technological systems in health care are often highly complex in the sense that experts rarely have a full understanding of system design or functionality.

    “Technological systems in health care are often highly complex in the sense that experts rarely have a full understanding of system design or functionality.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Quantitative methods like CPS omitted the complex social, ethical and political issues that chatbots bring with them to health care.

    “However, these kinds of quantitative methods omitted the complex social, ethical and political issues that chatbots bring with them to health care.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    People seem to be more willing to share medical information with chatbots-e.g. people use chatbots to diagnose more intimate conditions, such as STDs.

    “Thus, people seem to be more willing to share medical information with them—e.g. people use chatbots to diagnose more intimate conditions, such as STDs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    In most cases, it seems that chatbots have had a positive effect in precisely the same tasks performed in other industries (e.g. customer service).

    “In most cases, it seems that chatbots have had a positive effect in precisely the same tasks performed in other industries (e.g. customer service).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    It was still not entirely clear whether chatbots were better overall at improving the various clinical health outcomes of patients and why they should be adopted compared with traditional methods of care.

    “Palanica et al. noted that it was still not entirely clear whether chatbots were better overall at improving the various clinical health outcomes of patients and why they should be adopted compared with traditional methods of care, that is, information coming from a human physician.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Palanica et al. 2019 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationcitation

    Because chatbots lack the intelligence to accurately assess patients, they cannot provide detailed clarifications regarding patient assessments, are unable to assess emergency health situations or may indirectly harm patients by not knowing all the personal factors associated with specific patients.

    “Because chatbots lack the intelligence to accurately assess patients, they cannot provide detailed clarifications regarding patient assessments, are unable to assess emergency health situations or may indirectly harm patients by not knowing all the personal factors associated with specific patients.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Palanica et al. 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Trust in chatbots-when they function as they should-has positive effects on trust in HCPs.

    “It seems that trust in chatbots—when they function as they should—has positive effects on trust in HCPs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    When problems are recognised, they can rarely be traced to a single factor because different agents are involved in the configuration of the system.

    “When problems are recognised, they can rarely be traced to a single factor because different agents are involved in the configuration of the system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Chatbots are more efficient in the sense that they can function 24 h per day and do not tire or fall ill.

    “Chatbots are more efficient in the sense that they can function 24 h per day and do not tire or fall ill.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    It is difficult to assess the legitimacy of particular applications and their underlying business interests using concepts drawn from universal AI ethics or traditional professional ethics inherited from bioethics.

    “It is difficult to assess the legitimacy of particular applications and their underlying business interests using concepts drawn from universal AI ethics or traditional professional ethics inherited from bioethics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    This leads to 'cognitive load' (Zhou et al. 2017), which affects work memory and, thus, delimits cognitive resources, further increasing the level of distrust and misreading of uncertainties.

    “This leads to ‘cognitive load’ (Zhou et al. 2017), which affects work memory and, thus, delimits cognitive resources, further increasing the level of distrust and misreading of uncertainties.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Zhou et al. 2017 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Insufficient consideration regarding the implementation of chatbots in health care can lead to poor professional practices, creating long-term side effects and harm for professionals and their patients.

    “Insufficient consideration regarding the implementation of chatbots in health care can lead to poor professional practices, creating long-term side effects and harm for professionals and their patients.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The many perceived challenges and risks associated with healthcare chatbots would need to be addressed before the technology is widely endorsed by practicing physicians.

    “The many perceived challenges and risks associated with healthcare chatbots would need to be addressed before the technology is widely endorsed by practicing physicians.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Palanica et al. 2019 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Machines do not have the human capabilities of prudence and practical wisdom or the flexible, interpretive capacity to correct mistakes and wrong decisions.

    “However, machines do not have the human capabilities of prudence and practical wisdom or the flexible, interpretive capacity to correct mistakes and wrong decisions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Pasquale 2020, p. 57 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Chatbots can become entangled with clinical practices in complex ways.

    “We suggest that new ethico-political approaches are required in professional ethics because chatbots can become entangled with clinical practices in complex ways.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    AI technologies, especially ML, have increasingly been occupying other industries; thus, these technologies are arguably naturally adapted to the healthcare sector.

    “AI technologies, especially ML, have increasingly been occupying other industries; thus, these technologies are arguably naturally adapted to the healthcare sector.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    While benefits of chatbots can be broad, whether they outweigh the potential risks to both patients and physicians has yet to be seen.

    “While we acknowledge that the benefits of chatbots can be broad, whether they outweigh the potential risks to both patients and physicians has yet to be seen.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Due to the rapid digital leap caused by the Coronavirus pandemic in health care, there are currently no established ethical principles to evaluate healthcare chatbots.

    “Due to the rapid digital leap caused by the Coronavirus pandemic in health care, there are currently no established ethical principles to evaluate healthcare chatbots.”
    Quote in the paperNot a substantial claim
81 of 81

  • The paper relies on this without support.

    Diagnostic practice requires practical wisdom and collaboration between different specialists as well as close communication with patients.

    “Rapid diagnoses by chatbots can erode diagnostic practice, which requires practical wisdom and collaboration between different specialists as well as close communication with patients.”

  • The paper relies on this without support.

    HCP expertise relies on the intersubjective circulation of knowledge, a pool of dynamic knowledge and intersubjective criticism of data, knowledge and processes.

    “HCP expertise relies on the intersubjective circulation of knowledge, that is, a pool of dynamic knowledge and the intersubjective criticism of data, knowledge and processes.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The integration of AI into healthcare alters traditional decision-making practices based on epistemic probability and prudence and raises ethical, legal, and medical dilemmas when making critical decisions about human lives and medical conditions.

    “Additionally, the integration of AI into healthcare alters traditional decision-making practices based on epistemic probability and prudence (Parviainen, Rantala (2022)) and raises ethical, legal, and medical dilemmas when it comes to making critical decisions about human lives and medical conditions (Shaban-Nejad et al., 2021;Vilaza & McCashin, 2021).”

    From Navigating the perils of artificial intelligence: a focused review on ChatGPT and responsible research and innovation (2024) · cites this paper as Parviainen, Rantala (2022)

Cites

Flagged first, then by citations

1 of the 62 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Pynnönen, T., K. Rantala, and R. Räsänen. 2020. Kokemuksia Omaolo-palvelusta (Experiences with the Omaolo Service). Bachelor’s thesis. Tampere: Tampere University of Applied Sciences.
  2. Topol, E.J. 2019. Deep medicine. New York: Basic Books.

Cited by

Sorted by citations

Related

Authors
Parviainen, J., & Rantala, J.
Affiliations
Tampere University (FI)
Corresponding author
Parviainen, Jaana
ORCID
Venue
Medicine, Health Care and Philosophy
Publisher
Springer Science and Business Media LLC
Published
2021-09-04
Volume
25
Issue
1
Pages
61-71
ISSN
  • 1386-7423 (ISSN-L)
  • 1572-8633
DOI
10.1007/s11019-021-10049-w
PMC
PMC8416570
OpenAlex
W3197603620
Semantic Scholar
0475fb5c81992147f30999c7014340377cb3d4fb
Cited by
146
References
62
Citations by year
2026: 17 · 2025: 32 · 2024: 36 · 2023: 49 · 2022: 12
FWCI
7.07
Type
journal article
Language
en
Open access
Licence
Locations
  • Medicine Health Care and Philosophy (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Tampere University Institutional Repository (Tampere University) (submittedVersion, cc-by)
Funders
Ella ja Georg Ehrnroothin Säätiö
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics and bioethics in healthcare
Keywords
chatbots, artificial intelligence in healthcare, clinical decision-making, rationality, epistemic probability, prudence, COVID-19 pandemic, task-oriented chatbots
MeSH terms
COVID-19, SARS-CoV-2, Artificial Intelligence, Delivery of Health Care, Humans, Referral and Consultation, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review