CoChat
Sign Up
Cited by 103Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsSocial Robot Interaction and HRI

Conversational Agents as Mediating Social Actors in Chronic Disease Management Involving Health Care Professionals, Patients, and Family Members: Multisite Single-Arm Feasibility Study

Open access
Kowatsch, Tobias, Schachner, Theresa, Harperink, Samira, Barata, Filipe, Dittler, Ullrich, Xiao, Grace, Stanger, Catherine, v Wangenheim, Florian +3 more·2021·Journal of Medical Internet Research
DOI 10.2196/25060PMC PMC7929753

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Successful management of chronic diseases requires a trustful collaboration between health care professionals, patients, and family members. Scalable conversational agents, designed to assist health care professionals, may play a significant role in supporting this collaboration in a scalable way by reaching out to the everyday lives of patients and their family members. However, to date, it remains unclear whether conversational agents, in such a role, would be accepted and whether they can support this multistakeholder collaboration. Objective: With asthma in children representing a relevant target of chronic disease management, this study had the following objectives: (1) to describe the design of MAX, a conversational agent–delivered asthma intervention that supports health care professionals targeting child-parent teams in their everyday lives; and (2) to assess the (a) reach of MAX, (b) conversational agent–patient working alliance, (c) acceptance of MAX, (d) intervention completion rate, (e) cognitive and behavioral outcomes, and (f) human effort and responsiveness of health care professionals in primary and secondary care settings. Methods: MAX was designed to increase cognitive skills (ie, knowledge about asthma) and behavioral skills (ie, inhalation technique) in 10-15-year-olds with asthma, and enables support by a health professional and a family member. To this end, three design goals guided the development: (1) to build a conversational agent–patient working alliance; (2) to offer hybrid (human- and conversational agent–supported) ubiquitous coaching; and (3) to provide an intervention with high experiential value. An interdisciplinary team of computer scientists, asthma experts, and young patients with their parents developed the intervention collaboratively. The conversational agent communicates with health care professionals via email, with patients via a mobile chat app, and with a family member via SMS text messaging. A single-arm feasibility study in primary and secondary care settings was performed to assess MAX. Results: Results indicated an overall positive evaluation of MAX with respect to its reach (49.5%, 49/99 of recruited and eligible patient-family member teams participated), a strong patient-conversational agent working alliance, and high acceptance by all relevant stakeholders. Moreover, MAX led to improved cognitive and behavioral skills and an intervention completion rate of 75.5%. Family members supported the patients in 269 out of 275 (97.8%) coaching sessions. Most of the conversational turns (99.5%) were conducted between patients and the conversational agent as opposed to between patients and health care professionals, thus indicating the scalability of MAX. In addition, it took health care professionals less than 4 minutes to assess the inhalation technique and 3 days to deliver related feedback to the patients. Several suggestions for improvement were made. Conclusions: This study provides the first evidence that conversational agents, designed as mediating social actors involving health care professionals, patients, and family members, are not only accepted in such a “team player” role but also show potential to improve health-relevant outcomes in chronic disease management.

AI snapshot

Written from the paper’s full text.

TL;DR

In a multisite single-arm feasibility study, MAX, a conversational agent designed as a mediating social actor for pediatric asthma, was feasible and positively evaluated across patients, family members, and health care professionals. It showed high reach, strong patient-agent working alliance, 75.5% completion, improved asthma knowledge, and scalable delivery with most conversational turns handled by the agent.

Key findings

  • Reach was 49.5% (49/99 recruited and eligible patient-family member teams started interacting with MAX); intervention completion was 75.5% (37/49).
  • Patient-MAX working alliance was high across coaching sessions 2, 8, and 14 (means 6.34, 6.14, and 6.34, respectively), and acceptance ratings were above the neutral scale value of 4 (perceived usefulness 6.42, ease of use 6.75, enjoyment 6.47, control 5.53, intention to continue 5.58).
  • Asthma knowledge increased significantly from pretest to posttest (7.73 to 8.79 with baseline observation carried forward, n=48, P<.001, d=0.91; 9.43 in complete cases, n=37, P<.001, d=1.19).
  • Family members supported patients in 269/275 (97.8%) coaching sessions; 99.5% (15,078/15,152) of conversational turns were between patients and MAX versus 0.5% (74/15,152) between patients and health care professionals. Health care professionals took an average of 221 seconds to assess inhalation videos and 2.34 days to provide feedback.

Methods

  • Single-arm feasibility study conducted in two home care settings and four secondary care settings in the German-speaking part of Switzerland; recruitment occurred January-April 2019.
  • Participants were 10- to 15-year-old German-speaking patients with asthma who had smartphone access and a German-speaking family member able to receive SMS messages and support the patient every second intervention day; 99 patients were approached and 49 started interacting with MAX.
  • Outcomes included reach, patient-conversational agent working alliance, acceptance, intervention completion, cognitive and behavioral skills, and human effort/responsiveness; analyses included descriptive statistics, paired-sample t tests for asthma knowledge, and qualitative feedback.

Limitations

  • The paper states that this was a feasibility study with a limited sample, so results are not representative and must be interpreted with caution; only health care professionals from four hospitals and two cantonal patient organizations participated, leaving unclear whether MAX would work similarly with general practitioners.
  • The paper states that qualitative open coding was performed by one author, social support was self-reported and incentivized and thus likely overestimated, and the MAX interface was not integrated into hospital or patient-organization information systems, likely overestimating efforts; only costs and efforts were reported, so no cost-effectiveness implications can be drawn.
  • Evident from the text: the single-arm design without a control group limits causal conclusions about intervention effects, and the small, exploratory sample constrains generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/25060

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 97 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Most conversational turns (99.5%) were conducted between patients and the conversational agent as opposed to between patients and health care professionals.

    “Most of the conversational turns (99.5%) were conducted between patients and the conversational agent as opposed to between patients and health care professionals”
    Quote in the paperSubstantial claim
  • Findingown data

    The intervention completion rate was 75.5%.

    “an intervention completion rate of 75.5%”
    Quote in the paperNot a substantial claim
  • Findingown data

    Family members supported the patients in 269 out of 275 (97.8%) coaching sessions.

    “Family members supported the patients in 269 out of 275 (97.8%) coaching sessions.”
    Quote in the paperSubstantial claim
  • Findingown data

    It took health care professionals less than 4 minutes to assess the inhalation technique.

    “it took health care professionals less than 4 minutes to assess the inhalation technique”
    Quote in the paperNot a substantial claim
  • Findingown data

    MAX achieved a strong patient-conversational agent working alliance.

    “a strong patient-conversational agent working alliance”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 32
  • May combine two claims. 29

  • The quote only partly states this claim.

    Findingown data

    MAX led to improved behavioral skills (inhalation technique) in patients.

    “MAX led to improved cognitive and behavioral skills”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    It took health care professionals 3 days to deliver related feedback to the patients.

    “and 3 days to deliver related feedback to the patients”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    MAX led to improved cognitive skills (knowledge about asthma) in patients.

    “MAX led to improved cognitive and behavioral skills”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants who finished the intervention (n=37) did so on average within 3 weeks, which was within the incentivized duration of 4 weeks.

    “Participants who finished the intervention (n=37) did so on average within 3 weeks, which was within the incentivized duration of 4 weeks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All mean values of technology acceptance perceptions lie clearly above the neutral scale value of 4, indicating positive evaluations of the mobile app.

    “All mean values lie clearly above the neutral scale value of 4, indicating positive evaluations of the mobile app.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The single-item measures used to assess technology acceptance are sufficient to validly measure the constructs.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    For two video clips, three serious inhalation mistakes were identified, eliciting a feedback to resend a corrected video clip; after resubmission, no severe inhalation mistakes could be identified in the second video clip.

    “For two video clips, three serious inhalation mistakes were identified, eliciting a feedback to resend a corrected video clip. After resubmission, no severe inhalation mistakes could be identified in the second video clip.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    99.5% (15,078/15,152) of conversational turns took place between patients and the MAX conversational agent, and only 0.5% (74/15,152) occurred between patients and health care professionals.

    “99.5% (15,078/15,152) took place between patients and the MAX conversational agent, and only 0.5% (74/15,152) occurred between patients and health care professionals (Table 1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The intervention completion rate was 75.5%, as 37 out of the 49 patients finished the intervention.

    “The intervention completion rate was 75.5%, as 37 out the 49 patients finished the intervention.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Patients learned new aspects about asthma management and enjoyed the coaching sessions.

    “Moreover, patients learned new aspects about asthma management and enjoyed the coaching sessions.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The single-item measures used to assess technology acceptance are sufficient to validly measure the constructs.

  • May combine two claims.

    Findingown data

    It took health care professionals less than 4 minutes to assess the inhalation technique and 3 days to deliver that feedback to the patients.

    “In addition, it took them less than 4 minutes to assess the inhalation technique and 3 days to deliver that feedback to the patients.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Overall, 42 inhalation video clips were recorded and submitted to the health care professionals.

    “Overall, 42 inhalation video clips were recorded and submitted to the health care professionals (Table 2).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    It took health care professionals an average of 221 seconds to assess the video clips, with a clear difference observed between health care settings (average of 410 seconds in the primary care setting and 127 seconds in the secondary care setting).

    “it took health care professionals an average of 221 seconds to assess the videos clips, with a clear difference observed between health care settings (average of 410 seconds in the primary care setting and 127 seconds in the secondary care setting; see Table 2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    For the gender-specific choices of the MAX conversational agent, all participants chose the character corresponding to their own indicated gender.

    “For the gender-specific choices of the MAX conversational agent, all participants chose the character corresponding to their own indicated gender.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Reach of the MAX intervention was 49.5%, with 49 out of 99 approached patients downloading the app and starting to interact with the MAX conversational agent.

    “Reach was 49.5% with 49 out of 99 approached patients downloading the app and starting to interact with the MAX conversational agent.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    For each patient, this intervention involved an average of 20 minutes of human effort, 10 automated SMS text messages, including three reminders, and additional costs for gift vouchers, including lottery winnings.

    “For each patient, this intervention involved an average of 20 minutes of human effort, 10 automated SMS text messages, including three reminders, and additional costs for gift vouchers, including lottery winnings.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The text-based conversational agent was perceived as positive and engaging.

    “the text-based conversational agent was perceived as positive and engaging”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Conversational agents have shown promising results with respect to therapeutic alliance.

    “therapeutic alliance [16,17]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [16] source

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Chronic conditions present a significant risk to the global population and cause substantial financial and health-related burdens, resulting in low quality of life of those affected.

    “Chronic conditions present a significant risk to the global population, and cause substantial financial and health-related burdens, resulting in low quality of life of those affected [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Successful disease management often requires a trustful collaboration among health care professionals, patients, and their families.

    “Successful disease management often requires a trustful collaboration among health care professionals, patients, and their families [3].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several recent studies have provided evidence supporting the patient benefits of digital interventions, particularly in children and adolescents.

    “Indeed, several recent studies have provided evidence supporting the patient benefits of such digital interventions, particularly in children and adolescents [10-14].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [10] source

      Not checked in the free run.

    • cites [11] source

      Not checked in the free run.

    • cites [12] source

      Not checked in the free run.

    • cites [13] source

      Not checked in the free run.

    • cites [14] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Conversational agents have shown promising results with respect to patient satisfaction.

    “conversational agents (ie, computer programs that imitate interaction with human beings) have shown promising results with respect to patient satisfaction [15]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Asthma is associated with high financial and health costs, with total annual asthma costs in the United States estimated at US $56 billion in 2011.

    “Asthma is associated with high financial and health costs, with total annual asthma costs in the United States estimated at US $56 billion in 2011 [23].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    MobileCoach has already been used successfully for various clinical and public health interventions.

    “The intervention was developed with the open-source software platform MobileCoach [67,83,84], which has already been used successfully for various clinical and public health interventions [17,68,77-79,85,86]”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [67] source

      Not checked in the free run.

    • cites [83] source

      Not checked in the free run.

    • cites [84] MobileCoach2021-02-06www.mobile-coach.eu

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

    • cites [68] source

      Not checked in the free run.

    • cites [77] source

      Not checked in the free run.

    • cites [78] source

      Not checked in the free run.

    • cites [79] source

      Not checked in the free run.

    • cites [85] source

      Not checked in the free run.

    • cites [86] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Asthma affects approximately 235 million people and is one of the most common chronic diseases worldwide.

    “Affecting approximately 235 million people, asthma is one of the most common chronic diseases worldwide [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] Asthma Fact SheetWorld Health Organization2020-04-13https://www.who.int/en/news-room/fact-sheets/detail/asthma

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Conversational agents have shown promising results with respect to treatment success.

    “and treatment success [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Poor technique during medication inhalation leads to reduced dispersion of the drug in the lungs, and subsequent decreased asthma stability and lowered clinical effectiveness of the delivered drug.

    “Another common concern is poor technique during medication inhalation, leading to reduced dispersion of the drug in the lungs, and subsequent decreased asthma stability and lowered clinical effectiveness of the delivered drug [41-44].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [41] source

      Not checked in the free run.

    • cites [42] source

      Not checked in the free run.

    • cites [43] source

      Not checked in the free run.

    • cites [44] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Compared to very similar conversational agent research targeting childhood obesity, physical inactivity, or the management of chronic pain, this intervention resulted in a high overall therapeutic goal achievement rate (75.5%) but also in improved asthma knowledge test scores and behavioral skills.

    “Compared to very similar conversational agent research targeting childhood obesity [68], physical inactivity [77], or the management of chronic pain [17], this intervention resulted in a high overall therapeutic goal achievement rate (75.5%) but also in improved asthma knowledge test scores and behavioral skills (ie, no identified inhalation mistakes occurred after the feedback from health care”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [68] source

      Not checked in the free run.

    • cites [77] source

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Digital health interventions in the form of mobile apps can be particularly effective for children as they provide an attractive channel for education and management through the possible integration of multimedia content such as audio or video.

    “Digital health interventions in the form of mobile apps can be particularly effective for children as they provide an attractive channel for education and management through the possible integration of multimedia content such as audio or video [19].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The results of this first feasibility study indicate an overall positive evaluation with respect to the reach of the intervention, the strong working alliance between patients and the MAX conversational agent, and high acceptance of the intervention by all relevant stakeholders.

    “The results of this first feasibility study indicate an overall positive evaluation with respect to the reach of the intervention (ie, 49.5% of 99 young patients approached did install the app and started to interact with the MAX conversational agent), the strong working alliance between patients and the MAX conversational agent, and high acceptance of the intervention by all relevant stakeholders”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The single-item measures used to assess technology acceptance are sufficient to validly measure the constructs.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Knowledge about asthma was assessed by a validated health literacy quiz for children with asthma, with a quiz score ranging from 0 to 11.

    “Knowledge about asthma (ie, cognitive skill) was assessed at the beginning of the intervention (ie, at the end of the introductory chat with the conversational agent MAX) and in the last session by a validated health literacy quiz for children with asthma, with a quiz score ranging from 0 (no knowledge) to 11 (good knowledge) [80,94] (see Multimedia Appendix 19).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [80] SchnyderMASantschiRAuf Spurensuche im FerienlagerEin Lerncomic über Asthma20122020-04-13https://www.lungenliga.ch/uploads/tx_pubshop/Lerncomic_Spurensuche.pdf

      Not checked in the free run.

    • cites [94] Schnyder, M, 2014, Begleitbericht Zur Entwicklung eines Lernmittels für Asthmakinder im Alter von 7 bis 12 Jahren

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

    Premisecitation

    Conversational agents can be influential and engaging for young patients.

    “It was previously suggested that conversational agents can be influential and engaging for young patients, and that open-text answers are highly appreciated [104].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Acceptance of the intervention was assessed with instruments adapted from information systems research.

    “perceived usefulness (“The app helped me to increase my knowledge about my asthma”), ease of use (“The app was easy to use”), enjoyment (“I found the app enjoyable”), control (“I could control many aspects of the app”), and usage intention (“How much would you like to continue working with MAX?”) were assessed by patients at the end of the intervention with instruments adapted from information”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [50] source

      Not checked in the free run.

    • cites [93] source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    The study minimized the risk for smartphone addiction by limiting the amount of possible sessions to one per day and further including active exercises outside the digital environment of the app to increase social interaction and to counteract increasing smartphone usage among children.

    “We minimized the risk for smartphone addiction [76] by limiting the amount of possible sessions to one per day and further including active exercises outside the digital environment of the app to increase social interaction and to counteract increasing smartphone usage among children [103].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [76] source

      Not checked in the free run.

    • cites [103] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study was approved by the institutional review board of ETH Zurich (reference number: EK 2018-N-59).

    “The study was approved by the institutional review board of ETH Zurich (reference number: EK 2018-N-59).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Digital health interventions for asthma include numerous mobile health apps that provide patients with information and help them track symptoms or medications, often using a gamification component.

    “Digital health interventions for asthma include numerous mobile health apps that provide patients with information and help them track symptoms or medications, often using a gamification component [45-47].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [45] source

      Not checked in the free run.

    • cites [46] source

      Not checked in the free run.

    • cites [47] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    General ongoing costs include keeping the intervention content updated according to recent asthma management guidelines at a cost of approximately US $10,000 every 3 years.

    “General ongoing costs include keeping the intervention content updated according to recent asthma management guidelines (at a cost of approximately US $10,000 every 3 years) and maintaining technical software (at a cost of approximately US $10,000 per year).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The MAX project team's cost estimates are accurate.

  • May combine two claims.

    Study design and conductreasoning

    The study was exploratory by nature and did not include a detailed power analysis to determine a particular sample size.

    “Therefore, the study was exploratory by nature and did not include a detailed power analysis to determine a particular sample size.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Privacy issues with conversational agents and open-text answers were pointed out by prior work.

    “However, privacy issues with conversational agents and open-text answers were pointed out by prior work [105], as conversational agents that are responsive to such inputs could potentially and unintentionally retrieve more and more personal information.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [105] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Open-text answers are highly appreciated.

    “It was previously suggested that conversational agents can be influential and engaging for young patients, and that open-text answers are highly appreciated [104].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Some data had to be stored in a redundant way in the MAX system, which likely resulted in an overestimation of efforts.

    “Specifically, some data had to be stored in a redundant way (eg, contact number, patient name) in the MAX system, which likely resulted in an overestimation of efforts (eg, the duration of the onboarding process).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Based on the limited sample, the results are not representative and must be interpreted with caution.

    “Based on our limited sample, it is clear that the results are not representative and must be interpreted with caution.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    It is not clear whether and to what degree the MAX intervention would work the same way with other relevant health care professionals such as a general practitioner.

    “Therefore, it is not clear whether and to what degree the MAX intervention would work the same way with other relevant health care professionals such as a general practitioner.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Estimates of the MAX project team indicate that the development of the current MAX intervention into a "product" would cost another US $100,000.

    “In case none of these additional intervention components or studies is considered, estimates of the MAX project team indicate that the development of the current MAX intervention into a “product” would cost another US $100,000.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The MAX project team's cost estimates are accurate.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nocturnal cough in adult asthma patients can be detected reliably with the microphone of a smartphone.

    “For example, there is evidence that the number of nocturnal cough events is negatively correlated with asthma control [106,107], and that nocturnal cough in adult asthma patients can be detected reliably with the microphone of a smartphone [89,108].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [89] source

      Not checked in the free run.

    • cites [108] BarataFKipferKWeberMTinschertPFleischEKowatschTTowards device-agnostic mobile cough detection with convolutional neural networks2019IEEE International Conference on Healthcare Informatics (ICHI)2019China111 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Conversational agents can take over the role of a mediating social actor in a complex health care setting with various stakeholders, and deliver a digital health intervention in a scalable way into the everyday life of patients and their family members.

    “We have demonstrated that conversational agents can take over the role of a mediating social actor in a complex health care setting with various stakeholders, and deliver a digital health intervention in a scalable way into the everyday life of patients and their family members.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The MAX intervention can be delivered in a scalable way into the everyday life of patients and their family members.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    We have only reported the costs and efforts related to the MAX intervention, and therefore no implications regarding cost-effectiveness can be discussed.

    “Finally, we have only reported the costs and efforts related to the MAX intervention, and therefore no implications regarding cost-effectiveness can be discussed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Working alliance between the patient and the conversational agent MAX was assessed with a German-adapted version of the Session Alliance Inventory after coaching sessions 2, 8, and 14.

    “Working alliance between the patient and the conversational agent MAX was assessed with a German-adapted version of the Session Alliance Inventory [92] after coaching sessions 2, 8, and 14 (eg, “MAX and I respect each other” with answers anchored on a 7-point Likert scale ranging from 1 [“never”] to 8 [“always”]; see Multimedia Appendix 18).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [92] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The inhalation technique of each patient was systematically assessed by the patient's responsible health care professional with the help of predefined evaluation criteria developed by health care professionals of the Swiss Lung Association and the participating pediatric pneumologists.

    “The inhalation technique of each patient (ie, behavioral skill) was systematically assessed by the patient’s responsible health care professional with the help of predefined evaluation criteria These criteria were developed by health care professionals of the Swiss Lung Association and the participating pediatric pneumologists.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The predefined evaluation criteria used by health care professionals validly assess inhalation technique.
    • Rests on an assumption: The health care professionals' assessments of inhalation technique are reliable.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There is evidence that the number of nocturnal cough events is negatively correlated with asthma control.

    “For example, there is evidence that the number of nocturnal cough events is negatively correlated with asthma control [106,107], and that nocturnal cough in adult asthma patients can be detected reliably with the microphone of a smartphone [89,108].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [106] source

      Not checked in the free run.

    • cites [107] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Conversational agents that are responsive to open-text inputs could potentially and unintentionally retrieve more and more personal information.

    “However, privacy issues with conversational agents and open-text answers were pointed out by prior work [105], as conversational agents that are responsive to such inputs could potentially and unintentionally retrieve more and more personal information.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [105] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The MAX intervention extended the reach of health care professionals into the everyday lives of patients in an efficient way without compromising the quality of care and working alliance.

    “With the MAX conversational agent, we were also able to show how to extend the reach of health care professionals into the everyday lives of patients in an efficient way without compromising the quality of care and working alliance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The MAX conversational agent uses several communication channels (eg, in-app chat, email, and SMS), and therefore "lives" not only on a smartphone in the pocket of a patient but is rather omnipresent.

    “For this purpose, the MAX conversational agent uses several communication channels (eg, in-app chat, email, and SMS), and therefore “lives” not only on a smartphone in the pocket of a patient but is rather omnipresent (ie, MAX appears on the phones of family members, such as via SMS, or on desktop or tablet computers of health care professionals, such as via emails and the web-based MAX interface”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Maintaining technical software costs approximately US $10,000 per year.

    “General ongoing costs include keeping the intervention content updated according to recent asthma management guidelines (at a cost of approximately US $10,000 every 3 years) and maintaining technical software (at a cost of approximately US $10,000 per year).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The MAX project team's cost estimates are accurate.

  • May combine two claims.

    Study design and conductnone given

    This study was designed as a feasibility study with a limited number of participants.

    “This study was designed as a feasibility study with a limited number of participants.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Nonspecialized health care experts may require significantly more time for the assessment of the inhalation video clips or would not have the expertise to do so without additional educational efforts.

    “These nonspecialized health care experts may require significantly more time for the assessment of the inhalation video clips or would not have the expertise to do so without additional educational efforts.”
    Quote in the paperNot a substantial claim
55 of 55

  • The paper relies on this without support.

    Children often face problems with technological solutions when they are not integrated into existing health care systems and do not allow for explicit support by health care professionals or family members.

    “However, and in addition to shortcomings of asthma management related to cognitive and behavioral skills, children often face problems with such technological solutions when they are not integrated into existing health care systems and do not allow for explicit support by health care professionals or family members.”

  • The paper relies on this without support.

    Without a dedicated party or mediator, it often becomes a challenge to integrate all relevant stakeholders (health care professionals, family members, and patients) into the disease management process.

    “Without a dedicated party or mediator, it often becomes a challenge to integrate all of these relevant stakeholders (ie, health care professionals, family members, and the patients themselves) into the disease management process.”

  • The paper relies on this without support.

    Conversational agents can act as mediating social actors, taking over a significant amount of intervention delivery in a scalable manner and coordinating communication among health care professionals, family members, and patients if required.

    “Conversational agents, as part of such interventions, can act as mediating social actors; that is, they take over not only a significant amount of the intervention delivery in a scalable manner but also coordinate the communication among health care professionals, family members, and patients if required.”

  • The authors state this as an assumption.

    The four distinct intervention components enabled an experiential learning cycle and were assumed to influence the outcomes of the intervention.

    “Four distinct intervention components as depicted in Figure 1 enabled an experiential learning cycle [66], and were assumed to influence the outcomes of the intervention”

  • The authors state this as an assumption.

    The need to self-regulate one’s experiences and actions (as an important predictor of engagement) is also true for digital interventions.

    “Assuming that the need to self-regulate one’s experiences and actions (as an important predictor of engagement [70], as posited by self-determination theory) is also true for digital interventions”

  • The authors state this as an assumption.

    There is a positive/negative reinforcement link that connects the outcomes of the conceptual model with the perceived characteristics of the app and working alliance with the conversational agent.

    “Finally, we assume that there is a positive/negative reinforcement link that connects the outcomes of the conceptual model with the perceived characteristics of the app and working alliance with the conversational agent.”

  • The paper relies on this without support.

    Specific cognitive skills required for asthma management include knowledge about asthma triggers and the importance of medication inhalation adherence, as well as behavioral skills such as the application of correct inhalation techniques.

    “For asthma, specific cognitive skills required for disease management include knowledge about asthma triggers and the importance of medication inhalation adherence, as well as behavioral skills such as the application of correct inhalation techniques.”

  • The paper relies on this without support.

    The predefined evaluation criteria used by health care professionals validly assess inhalation technique.

    “These criteria were developed by health care professionals of the Swiss Lung Association and the participating pediatric pneumologists.”

  • The paper relies on this without support.

    The health care professionals' assessments of inhalation technique are reliable.

    “The inhalation technique of each patient (ie, behavioral skill) was systematically assessed by the patient’s responsible health care professional with the help of predefined evaluation criteria”

  • The paper relies on this without support.

    The single-item measures used to assess technology acceptance are sufficient to validly measure the constructs.

    “Single-item measures were used to reduce the burden of the intervention and answers were anchored on 7-point Likert scales ranging from “strongly disagree” (1) to “strongly agree” (7).”

  • The paper relies on this without support.

    The observed improvements in cognitive and behavioral skills are attributable to the MAX intervention rather than other factors.

    “We have shown that conversational agents framed as digital assistants of health care professionals have the potential to improve cognitive and behavioral skills in chronic disease management, with asthma in children as one specific example.”

  • The paper relies on this without support.

    The MAX intervention can be delivered in a scalable way into the everyday life of patients and their family members.

    “We have demonstrated that conversational agents can take over the role of a mediating social actor in a complex health care setting with various stakeholders, and deliver a digital health intervention in a scalable way into the everyday life of patients and their family members.”

  • The paper relies on this without support.

    The MAX project team's cost estimates are accurate.

    “In case none of these additional intervention components or studies is considered, estimates of the MAX project team indicate that the development of the current MAX intervention into a “product” would cost another US $100,000.”
13 of 13

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The same participant count is assigned two different percentages

    Minor

    Table 1 labels 12 nonfinishers as 25% in one row and 24% in another, although both rows use the same total of 49 participants.

  • Working-alliance response scale has conflicting endpoints

    Minor

    A scale with integer responses from 1 through 8 has eight points, not seven.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. 10.1043/1543-2165(2006)130[447:apap]2.0.co;2 · DOI
  2. Collins, LM, 2018, Optimization of Behavioral, Biobehavioral, and Biomedical Interventions
  3. Kolb, DA, 1984, Experience as the source of learning and development
  4. BoatengGSelsLKuppensPLüscherJScholzUKowatschTEmotion elicitation and capture among real couples in the lab2020ACM CHI Conference on Human Factors in Computing Systems2020Honolulu (cancelled)New York, NYAssociation for Computing Machinery
  5. NielsenJLandauerTKModel of the finding of usability problems1993ACM Conference on Human Factors in Computing Systems (CHI)1993Netherlands206213
  6. Schnyder, M, 2014, Begleitbericht Zur Entwicklung eines Lernmittels für Asthmakinder im Alter von 7 bis 12 Jahren
  7. 10.1145/3290607.3312853 · DOI · Paper page
  8. 10.2307/25148660 · DOI · Paper page
  9. HeilbronFCEscorciaVGhanemBNieblesJCActivityNet: A large-scale video benchmark for human activity understanding20152015 IEEE Conference on Computer Vision and Pattern RecognitionJune 7-12, 2015Boston, MA961970

Cited by

Sorted by citations

Related

Authors
Kowatsch, T., Schachner, T., Harperink, S., Barata, F., Dittler, U., Xiao, G., Stanger, C., v Wangenheim, F., Fleisch, E., Oswald, H., & Möller, A.
Affiliations
  • University of St.Gallen (CH)
  • ETH Zurich (CH)
  • Furtwangen University (DE)
  • Johns Hopkins University (US)
  • Dartmouth College (US)
  • Kantonsspital Winterthur (CH)
Corresponding author
Kowatsch, Tobias
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2021-02-17
Volume
23
Issue
2
Pages
e25060
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/25060
PMC
PMC7929753
OpenAlex
W3122673691
Semantic Scholar
85bb70a2f7f6a60e2b1aba483a3e298c4e2f1dd1
Cited by
103
References
102
Citations by year
2026: 15 · 2025: 11 · 2024: 23 · 2023: 20 · 2022: 29 · 2021: 5
FWCI
21.63
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • Repository of Futwangen University of Applied Sciences (Furtwangen University) (submittedVersion, cc-by)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • Alexandria (UniSG) (University of St.Gallen) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
Funders
  • National Institute on Drug Abuse (P30DA029926)
  • National Institute of Diabetes and Digestive and Kidney Diseases (R01DK124428)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
chatbots, chronic disease management, childhood asthma, asthma self-management, therapeutic relationship, healthcare workers, inhaler technique, text messaging, intervention acceptance, feasibility, primary and secondary care, patient-centered design, family involvement in care, patient knowledge and skills
MeSH terms
Chronic Disease, Communication, Family, Feasibility Studies, Female, Health Personnel, Humans, Male, Patients

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review