The DOI matches its Crossref record; 27 of the 29 references it cites were found in a registry or an index; 4 of the 27 sources its claims cite were checked for support.
A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing
Open accessPublished version of a preprint
Abstract
BACKGROUND: Individuals living with long-term physical health conditions frequently experience co-occurring mental health problems. This comorbidity has a significant impact on an individual's levels of emotional distress, health outcomes, and associated health care utilization. As health care services struggle to meet demand and care increasingly moves to the community, digital tools are being promoted to support patients to self-manage their health. One such technology is the autonomous virtual agent (chatbot, conversational agent), which uses artificial intelligence (AI) to process the user's written or spoken natural language and then to select or construct the corresponding appropriate responses. OBJECTIVE: This study aimed to co-design the content, functionality, and interface modalities of an autonomous virtual agent to support self-management for patients with an exemplar long-term condition (LTC; chronic pulmonary obstructive disease [COPD]) and then to assess the acceptability and system content. METHODS: We conducted 2 co-design workshops and a proof-of-concept implementation of an autonomous virtual agent with natural language processing capabilities. This implementation formed the basis for video-based scenario testing of acceptability with adults with a diagnosis of COPD and health professionals involved in their care. RESULTS: Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios for which they would like to receive support: at the time of diagnosis (information provision), during acute exacerbations (crisis support), during periods of low mood (emotional support), and for general self-management (motivation). From the scenario testing, 12 additional adults with COPD felt the system to be both acceptable and engaging, particularly with regard to internet-of-things capabilities. They felt the system would be particularly useful for individuals living alone. CONCLUSIONS: Patients did not explicitly separate mental and physical health needs, although the content they developed for the virtual agent had a clear psychological approach. Supported self-management delivered via an autonomous virtual agent was acceptable to the participants. A co-design process has allowed the research team to identify key design principles, content, and functionality to underpin an autonomous agent for delivering self-management support to older adults living with COPD and potentially other LTCs.
Overview
CoChat Summary
TL;DR
This study co-designed and tested the acceptability of Avachat, an autonomous virtual agent intended to support self-management for people with COPD and comorbid mental health needs. Participants identified four priority support scenarios and generally found the prototype acceptable and engaging, especially its multimodal and internet-of-things possibilities.
Key findings
- Co-design participants prioritized four self-management scenarios: at diagnosis (information provision), during acute exacerbations (crisis support), during periods of low mood (emotional support), and for general self-management (motivation).
- In video-based scenario testing, 8 patients/carers viewed the film; the System Usability Scale median was 73.75, with 5/8 scores above 68 (72.5, 75, 75, 80, 85) and 3/8 below (52.2, 62.5, 65).
- 4/8 participants strongly agreed they would like to use the system frequently, 7/8 strongly agreed it was easy to use, and all but 1 strongly agreed they would feel confident using it.
- Participants did not explicitly separate mental and physical health needs, but the content they developed had a clear psychological approach; they valued multimodal access, internet-of-things capabilities, and thought the system would be particularly useful for individuals living alone.
Methods
- Two co-design workshops were held with adults with COPD/carers and health professionals, using day-in-the-life mapping, missing-sentence prompts, mood boards, vignettes, role play, and script development; data were analyzed with content analysis and thematic analysis.
- A semiautonomous Avachat prototype was developed with spoken and written interaction, then filmed in a Wizard of Oz setup as a short video (<8 min) showing two scenarios: crisis support and motivational support.
- Acceptability testing screened the video with 8 patients/carers, followed by discussion around design, functionality, and content, plus self-completed System Usability Scale questionnaires.
Limitations
- Stated: the sample was small, white, British, medically stable, and regional; no participants reported a current mental health problem, though all referred to past low mood or worry; recruitment through support groups likely selected people already experienced in self-management.
- Stated: health practitioners contributed relatively little compared with patients, so it is unclear whether they endorsed the final scenarios as clinical priorities or best practice; the project was not a software development project, limiting development effort.
- Stated: possible negative consequences, incorrect advice, and wider ethical implications were not discussed; evident from the text: the abstract reports 12 additional adults with COPD in scenario testing, while the Results section reports 8 patients/carers screened, and the prototype was Wizard-of-Oz rather than fully autonomous, so acceptability of the actual autonomous system was not directly tested.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
2 Valid · 1 Not valid · 9 Not checked · 83 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“A blended approach to care has been shown to be most effective and acceptable for patients when managing both physical and emotional health [19];”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Disputedcites [19] sourceExperimental — accuracy not yet measured
The Discussion (Conclusions and policy implications; page unavailable) says that supported computerised CBT added to usual GP care offered modest or no benefit for depression, contrary to the claim that a blended approach was shown to be most effective. The paper also reports limited uptake despite telephone support; it does not establish patient acceptability or effectiveness for managing both physical and emotional health. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Could not verifycites [37] sourceExperimental — accuracy not yet measured
The Results (Description of included studies; Evaluation measures) describe studies of conversational agents, including embodied agents, but do not report whether their samples included older adults. Searches for older adults, participant age, and comorbidities found no passage addressing the claimed age-specific testing gap or the parenthetical about comorbidities. (LLM, conf=0.70)
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The Results (Autism Spectrum Disorders; page unavailable) say that over half of the 49 reviewed studies targeted autism and that only three autism applications focused on adults, supporting the point that much testing involved younger people. The review does not quantify testing in older adults or address the parenthetical claim about where most comorbidities are evident. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Taken together, this suggests that the potential exists for the development of a relationship of sorts between human and autonomous agents, a suggestion supported by a recent study investigating the use of text-based AI to deliver psychological therapy [42].”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Supportedcites [42] source
The Discussion ('Main findings', this volume) states that 'many of the participants perceived the app as somewhat a living character, with comments that referred to the app as "a (real) person" and the interaction as a "relationship", indicating an ability in the chatbot to mimic human guidance', and the Results ('Qualitative data') report participants talking about the depth of 'our relationship' with Shim (Participants 5, 7, 8). The cited work is indeed the recent study the sentence identifies: a pilot RCT of a text-based, fully automated conversational agent (Shim) delivering positive-psychology and third-wave CBT strategies with therapeutic-process features such as empathic responses and accountability (Introduction; Methods, 'The intervention'). The paper additionally notes prior evidence of a bond with bots (Bickmore et al., 2005; Fitzpatrick et al., 2017), which reinforces rather than displaces its own finding. (LLM, conf=0.85)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although this development process resulted in a system that was functional, in practice, unless the user conformed quite closely to one of the developed scripts, a significant amount of input was required of the wizard.”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 62% (5/8) of the SUS scores were above the score of 68 (with scores of 72.5, 75, 75, 80, and 85), with the other 38% (3/8) participants rating the system as follows: 52.2, 62.5, and 65.”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
33 more claims
Create a free account to see all 95 claims and to verify the 23 references still open.
- Probably not a substantial claim. 47
- It may state two separate claims as one. 23
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Four studies found high levels of engagement with the conversational agent or reported the conversational agent to be engaging [50,52,53,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“In 7 studies, user experience was assessed. User experience was generally positive regarding the acceptability, understanding of the conversational agents, comprehensibility of the systems’ responses, interaction rates, or content relevance [51,53,54,56-59].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Disease-specific patients were involved in 3 studies [50,54,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“A total of 4 studies developed the artificial intelligence system internally [50,51,57,59], and 5 studies relied on external sources [52-54,56,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Four studies were classified as AI [53,56-58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“From the scenario testing, 12 additional adults with COPD felt the system to be both acceptable and engaging”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
“We then screened this video with 8 patients who had a diagnosis of COPD or were a carer of a spouse with COPD: 5 males and 3 females. Overall, 3 of these participants had been part of the co-design process, 5 were novel and new to the concept of Avachat.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“A total of 6 health professionals attended, all female, working across general practice, secondary care respiratory services, and specialist mental health care.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“Overall, 5 individuals with a diagnosis of COPD and a carer of an individual with COPD agreed to attend (n=6), of whom 5 attended”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
“5 patients who attended the first workshop also signed up for this second workshop. Of these, 4 attended (male n=2 and female n=2), with 1 drop out because of ill health (male). The median age of the participants was 73 years (range 66 to 80 years)”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“Participants had a median age of 82.5 years (range 69 to 86 years)”
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Weber, R. (1990) · doi:10.4135/9781412983488 · Cited by 7,750
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Depression, chronic diseases, and decrements in health: results from the World Health Surveys
Moussavi, S., Chatterji, S., Verdes, E., Tandon, A., Patel, V., & Ustun, B. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61415-9 · Cited by 4,070 · cited 1×
“These mental health conditions are distressing, have a clinically significant impact on health-related quality of life, and can lead to further health complications and greater consumption of costly health care [1,3,7].”
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“In recent years, several such autonomous virtual agents have been developed to support the delivery of psychological therapies, incorporating automated responses to text-based chat [23,24].”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 2×
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
Katon, W. J. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(03)00273-7 · Cited by 1,223
Bower, P., & Gilbody, S. (2005) · British Journal of Psychiatry · doi:10.1192/bjp.186.1.11 · Cited by 1,084
Embodied conversational agents in clinical psychology: a scoping review
Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6553 · Cited by 433 · cited 3×
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
Gilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409 · cited 2×
“…as engaging, as acceptable by patients, and as clinically effective as traditional face-to-face delivery of care, a blended or guided approach, which includes aspects of face-to-face or telephone support, is recommended rather than the use of digital tools in isolation from human contact [19-21].”
Research works printed but not linked: 6
- 2011, No Health Without Mental Health: A Cross-Government Mental Health Outcomes Strategy for People of All Ages
- de Silva, D, 2011, Evidence: Helping People Help Themselves
- Luxton, DD, 2015, Artificial Intelligence in Behavioural and Mental Healthcare
- Stickdorn, M, 2011, This Is Service Design Thinking
- Ortiz, A, 2007, Universal Access In Ambient Intelligence Environments: 9th Ercim Workshop On User Interfaces For All, Königswinter, Germany, September 27-28, 2006, Revised Papers (Lecture Notes In Computer Science)
- Carrasco, E, 2008, Computers Helping People With Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105
Cited by
Acceptability of digital health interventions: embracing the complexity
Perski, O., & Short, C. E. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab048 · Cited by 300
“ample, Easton and colleagues co-created an autonomous virtual agent for and with people living with chronic health conditions through a series of co-design workshops [31].”
BackgroundApplication Scenarios for Artificial Intelligence in Nursing Care: Rapid Review
Seibert, K., Domhoff, D., Bruch, D., Schulte-Althoff, M., Fürstenau, D., Biessmann, F., & Wolf-Ostermann, K. (2021) · Journal of Medical Internet Research · doi:10.2196/26522 · Cited by 314
Schachner, T., Keller, R., & v Wangenheim, F. (2020) · Journal of Medical Internet Research · doi:10.2196/20701 · Cited by 267
“Other types of chronic conditions included respiratory [51,58], cardiovascular [50], nervous system [56], rheumatic [57], and autoimmune/eye conditions [59].”
BackgroundMethodInfluentialPerceptions and Needs of Artificial Intelligence in Health Care to Increase Adoption: Scoping Review
Chew, H. S. J., & Achananuparp, P. (2022) · Journal of Medical Internet Research · doi:10.2196/32939 · Cited by 243
“The chatbot should be enriched by the ability to de-The de-The system should allow personalization Needed clarity on whether the chatbot was a physician or an adviser Easton et al [31] expressive.”
BackgroundInfluentialVandekerckhove, P., de Mul, M., Bramer, W. M., & de Bont, A. (2020) · Journal of Medical Internet Research · doi:10.2196/13780 · Cited by 151
“The most common recruitment strategy was purposive or convenience sampling [30,31,33,37,42,46,52,63,70,71,80,83] followed by snowball or in-person recruitment [40,65,71,79].”
Background
Metadata
- Authors
- Easton, K., Potter, S., Bec, R., Bennion, M., Christensen, H., Grindell, C., Mirheidari, B., Weich, S., de Witte, L., Wolstenholme, D., & Hawley, M. S.
- Affiliations
- University of Sheffield (GB)
- Royal Hallamshire Hospital (GB)
- Corresponding author
- Easton, Katherine
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-30
- Volume
- 21
- Issue
- 5
- Pages
- e12996
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6658240
- OpenAlex
- W2953947009
- Cited by
- 88
- References
- 29
- Citations by year
- 2026: 11 · 2025: 19 · 2024: 8 · 2023: 17 · 2022: 13 · 2021: 11 · 2020: 7 · 2019: 2
- FWCI
- 9.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- SHURA (Sheffield Hallam University Research Archive) (Sheffield Hallam University) (acceptedVersion)
- Funders
- Medical Research Council (MC_PC_14115)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- virtual agents, chatbots, chronic obstructive pulmonary disease, self-management support, co-design, acceptability, natural language processing, Internet of Things, older adults, emotional support, crisis support, living alone, chronic disease
- MeSH terms
- Adult, Female, Humans, Male, Mental Health, Middle Aged, Social Support, Comorbidity, Pulmonary Disease, Chronic Obstructive, Virtual Reality Exposure Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing. https://app.cochat.ai/papers/10.2196/12996
- The paper
- Easton, K., Potter, S., Bec, R., Bennion, M., Christensen, H., Grindell, C., Mirheidari, B., Weich, S., de Witte, L., Wolstenholme, D., & Hawley, M. S. (2019). A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing. Journal of Medical Internet Research. https://doi.org/10.2196/12996
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.