A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing
Open accessPublished version of a preprint
The DOI matches its Crossref record; 27 of the 29 references it cites were found in a registry or an index; 4 of the 27 sources its claims cite were checked for support.
Abstract
BACKGROUND: Individuals living with long-term physical health conditions frequently experience co-occurring mental health problems. This comorbidity has a significant impact on an individual's levels of emotional distress, health outcomes, and associated health care utilization. As health care services struggle to meet demand and care increasingly moves to the community, digital tools are being promoted to support patients to self-manage their health. One such technology is the autonomous virtual agent (chatbot, conversational agent), which uses artificial intelligence (AI) to process the user's written or spoken natural language and then to select or construct the corresponding appropriate responses. OBJECTIVE: This study aimed to co-design the content, functionality, and interface modalities of an autonomous virtual agent to support self-management for patients with an exemplar long-term condition (LTC; chronic pulmonary obstructive disease [COPD]) and then to assess the acceptability and system content. METHODS: We conducted 2 co-design workshops and a proof-of-concept implementation of an autonomous virtual agent with natural language processing capabilities. This implementation formed the basis for video-based scenario testing of acceptability with adults with a diagnosis of COPD and health professionals involved in their care. RESULTS: Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios for which they would like to receive support: at the time of diagnosis (information provision), during acute exacerbations (crisis support), during periods of low mood (emotional support), and for general self-management (motivation). From the scenario testing, 12 additional adults with COPD felt the system to be both acceptable and engaging, particularly with regard to internet-of-things capabilities. They felt the system would be particularly useful for individuals living alone. CONCLUSIONS: Patients did not explicitly separate mental and physical health needs, although the content they developed for the virtual agent had a clear psychological approach. Supported self-management delivered via an autonomous virtual agent was acceptable to the participants. A co-design process has allowed the research team to identify key design principles, content, and functionality to underpin an autonomous agent for delivering self-management support to older adults living with COPD and potentially other LTCs.
Overview
CoChat Summary
TL;DR
This study co-designed and tested the acceptability of Avachat, an autonomous virtual agent intended to support self-management for people with COPD and comorbid mental health needs. Participants identified four priority support scenarios and generally found the prototype acceptable and engaging, especially its multimodal and internet-of-things possibilities.
Key findings
- Co-design participants prioritized four self-management scenarios: at diagnosis (information provision), during acute exacerbations (crisis support), during periods of low mood (emotional support), and for general self-management (motivation).
- In video-based scenario testing, 8 patients/carers viewed the film; the System Usability Scale median was 73.75, with 5/8 scores above 68 (72.5, 75, 75, 80, 85) and 3/8 below (52.2, 62.5, 65).
- 4/8 participants strongly agreed they would like to use the system frequently, 7/8 strongly agreed it was easy to use, and all but 1 strongly agreed they would feel confident using it.
- Participants did not explicitly separate mental and physical health needs, but the content they developed had a clear psychological approach; they valued multimodal access, internet-of-things capabilities, and thought the system would be particularly useful for individuals living alone.
Methods
- Two co-design workshops were held with adults with COPD/carers and health professionals, using day-in-the-life mapping, missing-sentence prompts, mood boards, vignettes, role play, and script development; data were analyzed with content analysis and thematic analysis.
- A semiautonomous Avachat prototype was developed with spoken and written interaction, then filmed in a Wizard of Oz setup as a short video (<8 min) showing two scenarios: crisis support and motivational support.
- Acceptability testing screened the video with 8 patients/carers, followed by discussion around design, functionality, and content, plus self-completed System Usability Scale questionnaires.
Limitations
- Stated: the sample was small, white, British, medically stable, and regional; no participants reported a current mental health problem, though all referred to past low mood or worry; recruitment through support groups likely selected people already experienced in self-management.
- Stated: health practitioners contributed relatively little compared with patients, so it is unclear whether they endorsed the final scenarios as clinical priorities or best practice; the project was not a software development project, limiting development effort.
- Stated: possible negative consequences, incorrect advice, and wider ethical implications were not discussed; evident from the text: the abstract reports 12 additional adults with COPD in scenario testing, while the Results section reports 8 patients/carers screened, and the prototype was Wizard-of-Oz rather than fully autonomous, so acceptability of the actual autonomous system was not directly tested.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
2 Valid · 1 Not valid · 9 Not checked · 83 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“A blended approach to care has been shown to be most effective and acceptable for patients when managing both physical and emotional health [19];”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Disputedcites [19] sourceExperimental — accuracy not yet measured
The Discussion (Conclusions and policy implications; page unavailable) says that supported computerised CBT added to usual GP care offered modest or no benefit for depression, contrary to the claim that a blended approach was shown to be most effective. The paper also reports limited uptake despite telephone support; it does not establish patient acceptability or effectiveness for managing both physical and emotional health. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Could not verifycites [37] sourceExperimental — accuracy not yet measured
The Results (Description of included studies; Evaluation measures) describe studies of conversational agents, including embodied agents, but do not report whether their samples included older adults. Searches for older adults, participant age, and comorbidities found no passage addressing the claimed age-specific testing gap or the parenthetical about comorbidities. (LLM, conf=0.70)
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The Results (Autism Spectrum Disorders; page unavailable) say that over half of the 49 reviewed studies targeted autism and that only three autism applications focused on adults, supporting the point that much testing involved younger people. The review does not quantify testing in older adults or address the parenthetical claim about where most comorbidities are evident. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Taken together, this suggests that the potential exists for the development of a relationship of sorts between human and autonomous agents, a suggestion supported by a recent study investigating the use of text-based AI to deliver psychological therapy [42].”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Discussion > Principal Findings
What the sources say
- Supportedcites [42] source
The Discussion ('Main findings', this volume) states that 'many of the participants perceived the app as somewhat a living character, with comments that referred to the app as "a (real) person" and the interaction as a "relationship", indicating an ability in the chatbot to mimic human guidance', and the Results ('Qualitative data') report participants talking about the depth of 'our relationship' with Shim (Participants 5, 7, 8). The cited work is indeed the recent study the sentence identifies: a pilot RCT of a text-based, fully automated conversational agent (Shim) delivering positive-psychology and third-wave CBT strategies with therapeutic-process features such as empathic responses and accountability (Introduction; Methods, 'The intervention'). The paper additionally notes prior evidence of a bond with bots (Bickmore et al., 2005; Fitzpatrick et al., 2017), which reinforces rather than displaces its own finding. (LLM, conf=0.85)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although this development process resulted in a system that was functional, in practice, unless the user conformed quite closely to one of the developed scripts, a significant amount of input was required of the wizard.”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 62% (5/8) of the SUS scores were above the score of 68 (with scores of 72.5, 75, 75, 80, and 85), with the other 38% (3/8) participants rating the system as follows: 52.2, 62.5, and 65.”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
33 more claims
Create a free account to see all 95 claims and to verify the 23 references still open.
- Probably not a substantial claim. 47
- It may state two separate claims as one. 23
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Four studies found high levels of engagement with the conversational agent or reported the conversational agent to be engaging [50,52,53,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“In 7 studies, user experience was assessed. User experience was generally positive regarding the acceptability, understanding of the conversational agents, comprehensibility of the systems’ responses, interaction rates, or content relevance [51,53,54,56-59].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Disease-specific patients were involved in 3 studies [50,54,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“A total of 4 studies developed the artificial intelligence system internally [50,51,57,59], and 5 studies relied on external sources [52-54,56,58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Four studies were classified as AI [53,56-58].”
From Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review (2020) · cites this paper as [58]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“From the scenario testing, 12 additional adults with COPD felt the system to be both acceptable and engaging”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
“We then screened this video with 8 patients who had a diagnosis of COPD or were a carer of a spouse with COPD: 5 males and 3 females. Overall, 3 of these participants had been part of the co-design process, 5 were novel and new to the concept of Avachat.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“A total of 6 health professionals attended, all female, working across general practice, secondary care respiratory services, and specialist mental health care.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Abstract > Results
“Adults (n=6) with a diagnosis of COPD and health professionals (n=5) specified 4 priority self-management scenarios”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“Overall, 5 individuals with a diagnosis of COPD and a carer of an individual with COPD agreed to attend (n=6), of whom 5 attended”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 2
“5 patients who attended the first workshop also signed up for this second workshop. Of these, 4 attended (male n=2 and female n=2), with 1 drop out because of ill health (male). The median age of the participants was 73 years (range 66 to 80 years)”
§ A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing > Results > Workshop 1
“Participants had a median age of 82.5 years (range 69 to 86 years)”
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Weber, R. (1990) · doi:10.4135/9781412983488 · Cited by 7,750
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Depression, chronic diseases, and decrements in health: results from the World Health Surveys
Moussavi, S., Chatterji, S., Verdes, E., Tandon, A., Patel, V., & Ustun, B. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61415-9 · Cited by 4,070 · cited 1×
“These mental health conditions are distressing, have a clinically significant impact on health-related quality of life, and can lead to further health complications and greater consumption of costly health care [1,3,7].”
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“In recent years, several such autonomous virtual agents have been developed to support the delivery of psychological therapies, incorporating automated responses to text-based chat [23,24].”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 2×
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
Katon, W. J. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(03)00273-7 · Cited by 1,223
Bower, P., & Gilbody, S. (2005) · British Journal of Psychiatry · doi:10.1192/bjp.186.1.11 · Cited by 1,084
Embodied conversational agents in clinical psychology: a scoping review
Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6553 · Cited by 433 · cited 3×
“These have rarely been tested with samples of older adults (where most comorbidities are evident) [37,38].”
Gilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409 · cited 2×
“…as engaging, as acceptable by patients, and as clinically effective as traditional face-to-face delivery of care, a blended or guided approach, which includes aspects of face-to-face or telephone support, is recommended rather than the use of digital tools in isolation from human contact [19-21].”
Research works printed but not linked: 6
- 2011, No Health Without Mental Health: A Cross-Government Mental Health Outcomes Strategy for People of All Ages
- de Silva, D, 2011, Evidence: Helping People Help Themselves
- Luxton, DD, 2015, Artificial Intelligence in Behavioural and Mental Healthcare
- Stickdorn, M, 2011, This Is Service Design Thinking
- Ortiz, A, 2007, Universal Access In Ambient Intelligence Environments: 9th Ercim Workshop On User Interfaces For All, Königswinter, Germany, September 27-28, 2006, Revised Papers (Lecture Notes In Computer Science)
- Carrasco, E, 2008, Computers Helping People With Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105
Cited by
Acceptability of digital health interventions: embracing the complexity
Perski, O., & Short, C. E. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab048 · Cited by 300
“ample, Easton and colleagues co-created an autonomous virtual agent for and with people living with chronic health conditions through a series of co-design workshops [31].”
BackgroundApplication Scenarios for Artificial Intelligence in Nursing Care: Rapid Review
Seibert, K., Domhoff, D., Bruch, D., Schulte-Althoff, M., Fürstenau, D., Biessmann, F., & Wolf-Ostermann, K. (2021) · Journal of Medical Internet Research · doi:10.2196/26522 · Cited by 314
Schachner, T., Keller, R., & v Wangenheim, F. (2020) · Journal of Medical Internet Research · doi:10.2196/20701 · Cited by 267
“Other types of chronic conditions included respiratory [51,58], cardiovascular [50], nervous system [56], rheumatic [57], and autoimmune/eye conditions [59].”
BackgroundMethodInfluentialPerceptions and Needs of Artificial Intelligence in Health Care to Increase Adoption: Scoping Review
Chew, H. S. J., & Achananuparp, P. (2022) · Journal of Medical Internet Research · doi:10.2196/32939 · Cited by 243
“The chatbot should be enriched by the ability to de-The de-The system should allow personalization Needed clarity on whether the chatbot was a physician or an adviser Easton et al [31] expressive.”
BackgroundInfluentialVandekerckhove, P., de Mul, M., Bramer, W. M., & de Bont, A. (2020) · Journal of Medical Internet Research · doi:10.2196/13780 · Cited by 151
“The most common recruitment strategy was purposive or convenience sampling [30,31,33,37,42,46,52,63,70,71,80,83] followed by snowball or in-person recruitment [40,65,71,79].”
BackgroundBradwell, H. L., Edwards, K. J., Winnington, R., Thill, S., & Jones, R. B. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032468 · Cited by 161
Chatbots to Support Young Adults’ Mental Health: An Exploratory Study of Acceptability
Koulouri, T., Macredie, R. D., & Olakitan, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3485874 · Cited by 127
“Finally, Wizard of Oz studies were excluded, as system responses in those setups were generated by another human (for example, the study by Easton et al. [2019]). (3) They performed a user evaluation that measured any aspect linked to acceptability. Studies in which the user evaluation reported only health-related mea…”
MethodInfluentialZidaru, T., Morrow, E. M., & Stockley, R. (2021) · Health Expectations · doi:10.1111/hex.13299 · Cited by 169
Duffy, A., Christie, G. J., & Moreno, S. (2022) · JMIR Human Factors · doi:10.2196/35693 · Cited by 114
“This approach is supported by the UK National Institute for Health Research [50], with the foundation being that the intersection of various sciences and learned experiences harmonizes DHIs [3] in the form of social innovation [51].”
BackgroundBin Sawad, A., Narayan, B., Alnefaie, A., Maqbool, A., Mckie, I., Smith, J., Yuksel, B., Puthal, D., Prasad, M., & Kocaballi, A. B. (2022) · Sensors · doi:10.3390/s22072625 · Cited by 113
“Other conditions included hypertension [43], heart failure [38], HIV [53–56], long‐term conditions [35], chronic pain [36], chronic problems in oral health [41], urinary incontinence [31], and post‐traumatic stress disorder [49].”
BackgroundThorn, P., Hill, N. T. M., Lamblin, M., Teh, Z., Battersby-Coulter, R., Rice, S., Bendall, S., Gibson, K. L., Finlay, S. M., Blandon, R., et al. (2020) · JMIR Mental Health · doi:10.2196/17520 · Cited by 122
“Although participatory approaches are becoming increasingly common in mental health [35-43], to the best of our knowledge, they have rarely been applied to the development of youth suicide prevention interventions.”
BackgroundHealth information technology to improve care for people with multiple chronic conditions
Samal, L., Fu, H. N. C., Camara, D. S., Wang, J., Bierman, A. S., & Dorr, D. A. (2021) · Health Services Research · doi:10.1111/1475-6773.13860 · Cited by 136
“78”
BackgroundResultWutz, M., Hermes, M., Winter, V., & Köberlein–Neu, J. (2023) · Journal of Medical Internet Research · doi:10.2196/46548 · Cited by 137
“In addition, the system should have the ability to interact with other digital services and home devices, such as calendars, smart home technology, and existing medical devices or applications [21,24,26,28,55,113,120].”
InfluentialLoveys, K., Sagar, M. A., Pickering, I., & Broadbent, E. (2021) · JMIR Mental Health · doi:10.2196/31586 · Cited by 101
“This study achieved positive results that align with prior CA research showing good acceptability at delivering loneliness interventions to older people [38,68,69], and psychological support for well-being [70] and anxiety in adults with chronic health conditions [39].”
Current Implementation of Digital Health in Chronic Disease Management: Scoping Review
Pong, C. Y., Tseng, R. M. W. W., Tham, Y. C., & Lum, E. (2024) · Journal of Medical Internet Research · doi:10.2196/53576 · Cited by 124
“…future research in other contexts, such as different countries or cities [43,59,68,88], different health care systems [59,68] or settings [47,211,264-266], different stakeholder groups involved in the implementation [43,80,96,267], different demographics in terms of age [268] or socioeconomic…”
Anisha, S. A., Sen, A., & Bain, C. (2024) · Journal of Medical Internet Research · doi:10.2196/56114 · Cited by 102
“%) were relatively heterogeneous, with older adults [10,58,72,73,91,95] and middle-aged adults [24,99] being the most frequently targeted user group (8/43, 19%).”
InfluentialJúnior, J. L. A. D. S., Biduski, D., Bellei, E. A., Becker, O. H. C., Daroit, L., Pasqualotti, A., Filho, H. T., & Marchi, A. C. B. D. (2021) · JMIR Serious Games · doi:10.2196/23423 · Cited by 62
“A co-design process can significantly contribute to product development, reducing social stigma, empowering users [28], and giving voice to patient groups in the creation process [29].”
Zipfel, N., Horreh, B., Hulshof, C. T. J., de Boer, A. G. E. M., & van der Burg-Vermeulen, S. J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058630 · Cited by 73
Potts, C., Ennis, E., Bond, R., Mulvenna, M. D., McTear, M., Boyd, K., Broderick, T., Malcolm, M., Kuosmanen, L., Nieminen, H., et al. (2021) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-021-00222-6 · Cited by 70
“reported on co-designing content for a health chatbot by involving patients with lived experiences (Easton et al., 2019).”
BackgroundKim, H. K. (2024) · Healthcare · doi:10.3390/healthcare12050534 · Cited by 78
“AI chatbots overcome the physical limitations of human therapists, such as the ability to operate without fatigue and to be available almost anywhere and anytime [38].”
BackgroundReview of artificial intelligence‐based question‐answering systems in healthcare
Cilar, L., Gosak, L., & Štiglic, G. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1487 · Cited by 86
Pal, D., Vanijja, V., Thapliyal, H., & Zhang, X. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107788 · Cited by 65
Participatory Development and Pilot Testing of an Adolescent Health Promotion Chatbot
Maenhout, L., Peuters, C., Cardon, G., Compernolle, S., Crombez, G., & DeSmet, A. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.724779 · Cited by 53
Luca, S., Clausen, M., Shaw, A., Lee, W., Krishnapillai, S., Adi-Wauran, E., Faghfoury, H., Costain, G., Jobling, R. K., Aronson, M. D., et al. (2023) · Human Genetics · doi:10.1007/s00439-022-02512-2 · Cited by 37
“Patient acceptability of chatbots in healthcare has been studied in other disciplines like psychology, (Easton et al. 2019) (Goonesekera and Donkin 2022), oncology (Bibault et al. 2019; Chaix et al. 2019), and public health (Luk et al. 2022).”
BackgroundAI Applications for Chronic Condition Self-Management: Scoping Review
Hwang, M., Zheng, Y., Cho, Y., & Jiang, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/59632 · Cited by 71
“An AI chatbot (Avachat [52]) was tested to provide motivational support for patients with COPD to engage in general self-management during periods of low moods.”
InfluentialBartle, V., Lyu, J., El Shabazz-Thompson, F., Oh, Y., Chen, A. A., Chang, Y.-J., Holstein, K., & Dell, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517683 · Cited by 35
Co-designing care for multimorbidity: a systematic review
Sumner, J., Ng, C. W. T., Teo, K., Peh, A. L. T., & Lim, Y. W. (2024) · BMC Medicine · doi:10.1186/s12916-024-03263-9 · Cited by 46
“2 Facilitators and barriers of co-design identified in our included studies online web portals [27, 29] and other digital media [36].”
BackgroundMethodInfluentialCho, H., Oh, O., Greene, N., Gordon, L., Morgan, S., Walke, L. M., & Demiris, G. (2025) · The Journals of Gerontology Series A · doi:10.1093/gerona/glaf024 · Cited by 41
Gunn, K. M., Skaczkowski, G., Dollman, J., Vincent, A. D., Short, C. E., Brumby, S. A., Barrett, A., Harrison, N. J., & Turnbull, D. A. (2022) · JMIR Human Factors · doi:10.2196/27631 · Cited by 29
“opinion with farmers’ wants and preferences was used in a process similar to that described by Easton et al [30].”
MethodUnderstanding the Benefits and Design of Chatbots to Meet the Healthcare Needs of Migrant Workers
Tseng, Y., Jarupreechachan, W., & Lee, T.-H. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610106 · Cited by 43
“The first design insight of the conceptual chatbot was based on the findings of related studies that the chatbot application should have problem-related information and the capacity to connect with other digital services and domestic services [33, 79].”
BackgroundMethodInfluentialDu, Y., Yang, P., Liu, Y., Deng, C., & Li, X. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1689911 · Cited by 51
Kruse, L., Hertel, J., Mostajeran, F., Schmidt, S., & Steinicke, F. (2023) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar59233.2023.00082 · Cited by 28
“Furthermore, they should be accessible [8,17,62], affordable [75] and validated by credible institutions or therapists or by using evidence-based data [1,3,78, 80,90].”
BackgroundInfluentialAn, Q., Sandlund, M., Agnello, D. M., McCaffrey, L., CHASTIN, S. F. M., Helleday, R., & Wadell, K. (2023) · Respiratory Medicine · doi:10.1016/j.rmed.2023.107193 · Cited by 21
“…studies that included participants from various linguistic groups both mentioned the use of translators as part of the co-creation process [58,61]. d) Employment of creative methods and the use of generative tools was considered useful and contributed positively to the co-creation process [50,60].”
InfluentialThe Value-Sensitive Conversational Agent Co-Design Framework
Sadek, M., Calvo, R. A., & Mougenot, C. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2426737 · Cited by 32
“…they will then roleplay CA creators’ and CA users’ different versions of a scenario, which is a very commonly used technique in CA co-design (Garg & Sengupta, 2020; Easton et al., 2019; Candello et al., 2020; Lee et al., 2017; Weber et al., 2021; Luria et al., 2020; Lopatovska et al., 2022).”
Vázquez, A., Zorrilla, A. L., Olaso, J. M., & Torres, M. I. (2023) · Sensors · doi:10.3390/s23031423 · Cited by 20
“However, the systems presented in these reviews (Woebot [32], Tess [33], MMD agent [34], Avachat [35] or JADE [36]) do not base their coaching methodology on conversations.”
BackgroundThe Role of a Smart Health Ecosystem in Transforming the Management of Chronic Health Conditions
Nourse, R. A., Dingler, T., Kelly, J. T., Kwaśnicka, D., & Maddison, R. (2023) · Journal of Medical Internet Research · doi:10.2196/44265 · Cited by 17
Künstliche Intelligenz in der Radiologie: Was ist in den nächsten Jahren zu erwarten?
Haubold, J. (2019) · Der Radiologe · doi:10.1007/s00117-019-00621-0 · Cited by 15
Ung, T. X., O’Reilly, C. L., Moles, R. J., & El‐Den, S. (2023) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-023-01622-9 · Cited by 15
“Similar to previous co-design studies [52–54], consumers met to brainstorm ideas, further developed by the research team.”
ResultEhtesham-Ul-Haque, M., D’Rozario, J., Adnin, R., Utshaw, F. T., Tasneem, F., Shefa, I. J., & Islam, A. B. M. A. A. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101168 · Cited by 22
Intelligent Digital Human Agent Service With Deep Learning Based-Face Recognition
Kim, B., & Seo, S. (2022) · IEEE Access · doi:10.1109/access.2022.3188852 · Cited by 14
Salomé, S., du Bousquet, L., & Monfort, E. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100606 · Cited by 17
A Survey of Conversational Agents and Their Applications for Self-Management of Chronic Conditions
Park, M. S., Upama, P. B., Anik, A. A., Ahamed, S. I., Luo, J., Tian, S., Rabbani, M., & Oh, H. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00162 · Cited by 18
“Other conditions include obesity [77], [78], diabetes [79], [80], cancer [81], asthma [82], [83], neuropsychiatric disorders [74], [84], cardiovascular diseases [85]–[87], neurological disorders [88], chronic obstructive pulmonary disease (COPD) [83], arthritis [89], hypertension [90], and hearing…”
BackgroundVirtual Agents with Augmented Reality in Digital Healthcare
Jeong, H., Yoo, J. H., & Song, H. (2022) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc55196.2022.9952909 · Cited by 11
“In the case of digital healthcare, it is beneficial that patients can self-manage through digital tools such as chatbots or conversational agents and over through the limited communication between patients and medical staff [1].”
BackgroundReview of innovative models of aged care
Dyer, S. M., van den Berg, M., Brown, A., Johnstone, G., Laver, K., Lowthian, J. A., Maeder, A., Meyer, C., Moores, C. J., Ogrin, R., et al. (2020) · Cited by 20
Lee, Y. H., Choi, H., & Lee, S. (2025) · JMIR Formative Research · doi:10.2196/67272 · Cited by 16
Talking Abortion (Mis)information with ChatGPT on TikTok
Sharevski, F., Loop, J. V., Jachim, P., Devine, A., & Pieroni, E. (2023) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw59978.2023.00071 · Cited by 21
“For example, P16FL30 refers to participant 16, female, leftleaning, age bracket [21-30].”
BackgroundInfluentialDesign Validation of a Relational Agent by COVID-19 Patients: Mixed Methods Study
Islam, A., & Chaudry, B. (2022) · JMIR Human Factors · doi:10.2196/42740 · Cited by 11
Wang, J., Ly, L., Barson, E., & Smallwood, N. (2025) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-025-00430-0 · Cited by 12
“Our fi ndings also point to the value of health professionals working collaboratively with patients to tailor self-management strategies, and to consider a more ‘ blended ’ approach to mental health care 6,14,43 .”
Stakeholder engagement: the path to trustworthy AI in healthcare
Camacho, M., Llussà, J. P., Puig-Bosch, X., Dang, V. N., Díaz, O., & Lekadir, K. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00038-9 · Cited by 14
Understanding People’s Expectations When Designing a Chatbot for Cancer Patients
Félix, B., & Ribeiro, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-94890-0_3 · Cited by 9
Developing a chatbot/intelligent system for neurological diagnosis and management
Das, A., Sen, V., & Rose, A. C. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90037-9.00010-2 · Cited by 10
Altom, D. S., Taha, A. I. A., Hussein, A. A. A. M., Elshiekh, M. A. I., Abdelmajed, A. H., Ibrahim, F. I. A., Mohammed, S., & Tifoor, M. M. E. E. (2025) · Cureus · doi:10.7759/cureus.81001 · Cited by 16
“They are employed in the diagnosis, coaching, data gathering, support, and educating of patients with long-term illnesses [16].”
InfluentialVeneziani, I., Grimaldi, A., Marra, A., Morini, E., Culicetto, L., Marino, S., Quartarone, A., & Maresca, G. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13082307 · Cited by 8
“Furthermore, chatbots for supporting hypertension medication self-management were found useful in managing patient’s medications, including reminders, refills, and even communication with healthcare providers [40].”
BackgroundMonteiro, M. D. S., Salgado, L. C. D. C., Seixas, F. L., & Santana, R. F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50249-2_12 · Cited by 8
What patients want from healthcare chatbots: insights from a mixed-methods study
Dellavalle, N. S., Ellis, J. R., Moore, A., Akerson, M., Andazola, M., Campbell, E. G., & DeCamp, M. W. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf164 · Cited by 14
Cao, J., Pai, Y. S., Li, C., Hoermann, S., & Billinghurst, M. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720086 · Cited by 8
“Recent advances in Artificial Intelligence (AI) driven virtual agents could enable them to provide accessible mental health support, such as using conversational interfaces to offer ongoing emotional support [8, 22].”
Sezgın, E., Jackson, D., Kaufman, K. M., Skeens, M. A., Gerhardt, C. A., & Moscato, E. L. (2025) · Digital Health · doi:10.1177/20552076251326160 · Cited by 11
Professional representation of conversational agents for health care: a scoping review protocol
MacNeill, A. L., MacNeill, L., Doucet, S. A., & Luke, A. (2021) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00589 · Cited by 6
The professional representation of conversational agents for health care: A scoping review protocolPreprint
MacNeill, A. L., MacNeill, L., Luke, A., & Doucet, S. A. (2021) · doi:10.17605/osf.io/d8v2q · Cited by 4
Chew, H. S. J., & Achananuparp, P. (2021) · doi:10.2196/preprints.32939 · Cited by 4
Towards a Design Framework for Conversational Agents for Diabetes Prevention
Sawad, A. B., Alakhtar, R. A., Alturki, B., Narayan, B., Lin, S., Prasad, M., & Kocaballi, A. B. (2023) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3638380.3638421 · Cited by 3
“Easton et al. [14] investigated a web-based conversational AI that utilised NLP and speech recognition.”
BackgroundMarcoux, A., Tessier, M.-H., Grondin, F., Reduron, L. R., & Jackson, P. L. (2021) · Santé mentale au Québec · doi:10.7202/1081509ar · Cited by 6
Schachner, T., Keller, R., & von Wangenheim, F. (2020) · doi:10.2196/preprints.20701 · Cited by 3
Künstliche Intelligenz in der Radiologie: Was ist in den nächsten Jahren zu erwarten?
Haubold, J. (2020) · Der Radiologe · Cited by 3
Baptista, S., Wadley, G., Bird, D., Oldenburg, B., Speight, J., & The My Diabetes Coach Research Group (2020) · JMIR mHealth and uHealth · doi:10.2196/17038 · Cited by 3
“Although participants reported a clear preference for a human-like character, which is supported by previous research [15], her presentation of a backstory might be a step too far as it did not seem credible to some participants, possibly because of an uncanny valley phenomenon [25].”
BackgroundResultFerrarini, F., & Leonelli, S. (2025) · Industrial Management & Data Systems · doi:10.1108/imds-05-2025-0661 · Cited by 8
Virtual humans in geriatric care: an integrative review
Diño, M. J. S., Dion, K. W., Abadir, P. M., Budhathoki, C. B., Huang, C., Vital, J. C., Rivero, J. A., Malacas, M. K. G., Hernandez, R., Balbin, P. T., et al. (2025) · The Journals of Gerontology Series A · doi:10.1093/gerona/glaf145 · Cited by 2
Thorn, P., Hill, N. T. M., Lamblin, M., et al. (2019) · doi:10.2196/preprints.17520 · Cited by 2
Cao, J., Li, C., Gao, W., Hoermann, S., Baghaei, N., & Billinghurst, M. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817039 · Cited by 3
“This shift encompasses both the transition from traditional face-to-face therapy to teletherapy via platforms like Zoom [8], as well as the exploration of novel approaches to remote mental healthcare delivery [14, 54].”
A scoping review on conversational AI in mental health: A human-centered perspective
Li, J., Li, H., Zhu, H., Walter, H., Banaschewski, T., Li, C., Li, X., & Zhang, Y. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106460 · Cited by 4
Toward Natural and Companionable Virtual Agents via Cross-Temporal Emotional Modeling
Zheng, Y., Qin, F., Li, X., Huang, H., Wang, H., Wang, X., Lu, Y., & Zhang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790917 · Cited by 1
“Stable identity traits foster immersion in education [5, 34], services [9, 51, 53, 72], and healthcare [2, 19].”
The relationship between mental health disorders and LUTS
Janssen, J. M. W., den Ende, M. V., Kheir, G. B., Leue, C., & Vrijens, D. (2026) · Autonomic Neuroscience · doi:10.1016/j.autneu.2026.103422 · Cited by 3
Pröbsting, J. S., Rießenberger, K. A., & Fischer, F. (2026) · BMC Digital Health · doi:10.1186/s44247-026-00254-z · Cited by 1
Zeng, J., Xu, Y., Chen, P., Hong, M., & Zhang, Q. (2026) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2026.105394 · Cited by 2
Lee, J., Dobmeier, C. M., Heo, M., & Woo, S. S. (2025) · Health Communication · doi:10.1080/10410236.2025.2599476 · Cited by 1
“…harnessed the versatility of digital platforms by employing autonomous virtual agents (e.g., chatbots) designed to simulate empathetic interactions (Easton et al., 2019), digital avatars tailored for therapist – client communication (Rehm et al., 2016), celeb-rities whose established public…”
Conversational Agents for Managing Chronic Diseases: A Systematic Review
Zakerabasali, S., Amiri, P., & Alipour, J. (2025) · Health Scope · doi:10.5812/healthscope-163131 · Cited by 1
Scoping Review on Lead User Engagement Methods in the Development of Telemedicine Products
Bhebhe, M., Grobbelaar, S. S. S., & Herselman, M. E. (2023) · International Conference on Information Science and Systems (ICISS) · doi:10.1145/3625156.3625195 · Cited by 1
Hassan, N. B., Teo, T. P., Liew, M. F., Tan, W. H., Wu, W., Tan, T. M., Mustafah, S. A. B., Poh, D. W. Q., He, H. G., Khor, J. H., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/98338
“Several included studies did involve patients or caregivers in co-design [37,45,46], and these provided the most detailed and actionable insights into user requirements in this review.”
InfluentialGalland, L., Clavel, C., & Ochs, M. (2026) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3806774.3827976
Léveillée, A.-K., Dellazizzo, L., & Savard, I. (2026) · Médiations et médiatisations · doi:10.52358/mm.vi23.501
User-centered service design approach for self-care using earable devices
Choi, J.-Y., Lee, S., Park, J., Ha, J., Lee, T., Kim, J., Cho, S., & Jeon, G. (2026) · Digital Health · doi:10.1177/20552076261478392
Shu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13022-8_28
Zhang, W., & Tao, D. (2025) · IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) · doi:10.1109/ieem63636.2025.11357573
“Therefore, in doctor-patient consultations, it has been found that participants often express a desire to customize the characteristics of the avatar to meet their personal preferences, e.g., in terms of age, gender, and visualization [12-15].”
A Narrative Review of Virtual Human Twins (VHTs) of Healthcare Professionals
Gargiulo, L., Saritas, S. N., Arar, F. E., Saitta, L., de Luca, M., Vallefuoco, E., Arlati, S., Mondellini, M., Moccaldi, N., d'Errico, G., et al. (2025) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine66377.2025.11340559
MacNeill, A. L., MacNeill, L., Luke, A., & Doucet, S. A. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1590514
The innovator’s cookbook: engaging lead users in healthcare product development
Bhebhe, M., & Grobbelaar, S. S. S. (2025) · Management Review Quarterly · doi:10.1007/s11301-025-00505-9 · Cited by 1
Lee, Y. H., Choi, H., & Lee, S. (2024) · doi:10.2196/preprints.67272
AI Applications for Chronic Condition Self-Management: Scoping Review (Preprint)Preprint
Hwang, M., Zheng, Y., Cho, Y., et al. (2024) · doi:10.2196/preprints.59632
Turntable: The Co-creation of a Digital Solution Supporting Older Adults in Active Aging
Szakonyi, B., Vassányi, I., Mantur-Vierendeel, A., Loi, D., Correia, G., & Blažica, B. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-37496-8_1 · Cited by 2
An, Q., Gebart-Hedman, K., & Wadell, K. (2023) · International Conference on Engineering and Product Design Education (EPDE) · doi:10.35199/epde.2023.40
Nourse, R. A., Dingler, T., Kelly, J. T., et al. (2022) · doi:10.2196/preprints.44265
Design Validation of a Relational Agent by COVID-19 Patients: Mixed Methods Study (Preprint)Preprint
Islam, A., & Chaudhry, B. M. (2022) · doi:10.2196/preprints.42740
Research on 3D Virtual Emotional Interaction Design Based on Network Digitization
Jing, X., & Xie, X. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-2452-1_19
Loveys, K., Sagar, M. A., Pickering, I., et al. (2021) · doi:10.2196/preprints.31586
Gunn, K. M., Skaczkowski, G., Dollman, J., et al. (2021) · doi:10.2196/preprints.27631
(2019) · doi:10.2196/preprints.17038 · Cited by 3
Metadata
- Authors
- Easton, K., Potter, S., Bec, R., Bennion, M., Christensen, H., Grindell, C., Mirheidari, B., Weich, S., de Witte, L., Wolstenholme, D., & Hawley, M. S.
- Affiliations
- University of Sheffield (GB)
- Royal Hallamshire Hospital (GB)
- Corresponding author
- Easton, Katherine
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-30
- Volume
- 21
- Issue
- 5
- Pages
- e12996
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6658240
- OpenAlex
- W2953947009
- Cited by
- 88
- References
- 29
- Citations by year
- 2026: 11 · 2025: 19 · 2024: 8 · 2023: 17 · 2022: 13 · 2021: 11 · 2020: 7 · 2019: 2
- FWCI
- 9.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- SHURA (Sheffield Hallam University Research Archive) (Sheffield Hallam University) (acceptedVersion)
- Funders
- Medical Research Council (MC_PC_14115)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- virtual agents, chatbots, chronic obstructive pulmonary disease, self-management support, co-design, acceptability, natural language processing, Internet of Things, older adults, emotional support, crisis support, living alone, chronic disease
- MeSH terms
- Adult, Female, Humans, Male, Mental Health, Middle Aged, Social Support, Comorbidity, Pulmonary Disease, Chronic Obstructive, Virtual Reality Exposure Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.