Understanding the Benefits and Challenges of Deploying Conversational AI Leveraging Large Language Models for Public Health Intervention
Open accessThe DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Abstract
Recent large language models (LLMs) have advanced the quality of open-ended conversations with chatbots. Although LLM-driven chatbots have the potential to support public health interventions by monitoring populations at scale through empathetic interactions, their use in real-world settings is underexplored. We thus examine the case of CareCall, an open-domain chatbot that aims to support socially isolated individuals via check-up phone calls and monitoring by teleoperators. Through focus group observations and interviews with 34 people from three stakeholder groups, including the users, the teleoperators, and the developers, we found CareCall offered a holistic understanding of each individual while offloading the public health workload and helped mitigate loneliness and emotional burdens. However, our findings highlight that traits of LLM-driven chatbots led to challenges in supporting public and personal health needs. We discuss considerations of designing and deploying LLM-driven chatbots for public health intervention, including tensions among stakeholders around system expectations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 1×
“We used thematic analysis [8] to qualitatively analyze both interview transcripts and observational notes.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“, BERT [16]) required additional fne-tuning steps to perform a specifc task, recent large language models (e.”
MethodOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Also, there exists a possibility that the unethical or biased phrases ingrained in the models’ pre-training datasets might be exposed in the model’s output, causing the chatbot to say socially biased [6, 7, 21, 67, 68] or toxic [22] messages.”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 4×
“, 13–175B for GPT-3, 82B for HyperCLOVA), have enabled a new paradigm of in-context learning [9, 43].”
BackgroundInfluentialLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 4×
“, 13–175B for GPT-3, 82B for HyperCLOVA), have enabled a new paradigm of in-context learning [9, 43].”
BackgroundInfluentialPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 2×
“, GPT-3 [9], HyperCLOVA [29], PaLM [12], OPT [85]) with a larger number of parameters (e.”
BackgroundEvaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Heat wave: a social autopsy of disaster in Chicago
(2003) · Choice Reviews Online · doi:10.5860/choice.40-4319 · Cited by 1,273
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
Granny and the robots: ethical issues in robot care for the elderly
Sharkey, A., & Sharkey, N. (2012) · Ethics and Information Technology · doi:10.1007/s10676-010-9234-6 · Cited by 985 · cited 2×
“[66] pointed out that such a reduction in human contact is unethical because it might have a negative impact on the health and wellbeing of the individuals.”
BackgroundRecipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Recipes for building an open-domain chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Shuster, K., Smith, E. M., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.13637 · Cited by 784 · cited 2×
“Recent LLMs, however, have brought breakthroughs in opendomain dialog systems thanks to their capabilities in generating coherent and contextual responses through in-context learning [3, 63].”
BackgroundEmotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory
Zhou, H., Huang, M., Zhang, T., Zhu, X., & Liu, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11325 · Cited by 763
Donovan, N. J., & Blazer, D. (2020) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2020.08.005 · Cited by 739
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Social Isolation and Loneliness in Older Adults: Opportunities for the Health Care System
Committee on the Health and Medical Dimensions of Social Isolation and Loneliness in Older Adults, Board on Health Sciences Policy, Board on Behavioral, C., Division, H. A. M., Education, D. O. B. A. S. S. A., & National Academies of Sciences, E. (2020) · National Academies Press eBooks · doi:10.17226/25663 · Cited by 483
Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot
Liu, B., & Sundar, S. S. (2018) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2018.0110 · Cited by 447 · cited 1×
“In particular, chatbots have been proposed as efective tools for scaling abilities to provide informational and emotional support around health [42, 76].”
MethodLee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
"I Hear You, I Feel You": Encouraging Deep Self-disclosure through a Chatbot
Lee, Y.-C., Yamashita, N., Huang, Y., & Fu, W. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376175 · Cited by 339 · cited 2×
“Specifcally, studies have designed chatbots for eliciting self-disclosure [40, 41, 55] or increasing self-compassion by taking care of chatbots that experience distress [30, 38].”
BackgroundChallenges in building intelligent open-domain dialog systems
Huang, M., Zhu, X., & Gao, J. (2020) · ACM Transactions on Information Systems · doi:10.1145/3383123 · Cited by 288 · cited 5×
“suggest that open-dialog systems should aim to (1) understand the semantics of what the user said, (2) behave consistently with their predefned persona, conversation history, and speaking style, and (3) interact with the user emotionally [23].”
BackgroundInfluentialKorngiebel, D. M., & Mooney, S. D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00464-x · Cited by 259
Digital Family Portrait Field Trial: Support for Aging in Place
Rowan, J., & Mynatt, E. D. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055044 · Cited by 254 · cited 3×
“A core concern is that existing technologies have predominantly targeted individuals who have readily accessible social contacts, such as informal caregivers [15, 51, 64, 74].”
BackgroundNeural Text Generation with Unlikelihood Training
Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2019) · arXiv · doi:10.48550/arxiv.1908.04319 · Cited by 254 · cited 1×
“, the agent acts as if it was a child of the user or speaks impolitely) or system policies in a way which decreased the probability of them being selected [3, 77].”
BackgroundIncreasing the opportunities for aging in place
Mynatt, E. D., Essa, I., & Rogers, W. (2000) · Proceedings on the 2000 conference on Universal Usability - CUU '00 · doi:10.1145/355460.355475 · Cited by 248
The CareNet Display: Lessons Learned from an In Home Evaluation of an Ambient Display
Consolvo, S., Roessler, P., & Shelton, B. E. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30119-6_1 · Cited by 227 · cited 3×
“A core concern is that existing technologies have predominantly targeted individuals who have readily accessible social contacts, such as informal caregivers [15, 51, 64, 74].”
BackgroundFine-grained Sentiment Classification using BERT
Munikar, M., Shakya, S., & Shrestha, A. (2019) · 2019 Artificial Intelligence for Transforming Business and Society (AITB) · doi:10.1109/aitb48515.2019.8947435 · Cited by 226
Pradhan, A., Findlater, L., & Lazar, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359316 · Cited by 223
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Also, there exists a possibility that the unethical or biased phrases ingrained in the models’ pre-training datasets might be exposed in the model’s output, causing the chatbot to say socially biased [6, 7, 21, 67, 68] or toxic [22] messages.”
BackgroundTaleBrush: Sketching Stories with Generative Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501819 · Cited by 206
Mobile-izing health workers in rural India
Ramachandran, D., Canny, J., Das, P. D., & Cutrell, E. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753610 · Cited by 200 · cited 1×
“Researchers reiterate that these systems should thus not fully replace public health workers in collection roles but aim to be complementary support [60, 81].”
BackgroundBeyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.356 · Cited by 188
Caring for Vincent: A Chatbot for Self-Compassion
Lee, M., Ackermans, S., van As, N., Chang, H., Lucas, E., & IJsselsteijn, W. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300932 · Cited by 186 · cited 2×
“Specifcally, studies have designed chatbots for eliciting self-disclosure [40, 41, 55] or increasing self-compassion by taking care of chatbots that experience distress [30, 38].”
BackgroundLee, Y.-C., Yamashita, N., & Huang, Y. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392836 · Cited by 185 · cited 1×
“Specifcally, studies have designed chatbots for eliciting self-disclosure [40, 41, 55] or increasing self-compassion by taking care of chatbots that experience distress [30, 38].”
BackgroundTowards Making the Most of BERT in Neural Machine Translation
Yang, J., Wang, M., Zhou, H., Zhao, C., Zhang, W., Yu, Y. H., & Li, L. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6479 · Cited by 179
Designing and Evaluating mHealth Interventions for Vulnerable Populations: A Systematic Review
Stowell, E., Lyson, M. C., Saksono, H., Wurth, R. C., Jimison, H., Pavel, M., & Parker, A. G. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173589 · Cited by 169 · cited 1×
“However, studies have pointed out that compared to high socioeconomic status (SES) individuals, low-SES individuals living alone tend to have fewer social contacts that they can reach out to in emergency situations [1, 78], refecting important diferences in how to approach designing technology to support this more vul…”
BackgroundPersonalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · arXiv · doi:10.48550/arxiv.1801.07243 · Cited by 165
Towards Exploiting Background Knowledge for Building Conversation Systems
Moghe, N., Arora, S., Banerjee, S., & Khapra, M. M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1255 · Cited by 157 · cited 1×
“, [49]), with an overarching goal of providing empathy and enhancing feelings of social belonging (e.”
BackgroundBERT with History Answer Embedding for Conversational Question Answering
Qu, C., Yang, L., Qiu, M., Croft, W. B., Zhang, Y., & Iyyer, M. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331341 · Cited by 155 · cited 1×
“, [59])—through fne-tuning with task-specifc datasets [9, 43].”
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07567 · Cited by 148
FeedFinder: A Location-Mapping Mobile Application for Breastfeeding Women
Balaam, M., Comber, R., Jenkins, E., Sutton, S., & Garbett, A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702328 · Cited by 131 · cited 2×
“One major line of the research on technology interventions in public health settings has focused on automating aspects of care that public health workers typically have to provide manually, such as answering common questions [81] and identifying public resources [4, 69].”
BackgroundSocietal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131 · cited 1×
“Also, there exists a possibility that the unethical or biased phrases ingrained in the models’ pre-training datasets might be exposed in the model’s output, causing the chatbot to say socially biased [6, 7, 21, 67, 68] or toxic [22] messages.”
BackgroundGarrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., & Ureña-López, L. A. (2021) · Applied Sciences · doi:10.3390/app11073184 · Cited by 123
Feedpal: Understanding Opportunities for Chatbots in Breastfeeding Education of Women in India
Yadav, D., Malik, P., Dabas, K., & Singh, P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359272 · Cited by 90 · cited 6×
“Researchers reiterate that these systems should thus not fully replace public health workers in collection roles but aim to be complementary support [60, 81].”
BackgroundInfluentialLeveling Up: On the Potential of Upstream Health Informatics Interventions to Enhance Health Equity
Veinot, T. C., Ancker, J. S., Cole-Lewis, H., Mynatt, E. D., Parker, A. G., Siek, K. A., & Mamykina, L. (2019) · Medical Care · doi:10.1097/mlr.0000000000001032 · Cited by 83
Calling for Care: ‘Disembodied’ Work, Teleoperators and Older People Living at Home
Roberts, C., Mort, M., & Milligan, C. (2012) · Sociology · doi:10.1177/0038038511422551 · Cited by 80
Dignity, Autonomy, and Style of Company: Dimensions Older Adults Consider for Robot Companions
Coghlan, S., Waycott, J., Lazar, A., & Barbosa Neves, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449178 · Cited by 78
Emotional Dialogue Generation using Image-Grounded Language Models
Huber, B., McDuff, D., Brockett, C., Galley, M., & Dolan, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173851 · Cited by 76
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2018) · Proceedings of ACL 2018, Tutorial Abstracts · doi:10.18653/v1/p18-5002 · Cited by 73
Kim, B., Kim, H., Lee, S.-W., Lee, G., Kwak, D., Dong Hyeon, J., Park, S., Kim, S., Kim, S., Seo, D., et al. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.274 · Cited by 72 · cited 5×
“, GPT-3 [9], HyperCLOVA [29], PaLM [12], OPT [85]) with a larger number of parameters (e.”
BackgroundMethodInfluentialHow In-Home Technologies Mediate Caregiving Relationships in Later Life
Huber, L. L., Shankar, K., Caine, K. E., Connelly, K., Camp, L. J., Walker, B. A., & Borrero, L. (2012) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2012.715990 · Cited by 67 · cited 1×
“Prior work highlighted the concern that the introduction of AI technology that supports aging in place could lead to unintended consequences such as reducing human contact with their formal and informal caregivers [25, 37, 66].”
BackgroundRyu, H., Kim, S., Kim, D., Han, S., Lee, K., & Kang, Y. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415223 · Cited by 66 · cited 1×
“Relevant to our work, Yeonheebot performs conversations with older adults to mitigate their depression and anxiety [65].”
BackgroundCASS: Towards Building a Social-Support Chatbot for Online Health Community
Wang, L., Wang, D., Tian, F., Peng, Z., Fan, X., Zhang, Z., Yu, M., Ma, X., & Wang, H. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449083 · Cited by 65 · cited 1×
“In particular, chatbots have been proposed as efective tools for scaling abilities to provide informational and emotional support around health [42, 76].”
MethodLong Time No See! Open-Domain Conversation with Long-Term Persona Memory
Xu, X., Gou, Z., Wu, W., Niu, Z.-Y., Wu, H., Wang, H., & Wang, S. (2022) · arXiv · doi:10.48550/arxiv.2203.05797 · Cited by 65
Making family care work: dependence, privacy and remote home monitoring telecare systems
Vines, J., Lindsay, S., Pritchard, G. W., Lie, M., Greathead, D., Olivier, P., & Brittain, K. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493469 · Cited by 63 · cited 2×
“A core concern is that existing technologies have predominantly targeted individuals who have readily accessible social contacts, such as informal caregivers [15, 51, 64, 74].”
BackgroundAI in Global Health: The View from the Front Lines
Ismail, A., & Kumar, N. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445130 · Cited by 59 · cited 4×
“A range of systems, including chatbots [27, 81] and mobile apps [44, 45], have been proposed and examined to support care recipients in self-reporting aspects of their health and well-being to public and community health infrastructures.”
BackgroundInfluentialPendse, S. R., Sharma, A., Vashistha, A., De Choudhury, M., & Kumar, N. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445410 · Cited by 59 · cited 1×
“[56] highlighted that institutional limitations often interfere with providing support through helpline systems, suggesting that automating some aspects of these systems could help care recipients better navigate the barriers.”
BackgroundPark, S., Thieme, A., Han, J., Lee, S., Rhee, W., & Suh, B. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462143 · Cited by 57 · cited 2×
“Specifcally, studies have designed chatbots for eliciting self-disclosure [40, 41, 55] or increasing self-compassion by taking care of chatbots that experience distress [30, 38].”
BackgroundFoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input
Luo, Y., Kim, Y.-H., Lee, B., Hassan, N., & Choe, E. K. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462074 · Cited by 56
Self-citationDivided We Stand: The Collaborative Work of Patients and Providers in an Enigmatic Chronic Disease
Pichon, A., Schiffer, K., Horan, E. N., Massey, B., Henry, S. B., Mamykina, L., & Elhadad, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3434170 · Cited by 54
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models
Bae, S., Kwak, D., Kim, S., Ham, D., Kang, S., Lee, S., & Park, W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.155 · Cited by 54
Bridging Disconnected Knowledges for Community Health
Ismail, A., Karusala, N., & Kumar, N. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274344 · Cited by 53 · cited 1×
“Further, research often does not capture the attitudes of the people on the front lines of using these technologies, such as community health workers, to understand the technology’s benefts and tradeofs [26].”
BackgroundFrom checking on to checking in: designing for low socio-economic status older adults
Arreola, I., Morris, Z., Francisco, M., Connelly, K., Caine, K., & White, G. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557084 · Cited by 46 · cited 1×
“However, studies have pointed out that compared to high socioeconomic status (SES) individuals, low-SES individuals living alone tend to have fewer social contacts that they can reach out to in emergency situations [1, 78], refecting important diferences in how to approach designing technology to support this more vul…”
BackgroundDesigning and Evaluating a Chatbot for Survivors of Image-Based Sexual Abuse
Maeng, W., & Lee, J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517629 · Cited by 45 · cited 1×
“Prior work on chatbots for mental health indicated that expectation management around the system capabilities is challenging but critical [41, 48, 55].”
BackgroundLuo, Y., Lee, B., & Choe, E. K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376616 · Cited by 44
Lazar, A., Thompson, H. J., Lin, S.-Y., & Demiris, G. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274372 · Cited by 42 · cited 2×
“One subset of these systems is often referred to as telecare systems, which seek to mediate care among individuals living alone, formal and informal caregivers, and emergency services [37, 62].”
BackgroundKuoppamäki, S. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06944-w · Cited by 39 · cited 2×
“Prior work has indicated that municipalities frequently have diferent public health needs from others based on their demographics and organizational capacity [17, 36].”
BackgroundMyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech
Kim, Y.-H., Chou, D., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517457 · Cited by 39 · cited 1×
“[31] have proposed a speech-based smartwatch app to assist older adults in labeling physical activities with a low capture burden.”
BackgroundSelf-citationDo Neural Language Models Overcome Reporting Bias?
Shwartz, V., & Choi, Y. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.605 · Cited by 37 · cited 1×
“Also, there exists a possibility that the unethical or biased phrases ingrained in the models’ pre-training datasets might be exposed in the model’s output, causing the chatbot to say socially biased [6, 7, 21, 67, 68] or toxic [22] messages.”
BackgroundKim, T., Ruensuk, M., & Hong, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376743 · Cited by 36 · cited 1×
“Specifcally, studies have designed chatbots for eliciting self-disclosure [40, 41, 55] or increasing self-compassion by taking care of chatbots that experience distress [30, 38].”
Background(2022) · doi:10.1145/3491102.3501868 · Cited by 36 · cited 1×
“Furthermore, collecting private data, such as everyday conversations, for machine learning purposes involves privacy concerns [71], particularly with marginalized populations [53].”
BackgroundBartle, V., Lyu, J., El Shabazz-Thompson, F., Oh, Y., Chen, A. A., Chang, Y.-J., Holstein, K., & Dell, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517683 · Cited by 35 · cited 2×
“Prior work on HCI and CSCW has highlighted the challenges in balancing multiple stakeholders’ needs when using new technology in complex care settings [28, 57, 61], suggesting the need for mechanisms to assist each stakeholder in voicing and negotiating their needs [5, 28].”
BackgroundJo, E., Park, S., Bang, H., Hong, Y., Kim, Y., Choi, J., Kim, B.-N., Epstein, D. A., & Hong, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512939 · Cited by 33 · cited 1×
“Prior work on HCI and CSCW has highlighted the challenges in balancing multiple stakeholders’ needs when using new technology in complex care settings [28, 57, 61], suggesting the need for mechanisms to assist each stakeholder in voicing and negotiating their needs [5, 28].”
BackgroundSelf-citationKeep Me Updated! Memory Management in Long-term Conversations
Bae, S., Kwak, D., Kang, S., Lee, M. Y., Kim, S., Jeong, Y., Kim, H., Lee, S.-W., Park, W., & Sung, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.276 · Cited by 29
GPT3-to-plan: Extracting plans from text using GPT-3
Olmo, A., Sreedharan, S., & Kambhampati, S. (2021) · arXiv · doi:10.48550/arxiv.2106.07131 · Cited by 27 · cited 1×
“, [32, 54]), and writing programming code (e.”
Shared Understanding in Care Coordination for Children's Behavioral Health
Richards, O. K., Choi, A., & Marcu, G. M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449095 · Cited by 24
An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers
Wang, L., Mujib, M. I., Williams, J., Demiris, G., & Huh-Yoo, J. (2021) · arXiv · doi:10.48550/arxiv.2107.13115 · Cited by 21 · cited 2×
“found that the chatbot was likely to provide more negative comments than the human therapists would [75].”
BackgroundLimited but satisfied: Low SES older adults experiences of aging in place
White, G., Singh, T. G., Caine, K. E., & Connelly, K. (2015) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/icst.pervasivehealth.2015.259095 · Cited by 20 · cited 1×
“However, studies have pointed out that compared to high socioeconomic status (SES) individuals, low-SES individuals living alone tend to have fewer social contacts that they can reach out to in emergency situations [1, 78], refecting important diferences in how to approach designing technology to support this more vul…”
BackgroundExperiences of Delivering a Public Health Data Service
Simpson, E. L., Comber, R., Garbett, A., Jenkins, E., & Balaam, M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025881 · Cited by 13 · cited 3×
“One major line of the research on technology interventions in public health settings has focused on automating aspects of care that public health workers typically have to provide manually, such as answering common questions [81] and identifying public resources [4, 69].”
BackgroundUnderstanding Cultural Influence on Perspectives Around Contact Tracing Strategies
Lu, X., Jo, E., Park, S., Hong, H., Chen, Y., & Epstein, D. A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555569 · Cited by 12 · cited 5×
“In the HCI and CSCW communities, a range of systems, including chatbots [27, 81] and mobile apps [44, 45] have been proposed and examined to support public health monitoring and intervention at scale.”
BackgroundInfluentialSelf-citationDiasso, D., Doudou, M. H., Levrak, M. C., Sedutto, H. D., & Savadogo, A. (2021) · Global Health Action · doi:10.1080/16549716.2021.1979279 · Cited by 3 · cited 1×
“Prior work has indicated that municipalities frequently have diferent public health needs from others based on their demographics and organizational capacity [17, 36].”
BackgroundLeveraging Pre-Trained Language Models to Streamline Natural Language Interaction for Self-TrackingPreprint
Kim, Y., Kim, S., Chang, M., & Lee, S.-W. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.15503 · Cited by 3 · cited 1×
“, [32, 54]), and writing programming code (e.”
Self-citationArtifically intelligent physics solver: This AI understands Newtons law
Nirmala, V., & Rajagopal, A. R. (2019) · Science & Technology Journal · doi:10.22232/stj.2019.07.01.03 · Cited by 1
Barbero (2014) · doi:10.1016/j · Cited by 1
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068 · cited 2×
“, GPT-3 [9], HyperCLOVA [29], PaLM [12], OPT [85]) with a larger number of parameters (e.”
Background
Research works printed but not linked: 15
- 10.18653/v1 · doi:10.18653/v1
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems (NeurIPS ’20), H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
- Hye-jin Byun. 2022. NAVER launches AI call service aimed at seniors - The Korea Herald. Retrieved Sep 15, 2022 from https://www.koreaherald.com/view.php?ud=20220530000643
- Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur. 2020. MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, Marseille, France, 422–428. https://aclanthology.org/2020.lrec-1.53
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.1145/3555569 · DOI
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.5555/3295222.3295349 · doi:10.5555/3295222.3295349
- 10.18653/v1 · doi:10.18653/v1
- Yonhap. 2017. Lonely deaths of middle-aged, youth brackets stand out amid single-person households - The Korea Herald. Retrieved Sep 15, 2022 from https://www.koreaherald.com/view.php?ud=20171207000623
- 10.18653/v1 · doi:10.18653/v1
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
“Much of the recent work on generative AI with human participants, however, tends to focus on attitudes towards the tool 44,45 , interviews or think-alouds with participants 46–48 , or user experience instead of task performance 49–51 .”
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3643540 · Cited by 278
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E., Malin, B., & Yin, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/22769 · Cited by 165
“Jo et al [2] Figure 2.”
InfluentialEnhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
Yang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Metadata
- Authors
- Jo, E., Epstein, D. A., Jung, H., & Kim, Y.-H.
- Affiliations
- University of California, Irvine (US)
- NAVER Cloud (South Korea) (KR)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-16
- OpenAlex
- W4366591012
- Cited by
- 174
- References
- 70
- Citations by year
- 2026: 30 · 2025: 66 · 2024: 68 · 2023: 9
- FWCI
- 86.45
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- large language models, conversational AI, chatbots, public health interventions, loneliness, empathetic interaction, stakeholder conflict, emotional burden
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.