CoChat

Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Intervention

Jo, Eunkyung, Jeong, Yuin, Park, Sohyun, Epstein, Daniel A., Kim, Young-Ho · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 84

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.

Abstract

Recent large language models (LLMs) offer the potential to support public health monitoring by facilitating health disclosure through open-ended conversations but rarely preserve the knowledge gained about individuals across repeated interactions. Augmenting LLMs with long-term memory (LTM) presents an opportunity to improve engagement and self-disclosure, but we lack an understanding of how LTM impacts people’s interaction with LLM-driven chatbots in public health interventions. We examine the case of CareCall—an LLM-driven voice chatbot with LTM—through the analysis of 1,252 call logs and interviews with nine users. We found that LTM enhanced health disclosure and fostered positive perceptions of the chatbot by offering familiarity. However, we also observed challenges in promoting self-disclosure through LTM, particularly around addressing chronic health conditions and privacy concerns. We discuss considerations for LTM integration in LLM-driven chatbots for public health monitoring, including carefully deciding what topics need to be remembered in light of public health goals.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642420

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 50

  1. 10.18653/v1 · doi:10.18653/v1
  2. 10.18653/v1 · doi:10.18653/v1
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems (NeurIPS ’20), H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  4. Hye-jin Byun. 2022. NAVER launches AI call service aimed at seniors - The Korea Herald. Retrieved Sep 14, 2023 from https://www.koreaherald.com/view.php?ud=20220530000643
  5. Character AI. 2023. Character AI. Retrieved Sep 14, 2023 from https://character.ai/
  6. 10.13140/RG.2.2.35753.70240 · DOI
  7. Samuel Rhys Cox Yi-Chieh Lee and Wei Tsang Ooi. 2023. Comparing How a Chatbot References User Utterances from Previous Chatting Sessions: An Investigation of Users’ Privacy Concerns and Perceptions. http://arxiv.org/abs/2308.04879 arXiv:2308.04879 [cs]. · arXiv
  8. 10.1145/3543829.3543831 · DOI · Paper page
  9. 10.1177/160940691301200137 · DOI · Paper page
  10. 10.13140/RG.2.2.36112.92165 · DOI
  11. Google, Inc. 2023. Bard - Chat Based AI Tool from Google, Powered by PaLM 2. Retrieved Sep 14, 2023 from https://bard.google.com/
  12. 10.1145/3308558.3313432 · DOI
  13. Robert B Hays. 1985. A Longitudinal Study of Friendship Development. (April 1985).
  14. 10.1093/joc · doi:10.1093/joc
  15. Inflection AI. 2023. Pi, your personal AI. Retrieved Sep 14, 2023 from https://pi.ai/talk
  16. 10.1145/3274344 · DOI · Paper page
  17. 10.1145/3411764.3445130 · DOI · Paper page
  18. 10.1145/3610203 · DOI
  19. 10.3115/1620754.1620847 · DOI · Paper page
  20. 10.1002/cav.345 · DOI · Paper page
  21. Evangelos Karapanos Jens Gerken Jesper Kjeldskov and Mikael B. Skov (Eds.). 2021. Advances in Longitudinal HCI Research. Springer International Publishing Cham. https://doi.org/10.1007/978-3-030-67322-2 · doi:10.1007/978-3-030-67322-2
  22. 10.18653/v1 · doi:10.18653/v1
  23. 10.1145/3491102.3502116 · DOI · Paper page
  24. Korean Statistical Information Service. 2023. Resident Population in Five-Year Age Groups. Retrieved Nov 14, 2023 from https://kosis.kr/statHtml/statHtml.do?orgId=101%26tblId=DT_1B04005N%26vw_cd=MT_ETITLE%26list_id=A_7%26scrId=%26language=en%26seqNo=%26lang_mode=en%26obj_var_id=%26itm_id=%26conn_path=MT_ETITLE%26path=%2Feng%2FstatisticsList%2FstatisticsListIndex.do
  25. 10.3389/fpsyg.2019.00558 · DOI · Paper page
  26. 10.18653/v1 · doi:10.18653/v1
  27. 10.1145/3290605.3300932 · DOI · Paper page
  28. 10.1145/3392836 · DOI · Paper page
  29. 10.1177/10944281221134096 · DOI
  30. 10.1145/3555569 · DOI
  31. 10.1145/3411764.3445669 · DOI
  32. 10.1007/978-3-319-67744-6_28 · DOI · Paper page
  33. 10.1145/3411764.3445420 · DOI · Paper page
  34. OpenAI, Inc. 2022. ChatGPT. Retrieved Sep 14, 2023 from https://chat.openai.com
  35. OpenAI, Inc. 2023. Replika. Retrieved Nov 14, 2023 from https://replika.com/
  36. 10.2196/12231 · DOI · Paper page
  37. 10.1145/3461778.3462143 · DOI · Paper page
  38. 10.1145/3544548.3581346 · DOI
  39. 10.1016/j.chb.2016.08.004 · DOI
  40. Manuel Portela and Carlos Granell-Canut. 2017. A new friend in our Smartphone? Observing Interactions with Chatbots in the search of emotional engagement. (2017). · DOI · Paper page
  41. 10.18653/v1 · doi:10.18653/v1
  42. 10.18653/v1 · doi:10.18653/v1
  43. 10.1145/3041021.3054172 · DOI · Paper page
  44. 10.2105/AJPH.2015.302696 · DOI · Paper page
  45. 10.1109/TVCG.2015.2467831 · DOI
  46. 10.1037/0022-0167.50.3.351 · DOI · Paper page
  47. 10.18653/v1 · doi:10.18653/v1
  48. Yonhap. 2017. Lonely deaths of middle-aged, youth brackets stand out amid single-person households - The Korea Herald. Retrieved Sep 14, 2023 from https://www.koreaherald.com/view.php?ud=20171207000623
  49. Zhonghua Zheng Lizi Liao Yang Deng and Liqiang Nie. 2023. Building Emotional Support Chatbots in the Era of LLMs. arxiv:2308.11584 [cs.CL] · arXiv
  50. Wanjun Zhong, Lianghong Guo, Qiqi Gao, and Yanlin Wang. 2023. MemoryBank: Enhancing Large Language Models with Long-Term Memory. arXiv preprint arXiv:2305.10250 (2023). · arXiv

Cited by

See all 84

Metadata

Authors
Jo, E., Jeong, Y., Park, S., Epstein, D. A., & Kim, Y.-H.
Affiliations
  • University of California, United States
  • NAVER Cloud, Korea, Republic of
  • Informatics, University of California, United States
  • NAVER AI Lab, Korea, Republic of
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-21
Cited by
84
References
30

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.