Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Open accessThe DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index; 6 of the 60 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs.
Overview
CoChat Summary
TL;DR
This paper explores whether GPT-3-powered chatbots can collect self-reported health data through naturalistic conversation by varying prompt structure and persona. In an online study (N=48), zero-shot chatbots covered 79% of desired information slots, with prompt design, topic, and conversation path significantly affecting slot filling and dialogue style.
Key findings
- Zero-shot GPT-3 chatbots covered 79% of desired information slots overall; slot-filling rates were SP 83%, SN 77%, DP 72%, DN 83%, with the highest SP-Exercise condition at 93% and the lowest SN-Work condition at 64%.
- Prompt design affected question-asking: structured format increased required-question turn ratio (p = .03) and decreased secondary-question turn ratio (p = .002); descriptive format produced more acknowledging and appreciating turns; the personality modifier increased sympathizing turns (p = .002).
- Topic and conversation path mattered: Exercise had the highest slot-filling rate (88.4%), significantly higher than Sleep (p = .01), Work (p < .0001), and Food (p = .02); the Positive path had higher slot filling (p = .01) and required-question ratio (p = .01), while the Negative path had higher acknowledging (p < .001) and sympathizing (p < .0001) turn ratios.
- Errors were present: 6.7% of chatbot turns (257/3,916) were erroneous, including repetition (3.8%), self-talk (1.1%), incorrect phrases (0.79%), and miscellaneous errors (0.9%); 75.7% of conversations ended organically. In exit surveys, 31.3% rated 5 and 50.0% rated 4 on understanding; 22.9% rated 5 and 64.6% rated 4 on acknowledging; empathy ratings had a median of 3, with no significant differences among conditions.
Methods
- Built 16 GPT-3 (davinci-text-002) chatbots: 4 topics (sleep, food intake, work/productivity, exercise) × 2 information formats (structured vs. descriptive) × 2 personality modifiers (present vs. absent); prompts included job identity and information slots, and ran on a web chat interface.
- Online between-subject study: 48 participants (aged 19–56, 18 females; 12 per condition) each assigned one prompt design and completed 8 conversations (4 topics × positive/negative paths), yielding 374 dialogues and 7,442 turns.
- Analysis: manual slot-filling coding; dialogue-act coding with Cohen’s Kappa 0.96 for essential acts and 0.935 for empathy/engagement; mixed-effect models with backward elimination and post-hoc least-squared means; Kruskal-Wallis tests for exit survey ratings.
Limitations
- Paper-stated: limited participants and imperfect counterbalancing; Food and Sleep always came after Work and Exercise, and Positive always preceded Negative, so fatigue/order effects may remain; some participants may not have followed paths; slots covered only time, scale, binary, and open-ended types; topic differences may reflect lexicon and data-type composition; participant conversation style was uncontrolled.
- Paper-stated: only GPT-3 davinci-text-002 and zero-shot prompts were tested, so prompts may not generalize to other LLMs; the study focused on chatbot performance/behaviors rather than user experience.
- Evident from the text: single LLM and small sample limit generalizability; no neutral path was tested; no ending-detection algorithm was implemented (participants clicked “Next”); the system went offline 5 times due to API calls, and repetition/errors may affect data quality.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 9 Not valid · 37 Not checked · 251 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Recent Large Language Models (LLMs; e.g. , GPT-3 [17], PaLM [23], OPT [118], HyperCLOVA [41]), with billions of parameters pre-trained on a large amount of language corpora, provide new opportunities for conversational agents.”
What the sources say
- Not checked yetcites [17] [17] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877-1901. -
- Supportedcites [23] [23] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, H source
The citation is an identification one: PaLM is listed as an example of a recent LLM with billions of parameters pre-trained on a large language corpus. Our copy states exactly that in the Introduction (p. 3): a 540 billion parameter, densely activated autoregressive Transformer trained on 780 billion tokens of high-quality text, with state-of-the-art few-shot results. The related-work section (p. 47) also ties scale to conversational/dialogue capability ('dialogue applications have advanced through scale'), which covers the framing half of the citing sentence; the 'new opportunities for conversational agents' conclusion itself is the citing authors' inference rather than PaLM's claim. (LLM, conf=0.85)
- Supportedcites [118] source
OPT is exactly the kind of thing the citing sentence lists it as: the Abstract (p. 1) and Section 2.1 (p. 2) describe OPT as decoder-only transformers from 125M to 175B parameters, and Section 2.3 (p. 2) documents a pre-training corpus of roughly 180B tokens drawn from RoBERTa, the Pile and Reddit data. The conversational-agent part is treated in Section 3.2 Dialogue (p. 5), which states LLMs 'are known to be an integral component of modern dialogue models' and shows unsupervised OPT-175B competitive with the fully supervised BlenderBot 1 on ConvAI2 and other dialogue datasets. The paper does not use the phrase 'new opportunities' — that framing is the citing authors' — and it adds safety caveats (Section 4.5, p. 8), but nothing contradicts the attribution. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While the HCI and CSCW community have actively explored the use of LLMs in various domains ( e.g. , [24, 53, 114]), research that leverages LLMs for powering chatbots, particularly taskoriented ones [12, 72, 103], is still sparse.”
What the sources say
- Not checked yetcites [12] [12] Sanghwan Bae, Donghyun Kwak, Sungdong Kim, Donghoon Ham, Soyoung Kang, Sang-Woo Lee, and Woomyoung Park. 2022. Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models. arXiv preprint arXiv:2205.00176 (2022). Jing Wei, Sungdong Kim, Hyunhoon Jung, and Young-Ho Kim -
- Not checked yetcites [72]
- Partly supportedcites [103] sourceExperimental — accuracy not yet measured
Our copy supports only the existence/description of LLM-powered chatbot work with task-oriented capability: the Abstract (p. 1) shows a Codex (GPT-3 finetuned on GitHub) agent that holds multi-turn conversations and answers questions about structured Minecraft data, and the Conclusion (p. 7/9) states 'Codex-powered NPCs can integrate both conversational and task-oriented language interactions almost seamlessly with code generation.' Nothing in our copy asserts that such research is 'still sparse'; the closest field-level statement, in Related Work (p. 3), runs the other way — that prompting for dialog generation 'is rapidly being mainstreamed' — and the related-work survey (pp. 2-3) lists several dialogue/prompting systems (DialoGPT, Meena, PLATO-XL, LaMDA, Madotto et al. 2021, Zheng and Huang 2021) rather than a scarcity of task-oriented chatbot work. So the sparsity characterisation attributed here is the citing authors' own judgement, not this paper's claim. (LLM, conf=0.68)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“In terms of utilizing mainstream chatbot platforms to build voice applications, it is of great human effort to come up with user intents and example phrases as well as design conversation flows and logic [27, 68].”
What the sources say
- Could not verifycites [27] [27] Tilman Dingler, Dominika Kwasnicka, Jing Wei, Enying Gong, and Brian Oldenburg. 2021. The use and promise of conversational agents in digital health. Yearbook of Medical Informatics 30, 01 (2021), 191-199. - sourceExperimental — accuracy not yet measured
Section 2 describes tools such as Dialogflow and says they do not require specific programming knowledge. Section 3.3 says custom agents built with open-source frameworks require expertise and development resources, but neither passage addresses whether mainstream platforms require substantial human effort to devise intents, example phrases, and conversation flows. Searches for those terms and for manual design effort found no supporting passage. (LLM, conf=0.72)
- Not checked yetcites [68]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Despite drawbacks discussed above, LLMs-based chatbots can become a valuable and scalable tool for researchers to collect data for personal informatics [35].”
What the sources say
- Could not verifycites [35] sourceExperimental — accuracy not yet measured
Our copy of Homewood et al. (2020) is a fourth-wave HCI methods paper: it proposes 'removal as a method' built on autoethnography, interviews and participant reflection on menstrual-cycle tracking apps (INTRODUCTION p. 1; 'How to use the method' pp. 9-10), and discusses personal informatics data collection only via existing approaches such as photo elicitation, surveys, diaries and autoethnographies (p. 2). No passage mentions LLMs, chatbots, conversational agents or scalability of data collection (searched: chatbot/LLM/conversational agent, artificial intelligence, scalable data collection, valuable method for researchers). The closest statement even cautions against such a generalisation: 'this method would have to be applied to many different cases before we could make generalized statements about its value, limitations and implications as a fourth-wave method' (EXPANDING AND APPLYING REMOVAL AS A METHOD, p. 9). So the claim about LLM-based chatbots cannot be checked against this source. (LLM, conf=0.80)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 1,482 out of 7,442 turns ( 𝑎𝑣𝑔. 18.45% per dialogue) were classified as Statement (See Table 5).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
87 more claims
Create a free account to see all 300 claims and to verify the 54 references still open.
- Probably not a substantial claim. 154
- It may state two separate claims as one. 91
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As pointed out by existing work in LLMs and natural language processing, there are restrictions to the number of tokens input to the LLMs [6], limiting the length of conversation history texts that could be included in LLM prompting [12,70,116].”
From Talk2Care: An LLM-based Voice Assistant for Communication between Healthcare Providers and Older Adults (2024) · cites this paper as [116]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.2 Slot Filling Rate · p. 12
“On the other hand, there were significant randomeffects of the topics( 𝑝 < . 0001), the conversation paths ( 𝑝 = . 01), and the interaction between the information formats and personality modifiers ( 𝑝 < . 001).”
§ 3.4 Data Analysis · p. 11
“Starting from a full model containing participants as a random effect and the four main study factors-information format, personality modifier, topic, and path-and their interactions as fixed effects”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“In total, 6.7% of the chatbot turns (257 out of 3,916) were tagged erroneous”
Reported 6.7 · recomputed 6.562819
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.4 Data Analysis · p. 11
“To assess the difference among the experimental conditions, we conducted Kruskal-Wallis tests over the four rating questions.”
§ 3.2 Online Study · p. 9
“The survey consisted of three 5-point Likert scale questions and one open-ended feedback textfield.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“in 31 dialogues (SP: 3, SN: 9, DP: 8, DN:10).”
Reported 31 · recomputed 30
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 10×
“LLMs work differently from traditional task-specific models as they accept prompts written in natural language as input to perform various tasks [14, 60].”
BackgroundMethodInfluentialMixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187
emmeans: Estimated Marginal Means, aka Least-Squares Means
Lenth, R. V., & Piaskowski, J. (2017) · CRAN: Contributed Packages · doi:10.32614/cran.package.emmeans · Cited by 8,143
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Self-citationLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 2×
“, [55]), given that LLMs can be sensitive to prompt designs [60], we reckon that our proposed prompts may not yield similar performance on other LLMs due to the differences in the training corpora and the model architecture.”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 2×
“, [55]), given that LLMs can be sensitive to prompt designs [60], we reckon that our proposed prompts may not yield similar performance on other LLMs due to the differences in the training corpora and the model architecture.”
BackgroundFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“to help people develop long-term adoptions for health monitoring [23, 26, 72].”
BackgroundPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 2×
“, GPT-3 [17], PaLM [23], OPT [118], HyperCLOVA [41]), with billions of parameters pre-trained on a large amount of language corpora, provide new opportunities for conversational agents.”
BackgroundSelf-citationLarge language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · cited 1×
“We kept the temperature and the presence penalty unchanged based on OpenAI’s suggestions and increased the frequency penalty to reduce the re-use of words.”
MethodLarge Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Research works printed but not linked: 23
- Apple Inc. 2022. SIRI shortcuts boost health and fitness routines - Apple News Room. Retrieved Dec 04, 2022 from https://www.apple.com/newsroom/2019/03/siri-shortcuts-boost-health-and-fitness-routines/
- Florian Bemmann, Ramona Schödel, Niels Van Berkel, and Daniel Buschek. 2021. Chatbots for Experience Sampling-Initial Opportunities and Challenges.. In IUI Workshops.
- Susan E Brennan. 1990. Conversation as direct manipulation: An iconoclastic view. The art of human-computer interface design (1990), 393--404.
- John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Sketching Stories with Generative Pretrained Language Models. In CHI Conference on Human Factors in Computing Systems. 1--19.
- Google. 2022. Google Assistant, Your Own Personal Google. Retrieved Dec 04, 2022 from https://assistant.google.com
- IBM. 2022. IBM Watson. Retrieved Dec 04, 2022 from https://www.ibm.com/watson
- Jae Ho Jeon. 2016. OmniTrack: Designing Flexible and Highly Customizable Quantified-Self Tool. MS thesis. Seoul National University Seoul Korea. http://www.riss.kr/link?id=T14226449
- Juju, inc. 2022. Cognitive AI Chatbot. Retrieved Dec 04, 2022 from https://juji.io/
- Min Kyung Lee, Sara Kiesler, Jodi Forlizzi, Siddhartha Srinivasa, and Paul Rybski. 2010. Gracefully mitigating breakdowns in robotic services. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 203--210.
- Yuhan Luo, Young-Ho Kim, Bongshin Lee, Naeemul Hassan, and Eun Kyoung Choe. 2021. FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input. In Designing Interactive Systems Conference 2021. 606--618.
- Michael McTear. 2018. Conversational modelling for chatbots: current approaches and future directions. Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung (2018), 175--185.
- Shikib Mehri, Yasemin Altun, and Maxine Eskenazi. 2022. LAD: Language Models as Data for Zero-Shot Dialog. arXiv preprint arXiv:2207.14393 (2022). · arXiv
- Meta Platforms, Inc. 2022. React -- A JavaScript library for building user interfaces. Retrieved Dec 04, 2022 from https://reactjs.org/
- Microsoft. 2022. TypeScript. Retrieved Dec 04, 2022 from https://www.typescriptlang.org
- Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? arXiv preprint arXiv:2202.12837 (2022). · arXiv
- Elliot Mitchell, Noemie Elhadad, and Lena Mamykina. 2022. Examining AI Methods for Micro-Coaching Dialogs. In CHI Conference on Human Factors in Computing Systems. 1--24.
- Ryan Morrison. 2022. GPT-3 developer OpenAI releases new Davinci generative text model. Retrieved Dec 04, 2022 from https://techmonitor.ai/technology/ai-and-automation/gpt-3-openai-davinci-generative-text
- OpenAI. 2022. OpenAI API. Retrieved Dec 04, 2022 from https://openai.com/api/
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207 (2018). · arXiv
- Kathryn Roulston and Myungweon Choi. 2018. Qualitative interviews. The SAGE handbook of qualitative data collection (2018), 233--249.
- Woosuk Seo Chanmo Yang and Young-Ho Kim. 2023. ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Events. arXiv:2309.12244 [cs.HC] · arXiv
- The OpenJS Foundation. 2022. Node.js. Retrieved Dec 04, 2022 from https://nodejs.org
- Anuradha Welivita and Pearl Pu. 2020. A taxonomy of empathetic response intents in human social conversations. arXiv preprint arXiv:2012.04080 (2020). · arXiv
Cited by
A Systematic Review of Generative AI for Teaching and Learning Practice
Ogunleye, B., Zakariyyah, K. I., Ajao, O., Olayinka, O., & Sharma, H. (2024) · Education Sciences · doi:10.3390/educsci14060636 · Cited by 251
Zhong, W., Luo, J., & Zhang, H. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.04.057 · Cited by 173
MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journaling
Kim, T., Bae, S., Kim, H. A., Lee, S.-W., Hong, H., Yang, C., & Kim, Y.-H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642937 · Cited by 112
“Recent LLMs have suggested a new paradigm of bootstrapping chatbots [93, 94].”
BackgroundInfluentialSelf-citationYang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Pandey, S., & Sharma, S. (2023) · Healthcare Analytics · doi:10.1016/j.health.2023.100198 · Cited by 77
Jo, E., Jeong, Y., Park, S., Epstein, D. A., & Kim, Y.-H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642420 · Cited by 84
“With their large-scale ( i.e. , over billions of) parameters trained with a tremendous amount of human-produced text corpus, LLMs tend to generate responses that coherently and organically follow up the conversation and flexibly respond to emergent topics [76].”
BackgroundSelf-citationSeo, W., Yang, C., & Kim, Y.-H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642152 · Cited by 66
“As long and complex input prompts tend to drop the task performance of LLMs [7, 79, 84], we used dedicated prompts for each phase instead of combining instructions for all phases in a single prompt [84].”
BackgroundMethodInfluentialSelf-citationEvolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Education
Krause, S., Panchal, B. H., & Ubhe, N. (2025) · Frontiers of digital education. · doi:10.1007/s44366-025-0058-7 · Cited by 89
Wu, R., Yu, C., Pan, X., Liu, Y., Zhang, N., Fu, Y., Wang, Y., Zheng, Z., Chen, L. Z., Jiang, Q., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642790 · Cited by 87
“Recent studies have explored various health applications supported by LLMs, such as health information seeking [91], mental health support [47, 51], personal health coaching [62, 83], and health education [48, 62].”
BackgroundPathways linking expectations for AI chatbots to loyalty: A moderated mediation analysis
Yao, X., & Xi, Y. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102625 · Cited by 50
Talking technology: exploring chatbots as a tool for cataract patient education
Yılmaz, İ. E., & Doğan, L. (2024) · Clinical and Experimental Optometry · doi:10.1080/08164622.2023.2298812 · Cited by 42
“35 However, one should keep in mind that challenges remain in real-world tasks, such as hallucinations and external knowledge.”
Abubakar, A. M., Gupta, D., & Parida, S. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.04.087 · Cited by 44
'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research Practices
Kapania, S., Wang, R., Li, T. J.-J., Li, T., & Shen, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711000 · Cited by 80
Song, I., Park, S., Pendse, S. R., Schleider, J. L., Choudhury, M. D., & Kim, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713883 · Cited by 78
“However, different types of guidance and interaction may be more effective for those who have progressed to the preparation stage, where they began taking small steps towards change.”
Self-citationFagbohun, O., Harrison, R. M., & Dereventsov, A. (2023) · Journal of Artificial Intelligence Machine Learning and Data Science · doi:10.51219/jaimld/oluwole-fagbohun/15 · Cited by 69
“Conversational prompting involves crafting prompts that mimic natural human conversation 15,47 .”
Taking the Chat out of Chatbot? Collecting User Reviews with Chatbots and Web Forms
Sachdeva, A., Kim, A., & Dennis, A. R. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2023.2301175 · Cited by 39
“Such LLM-based chatbots are also swiftly gaining traction [97].”
BackgroundChen, Z., Wang, Q., Sun, Y., Cai, H., & Lü, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104651 · Cited by 27
Xu, S., Chen, C., Liu, Z., Jin, X., Yuan, L.-P., Yan, Y., & Qu, H. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676336 · Cited by 37
“We did not directly prompt GPT-4V with the raw images of the entire photo collection due to token limits [81] and the negative impact of long input prompts on performance [13, 72, 76].”
Reviving Static Charts Into Live Charts
Ying, L., Wang, Y., Li, H., Dou, S., Zhang, H., Jiang, X., Qu, H., & Wu, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3397004 · Cited by 19
“Moreover, we follow established design guidelines [61], [62], [63] and develop distinct prompts tailored to each specific task.”
Gómez, C., Yin, J., Huang, C., & Unberath, M. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1427463 · Cited by 28
A review on polyadic chatbots: trends, challenges, and future research directions
Kuhail, M. A., Taj, I., Alimamy, S., & Shawar, B. A. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02287-0 · Cited by 32
Comparing Large Language Model AI and Human-Generated Coaching Messages for Behavioral Weight Loss
Huang, Z., Berry, M. P., Chwyl, C., Hsieh, G., Wei, J., & Forman, E. M. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00491-5 · Cited by 26
“Additionally, participants chose which messages they believed were written by ChatGPT and shared the strategies they used to identify them.”
Self-citationThe Impact of Prompt Programming on Function-Level Code Generation
Khojah, R., Neto, F. G. D. O., Mohamad, M., & Leitner, P. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3587794 · Cited by 76
“• Persona allows the LLM to play a specific role and consider its perspective when solving a problem [35], [36].”
Luo, Y., Gui, X., Ding, X., Zheng, X., Hernandez, R. H., Li, Z., & Song, Q. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735746 · Cited by 23
Chan, S., Li, J., Yao, B., Mahmood, A., Huang, C.-M., Jimison, H., Mynatt, E. D., & Wang, D. (2023) · arXiv · doi:10.48550/arxiv.2310.05853 · Cited by 16
The Evolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Education
Krause, S., Panchal, B. H., & Ubhe, N. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-97-9255-9_24 · Cited by 48
Micro-narratives: A Scalable Method for Eliciting Stories of People’s Lived Experience
Skeggs, A., Mehta, A., Yap, V., Ibrahim, S. B., Rhodes, C., Gross, J. J., Munson, S. A., Klasnja, P., Orben, A., & Slovak, P. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713999 · Cited by 14
Reimagining patient-reported outcomes in the age of generative AI
Boyer, L., Fernandes, S., Auquier, P., Falissard, B., & Panch, T. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02006-1 · Cited by 19
Nguyen, M. H., Sedoc, J., & Taylor, C. O. (2024) · Journal of Medical Internet Research · doi:10.2196/55164 · Cited by 20
“In a study exploring the use of LLMs to support chatbot collection of health self-reported data, Wei et al [63] present potential benefits of using an LLM-driven chatbot due to its abilities to ask specific follow-up questions, include social attributes, recover from errors, and track…”
Chopra, S., Juarez, K., Fogarty, J. A., & Munson, S. A. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749503 · Cited by 25
Chatbots talk Strabismus: Can AI become the new patient Educator?
Yılmaz, İ. E., Berhuni, M., Özcan, Z. Ö., & Doğan, L. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105592 · Cited by 14
Li, Z., Zhu, Z., Gui, X., & Luo, Y. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103555 · Cited by 28
AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking
Rashik, M., Sweth, S., Agrawal, N. A., Kochar, S., Smith, K. M., Rajabiyazdi, F., Setlur, V., Mahyar, N., & Sarvghad, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714280 · Cited by 20
“The rule-based classifier predicts a symptom based on the user’s input and then asks follow-up questions designed to probe specific topics related to that symptom.”
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“…task-oriented dialogue systems has shown to underperform in comparison with task-specific models, careful application is required for LLMs to be useful—as shown by [198] in automated scoring of user simulations, [199] in dialogue state tracking, and [200] in data collection via prompt engineering.”
BackgroundChatbot in education: trends, personalisation, and techniques
Jusoh, S., & Kadir, R. A. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20659-8 · Cited by 12
Foody Talk: Exploring Opportunities for Conversational Food Journaling
Silva, L. M., Lu, X., Liang, E. X., & Epstein, D. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713875 · Cited by 18
Sun, X., Liu, Y., Bosch, J. A., & Li, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711014 · Cited by 18
“These CUIs held the potential to revolutionize frontline medical services through automated data collection [162], patient consultations, and preliminary diagnostics [66], suggesting a transformative impact on healthcare delivery.”
Prompt Engineering Guidelines for Using Large Language Models in Requirements Engineering
Ronanki, K., Arvidsson, S., & Axell, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04207-1_17 · Cited by 30
Autorepo: A General Framework for Multimodal Llm-Based Automated Construction ReportingPreprint
Pu, H., Yang, X., Li, J., & Guo, R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4678046 · Cited by 10
RemVerse: Supporting Reminiscence Activities for Older Adults through AI-Assisted Virtual Reality
Li, R., Li, J., Sun, J., Wu, Z., Li, Z., Wang, Z., Lin, G., & Fan, M. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749505 · Cited by 17
Haberal, M., & Hançerlioğulları, D. (2026) · BMC Oral Health · doi:10.1186/s12903-025-07612-9 · Cited by 11
“In this study, the observed weak yet statistically significant positive correlation between question length and response time aligns with findings from numerous investigations into the operational mechanisms of language models [35–37].”
Influence of LLMs on Learning and Teaching in Higher EducationPreprint
Upadhyay, A., Farahmand, E., Muñoz, I., Khan, M. A., & Witte, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4716855 · Cited by 15
Near-Memory LLM Inference Processor based on 3D DRAM-to-logic Hybrid Bonding
Han, S., Yoon, B., Park, G., Song, C., Kim, D., & Kim, J.-J. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132870 · Cited by 10
“Transformer-based large language models (LLMs) [1]–[3] have emerged as pivotal components in various language processing applications, ranging from chatbots to machine translations [4]–[6].”
Guo, Z., Lai, A. G., Deng, Z., & Li, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-67278-1_20 · Cited by 11
SmartEats: Investigating the Effects of Customizable Conversational Agent in Dietary Recommendations
Liang, M., Wang, J., & Luo, Y. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736635 · Cited by 12
Chen, Y., Xiong, Y., Ding, X. X., Yang, F., Deng, Y., Ma, N., & Li, Y. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2527868 · Cited by 5
“These advanced models can effectively bootstrap chatbots with significant efficacy using only a few examples, or even none at all (Wei et al., 2024), thereby overcoming the limitations of traditional rule-based and retrieval-based approaches (Huang et al., 2020).”
Saturn Platform: Foundation Model Operations and Generative AI for Financial Services
Busson, A. J. G., Gaio, R., Rocha, R. H., Evangelista, F., Rizzi, B., Carvalho, L., Miceli, R., Rabaioli, M., & Favaro, D. (2023) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia_estendido.2023.234354 · Cited by 7
Lim, G., & Perrault, S. T. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681867 · Cited by 20
TellTime: An AI-Augmented Calendar with a Voice Interface for Collecting Time-Use Data
Hoefer, M. J. D., Rychecky, R., Gong, M., & Voida, S. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712116 · Cited by 10
SeSaMe: A Framework to Simulate Self-Reported Ground Truth for Mental Health Sensing Studies
Choube, A., Swain, V. D., & Mishra, V. (2024) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii63134.2024.00031 · Cited by 11
“In the past, researchers have attempted to make the questionnaire interfaces more engaging by using chatbots [23], [24], large language models (LLMs) [25], [26], and virtual agents [27], [28].”
Fu, Y., Li, B., Li, L., Zhang, W., & Xie, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728947 · Cited by 14
“The preliminary experiment compares the original manually designed prompt with three variants: COT (Chain of Thought) [72], Persona (assigning a specific role with its perspective to LLMs) [66], and Few-shot Learning [7].”
Fageeh, A. I. (2025) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2025.101996 · Cited by 12
Kishore, Y. (2024) · Intelligent Information Management · doi:10.4236/iim.2024.162005 · Cited by 4
You, C.-W., Yuan, C. W., Bi, N., Lin, H.-W., Lai, W.-N., Lu, H.-S., & Huang, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710975 · Cited by 3
“These devices could be configured to provide tips, unobtrusive reminders, or suggestions, based on the status self-reported by the patient during chatbot interactions [104].”
Pilli, S. (2023) · International Conference on Behavioral, Economic, and Socio-Cultural Computing · doi:10.1109/besc59560.2023.10386809 · Cited by 8
“We modify the Natural Language Generation (NLG) unit, which essentially uses a traditional (template or hand-crafted) [24] or contemporary approaches (LLMs) [25] [26] to generate decision tasks as utterance.”
MethodHalliday, S., Lavis, T., Callaghan, P., & Chur-Hansen, A. (2026) · Australian Psychologist · doi:10.1080/00050067.2025.2580236 · Cited by 2
“GenAI models extend on artificial intelligence models by being able to generate new and original content (in various forms like text, photos, videos) in response to user prompts (Wei et al., 2024).”
Soulbot: Chatbots for Mental Health Support using Gemini
Rakhma, N. A., & Sudianto, S. (2025) · International Conference on Computer Sciences, Engineering, and Technology Innovation (ICoCSETI) · doi:10.1109/icocseti63724.2025.11020143 · Cited by 2
“Previous studies have shown that LLM-based chatbots are effective in collecting user-reported self-data with high accuracy and relevance [15].”
Liang, M., & Luo, Y. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3737635 · Cited by 3
“First, the challenge lies in structuring the conversation to gather sufficient information and respond to users in a coherent and engaging flow, while encouraging them to share more relevant information [13, 26]—balancing these objectives may exceed the capacity of a single prompt.”
InfluentialWho Is Talking to the AI Assistants? A Consumer Segmentation Analysis
Fares, O. H., Yang, X., & Lee, S. H. (2026) · International Journal of Consumer Studies · doi:10.1111/ijcs.70219 · Cited by 2
Lin, Y.-C., Chen, Y.-M., Chuang, C. F., Yuan, C. W., Bi, N., Chang, H., Huang, M., & You, C.-W. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720128 · Cited by 1
“Chatbots can be personalized and foster trust through anthropomor-phism [48], linguistic style adaptation [2], and prompt engineering [49].”
SafetyBuilder: An AR-based Framework for In-situ AI-assisted Creation of Child Safety Protection
Li, J., Li, Z., Chen, S., Wang, Z., Zhang, Y., Fan, M., & He, L. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3789690 · Cited by 2
OntoChat Assistant for User Story Generation in Ontology Engineering
Zhao, Y., Kurteva, A., Meroño-Peñuela, A., & Simperl, E. P. B. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3810943 · Cited by 2
Tijani, O., Mahoney, J., Vijaykumar, S., & Nicholson, J. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817016 · Cited by 1
Rodríguez‐Sánchez, M. J., Callejas, Z., Ruíz-Zafra, Á., & Benghazi, K. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0805-8_13 · Cited by 1
IntervEEG-LLM: Exploring EEG-Based Multimodal Data for Customized Mental Health Interventions
Zhang, S., Hu, Y. '., Yi, X., Nanayakkara, S. C., & Chen, X. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717550 · Cited by 7
“Researchers have explored the application of LLMs in various healthcare user modeling and intervention tasks, such as mental health support [21, 22], health information seeking [25, 38, 45], public health interventions [15] and home-care support [44].”
Harnessing generative AI for quality of life assessment: Foundations for a new research agenda
Zhang, S., Fernandes, S., Boussat, B., Sollari, J., Tran, B. X., Do, H. P., Nguyen, C. T., Auquier, P., & Boyer, L. (2025) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2025.203156 · Cited by 1
Evaluating LLMs in ROS robotic software code generation
Zhao, Y., Mao, X., Zhang, T., Li, T., & Xiao, Z. (2027) · Empirical Software Engineering · doi:10.1007/s10664-026-10910-7
“The system prompt enables the LLM to adopt a specific role and approach problems from a designated perspective (Wei et al. 2024 ).”
A Sentence‐Level Encoder–Decoder Architecture for Designing an Administrative Roman Urdu Chatbot
Maqbool, M. N., Saleem, R. M., Sarwar, N., Ibrahim, M., & Hashmi, M. S. A. (2025) · Applied Computational Intelligence and Soft Computing · doi:10.1155/acis/4728280 · Cited by 1
Generative AI for Personalized Learning Experiences in Education
Sarkar, S. B., Robin, S., Gokulrajan, R., Jitheshvar, M., & Kudiyarasu, J. (2025) · doi:10.1109/iccct63501.2025.11020445
“This makes it easier for students to do their work [7].”
AI-Associate: A Lightweight Architecture for Conversational Agents
Gani, U. A. S., Madankar, A., Shinde, A., Sonwane, A., & Durbade, S. (2025) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon65728.2025.11070569
A Conceptual NLP Based Chatbot for Enhancing Sustainability Among Craftsmen
Tejas, H., & Gururaja, B. L. (2025) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom66777.2025.11115634
“The process for the training of Kalasrijan Chatbot starts with the collection and preprocessing of data [40].”
Evaluating the Potential of Data-Driven Surveys for Fitness-Tracking Research
Velykoivanenko, L., Niksirat, K. S., Abdrabo, A., & Huguenin, K. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770665
SurveyTalk: Voice-to-UI Mapping for In-Situ ESM via LLMs
Lee, H., Kim, E. G., Lee, J., Joo, H., & Kwak, S. S. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746058.3758377
LISLLM: Long Context Inference of Large Language Models with Short KV Cache
Liu, T., Li, G., Zhu, Z., Mo, Z., Yao, X., & Cheng, J. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323105
“They have been widely applied in dialogue systems [10], [11], chatbots [12], [13], text summarization [14], code completion [8], [9], healthcare [5]–[7], etc.”
Shih, M. T., Wu, P. Y., Chang, Y. C., Yen, G. Y.-C., & Chan, L. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789171 · Cited by 2
“Advances in LLMs have transformed how researchers analyze and collect survey data [31, 43, 52, 67].”
Li, Z., Li, R., Li, J., Liu, C., Zhu, J., Rus, D. L., Liang, C., & Fan, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790590 · Cited by 2
Conversational Inoculation to Enhance Resistance to Misinformation
Szabó, D., Yang, C.-L., Visuri, A., Oppenlaender, J., Sekar, B., Yatani, K., & Hosio, S. J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790954 · Cited by 1
The Effectiveness of Personal Data Detection in LLM-Based Conversational Agents
Paracuellos, D., Such, J. M., del Val, E., & García‐Fornes, A. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19770-2_9
Liu, J., Chen, Z., Wang, Q., Xie, T., Zheng, H., & Lv, X. (2025) · International Conference on Computational Biology and Bioinformatics (ICCBB) · doi:10.1145/3789938.3789954
“Wei et al. [13] showed that prompt design significantly affects the performance of self-report data collection.”
Yang, Z., Lu, Y., Bagdasarian, J. L., Swain, V. D., Agarwal, R., Campbell, C., Al‐Refaie, W. B., El-Bayoumi, J., Gao, G. G., Wang, D., et al. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3788068 · Cited by 7
Huang, M., Lin, H., & Zhou, H. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2686796
PersuAgent: A Dual-Route Expressive Agent Driven by LLM for Healthful Lifestyle Guidance
Yuan, H., Liu, R., Deng, C., Yang, J., Ge, Y., Yu, B., & Wang, Q. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-27582-0_2
Designing an LLM-Based Diary Platform as a Research Tool
Chen, Y.-M., & Tung, F.-W. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30836-8_12
Shiu, E., Tieu, A., Xiong, J., Qi, L., Munteanu, C., & White, K. S. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816275
Abbas, K. F., Malvankar‐Mehta, M. S., Juncal, V., Hooper, P. L., & Sheidow, T. G. (2026) · Journal of VitreoRetinal Diseases · doi:10.1177/24741264261460669
Iacobelli, F., Reyna, C. R., Antobre, A., Gonzalez, N., Fuente, C. D. L., & Balyan, R. (2026) · Health Education & Behavior · doi:10.1177/10901981261490851
Metadata
- Authors
- Wei, J., Kim, S., Jung, H., & Kim, Y.-H.
- Affiliations
- The University of Melbourne (AU)
- NAVER Cloud (South Korea) (KR)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2024-04-17
- Volume
- 8
- Issue
- CSCW1
- Pages
- 1-35
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4396219010
- Cited by
- 84
- References
- 74
- Citations by year
- 2026: 17 · 2025: 37 · 2024: 25
- FWCI
- 11.70
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (public-domain)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- large language models, chatbots, prompt engineering, naturalistic conversation, personas, conversation flow
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.