The DOI matches its Crossref record, with a small difference in title or year; 78 of the 80 references it cites were found in a registry or an index; 32 of the 75 sources its claims cite were checked for support.
Situated Understanding of Errors in Older Adults' Interactions with Voice Assistants: A Month-Long, In-Home Study
Open accessAbstract
Our work addresses the challenges older adults face with commercial Voice Assistants (VAs), notably in conversation breakdowns and error handling. Traditional methods of collecting user experiences—usage logs and post hoc interviews—do not fully capture the intricacies of older adults’ interactions with VAs, particularly regarding their reactions to errors. To bridge this gap, we equipped 15 older adults’ homes with smart speakers integrated with custom audio recorders to collect “in-the-wild” audio interaction data for detailed error analysis. Recognizing the growing use of Large Language Models (LLMs) to enhance the capabilities of VAs, our study also explored how this integration of LLMs changes older adults’ interaction dynamics, specifically during errors. Midway through our study, we deployed ChatGPT-powered VA to investigate its efficacy for older adults. Our research suggests that while technical improvements—such as leveraging vocal and verbal responses combined with LLMs’ contextual capabilities—can enhance error prevention and management in VAs, interaction-level challenges still remain, particularly those unique to older adults. We propose design considerations to better align future VAs with older adults’ expectations and lived experiences.
Overview
CoChat Summary
TL;DR
This four-week in-home study recorded older adults’ audio interactions with Amazon Alexa and found a true error rate of 24.76%, with most errors becoming conversational breakdowns and few resolved on the immediate retry. A ChatGPT-powered Alexa skill showed lower error rates and better resolution in limited use, but adoption was hindered by access complexity and unfamiliarity.
Key findings
- Across 2,552 user query–VA response pairs from 15 older adults, 632 queries (24.76%) had errors; 98.10% of these manifested as conversational breakdowns, and only 25.47% (160) were resolved on the immediate next attempt.
- The most frequent error type was intent recognition (n = 204, 32.3% of all errors), followed by VA limitation errors (n = 130, 20.6%); VA errors had the lowest resolution rate (20.44%), while human errors had the highest (38.9%).
- Participants recognized 79.84% of conversational breakdowns through reactions or corrective actions, but they reacted verbally or vocally to only 13.77% of errors (87 of 632); common reactions included negation (27), interruption (21), laughter (14), and negative comments (10).
- The ChatGPT+Alexa skill had a lower percentage of errors and a higher resolution rate of 14.1% versus 8.7% for standard information-seeking interactions, but only a few participants (P1, P3, P11, P15) interacted with it, mainly around deployment day.
Methods
- Four-week field deployment of a 5th-generation Amazon Echo Dot in 15 older adults’ homes, with a custom Arduino-based audio recorder that captured each interaction plus 10 seconds after Alexa’s blue light turned off.
- Data included usage logs, recorded audio, pre-study and exit semi-structured interviews, and a ChatGPT-3.5-powered Alexa skill deployed as a technology probe in Weeks 3–4.
- Audio was transcribed and matched to Alexa logs; 2,552 query–response pairs were coded for interaction purpose, error presence/type, resolution, recovery method, and whether errors led to breakdowns, using an inductive-then-deductive codebook.
Limitations
- The paper states the four-week duration may be too short for longer-term trends, the analysis is qualitative, the LLM-powered VA was tested only for generic information retrieval, and the sample was small and limited in location, cognitive ability, and ethnicity.
- The text also makes evident that audio data for participant P6 was missing due to a recorder malfunction, and that ChatGPT-skill findings are based on very limited use by only a few participants.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
27 Valid · 1 Not valid · 21 Not checked · 171 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“they often find it challenging to remember commands associated with VA features and skills [49, 57]-even those as simple as a wake word [1, 15, 49].”
What the sources say
- Partly supportedcites [49] [49] Alisha Pradhan, Amanda Lazar, and Leah Findlater. 2020. Use of intelligent voice assistants by older adults with low technology use. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 4 (2020), 1-27. - sourceExperimental — accuracy not yet measured
Section 4.3, “Challenges and Barriers to Use,” reports that five participants had difficulty remembering keywords for voice-assistant features and skills. It mentions problems after using the wake word, but does not report difficulty remembering the wake word itself. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [57] [57] Brodrick Stigall, Jenny Waycott, Steven Baker, and Kelly Caine. 2019. Older adults' perception and use of voice user interfaces: a preliminary review of the computing literature. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction . 423-427. - source
- Not checked — no full text of the sourcecites [1] [1] Leonardo Angelini, Maurizio Caon, Emmanuel Michielan, Omar Abou Khaled, and Elena Mugellini. 2021. Seniors' perception of smart speakers: challenges and opportunities elicited in the Silver&Home Living Lab. In Congress of the International Ergonomics Association . Springer, 137-144. - source
- Partly supportedcites [15] [15] Andrea Cuadra, Hyein Baek, Deborah Estrin, Malte Jung, and Nicola Dell. 2022. On Inclusion: Video Analysis of Older Adult Interactions with a Multi-Modal Voice Assistant in a Public Setting. In Proceedings of the 2022 International Conference on Information and Communication Technologies and Development . 1-17. - sourceExperimental — accuracy not yet measured
Section 4.1.1, “Overlooked by the machine” (page unavailable), reports that older participants frequently mispronounced, omitted, or misplaced the wake word. The paper supports difficulty using the wake word, but does not establish that the cause was difficulty remembering it or show a broader difficulty remembering commands for features and skills. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite older adults' desires to have conversational interactions with VAs, current commercial VA capabilities are far more limited [48].”
What the sources say
- Partly supportedcites [48] [48] Alisha Pradhan, Leah Findlater, and Amanda Lazar. 2019. " Phantom Friend" or" Just a Box with Information" Personification and Ontological Categorization of Smart Speaker-based Voice Assistants by Older Adults. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-21. - sourceExperimental — accuracy not yet measured
Section 4.4 reports that some older participants desired social contact through Alexa. Section 3.2.1 reports that Alexa sometimes failed to respond conversationally, particularly in its chat feature. The paper does not establish the broader claim that current commercial voice assistants’ capabilities are far more limited than older adults’ conversational desires; it also reports that brief exchanges with Alexa met some participants’ social needs. The indexed passages provide no page number. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“While researchers have investigated older adults' long-term interactions with VAs at home-focusing on their adoption of the technology, its benefits and challenges, and their perceptions of the experience [28, 48, 49, 51]-a significant research gap remains in the area of comprehensive error analysis.”
What the sources say
- Not checked — no full text of the sourcecites [28] [28] Sunyoung Kim and Abhishek Choudhury. 2021. Exploring older adults' perception and use of smart speaker-based voice assistants: A longitudinal study. Computers in Human Behavior 124 (2021), 106914. - source
- Supportedcites [48] [48] Alisha Pradhan, Leah Findlater, and Amanda Lazar. 2019. " Phantom Friend" or" Just a Box with Information" Personification and Ontological Categorization of Smart Speaker-based Voice Assistants by Older Adults. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-21. - source
The citation is attached to the statement that researchers have studied older adults’ in-home voice-assistant use, not to the citing authors’ conclusion about an error-analysis gap. The Methods section describes a three-week Echo Dot deployment in seven older adults’ homes, and §3.2 says the interviews examined their perceptions and use. (LLM, conf=0.87)
- Supportedcites [49] [49] Alisha Pradhan, Amanda Lazar, and Leah Findlater. 2020. Use of intelligent voice assistants by older adults with low technology use. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 4 (2020), 1-27. - source
The Methods section describes a three-week deployment of a voice assistant with seven older adults, using interviews, diaries, and usage logs to examine their perceptions and use. The Discussion reports benefits and challenges of using it at home. The claim that comprehensive error analysis remains a research gap is the citing authors’ own conclusion, not a finding attributed to this source. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [51] [51] Sandeep Purao, Haijing Hao, and Chenhang Meng. 2021. The use of smart home speakers by the elderly: exploratory analyses and potential for big data. Big Data Research 25 (2021), 100224. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“LLMs, on the other hand, have shown enhanced conversational capabilities, yielding robust and fluid interactions with users [9, 37].”
What the sources say
- Partly supportedcites [9] [9] Szeyi Chan, Jiachen Li, Bingsheng Yao, Amama Mahmood, Chien-Ming Huang, Holly Jimison, Elizabeth D Mynatt, and Dakuo Wang. 2023. " Mango Mango, How to Let The Lettuce Dry Without A Spinner?': Exploring User Perceptions of Using An LLM-Based Conversational Assistant Toward Cooking Partner. arXiv preprint arXiv:2310.05853 (2023). - sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says LLMs can understand lengthy input and accommodate multi-turn dialogue, and the findings describe conversations becoming more natural (section 4.2.2, p. 16). But that section also reports misunderstandings, wake-word constraints, and increased cognitive load, so the claim of robust and fluid user interactions overstates what the paper supports. (LLM, conf=0.86)
- Partly supportedcites [37] [37] Amama Mahmood, Junxiang Wang, Bingsheng Yao, Dakuo Wang, and Chien-Ming Huang. 2023. LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines. arXiv preprint arXiv:2309.13879 (2023). - sourceExperimental — accuracy not yet measured
The Conclusion (p. 24) says LLMs are better than traditional voice assistants at understanding context and handling breakdowns, and reports that ChatGPT absorbed 81% of intent-recognition failures. But it describes more fluid interactions as a goal; the Abstract (p. 1) likewise calls resilient and fluid interactions a potential benefit, rather than an established result. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Recent efforts to incorporate LLMs into voice-based interactions for older adults have shown improvement in conversational aspects [26, 63]”
What the sources say
- Not checked yetcites [26] [26] Eunkyung Jo, Daniel A. Epstein, Hyunhoon Jung, and Young-Ho Kim. 2023. Understanding the Benefits and Challenges of Deploying Conversational AI Leveraging Large Language Models for Public Health Intervention. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23) . Association for Computing Machinery, New York, NY, USA, Article 18, 16 pages. https://doi.org/10.1145/ 3544548.3581503 -
- Supportedcites [63] [63] Ziqi Yang, Xuhai Xu, Bingsheng Yao, Ethan Rogers, Shao Zhang, Stephen Intille, Nawar Shara, Guodong Gordon Gao, and Dakuo Wang. 2024. Talk2Care: An LLM-based Voice Assistant for Communication between Healthcare Providers and Older Adults. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 2 (2024), 1-35. - source
In Section 5.2.1 (page unavailable), older adults compared Talk2Care’s LLM-powered voice assistant with leaving a message and rated it as better supporting communication with providers. The paper attributes that benefit to its natural, interactive conversation, supporting the claim of improvement in conversational aspects. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
133 more claims · 10 with their cited sources checked
Create a free account to see all 220 claims and to verify the 8 references still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 58
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“in 238 instances they made at least one attempt to correct the error; of these, 107 errors (44 . 96%) were resolved after the first retry, 43 instances (15 . 19%) led to participants giving up”
Reported 15.19 · recomputed 18.067227
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Percentage of Errors and Their Resolution by Interaction Category · p. 14
“compared to human errors (6 . 7%), as shown in Table 2.”
§ 4 Findings · p. 9
“| Human errors | User made a mistake | 72 | 38.9% | 6.9% |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.5 Data and Analysis · p. 8
“| 13 | M | 81 | Caucasian | Vision issues | Good”
§ For each erroneous interaction · p. 30
“| Vision issues | Very good”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Analysis of Erroneous Interactions · p. 11
“health-related queries (52 . 38%)”
§ 4.1 Analysis of Erroneous Interactions · p. 10
“| Health | 43 | 55.81%”
Citations
Cites
Czaja, S. J., Charness, N., Fisk, A. D., Hertzog, C., Nair, S. N., Rogers, W. A., & Sharit, J. (2006) · Psychology and Aging · doi:10.1037/0882-7974.21.2.333 · Cited by 2,113 · cited 1×
“Design Consideration 4: VAs should employ context-aware responses with dynamic tutorials and real-time guidance in daily interactions to better align older adults’ mental models with evolving voice assistance technology.”
Technology probes: inspiring design for and with families
Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Druin, A., Plaisant, C., Beaudouin-Lafon, M., Conversy, S., Evans, H., Hansen, H., et al. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642616 · Cited by 1,471
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Vaportzis, E., Giatsi Clausen, M., & Gow, A. J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01687 · Cited by 916 · cited 1×
“Design Consideration 4: VAs should employ context-aware responses with dynamic tutorials and real-time guidance in daily interactions to better align older adults’ mental models with evolving voice assistance technology.”
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 4×
“These shifts to transactional dynamics have led researchers to debate whether such interactions should truly be considered “conversations” in the first place [59].”
InfluentialPradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448 · cited 1×
“Older adults find rigid keyword requirements frustrating [61, 62].”
"What can i help you with?": infrequent users' experiences of intelligent personal assistants
Cowan, B. R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., & Bandeira, N. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3098539 · Cited by 351
Communication Breakdowns Between Families and Alexa
Beneteau, E., Richards, O. K., Zhang, M., Kientz, J. A., Yip, J., & Hiniker, A. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300473 · Cited by 313 · cited 1×
“As users grow more accustomed to VA technology, they begin to value not just the systems’ operational ease, but also the sense of digital companionship they offer [34].”
Talk to me: Exploring user interactions with the Amazon Alexa
Lopatovska, I., Rink, K., Knight, I., Raines, K., Cosenza, K., Williams, H., Sorsche, P., Hirsch, D., Li, Q., & Martinez, A. (2019) · Journal of Librarianship and Information Science · doi:10.1177/0961000618759414 · Cited by 298 · cited 1×
“Overall, we observed a variety of interactions—from setting reminders to seeking information about various topics—as shown in Tables 3 and 4 and observed decline in interaction frequency over time, consistent with prior work [43, 61].”
Use of Intelligent Voice Assistants by Older Adults with Low Technology Use
Pradhan, A., Lazar, A., & Findlater, L. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3373759 · Cited by 285 · cited 8×
“While previous research has identified error types, estimated error rates, and examined recovery strategies in older adults’ VA use [34, 61], these studies do not address the sequential nature of how errors compound when users attempt to correct them.”
InfluentialBowling, A., Hankins, M., Windle, G., Bilotta, C., & Grant, R. (2013) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2012.08.012 · Cited by 247 · cited 1×
“Participants were compensated with either the option to keep their smart speaker or an amount equivalent to its market price ($50).”
Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224 · cited 1×
“As users grow more accustomed to VA technology, they begin to value not just the systems’ operational ease, but also the sense of digital companionship they offer [34].”
Pradhan, A., Findlater, L., & Lazar, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359316 · Cited by 223
Patterns for How Users Overcome Obstacles in Voice User Interfaces
Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173580 · Cited by 213 · cited 1×
“These findings align with prior research [50, 53].”
Kim, S., & Choudhury, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106914 · Cited by 200 · cited 10×
“While previous research has identified error types, estimated error rates, and examined recovery strategies in older adults’ VA use [34, 61], these studies do not address the sequential nature of how errors compound when users attempt to correct them.”
InfluentialJo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174
Age-related effects on speech production: A review
Mortensen, L., Meyer, A. S., & Humphreys, G. W. (2006) · Language and Cognitive Processes · doi:10.1080/01690960444000278 · Cited by 170 · cited 1×
“Errors are particularly prevalent in their interactions due to older adults’ verbose and disfluent speech patterns [52].”
"Alexa is a Toy": Exploring Older Adults' Reasons for Using, Limiting, and Abandoning Echo
Trajkova, M., & Martin-Hammond, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376760 · Cited by 154 · cited 2×
“Similar findings have been reported in prior work on older adults’ interactions with VAs [72].”
Voice‐Controlled Intelligent Personal Assistants to Support Aging in Place
O'Brien, K., Liggett, A., Ramirez‐Zohfeld, V., Sunkara, P., & Lindquist, L. A. (2020) · Journal of the American Geriatrics Society · doi:10.1111/jgs.16217 · Cited by 142 · cited 1×
“Common uses of VAs among older adults include seeking information, getting weather updates, listening to music, and setting reminders [2, 54].”
Older Adults and Voice Interaction: A Pilot Study with Google Home
Kowalski, J., Jaskulska, A., Skorupska, K., Abramczuk, K., Biele, C., Kopeć, W., & Marasek, K. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312973 · Cited by 125 · cited 3×
“Moreover, usage logs are unable to capture organic vocal reactions, interruptions, and overlapping speech during VA interactions.”
Yang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Mapping Perceptions of Humanness in Intelligent Personal Assistant Interaction
Doyle, P. R., Edwards, J., Dumbleton, O., Clark, L., & Cowan, B. R. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340116 · Cited by 100 · cited 1×
“Our data demonstrate that older adults adopt human dialogue as a metaphor for interaction with VAs [16, 20], which can be attributed to two factors: 1) speech interfaces are increasingly designed and marketed to emulate the nuances of human speech [23, 26] and 2) speech alone as interaction…”
Harrington, C. N., Garg, R., Woodward, A., & Williams, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501995 · Cited by 98 · cited 2×
“…this study, we developed a compact, packageable, and easily deployable tool to augment interaction data with real-world audio recordings, thus capturing older adults’ verbal and vocal reactions and responses to Amazon’s Alexa; this feedback also includes interruptions and overlapping speech,…”
Direct answers for search queries in the long tail
Bernstein, M. S., Teevan, J., Dumais, S., Liebling, D., & Horvitz, E. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207710 · Cited by 97 · cited 2×
“…( 64.29% error rate) and historical information ( 60.00% ) or those that did not fit into any category ( 44.74% )—had higher error rates, highlighting the VA’s limitations in handling less common or unique edge queries; this bears semblance to the “long tail problem” in information retrieval [6].”
InfluentialYang, B., Jiang, S., Xu, L., Liu, K., Li, H. M., Xing, G., Chen, H., Jiang, X., & Yan, Z. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699765 · Cited by 92
An Empirical Study of Older Adult’s Voice Assistant Use for Health Information Seeking
Brewer, R., Pierce, C., Upadhyay, P., & Park, L. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3484507 · Cited by 89 · cited 4×
“In this work, we explore if incorporating LLMs into VAs can reduce errors for older adults as seen in our prior work for young adults [48] and what new error and recovery patterns might emerge.”
InfluentialChoi, Y. K., Thompson, H. J., & Demiris, G. (2020) · JMIR Aging · doi:10.2196/21964 · Cited by 87 · cited 1×
“However, this method is constrained by transcription inaccuracies, which complicate error comprehension based solely on usage logs [34]—further suggesting that the usage logs are not always objective, either ( i.e., they are not always truly reflective of users’ interactions with VAs).”
Zargham, N., Reicherts, L., Bonfert, M., Voelkel, S. T., Schoening, J., Malaka, R., & Rogers, Y. (2022) · Proceedings of the 4th Conference on Conversational User Interfaces · doi:10.1145/3543829.3543834 · Cited by 84
Stigall, B., Waycott, J., Baker, S., & Caine, K. (2019) · Proceedings of the 31st Australian Conference on Human-Computer-Interaction · doi:10.1145/3369457.3369506 · Cited by 83 · cited 4×
“In this work, we explore if incorporating LLMs into VAs can reduce errors for older adults as seen in our prior work for young adults [48] and what new error and recovery patterns might emerge.”
InfluentialChen, C., Johnson, J. G., Charles, K., Lee, A., Lifset, E. T., Hogarth, M., Moore, A. A., Farcas, E., & Weibel, N. (2021) · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471218 · Cited by 67 · cited 1×
“…and easily deployable tool to augment interaction data with real-world audio recordings, thus capturing older adults’ verbal and vocal reactions and responses to Amazon’s Alexa; this feedback also includes interruptions and overlapping speech, which are not logged in traditional usage data.”
Hands free - care free: elderly people taking advantage of speech-only interaction
Wulf, L., Garschall, M., Himmelsbach, J., & Tscheligi, M. (2014) · Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational · doi:10.1145/2639189.2639251 · Cited by 64 · cited 2×
“Moreover, usage logs are unable to capture organic vocal reactions, interruptions, and overlapping speech during VA interactions.”
Arnold, A., Kolody, S., Comeau, A., & Miguel Cruz, A. (2024) · Disability and Rehabilitation: Assistive Technology · doi:10.1080/17483107.2022.2065369 · Cited by 59
Polite or Direct? Conversation Design of a Smart Display for Older Adults Based on Politeness Theory
Hu, Y., Qu, Y., Maus, A., & Mutlu, B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517525 · Cited by 54
Smart-speaker technology and intellectual disabilities: agency and wellbeing
Smith, E., Sumner, P., Hedge, C., & Powell, G. (2023) · Disability and Rehabilitation: Assistive Technology · doi:10.1080/17483107.2020.1864670 · Cited by 53 · cited 2×
“One primary approach is the combination of semi-structured interviews [34, 60, 61] and daily diary entries [61], enabling researchers to gather subjective user perceptions of their experiences with VAs.”
My Bad! Repairing Intelligent Voice Assistant Errors Improves Interaction
Cuadra, A., Li, S., Lee, H., Cho, J., & Ju, W. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449101 · Cited by 52 · cited 2×
“However, excessive correction can degrade user perceptions [18].”
Owning Mistakes Sincerely: Strategies for Mitigating AI Errors
Mahmood, A., Fung, J. W., Won, I., & Huang, C.-M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517565 · Cited by 52 · cited 1×
“Below, we discuss how these challenges are reflected in various speech recognition failures (partial listening, transcription, and wrong wake word errors) and propose design considerations to enhance the voice assistance experience for older adults.”
Self-citationStudying Exploration & Long-Term Use of Voice Assistants by Older Adults
Upadhyay, P., Heung, S., Azenkot, S., & Brewer, R. N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580925 · Cited by 51 · cited 1×
“One primary approach is the combination of semi-structured interviews [34, 60, 61] and daily diary entries [61], enabling researchers to gather subjective user perceptions of their experiences with VAs.”
User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants
Mahmood, A., Wang, J., Yao, B., Wang, D., & Huang, C.-M. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103406 · Cited by 51 · cited 9×
“This mental model contrasts sharply with that of younger adults in our previous work [11, 47, 48], who more readily understood the integration of LLMs and adapted their interactions accordingly.”
InfluentialSelf-citationPerceptions of Digital Assistants From Early Adopters Aged 55+
Koon, L. M., McGlynn, S. A., Blocker, K. A., & Rogers, W. A. (2020) · Ergonomics in Design: The Quarterly of Human Factors Applications · doi:10.1177/1064804619842501 · Cited by 51 · cited 1×
“Below, we discuss how these challenges are reflected in various speech recognition failures (partial listening, transcription, and wrong wake word errors) and propose design considerations to enhance the voice assistance experience for older adults.”
Mathur, N., Dhodapkar, K., Zubatiy, T., Li, J., Jones, B., & Mynatt, E. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544830 · Cited by 45 · cited 1×
“Recording ceases 10 seconds (tunable parameter) after the blue light turns off.”
Kobayashi, M., Kosugi, A., Takagi, H., Nemoto, M., Nemoto, K., Arai, T., & Yamada, Y. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-29390-1_4 · Cited by 45 · cited 2×
“Moreover, we observed cases where LLMs absorbed imperfect inputs and still advanced the task ( e.g., C6’s truncated utterance due to VA interruption), consistent with prior findings that LLMs can mitigate recognition issues or trigger recovery prompts [48].”
Voice Assistants for Health Self-Management: Designing for and with Older Adults
Mahmood, A., Cao, S., Stiber, M., Antony, V. N., & Huang, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713839 · Cited by 44
Self-citationDoyle, P. R., Gessinger, I., Edwards, J., Clark, L., Dumbleton, O., Garaialde, D., Rough, D. J., Bleakley, A., Branigan, H. P., & Cowan, B. R. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3729170 · Cited by 44 · cited 2×
“For older adults, these new conversational affordances often appeared unpredictable or “too human,” creating uncertainty about what the system could or should do, thus leading to mental model mismatches similar to the ones discussed in Section 5.3.1.”
InfluentialMavrina, L., Szczuka, J., Strathmann, C., Bohnenkamp, L. M., Krämer, N., & Kopp, S. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.791704 · Cited by 43 · cited 1×
“It’s too complicated, too cumbersome,” and further found the idea of chatting with a machine unusual: “I don’t want to try to have an artificial conversation.”
Liu, Y.-C., Chen, C.-H., Lin, Y.-S., Chen, H.-Y., Irianti, D., Jen, T.-N., Yeh, J.-Y., & Chiu, S. Y.-H. (2020) · JMIR mHealth and uHealth · doi:10.2196/20317 · Cited by 41 · cited 2×
“Moreover, usage logs are unable to capture organic vocal reactions, interruptions, and overlapping speech during VA interactions.”
Differences in Interactions with a Conversational Agent
Oh, Y. H., Chung, K., & Ju, D. Y. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17093189 · Cited by 32 · cited 1×
“Older adults frequently revert to a natural, conversational style, reflecting their preference for more social and informal interactions compared to younger [56] and middle-aged adults who adapt more easily to structured commands [14].”
CareJournal: A Voice-Based Conversational Agent for Supporting Care Communications
Rudnik, J., Raghuraj, S., Li, M., & Brewer, R. N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642163 · Cited by 30 · cited 4×
“However, VAs—even those powered by LLMs—often remain on the lower end of the partnership scale due to unintentional or superficial integration.”
InfluentialFan, M., Zhao, Q., & Tibdewal, V. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445680 · Cited by 29 · cited 1×
“…most errors faced by the older adults in our interaction data stemmed from the VA’s failure to recognize user intent or fulfill requests, a considerable portion of errors ( 𝑛 = 154 ) were related to automatic speech recognition technology, such as the partial listening and transcription errors.”
Exploring autonomy in the design of an intelligent health assistant for older adults
Sanders, J., & Martin-Hammond, A. (2019) · Companion Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3308557.3308713 · Cited by 28 · cited 1×
“Other works have explored health information-seeking [10, 25, 67], preventive care [76], and the potential use of reminders for medication administration [8, 12, 31].”
Chin, J., Desai, S., Lin, S. (.-H., & Mejia, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637365 · Cited by 27 · cited 1×
“Design Consideration 4: VAs should employ context-aware responses with dynamic tutorials and real-time guidance in daily interactions to better align older adults’ mental models with evolving voice assistance technology.”
Liu, C., Su, M., Xiang, Y., Huang, Y., Yang, Y., Kang Zhang, A., & Fan, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714228 · Cited by 27 · cited 5×
“To ensure transparency, a red LED light positioned on top of the recording device illuminates to indicate active audio recording, allowing the user to closely monitor recording status.”
InfluentialLeahu, L., Cohn, M., & March, W. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466455 · Cited by 24 · cited 1×
“Participants often found themselves forgetting and questioning the intent behind an alert, asking, “What am I supposed to do?” when a reminder rang without an accompanying message, indicating a mismatch between the VA’s functionalities and their needs.”
Chan, S., Li, J., Yao, B., Mahmood, A., Huang, C., Jimison, H. B., Mynatt, E. D., & Wang, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757442 · Cited by 24
Self-citationProgress and Prospects for Spoken Language Technology: What Ordinary People Think
Moore, R. K., Li, H., & Liao, S.-H. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-874 · Cited by 24 · cited 1×
“These factors often lead to an overestimation of VAs’ abilities resulting in communication breakdowns [37, 44, 51].”
Understanding the Needs and Challenges of Using Conversational Agents for Deaf Older Adults
Blair, J., & Abdullah, S. (2019) · Companion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3311957.3359487 · Cited by 23
Voice Assistant Use in Long-Term Care
Oewel, B., Ammari, T., & Brewer, R. N. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3597135 · Cited by 23
Yu, J. E., Parde, N., & Chattopadhyay, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581447 · Cited by 22 · cited 1×
“However, this method is constrained by transcription inaccuracies, which complicate error comprehension based solely on usage logs [34]—further suggesting that the usage logs are not always objective, either ( i.e., they are not always truly reflective of users’ interactions with VAs).”
Voice Assistant Reminders for Pain Self-Management Tasks in Aging Adults
Shade, M. Y., Rector, K., Soumana, R., & Kupzyk, K. (2020) · Journal of Gerontological Nursing · doi:10.3928/00989134-20200820-03 · Cited by 21 · cited 1×
“…VAs have shown significant promise as health aids; prior research focusing on specific needs such as pain management found VAs to be particularly beneficial for older adults [68], with their abilities to set reminders and promote accountability in health routines especially valuable [8, 68].”
Exploring the Design of LLM-based Agent in Enhancing Self-disclosure Among the Older Adults
Guo, Y., Wang, R., Huang, Z., Jin, T., Yao, X., Feng, Y.-L., Zhang, W., Yao, Y., & Mi, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713639 · Cited by 18 · cited 2×
“Additionally most of this work [24, 41, 45, 66, 69, 77] studies experiences or validates their systems in one off settings and more importantly studying breakdowns is not the focus, especially in longitudinal study and through analysis of interaction data and immediate user perceptions.”
Horstmann, A. C., Schubert, T., Lambrich, L., & Strathmann, C. (2023) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3570945.3607342 · Cited by 18
Social talk: making conversation with people and machine
Gilmartin, E., Collery, M., Su, K., Huang, Y., Elias, C., Cowan, B. R., & Campbell, N. (2017) · Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents · doi:10.1145/3139491.3139494 · Cited by 17 · cited 1×
“…[16, 20], which can be attributed to two factors: 1) speech interfaces are increasingly designed and marketed to emulate the nuances of human speech [23, 26] and 2) speech alone as interaction modality may inherently provide a false affordance for possible interactions—essentially tricking people…”
2021 Tech Trends and the 50+: Top 10 Biggest Trends
Kakulla, B. (2021) · doi:10.26419/res.00420.001 · Cited by 17 · cited 1×
“Several rigorous methodologies have been adopted to gather data about older adults’ use and perception of VAs in longitudinal at-home studies, which offer insights into their changing perceptions and interaction dynamics over time.”
Cuadra, A., Baek, H., Estrin, D., Jung, M., & Dell, N. (2022) · International Conference on Information & Communication Technologies and Development 2022 · doi:10.1145/3572334.3572371 · Cited by 16 · cited 4×
“In this work, we explore if incorporating LLMs into VAs can reduce errors for older adults as seen in our prior work for young adults [48] and what new error and recovery patterns might emerge.”
InfluentialAdapting a Virtual Assistant Device to Support the Interaction with Elderly People
Bolaños, M., Collazos, C. A., & Vela, F. L. G. (2020) · International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE) · doi:10.5220/0009840102910298 · Cited by 16 · cited 2×
“Therefore, in this study, we developed a compact, packageable, and easily deployable tool to augment interaction data with real-world audio recordings, thus capturing older adults’ verbal and vocal reactions and responses to Amazon’s Alexa; this feedback also includes interruptions and overlapping…”
Rewriting the Script: Adapting Text Instructions for Voice Interaction
Hwang, A., Oza, N., Callison-Burch, C., & Head, A. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596059 · Cited by 16
Chan, S., Li, J., Yao, B., Mahmood, A., Huang, C.-M., Jimison, H., Mynatt, E. D., & Wang, D. (2023) · arXiv · doi:10.48550/arxiv.2310.05853 · Cited by 16
Self-citationThe Use of Smart Home Speakers by the Elderly: Exploratory Analyses and Potential for Big Data
Purao, S., Hao, H., & Meng, C. (2021) · Big Data Research · doi:10.1016/j.bdr.2021.100224 · Cited by 15 · cited 3×
“A second approach relies on complementing understanding of user perceptions with a detailed analysis of usage logs to reveal usage frequency and evolving patterns in VA utilization [34, 55, 61, 63].”
Exploring Older Adults Personality Preferences for LLM-powered Conversational Companions
Shahid, A., Chung, J., & Heo, S. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719976 · Cited by 14 · cited 3×
“To ensure transparency, a red LED light positioned on top of the recording device illuminates to indicate active audio recording, allowing the user to closely monitor recording status.”
Yamada, S., Kitakoshi, D., Yamashita, A., Suzuki, K., & Suzuki, M. (2018) · 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI) · doi:10.1109/taai.2018.00020 · Cited by 8
Chiaradia, T. S., Seabra, R. D., & Mattedi, A. P. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-14070-0_60 · Cited by 7
Angelini, L., Caon, M., Michielan, E., Khaled, O. A., & Mugellini, E. (2021) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-030-74605-6_17 · Cited by 4 · cited 3×
“In this work, we explore if incorporating LLMs into VAs can reduce errors for older adults as seen in our prior work for young adults [48] and what new error and recovery patterns might emerge.”
Usability Assessment of Google Assistant and Siri Virtual Assistants Focusing on Elderly Users
Barros, T. C. G. M., & Seabra, R. D. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-43020-7_88 · Cited by 4
Data Capture and Analyses from Conversational Devices in the Homes of the Elderly
Purao, S., & Meng, C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-34146-6_14 · Cited by 3
Conversational Agents in Behavioral Sleep Medicine: Designing Self-Report and Analytics ToolsPreprint
Mahmood, A., Kim, B., Zhao, H.-H., Atwood, M. E., Buenaver, L., Smith, M. T., & Huang, C.-M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.15378 · Cited by 1
Self-citationChoi, Y. K., Thompson, H. J., & Demiris, G. (2020) · doi:10.2196/preprints.21964
Voice Assistant to Remind Pharmacologic Treatment in Elders
Jesús-Azabal, M., Rojo, J., Moguel, E., Flores-Martin, D., Berrocal, J., García-Alonso, J., & Murillo, J. M. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-41494-8_12
Liu, Y.-C., Chen, C.-H., Lin, Y.-S., Chen, H.-Y., Irianti, D., Jen, T.-N., Yeh, J.-Y., & Chiu, S. Y.-H. (2020) · doi:10.2196/preprints.20317
User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants
Mahmood, A., Wang, J., Yao, B., Wang, D., & Huang, C.-M. (2023) · arXiv · doi:10.48550/arxiv.2309.13879
Self-citationVoice Assistant for ageing people: Reminding drugs prescriptions without Internet connection
Jesús-Azabal, M., Berrocal, J., Murillo, J. M., & García-Alonso, J. (2020) · Gerontechnology · doi:10.4017/gt.2020.19.s.69640
Research works printed but not linked: 2
- Jiaan Li, Yulu Hu, Shihan Fu, Zhiqing Wu, Pan Hui, Hailiang Wang, and Mingming Fan. 2024. ‘It’s like having a friend’: Exploring the opportunity of large language models (LLMs) in assisting retirees in their daily lives. In Proceedings of the 12th International Symposium of Chinese CHI, 463–475.
- Cathy Pearl. 2016. Designing Voice User Interfaces: Principles of Conversational Experiences. O’Reilly Media, Inc.
Cited by
Public opinion on types of voice systems for older adults
Qu, N. Z., Li, J., Kongmanee, J., & Chignell, M. (2024) · Journal of Rehabilitation and Assistive Technologies Engineering · doi:10.1177/20556683241287414 · Cited by 1
Elderly Centric Mobile Assistance Platform with Voice and GPS Integration
N, P. K., S, A. H., Chouraddi, B. R., et al. (2026) · International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) · doi:10.1109/iitcee67948.2026.11394103
Summoogum, K., Lah, J. Y., Ansari, M., & Das, D. (2026) · doi:10.1109/iccms69991.2026.11642652
Metadata
- Authors
- Mahmood, A., Wang, J., & Huang, C.-M.
- Affiliations
- Johns Hopkins University (US)
- Carnegie Mellon University (US)
- Venue
- ACM Transactions on Accessible Computing
- Published
- 2026-03-18
- Volume
- 19
- Issue
- 1
- Pages
- 1-36
- ISSN
- 1936-7228 (ISSN-L)
- 1936-7236
- OpenAlex
- W7128159089
- Semantic Scholar
- 663b9fec2dbf8e29192d40503063e6dcd462f7bd
- Cited by
- 6
- References
- 52
- Citations by year
- 2026: 2
- FWCI
- 29.72
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- ACM Transactions on Accessible Computing (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1840088)
- Johns Hopkins University Institute for Assured Autonomy, and Malone Center for Engineering in Healthcare.
- Topics
- Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- voice assistants, older adults, error handling, smart speakers, large language models, ChatGPT, user experience, situated understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.