Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 10 of the 19 sources its claims cite were checked for support.
Abstract
The COVID-19 pandemic has expedited the integration of Smart Voice Assistants (SVA) among older people. The qualitative data derived from user commands on SVA is pivotal for elucidating the engagement patterns of older individuals with such systems. However, the sheer volume of user-generated voice interaction data presents a formidable challenge for manual coding. Compounding this issue, age-related cognitive decline and alterations in speech patterns further complicate the interpretation of older users’ SVA voice interactions. Conventional dictionary-based textual analysis tools, which count word frequencies, are inadequate in capturing the evolving and communicative essence of these interactions that unfold over a series of dialogues and modify with time. To address these challenges, our study introduces a novel, modified rule-based Natural Language Processing (MR-NLP) model augmented with human input. This reproducible approach capitalizes on human-derived insights to establish a lexicon of critical keywords and to formulate rules for the iterative refinement of the NLP model. English speakers, aged 50 or older and residing alone, were enlisted to engage with Amazon Alexa™ via predefined daily routines for a minimum of 30 min daily spanning three months ( N = 35, mean age = 77). We amassed time-stamped, textual data comprising participants’ user commands and responses from Alexa™. Initially, a subset constituting 20% of the data (1,020 instances) underwent manual coding by human coder, predicated on keywords and commands. Separately, a rule-based Natural Language Processing (NLP) methodology was employed to code the identical subset. Discrepancies arising between human coder and the NLP model programmer were deliberated upon and reconciled to refine the rule-based NLP coding framework for the entire dataset. The modified rule-based NLP approach demonstrated notable enhancements in efficiency and scalability and reduced susceptibility to inadvertent errors in comparison to manual coding. Furthermore, human input was instrumental in augmenting the NLP model, yielding insights germane to the aging adult demographic, such as recurring speech patterns or ambiguities. By disseminating this innovative software solution to the scientific community, we endeavor to advance research and innovation in NLP model formulation, subsequently contributing to the understanding of older people's interactions with SVA and other AI-powered systems.
Overview
CoChat Summary
TL;DR
This study introduces a modified rule-based NLP (MR-NLP) approach, refined with human input, to code older adults’ smart voice assistant interactions more efficiently and with fewer errors than manual coding. Using data from 35 older adults over 12 weeks, the authors report large time savings and mostly strong agreement between human and MR-NLP coding, though human coding produced a different factor structure.
Key findings
- Manual coding took an average of 3 h per participant (range 2–5 h), whereas the rule-based NLP method processed 35 participants in 9 h; the NLP method used only 9% of the time needed for manual coding.
- On the initial 20% subset (n = 7; 1,020 cases), manual vs. MR-NLP coding showed 85.49% agreement (872 cases) and Cohen’s kappa = 0.82.
- Of the 148 discrepancies (14.51%), 0.29% (3 cases) were due to program errors, 7.68% (67 cases) to human errors, and 6.54% (78 cases) to coding-definition differences.
- In factor analysis of the 12 interaction categories, MR-NLP coding and initial NLP coding each yielded four dimensions explaining 92% and 91.69% of variance, respectively, while manual coding yielded five dimensions; the authors say this supported H3.
- After expanding comparison, the paper reports 99% agreement and 0.99 Cohen’s kappa between human coding and NLP coding, with the remaining 1% disagreement attributed to human fatigue.
Methods
- The study recruited 35 English-speaking adults aged 50+ living alone in independent living facilities, with mean age 77; participants used Amazon Alexa for at least 30 min daily over 12 weeks.
- Participants’ Alexa commands and responses were extracted from time-stamped account data; a 20% subset (n = 7; 1,020 cases) was manually coded and then coded with a rule-based NLP system, with discrepancies discussed and used to refine the model.
- Analysis compared efficiency (coding time), error/consistency (agreement, Cohen’s kappa), and interpretive impact via principal-components factor analysis with varimax rotation on the 20% subset.
Limitations
- The authors state that only 20% of the data was manually coded initially, which could miss less frequent but meaningful interaction patterns and nuances in daily routines.
- They note limited generalizability because the sample was predominantly White, English-speaking, and from independent living facilities in one region.
- They also state that the study compared MR-NLP mainly with manual coding, not with other automated NLP models, so broader comparative performance and scalability remain untested.
- Evident from the text: the study’s factor-analysis comparison and reported agreement are based on a subset, so claims about the full dataset depend on extrapolation from that subset.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
8 Valid · 0 Not valid · 6 Not checked · 72 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Furthermore, older people with cognitive impairment may encounter difficulties in understanding speech (21).”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Introduction > Age-related changes in cognitive capacities and speech patterns
What the sources say
- Supportedcites [21] Banovic, 2018, Communication difficulties as a result of dementia, Mater Sociomed source
The Introduction says people with dementia can have difficulty understanding sentences. The Results section also reports lower hearing comprehension in people with Alzheimer’s dementia, supporting the hedged claim about older people with cognitive impairment. (LLM, conf=0.90)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Coder fatigue is a recognized threat to the trustworthiness of content analysis, particularly in cases involving repetitive and clerical coding (36).”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Discussion > Manual coding vs. NLP coding
What the sources say
- Supportedcites [36] Kleinheksel, 2020, Demystifying content analysis, Am J Pharm Educ source
The Ensuring Quality section calls coder fatigue a common threat to trustworthiness in qualitative analysis. It specifically says fatigue threatens manifest content analysis because its coding is clerical and repetitive. (LLM, conf=0.95)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Consistent with previous research (20, 21), the speech interactions between older people and SVA appear to be influenced by cognitive decline, resulting in repetitions in their speech patterns.”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Discussion > Manual coding vs. NLP coding
What the sources say
- Partly supportedcites [20] Szatloczki, 2015, Speaking in Alzheimer’s disease, is that an early sign? Importance of changes in language abilities in Alzheimer’s disease, Front Aging Neurosci sourceExperimental — accuracy not yet measured
The sections “The Relationship Between Language and Cognitive Functions in Alzheimer’s Disease” and “Alzheimer’s Disease and Language” link cognitive impairment with changes in speech; the supplied copy gives no page numbers. They do not discuss interactions with speech-based virtual assistants (SVA) or show that cognitive decline causes people to repeat themselves. The paper’s reference to “repetition abilities” concerns a language ability, not repetitive speech patterns. (LLM, conf=0.84)
- Partly supportedcites [21] Banovic, 2018, Communication difficulties as a result of dementia, Mater Sociomed sourceExperimental — accuracy not yet measured
The Results section (page not specified in our copy) says dementia-related cognitive decline affects language and communication. It does not discuss interactions with speech-based virtual assistants (SVA) or say that cognitive decline causes repetitive speech patterns. Its reference to poorer performance in “repetition” concerns a language ability, not repeated speech. (LLM, conf=0.86)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis revealed the presence of four dimensions of user-SVA voice interactions, accounting for 92% of the total variance of the items in data set 1 using the MR-NLP coding method”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Results > Hypothesis testing
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The modified rule-based NLP approach demonstrated notable enhancements in efficiency and scalability and reduced susceptibility to inadvertent errors in comparison to manual coding.”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Abstract The COVID-19 pandemic has expedited the integration of Smart Voice Assistants (SVA) among older people. The qualitative data derived from user commands on SVA is pivotal for elucidating the engagement patterns of older individuals with such systems. However, the sheer volume of user-generated voice interaction data presents a formidable challenge for manual coding. Compounding this issue, age-related cognitive decline and alterations in speech patterns further complicate the interpretation of older users’ SVA voice interactions. Conventional dictionary-based textual analysis tools, which count word frequencies, are inadequate in capturing the evolving and communicative essence of these interactions that unfold over a series of dialogues and modify with time. To address these challenges, our study introduces a novel, modified rule-based Natural Language Processing (MR-NLP) model augmented with human input. This reproducible approach capitalizes on human-derived insights to establish a lexicon of critical keywords and to formulate rules for the iterative refinement of the NLP model. English speakers, aged 50 or older and residing alone, were enlisted to engage with Amazon Alexa™ via predefined daily routines for a minimum of 30 min daily spanning three months (N = 35, mean age = 77). We amassed time-stamped, textual data comprising participants’ user commands and responses from Alexa™. Initially, a subset constituting 20% of the data (1,020 instances) underwent manual coding by human coder, predicated on keywords and commands. Separately, a rule-based Natural Language Processing (NLP) methodology was employed to code the identical subset. Discrepancies arising between human coder and the NLP model programmer were deliberated upon and reconciled to refine the rule-based NLP coding framework for the entire dataset. The modified rule-based NLP approach demonstrated notable enhancements in efficiency and scalability and reduced susceptibility to inadvertent errors in comparison to manual coding. Furthermore, human input was instrumental in augmenting the NLP model, yielding insights germane to the aging adult demographic, such as recurring speech patterns or ambiguities. By disseminating this innovative software solution to the scientific community, we endeavor to advance research and innovation in NLP model formulation, subsequently contributing to the understanding of older people's interactions with SVA and other AI-powered systems.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 5 assumptions not shown here.
33 more claims
Create a free account to see all 86 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 25
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Smart speakers that are intuitive and adaptable can foster empowerment and aging well, as demonstrated in Cluster 7, which ultimately elaborates on creating natural, empathetic, and engaging interactions through voice technology and human-like qualities that increase comfort and trust [83].”
From Smart Speakers for Health and Well-Being of Older Adults: A Mixed-Methods Review (2025) · cites this paper as [83]
Experimental — accuracy not yet measured
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(33)=−0.17, p = 0.88”
§ Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input > Method > Manual coding
Reported p = .88 · recomputed p = .866
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upt(33) = -0.17reported p = .88 · recomputed p = .866
“t(33)=−0.17, p = .88”
- Consistentt(33) = 0.46reported p = .65 · recomputed p = .649
“t(33) = 0.46, p = .65”
- Consistentt(33) = -0.69reported p = .49 · recomputed p = .495
“t(33)=−0.69, p = .49”
- Consistentt(33) = 0reported p = 1 · recomputed p = 1.000
“t(33) = 0, p = 1.00”
- Consistentt(33) = 1.12reported p = .27 · recomputed p = .271
“t(33) = 1.12, p = .27”
11 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53221.x · Cited by 25,553
The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · cited 1×
“Dictionary-based text analysis tools, such as the Linguistic Inquiry and Word Count (LIWC), have been used in computational linguistics (25) and psychology (26) for quantifying large volumes of textual information.”
MethodCharacter-level Convolutional Networks for Text Classification
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 3,205
The Impact of Age on Cognition
Murman, D. (2015) · Seminars in Hearing · doi:10.1055/s-0035-1555115 · Cited by 1,619
Kleinheksel, A. J., Rockich-Winston, N., Tawfik, H., & Wyatt, T. R. (2020) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7113 · Cited by 1,054
Learning Information Extraction Rules for Semi-Structured and Free Text
Soderland, S. (1999) · Machine Learning · doi:10.1023/a:1007562322031 · Cited by 934 · cited 1×
“While rule-based NLP relies on expert-crafted linguistic rules to extract structured information (29), machine learning based NLP uses statistical models to automate text classi fi cation (30).”
Method(2007) · Cited by 521
Portet, F., Vacher, M., Golanski, C., Roux, C., & Meillon, B. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-011-0470-5 · Cited by 421 · cited 1×
“…individuals seek ways to improve communication with family members, enjoy entertainment, and engage socially, many are turning to AI-powered SVA devices (13, 14), gradually replacing traditional information and communication technologies (ICT) like online platforms, email, smartphones, and iPads.”
BackgroundEvaluation of a decided sample size in machine learning applications
Rajput, D., Wang, W.-J., & Chen, C.-C. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05156-9 · Cited by 411 · cited 1×
“Past studies on older people ’ s use of SVA often involve small sample sizes (e.g., less than 100) and don ’ t meet the threshold of an adequate effect size for machine learning-based NLP, i.e., 0.5 or higher according to Cohen ’ s scale (33).”
BackgroundInfluential
Research works printed but not linked: 2
- 2021, 2021 Tech Trends and the 50-Plus
- Saldaña, 2016, The Coding Manual for Qualitative Researchers
Cited by
Toward sustainable smart cities: applications, challenges, and future directions
Ullah, A., Khan, S. H., Ullah, I., Mahmood, T., Nawazish, S., Ali, Z., & Rehman, A. U. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00856-2 · Cited by 87
“An example is a modified rule-based NLP model [109] that understands senior citizens’ voice interactions with smart voice assistants.”
Shankar, R., Bundele, A., & Mukhopadhyay, A. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100205 · Cited by 38
Vinay, R., Уэтова, Е., Tommila, N. C., Biller‐Andorno, N., & Kowatsch, T. (2025) · JMIR Aging · doi:10.2196/76489 · Cited by 14
“VAs have the potential to support older adults’ independence by facilitating daily activities through smooth, speech-based communication, offering a user-friendly interface that requires minimal technological expertise and can accommodate varying levels of digital literacy [17,32-34].”
Multimodal fusion-powered English speaking robot
Pan, R. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1478181 · Cited by 8
Smart Speakers for Health and Well-Being of Older Adults: A Mixed-Methods Review
Dino, M. J., Leinbach, C., Dino, G., Thiamwong, L., Villafuerte, C. M., Shattell, M., Pimentel, J., Zamora, M. A., Bautista, A., Vitug, J. P., et al. (2025) · Healthcare · doi:10.3390/healthcare13212772 · Cited by 5
“Smart speakers that are intuitive and adaptable can foster empowerment and aging well, as demonstrated in Cluster 7, which ultimately elaborates on creating natural, empathetic, and engaging interactions through voice technology and human-like qualities that increase comfort and trust [83].”
Metadata
- Authors
- Yan, Z., Dube, V., Heselton, J., Johnson, K., Yan, C., Jones, V., Blaskewicz Boron, J., & Shade, M.
- Affiliations
- University of California, Berkeley (US)
- University of Nebraska at Omaha (US)
- University of Nebraska–Lincoln (US)
- University of Nebraska Medical Center (US)
- Corresponding author
- Yan, Changmin
- Venue
- Frontiers in Digital Health
- Publisher
- Frontiers Media SA
- Published
- 2024-05-14
- Volume
- 6
- Pages
- 1329910-1329910
- ISSN
- 2673-253X (ISSN-L)
- PMC
- PMC11135128
- OpenAlex
- W4396900669
- Cited by
- 14
- References
- 37
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 1
- FWCI
- 11.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lincoln (University of Nebraska) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- older adults, voice interaction, rule-based natural language processing, human input, Amazon Alexa, user commands, age-related cognitive decline, speech patterns, COVID-19 pandemic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input. https://app.cochat.ai/papers/10.3389/fdgth.2024.1329910
- The paper
- Yan, Z., Dube, V., Heselton, J., Johnson, K., Yan, C., Jones, V., Blaskewicz Boron, J., & Shade, M. (2024). Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input. Frontiers in Digital Health. https://doi.org/10.3389/fdgth.2024.1329910
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

