Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study
Open accessThe DOI matches its Crossref record; 25 of the 27 references it cites were found in a registry or an index; 4 of the 9 sources its claims cite were checked for support.
Abstract
Abstract Background: early detection of dementia and Mild Cognitive Impairment (MCI) have an utmost significance nowadays, and smart conversational agents are becoming more and more capable. DigiMoCA, an Alexa-based voice application for the screening of MCI, was developed and tested. Objective: to evaluate the acceptability and usability of DigiMoCA, considering the perception of end-users and cognitive assessment administrators, through standard evaluation questionnaires. Method: a sample of 46 individuals and 24 evaluators participated in this study. End-users were fairly heterogeneous considering demographic and neuro-psychological characteristics. Evaluators were mostly health and social care professionals, relatively well-balanced in terms of gender, career background and years of experience. Results: end-users acceptability ratings were generally positive (rating above 3 in a 5-point scale for all dimensions) and it improved significantly after the interaction with DigiMoCA. Administrators also rated the usability of DigiMoCA, with an average score of 5.86/7 and with high internal consistency ( $$\alpha $$ α = 0.95). Conclusion: although there is still room for improvement in terms of user satisfaction and voice interface, DigiMoCA is perceived as an acceptable, accessible and usable cognitive screening tool, both by individuals being tested and test administrators. Graphical abstract
Overview
CoChat Summary
## TL;DR
This pilot study found that DigiMoCA, an Alexa-based conversational agent for cognitive screening, was generally acceptable to senior users and usable to administrators. Acceptability ratings were mostly positive across groups and improved after interaction, especially for conversational-agent-specific items.
## Key findings
- End-users (n = 46) gave DigiMoCA generally positive TAM scores, with all dimension means above 3 on a 5-point scale: perceived usefulness 3.87 ± 0.92, perceived ease of use 3.98 ± 0.96, and perceived satisfaction 3.27 ± 1.21.
- After interacting with DigiMoCA, acceptability for conversational agents improved significantly overall: at least one item in each TAM dimension showed a statistically significant positive change; the largest reported change was PU.1 at +59.14% (d = 1.06, p < 0.001).
- For the technology-in-general section, acceptability did not significantly change overall, though one item improved significantly: PEoU.2 at +9.00% (d = 0.44, p = 0.010).
- Administrators (n = 24) rated usability positively on a 7-point scale: overall 5.86 ± 1.24, system usefulness 5.96 ± 1.14, information quality 5.74 ± 1.44, and interface quality 5.81 ± 1.11; overall internal consistency was α = 0.95.
## Methods
- User-interaction pilot study with two samples: 46 senior end-users and 24 cognitive assessment administrators/professionals.
- End-users were recruited through two associations in Vigo, Spain; cognitive groups were reported as HC (n = 12), MCI (n = 18), and AD (n = 16), with mean age 78.61 ± 6.75 and 65% female.
- Study design used 3 sessions: baseline cognitive questionnaires (T-MoCA, MFE, IADL), DigiMoCA administration after at least 2 weeks, then a second DigiMoCA administration after at least 2 more weeks; TAM questionnaires were completed before and after DigiMoCA, and administrators completed a PSSUQ-based usability questionnaire.
- Analysis used descriptive statistics, Cronbach’s alpha for internal consistency, Student’s t-tests for pre/post comparisons, and Cohen’s d for effect sizes.
## Limitations
- The paper explicitly states that sample size is a main limitation; based on a 95% confidence interval and an estimated AD prevalence of 10.285% in Spain, it says an ideal sample would be n = 142 participants per study group, far beyond the achieved sample.
- The paper also says future work should better characterize the sample and assess acceptability/usability by technological training and life-long relationship with technology, which implies these were not fully addressed here.
- Evident from the text: the study is a pilot user-interaction study with small, uneven subgroup sizes, and some questionnaire subscales had low internal consistency (for example, MCI α = 0.42 for perceived usefulness and α = 0.28 for perceived ease of use).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 3 Not checked · 82 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Previous research demonstrated that it is possible to implement a voice-based version of a gold standard test for cognitive assessment using conversational agents [11].”
What the sources say
- Partly supportedcites [11] Pacheco-Lorenzo M, Fernández-Iglesias MJ, Valladares-Rodriguez S, Anido-Rifón LE (2023) Implementing scripted conversations by means of smart assistants. Software: Practice and Experience 53. https://doi.org/10.1002/spe.3182 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes a conversational agent implementing the telephone version of the Montreal Cognitive Assessment, and the Conclusion describes it as a standardized conversation-based test. This supports the feasibility of a voice-based cognitive test, but the paper calls the Mini-Mental State Examination—not MoCA—the reference test, so it does not establish that the implemented test was a gold standard. (LLM, conf=0.84)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find the biggest demographic differences between the health field group and the technological field group.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“most are concentrated on the MCI group: +85.84% (d = 1.29, p < 0.001) for PU.1”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the fact that the information and interface-related items obtain a slightly lower rating across all groups”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“none of the significant changes are in the HC group”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
39 more claims
Create a free account to see all 89 claims and every source verdict.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 35
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 2 Material and methods
“For additional information, we also show the results of the T-MoCA evaluation (16.25 ± 3.28 for healthy users (HC), 16.25 ± 3.28 for users with MCI and 16.25 ± 3.28 for users with dementia (AD))”
§ 2 Material and methods
“| T-MoCA | 16.25 ± 3.28 | 12.00 ± 2.95 | 9.00 ± 3.33 | 12.07 ± 4.20 |”
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53221.x · Cited by 25,553 · cited 1×
“Therefore, in later years other screening tools were created, e.g., the Montreal Cognitive Assessment (MoCA) [6] test, which has a 90% sensitivity for MCI detection (MMSE is not sensitive to MCI).”
Design and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · cited 1×
“According to previous relevant works [14, 15], in order to calculate the number of participants for a pilot study we need to take into account: (1) the parameters to be estimated; (2) that at least 30 participants are involved; (3) a minimum confidence interval of 80% is required.”
Application of Student's t-test, Analysis of Variance, and Covariance
Mishra, P., Singh, U., Pandey, C. M., Mishra, P., & Pandey, G. (2019) · Annals of Cardiac Anaesthesia · doi:10.4103/aca.aca_94_19 · Cited by 949
The Lawton instrumental activities of daily living scale
Graf, C. (2008) · AJN, American Journal of Nursing · doi:10.1097/01.naj.0000314810.46029.74 · Cited by 832
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810
Sample size calculations for pilot randomized trials: a confidence interval approach
Cocks, K., & Torgerson, D. J. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.09.002 · Cited by 431
Kueper, J. K., Speechley, M., & Montero-Odasso, M. (2018) · Journal of Alzheimer’s Disease · doi:10.3233/jad-170991 · Cited by 424
Research works printed but not linked: 2
- Gliem JA, Gliem RR (2003) Calculating, interpreting, and reporting cronbach’s alpha reliability coefficient for likert-type scales. https://hdl.handle.net/1805/344
- Thalheimer W, Cook S (2002)How to calculate effect sizes from published research: A simplified methodology. Work-Learning Research. https://api.semanticscholar.org/CorpusID:145490810
Cited by
A Systematic Review of NLP for Dementia: Tasks, Datasets, and Opportunities
Peled-Cohen, L., & Reichart, R. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.35 · Cited by 20
Cheema, M. N., Nazir, A., Harmanci, A. O., Harmanci, A. O., Cheema, Y., Masood, S., & Khokhar, F. A. (2026) · The Visual Computer · doi:10.1007/s00371-026-04385-2 · Cited by 2
Pacheco-Lorenzo, M. R., Priego, N. L., Iglesias, M. J. F., Anido, L., & Otero-González, I. (2023) · Electronics · doi:10.3390/electronics13010013 · Cited by 2
Self-citationValidation of DigiMoCA, A Conversational Agent for the Cognitive Assessment of Senior Adults
Pacheco-Lorenzo, M. R., Valladares‐Rodríguez, S., Iglesias, M. J. F., & Anido, L. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2537783 · Cited by 2
Self-citationEnhancing cognitive impairment screening by means of conversational agents and machine learning
Pacheco-Lorenzo, M. R., Valladares‐Rodríguez, S., Iglesias, M. J. F., & Anido, L. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132263
Self-citation
Metadata
- Authors
- Pacheco-Lorenzo, M. R., Anido-Rifón, L. E., Fernández-Iglesias, M. J., & Valladares-Rodríguez, S. M.
- Affiliations
- Universidade de Vigo (ES)
- Corresponding author
- Pacheco-Lorenzo, Moisés R.
- Venue
- Applied Intelligence
- Published
- 2024-06-15
- Volume
- 54
- Issue
- 17-18
- Pages
- 7897-7912
- ISSN
- 0924-669X (ISSN-L)
- 1573-7497
- OpenAlex
- W4399705270
- Cited by
- 8
- References
- 22
- Citations by year
- 2026: 4 · 2025: 3
- FWCI
- 2.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Applied Intelligence (publishedVersion, cc-by)
- Investigo Institutional repository of UVigo (Universidade de Vigo) (submittedVersion, cc-by)
- Funders
- Ministerio de Ciencia e Innovación (PID2020-115137RB-I00)
- Ministerio de Ciencia, Innovación y Universidades (FPU19/01981)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, voice assistants, Amazon Alexa, mild cognitive impairment, dementia screening, cognitive assessment, user acceptance, usability, end users, health and social care professionals, early detection of dementia, speech interfaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study. https://app.cochat.ai/papers/10.1007/s10489-024-05558-z
- The paper
- Pacheco-Lorenzo, M. R., Anido-Rifón, L. E., Fernández-Iglesias, M. J., & Valladares-Rodríguez, S. M. (2024). Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study. Applied Intelligence. https://doi.org/10.1007/s10489-024-05558-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.