CoChat

Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study

Open access
Pacheco-Lorenzo, Moisés R., Anido-Rifón, Luis E., Fernández-Iglesias, Manuel J., Valladares-Rodríguez, Sonia M. · 2024 · Applied Intelligence

The DOI matches its Crossref record; 25 of the 27 references it cites were found in a registry or an index; 4 of the 9 sources its claims cite were checked for support.

Abstract

Abstract Background: early detection of dementia and Mild Cognitive Impairment (MCI) have an utmost significance nowadays, and smart conversational agents are becoming more and more capable. DigiMoCA, an Alexa-based voice application for the screening of MCI, was developed and tested. Objective: to evaluate the acceptability and usability of DigiMoCA, considering the perception of end-users and cognitive assessment administrators, through standard evaluation questionnaires. Method: a sample of 46 individuals and 24 evaluators participated in this study. End-users were fairly heterogeneous considering demographic and neuro-psychological characteristics. Evaluators were mostly health and social care professionals, relatively well-balanced in terms of gender, career background and years of experience. Results: end-users acceptability ratings were generally positive (rating above 3 in a 5-point scale for all dimensions) and it improved significantly after the interaction with DigiMoCA. Administrators also rated the usability of DigiMoCA, with an average score of 5.86/7 and with high internal consistency ( $$\alpha $$ α = 0.95). Conclusion: although there is still room for improvement in terms of user satisfaction and voice interface, DigiMoCA is perceived as an acceptable, accessible and usable cognitive screening tool, both by individuals being tested and test administrators. Graphical abstract

Overview

CoChat Summary

## TL;DR

This pilot study found that DigiMoCA, an Alexa-based conversational agent for cognitive screening, was generally acceptable to senior users and usable to administrators. Acceptability ratings were mostly positive across groups and improved after interaction, especially for conversational-agent-specific items.

## Key findings

  • End-users (n = 46) gave DigiMoCA generally positive TAM scores, with all dimension means above 3 on a 5-point scale: perceived usefulness 3.87 ± 0.92, perceived ease of use 3.98 ± 0.96, and perceived satisfaction 3.27 ± 1.21.
  • After interacting with DigiMoCA, acceptability for conversational agents improved significantly overall: at least one item in each TAM dimension showed a statistically significant positive change; the largest reported change was PU.1 at +59.14% (d = 1.06, p < 0.001).
  • For the technology-in-general section, acceptability did not significantly change overall, though one item improved significantly: PEoU.2 at +9.00% (d = 0.44, p = 0.010).
  • Administrators (n = 24) rated usability positively on a 7-point scale: overall 5.86 ± 1.24, system usefulness 5.96 ± 1.14, information quality 5.74 ± 1.44, and interface quality 5.81 ± 1.11; overall internal consistency was α = 0.95.

## Methods

  • User-interaction pilot study with two samples: 46 senior end-users and 24 cognitive assessment administrators/professionals.
  • End-users were recruited through two associations in Vigo, Spain; cognitive groups were reported as HC (n = 12), MCI (n = 18), and AD (n = 16), with mean age 78.61 ± 6.75 and 65% female.
  • Study design used 3 sessions: baseline cognitive questionnaires (T-MoCA, MFE, IADL), DigiMoCA administration after at least 2 weeks, then a second DigiMoCA administration after at least 2 more weeks; TAM questionnaires were completed before and after DigiMoCA, and administrators completed a PSSUQ-based usability questionnaire.
  • Analysis used descriptive statistics, Cronbach’s alpha for internal consistency, Student’s t-tests for pre/post comparisons, and Cohen’s d for effect sizes.

## Limitations

  • The paper explicitly states that sample size is a main limitation; based on a 95% confidence interval and an estimated AD prevalence of 10.285% in Spain, it says an ideal sample would be n = 142 participants per study group, far beyond the achieved sample.
  • The paper also says future work should better characterize the sample and assess acceptability/usability by technological training and life-long relationship with technology, which implies these were not fully addressed here.
  • Evident from the text: the study is a pilot user-interaction study with small, uneven subgroup sizes, and some questionnaire subscales had low internal consistency (for example, MCI α = 0.42 for perceived usefulness and α = 0.28 for perceived ease of use).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s10489-024-05558-z

Cites

25 of 27 references verified2 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

2 Valid · 2 Not valid · 3 Not checked · 82 Own results and interpretations

39 more claims

Create a free account to see all 89 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 16
  • It may state two separate claims as one. 35
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 27 references

Research works printed but not linked: 2

  1. Gliem JA, Gliem RR (2003) Calculating, interpreting, and reporting cronbach’s alpha reliability coefficient for likert-type scales. https://hdl.handle.net/1805/344
  2. Thalheimer W, Cook S (2002)How to calculate effect sizes from published research: A simplified methodology. Work-Learning Research. https://api.semanticscholar.org/CorpusID:145490810

Cited by

See all 8

Metadata

Authors
Pacheco-Lorenzo, M. R., Anido-Rifón, L. E., Fernández-Iglesias, M. J., & Valladares-Rodríguez, S. M.
Affiliations
Universidade de Vigo (ES)
Corresponding author
Pacheco-Lorenzo, Moisés R.
Venue
Applied Intelligence
Published
2024-06-15
Volume
54
Issue
17-18
Pages
7897-7912
ISSN
  • 0924-669X (ISSN-L)
  • 1573-7497
OpenAlex
W4399705270
Cited by
8
References
22
Citations by year
2026: 4 · 2025: 3
FWCI
2.06
Type
journal article
Language
en
Open access
Licence
Locations
  • Applied Intelligence (publishedVersion, cc-by)
  • Investigo Institutional repository of UVigo (Universidade de Vigo) (submittedVersion, cc-by)
Funders
  • Ministerio de Ciencia e Innovación (PID2020-115137RB-I00)
  • Ministerio de Ciencia, Innovación y Universidades (FPU19/01981)
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
  • Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
chatbots, voice assistants, Amazon Alexa, mild cognitive impairment, dementia screening, cognitive assessment, user acceptance, usability, end users, health and social care professionals, early detection of dementia, speech interfaces

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study. https://app.cochat.ai/papers/10.1007/s10489-024-05558-z
The paper
Pacheco-Lorenzo, M. R., Anido-Rifón, L. E., Fernández-Iglesias, M. J., & Valladares-Rodríguez, S. M. (2024). Will senior adults accept being cognitively assessed by a conversational agent? a user-interaction pilot study. Applied Intelligence. https://doi.org/10.1007/s10489-024-05558-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.