Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 31 of the 33 references it cites were found in a registry or an index; 2 of the 31 sources its claims cite were checked for support.

Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise

Open access
Wargnier, Pierre, Carletti, Giovanni, Laurent-Corniquet, Yann, Benveniste, Samuel, Jouvelot, Pierre, Rigaud, Anne-Sophie · 2016 · 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH)

Abstract

We present the first experiment we conducted to evaluate the attention monitoring performance of Louise, following a Wizard of Oz method, during the interactions with a cohort of 8 elderly users in a day hospital environment. Louise is a new, semi-automatic prototype of an Embodied Conversational Agent (ECA), a virtual character interacting with users through social-like communication, adapted to the special needs of older adults with cognitive impairment; it is intended to ultimately provide assistance in their activities of daily living. We recorded and analyzed both videos of the conversation-like interactions and Louise's tracking data. In our experiment, Louise's attention estimation algorithm achieved about 80% accuracy; moreover, in almost all cases, the user's attention was successfully recaptured by Louise after a planned, experimenter-induced distraction. These results are in line with what was observed in previous experiments involving only younger adults, thus suggesting that attention measurement tools embedded in cognitive prostheses will not need to be adapted to elderly patients. Finally, to gain further insights on conversation management and provide evidence-based suggestions for future work, we performed an anthropological analysis of the whole experiment.

Overview

CoChat Summary

TL;DR

The paper evaluates attention monitoring and attention-recapture in the Embodied Conversational Agent Louise with 8 cognitively impaired and healthy older adults in a day hospital, using a Wizard of Oz setup. Louise’s attention estimator reached about 80% accuracy, with no significant difference from younger adults, and attention was successfully recaptured after almost all experimenter-induced distractions.

Key findings

  • Louise’s attention estimation algorithm achieved about 80% accuracy; overall performance was 84% on all data (AURC = 0.68), or 86% (AURC = 0.74) after removing skeleton tracking errors (3.3% of samples). Table I reports 76% on all samples, 80% with no tracking errors, 72% for participants with cognitive impairment only, and 77% for that group with no tracking errors.
  • Group comparison with 14 healthy younger adults showed no significant differences in medians (p > 0.05), suggesting attention measurement tools embedded in cognitive prostheses may not need adaptation for elderly patients.
  • Attention recapture was effective: across 19 distractions, the experimenter had to ask the participant to look at the screen only once; prompting phrases were triggered only 6 times in 19 distractions.
  • 6 of 8 participants successfully interacted with Louise; one had poor hearing, and one with the highest cognitive impairment (MMSE = 17) refused to continue after the first distraction. Anthropological indicators showed PWD vs control subjects averaged word count 60.7/26.5, new information 6.0/2.5, and silence length 1.32/0.83 seconds.

Methods

  • Wizard of Oz study with the semi-automated Louise ECA, using a Microsoft Kinect; the attention estimator combined shoulder orientation and head yaw/pitch into a weighted attention level (0–10) with hysteresis thresholds: engaged if >8, disengaged if <6.
  • Participants were 8 older adults (2 males, 6 females; ages 63–91, mean 78 ± 1 years); 6 had cognitive impairment (3 MCI, 3 AD; MMSE 17–29, mean 23 ± 3) and 2 did not. The interaction included 3 introduction utterances, 7 questions, 5 acknowledgments, 5 prompting utterances, a transition, and 2 conclusion utterances, with two experimenter-induced distractions at fixed moments.
  • Videos were annotated by three experts; algorithm decisions were compared with annotations using correct detection rates, ROC/AURC, point-biserial correlations, and a Wilcoxon rank sum test against 14 younger adults. An anthropological analysis used ELAN coding, expert interviews, and a data session.

Limitations

  • Stated: the sample was small, and the authors say findings should be confirmed with a larger sample; the anthropological silence findings also need further research due to the small number of participants.
  • Stated: the day hospital setting and experimenter presence likely produced a “white-blouse effect” and a three-party interaction, limiting ecological validity; future experiments in a more casual context with more participants are warranted.
  • Stated/evident: tracking failures occurred (one complete Kinect failure, brief body/face tracking loss, and major failures in the cognitive impairment group), and participant factors such as poor hearing and one refusal reduced usable data. Evident: no self-induced distractions due to losing track of the task were observed, so attention recapture for that scenario remains untested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/segah.2016.7586282

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

2 Valid · 2 Not valid · 15 Not checked · 73 Own results and interpretations

48 more claims

Create a free account to see all 92 claims and to verify the 1 reference still open.

See all claims — free account
  • The quote only partly states this claim. 1
  • It may state two separate claims as one. 16
  • Probably not a substantial claim. 29

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 33 references

Research works printed but not linked: 3

  1. wargnier, 0, List of utterances
  2. pino, 2014, User-driven innovation for dementia care in France: The LUSAGE living lab case study, J Interdisciplinary Studies, 3, 251
  3. 2012, Dementia A Public Health Priority

Cited by

See all 29

Metadata

Authors
Wargnier, P., Carletti, G., Laurent-Corniquet, Y., Benveniste, S., Jouvelot, P., & Rigaud, A.-S.
Affiliations
  • École Nationale Supérieure des Mines de Paris (FR)
  • Assistance Publique – Hôpitaux de Paris (FR)
  • Centre d’Expertise National en Stimulation Cognitive (FR)
  • Université Paris Descartes (FR)
Venue
2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH)
Publisher
IEEE
Published
2016-05
Pages
1-8
ISSN
2330-5649 (ISSN-L)
OpenAlex
W2255044263
Cited by
28
References
33
Citations by year
2025: 1 · 2024: 3 · 2023: 1 · 2022: 6 · 2021: 5 · 2020: 4 · 2019: 1 · 2018: 4 · 2017: 2 · 2016: 1
FWCI
4.50
Type
conference paper
Language
en
Licence
other-oa
Locations
  • SeGAH ... IEEE ... International Conference on Serious Games and Applications for Health/IEEE International Conference on Serious Games and Applications for Health (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • HAL (submittedVersion)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
Keywords
embodied conversational agents, attention monitoring, attention estimation, cognitive impairment, older adults, Wizard of Oz, day hospital, activities of daily living, cognitive prostheses, conversation management, anthropological research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise. https://app.cochat.ai/papers/10.1109/segah.2016.7586282
The paper
Wargnier, P., Carletti, G., Laurent-Corniquet, Y., Benveniste, S., Jouvelot, P., & Rigaud, A.-S. (2016). Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise. 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH). https://doi.org/10.1109/segah.2016.7586282

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.