Pay attention!: designing adaptive agents that monitor and improve user engagement
Abstract
Embodied agents hold great promise as educational assistants, exercise coaches, and team members in collaborative work. These roles require agents to closely monitor the behavioral, emotional, and mental states of their users and provide appropriate, effective responses. Educational agents, for example, will have to monitor student attention and seek to improve it when student engagement decreases. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from educational psychology to design adaptive agents that monitor student attention in real time using measurements from electroencephalography (EEG) and recapture diminishing attention levels using verbal and nonverbal cues. An experimental evaluation of our approach showed that an adaptive robotic agent employing behavioral techniques to regain attention during drops in engagement improved student recall abilities 43% over the baseline regardless of student gender and significantly improved female motivation and rapport. Our findings offer guidelines for developing effective adaptive agents, particularly for educational settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Measurements of EDA and derived metrics have been successfully applied, including user experience evaluation tools, detecting office workload, and driver-related tasks.
“Measurements of EDA and derived metrics have been successfully applied, including, e.g., user experience evaluation tools [76], detecting office workload [216] and driver-related tasks [202,217,224].”
From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [217]
Temperature-related measures such as heat flux are more robust, most likely due to the less noise-prone method to measure them.
“Here, temperature-related measures such as heat flux (heat transfer rate) are more robust [87,217], most likely due to the less noise-prone method to measure them.”
From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [217]
Cited by
Sorted by citations
Social robots for education: a review.
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018) · Science Robotics · doi:10.1126/scirobotics.aat5954 · Cited by 1,558 · see citing works
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304 · see citing works
“Cerebral EEG Frequencies [16, 17, 26, 52, 67, 70, 83, 86, 87, 100, 105, 106, 124, 130, 135, 138, 148, 150, 184, 197, 198, 207, 235, 243, 244, 268] EEG ERPs [23, 26, 59, 81, 184, 214, 227, 243] Hemodynamic Response (fNIRS) [3, 5, 32, 80, 96, 97, 104, 159, 161, 191, 192, 196, 221–223, 227, 261, 262]…”
Edwards, A., Edwards, C., Spence, P. R., Harris, C., & Gambino, A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.06.005 · Cited by 174 · see citing works
Engagement in human-agent interaction: An overview
Oertel, C., Castellano, G., Chetouani, M., Nasir, J., Obaid, M., Pelachaud, C., & Peters, C. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.00092 · Cited by 160 · see citing works
Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M., Afergan, D., Chang, R., & Jacob, R. J. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858388 · Cited by 137 · see citing works
“Szafir and Mutlu (2012) [47] used EEG to detect and recapture dropping attention levels in real-time.”
BackgroundLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116 · see citing works
The positive effects of verbal encouragement in mathematics education using a social robot
Brown, L. N., & Howard, A. M. (2014) · 2014 IEEE Integrated STEM Education Conference · doi:10.1109/isecon.2014.6891009 · Cited by 71 · see citing works
Brainatwork: logging cognitive engagement and tasks in the workplace using electroencephalography
Hassib, M., Khamis, M., Friedl, S., Schneegass, S., & Alt, F. (2017) · Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia · doi:10.1145/3152832.3152865 · Cited by 24 · see citing works
Modelling adaptive presentations in human-robot interaction using behaviour trees,
Axelsson, N., & Skantze, G. (2019) · Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w19-5940 · Cited by 15 · see citing works
Gupta, S., Kumar, P., & Tekchandani, R. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-18021-x · Cited by 14 · see citing works
- Authors
- Szafir, D., & Mutlu, B.
- Affiliations
- University of Wisconsin–Madison (US)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 11-20
- OpenAlex
- W2043736971
- Cited by
- 304
- References
- 0
- Citations by year
- 2026: 9 · 2025: 20 · 2024: 19 · 2023: 26 · 2022: 24 · 2021: 24 · 2020: 22 · 2019: 28 · 2018: 23 · 2017: 31 · 2016: 24 · 2015: 22 · 2014: 15 · 2013: 16 · 2012: 1
- FWCI
- 34.60
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- student attention monitoring, educational agents, EEG, brain-computer interface, attention restoration, verbal and nonverbal cues, student recall, student motivation, gender differences, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
