The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Using touchscreen interaction data to predict cognitive workload
Abstract
Although a great number of today’s learning applications run on devices with an interactive screen, the high-resolution interaction data which these devices provide have not been used for workload-adaptive systems yet. This paper aims at exploring the potential of using touch sensor data to predict user states in learning scenarios. For this purpose, we collected touch interaction patterns of children solving math tasks on a multi-touch device. 30 fourth-grade students from a primary school participated in the study. Based on these data, we investigate how machine learning methods can be applied to predict cognitive workload associated with tasks of varying difficulty. Our results show that interaction patterns from a touchscreen can be used to significantly improve automatic prediction of high levels of cognitive workload (average classification accuracy of 90.67% between the easiest and most difficult tasks). Analyzing an extensive set of features, we discuss which characteristics are most likely to be of high value for future implementations. We furthermore elaborate on design choices made for the used multiple choice interface and discuss critical factors that should be considered for future touch-based learning interfaces.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Building Intelligent Interactive Tutors : student-centered strategies for revolutionizing e-learning
Woolf, B. P. (2009) · HAL · Cited by 1,376
Combining SVMs with Various Feature Selection Strategies
Chen, Y.-W., & Lin, C.-J. (2008) · Studies in Fuzziness and Soft Computing · doi:10.1007/978-3-540-35488-8_13 · Cited by 980 · cited 1×
“For feature selection and generation of the final individual classifiers, we opted for an approach suggested by Chen et al. [3].”
MethodCognitive arithmetic: Evidence for retrieval and decision processes in mental addition.
Ashcraft, M. H., & Battaglia, J. (1978) · Journal of Experimental Psychology: Human Learning & Memory · doi:10.1037//0278-7393.4.5.527 · Cited by 350 · cited 1×
“Both, the so called problem size as well as the carry effect have been identified to robustly increase response latencies and/or error rates when solving multi-digit addition problems (e.g., [1, 10, 11]).”
BackgroundD’Mello, S. K., & Graesser, A. (2010) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9074-4 · Cited by 323
The neural substrate of arithmetic operations and procedure complexity
Kong, J., Wang, C., Kwong, K., Vangel, M., Chua, E., & Gollub, R. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.09.011 · Cited by 215 · cited 1×
“The carry effect has furthermore been repeatedly identified to be the source of increased cognitive workload (e.g, [8, 9]).”
BackgroundWhat Does Touch Tell Us about Emotions in Touchscreen-Based Gameplay?
Gao, Y., Bianchi-Berthouze, N., & Meng, H. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2395131.2395138 · Cited by 170 · cited 3×
“CCS Concepts •Human-centered computing → User models; Touch screens; Empirical studies in HCI; Keywords Multi-Touch, Machine Learning, Adaptive Learning Systems”
BackgroundMethodInfluentialGerjets, P. H., Walter, C., Rosenstiel, W., Bogdan, M., & Zander, T. O. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00385 · Cited by 160 · cited 2×
“We present the results of a user study with 30 fourthgraders from a primary school and discuss how our findings might influence the design of future adaptive systems that utilize multi-touch interaction data.”
BackgroundSelf-citationToward Measuring and Maintaining the Zone of Proximal Development in Adaptive Instructional Systems
Murray, T., & Arroyo, I. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47987-2_75 · Cited by 122 · cited 1×
“In order to maintain the so called zone of proximal development, cognitive and affective factors have to be taken into account [16].”
BackgroundImbo, I., Vandierendonck, A., & Rammelaere, S. D. (2006) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470210600762447 · Cited by 116
Saneiro, M., Santos, O. C., Salmerón-Majadas, S., & Boticario, J. G. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/484873 · Cited by 110 · cited 1×
“Audio and video recordings have also been analyzed to assess affective and/or cognitive states (e.g., [4, 12, 19, 23]).”
Background
Research works printed but not linked: 3
- 10.1037/0278-7393.4.5.527 · doi:10.1037/0278-7393.4.5.527 · Paper page
- Samsung. SUR40. http: //www.samsung.com/uk/b/p/LH40SFWTGC/EN , 2012 . Accessed : 2016-09-08. Samsung. SUR40. http: //www.samsung.com/uk/b/p/LH40SFWTGC/EN, 2012. Accessed: 2016-09-08.
- L. Vgotsky . Mind in society: The development of higher mental processes , 1978 . L. Vgotsky. Mind in society: The development of higher mental processes, 1978.
Cited by
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“Similarly, touch input [39] – especially finger trajectories [173] – were used to measure cognitive workload.”
Multimodal data capabilities for learning: What can multimodal data tell us about learning?
Sharma, K., & Giannakos, M. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12993 · Cited by 237
“( 2018 ) used skeleton positions and kinematics features to predict participants’ recall; Andrade, Danish, and ( 2016 ) used interaction logs from a touchscreen and the hand movements to predict the cognitive workload of the students.”
BackgroundMethodInfluentialNam, H., Seol, K., Lee, J., Cho, H., & Jung, S. W. (2021) · Sensors · doi:10.3390/s21144776 · Cited by 82
Cognitive load and self-regulation: Attempts to build a bridge
Boekaerts, M. (2017) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.07.001 · Cited by 73
“Gerjets and colleagues collected interaction patterns of primary school children while they solved math problems on a multi-touch table and then used machine learning methods on these high-resolution interactions (see Mock et al., 2016).”
BackgroundTian, F., Fan, X., Fan, J., Zhu, Y., Gao, J., Wang, D., Bi, X., & Wang, H. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300313 · Cited by 41
“The mean directional change in degrees of each point on the actual path [38].”
Background
Metadata
- Authors
- Mock, P., Gerjets, P., Tibus, M., Trautwein, U., Möller, K., & Rosenstiel, W.
- Affiliations
- Leibniz-Institut für Wissensmedien (DE)
- University of Tübingen (DE)
- Venue
- Proceedings of the 18th ACM International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2016-10-31
- Pages
- 349-356
- OpenAlex
- W2547090472
- Cited by
- 21
- References
- 23
- Citations by year
- 2025: 2 · 2023: 4 · 2022: 3 · 2021: 3 · 2020: 3 · 2019: 3 · 2018: 1 · 2017: 2
- FWCI
- 1.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- multi-touch devices, task difficulty, fourth grade, interaction patterns
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using touchscreen interaction data to predict cognitive workload. https://app.cochat.ai/papers/10.1145/2993148.2993202
- The paper
- Mock, P., Gerjets, P., Tibus, M., Trautwein, U., Möller, K., & Rosenstiel, W. (2016). Using touchscreen interaction data to predict cognitive workload. Proceedings of the 18th ACM International Conference on Multimodal Interaction. https://doi.org/10.1145/2993148.2993202
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.