Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes: A systematic literature review
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
This systematic review on data modalities synthesises the research findings in terms of how to optimally use and combine such modalities when investigating cognitive, motivational, and emotional learning processes. ERIC, WoS, and ScienceDirect databases were searched with specific keywords and inclusion criteria for research on data modalities, resulting in 207 relevant publications. We provide findings in terms of target journal, country, subject, participant characteristics, educational level, foci, type of data modality, research method, type of learning, learning setting, and modalities used to study the different foci. In total, 18 data modalities were classified. For the 207 multimodal publications, 721 occurrences of modalities were observed. The most popular modality was interview followed by survey and observation. The least common modalities were heart rate variability, facial expression recognition, and screen recording. From the 207 publications, 98 focused exclusively on the cognitive aspects of learning, followed by 27 publications that only focused on motivation, while only five publications exclusively focused on emotional aspects. Only 10 publications focused on a combination of cognitive, motivational, and emotional aspects of learning. Our results plea for the increased use of objective measures, highlight the need for triangulation of objective and subjective data, and demand for more research on combining various aspects of learning. Further, rather than researching cognitive, motivational, and emotional aspects of learning separately, we encourage scholars to tap into multiple learning processes with multimodal data to derive a more comprehensive view on the phenomenon of learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 1×
“The socio-cognitive theory posits that learning is an active constructive process in which individuals intentionally seek and process information (Bandura 2001; Pintrich 2000).”
BackgroundZimmerman, B. J. (2008) · American Educational Research Journal · doi:10.3102/0002831207312909 · Cited by 3,735
Universals and cultural differences in the judgments of facial expressions of emotion.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.53.4.712 · Cited by 1,763
Factors affecting students’ self-efficacy in higher education
van Dinther, M., Dochy, F., & Segers, M. (2011) · Educational Research Review · doi:10.1016/j.edurev.2010.10.003 · Cited by 967 · cited 1×
“In such a narrative analysis, the aim is to systematically analyse and integrate the state of knowledge in the field and also to highlight areas that research has left unresolved (Van Dinther et al. 2011).”
BackgroundToward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791 · cited 1×
“Multimodal approaches in learning research can help to tackle the constraints of typical single-channel data (e.g., subjective, objective, or physiological data), and help to draw more valid and reliable inferences about the learning processes (Harley et al. 2015; Pantic and Rothkrantz 2003).”
BackgroundInterpersonal autonomic physiology: a systematic review of the literature
Palumbo, R. V., Marraccini, M. E., Weyandt, L. L., Wilder-Smith, O., McGee, H. A., Liu, S., & Goodwin, M. S. (2017) · Personality and Social Psychology Review · doi:10.1177/1088868316628405 · Cited by 566
Psycho-physiological measures for assessing cognitive load
Haapalainen, E., Kim, S., Forlizzi, J. F., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864395 · Cited by 382 · cited 1×
“Several educational researchers have used objective data to infer various cognitive and affective states such as cognitive load (Cranford et al. 2014; Haapalainen et al. 2010), and emotions during learning (Chanel and Mühl 2015; D ’ Mello 2013; Fairclough et al. 2005).”
BackgroundAzevedo, R., & Gašević, D. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.03.025 · Cited by 345
Multimodal data to design visual learning analytics for understanding regulation of learning
Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.12.019 · Cited by 140
Self-citationPhysiological compliance for social gaming analysis: Cooperative versus competitive play
Chanel, G., Kivikangas, J. M., & Ravaja, N. (2012) · Interacting with Computers · doi:10.1016/j.intcom.2012.04.012 · Cited by 110
Cited by
Banihashem, S. K., Noroozi, O., den Brok, P., Biemans, H. J. A., & Kerman, N. T. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100803 · Cited by 102
Self-citationArtificial intelligence in multimodal learning analytics: a systematic literature review
Mohammadi, M., Tajik, E., Martinez-Maldonado, R., Sadiq, S., Tomaszewski, W., & Khosravi, H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100426 · Cited by 85
Celik, I., Gedrimiene, E., Siklander, S., & Muukkonen, H. (2024) · Australasian Journal of Educational Technology · doi:10.14742/ajet.9069 · Cited by 59
Metadata
- Authors
- Noroozi, O., Pijeira-Díaz, H. J., Sobocinski, M., Dindar, M., Järvelä, S., & Kirschner, P. A.
- Venue
- Education and Information Technologies
- Published
- 2020-05-30
- Volume
- 25
- Issue
- 6
- Pages
- 5499-5547
- ISSN
- 1360-2357 (ISSN-L)
- 1573-7608
- OpenAlex
- W3028965967
- Cited by
- 101
- References
- 10
- FWCI
- 4.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes: A systematic literature review. https://app.cochat.ai/papers/10.1007/s10639-020-10229-w
- The paper
- Noroozi, O., Pijeira-Díaz, H. J., Sobocinski, M., Dindar, M., Järvelä, S., & Kirschner, P. A. (2020). Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes: A systematic literature review. Education and Information Technologies. https://doi.org/10.1007/s10639-020-10229-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.