The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Multimodal data capabilities for learning: What can multimodal data tell us about learning?
Open accessAbstract
Most research on learning technology uses clickstreams and questionnaires as their primary source of quantitative data. This study presents the outcomes of a systematic literature review of empirical evidence on the capabilities of multimodal data (MMD) for human learning. This paper provides an overview of what and how MMD have been used to inform learning and in what contexts. A search resulted in 42 papers that were included in the analysis. The results of the review depict the capabilities of MMD for learning and the ongoing advances and implications that emerge from the employment of MMD to capture and improve learning. In particular, we identified the six main objectives (ie, behavioral trajectories, learning outcome, learning‐task performance, teacher support, engagement and student feedback) that the MMLA research has been focusing on. We also summarize the implications derived from the reviewed articles and frame them within six thematic areas. Finally, this review stresses that future research should consider developing a framework that would enable MMD capacities to be aligned with the research and learning design (LD). These MMD capacities could also be utilized on furthering theory and practice. Our findings set a baseline to support the adoption and democratization of MMD within future learning technology research and development. Practitioner Notes: What is already known about this topic Capturing and measuring learners’ engagement and behavior using MMD has been explored in recent years and exhibits great potential. There are documented challenges and opportunities associated with capturing, processing, analyzing and interpreting MMD to support human learning. MMD can provide insights into predicting learning engagement and performance as well as into supporting the process. What this paper adds Provides a systematic literature review (SLR) of empirical evidence on MMD for human learning. Summarizes the insights MMD can give us about the learning outcomes and process. Identifies challenges and opportunities of MMD to support human learning. Implications for practice and/or policy Learning analytics researchers will be able to use the SLR as a guide for future research. Learning analytics practitioners will be able to use the SLR as a summary of the current state of the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rasmussen, J. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313160 · Cited by 3,434
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · cited 1×
“We applied eight quality criteria (see Appendix A ) informed by related works (eg, Dybå & Dingsøyr, 2008 ).”
MethodAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893
Frontal theta activity in humans increases with memory load in a working memory task
Jensen, O., & Tesche, C. D. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2002.01975.x · Cited by 1,379
Baker, R. S. J. D., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.12.003 · Cited by 936
Eye tracking in HCI and usability research
Poole, A., & Ball, L. J. (2006) · Encyclopedia of Human Computer Interaction · doi:10.4018/978-1-59140-562-7.ch034 · Cited by 690 · cited 1×
“For example, gaze data can provide attentional information (Poole & Ball, 2006 ), physiological data can provide information about stress and arousal states of learners (Sharma, Pappas, et al ., 2019 ), and gesture, posture and motion data can tell us about the way learners interact/communicate…”
BackgroundEmpirically Building and Evaluating a Probabilistic Model of User Affect
Conati, C., & Maclaren, H. (2009) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-009-9062-8 · Cited by 328 · cited 1×
“Conati and MacLaren ( 2009 ) used emotions, detected by bodily expressions and game logs, to scaffold students’ emotional states based on their personal goals.”
BackgroundD’Mello, S. K., & Graesser, A. (2010) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9074-4 · Cited by 323
Spikol, D., Ruffaldi, E., Dabisias, G., & Cukurova, M. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12263 · Cited by 180
The promise and challenges of multimodal learning analytics
Cukurova, M., Giannakos, M., & Martinez‐Maldonado, R. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.13015 · Cited by 169
Self-citation
Cited by
Dwivedi, Y. K., Hughes, L., Baabdullah, A. M., Ribeiro-Navarrete, S., Giannakis, M., Al-Debei, M. M., Dennehy, D., Metri, B., Buhalis, D., Cheung, C. M. K., et al. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2022.102542 · Cited by 2,753
Human and artificial intelligence collaboration for socially shared regulation in learning
Järvelä, S., Nguyen, A., & Hadwin, A. (2023) · British Journal of Educational Technology · doi:10.1111/bjet.13325 · Cited by 342
Molenaar, I., Mooij, S. D., Azevedo, R., Bannert, M., Järvelä, S., & Gašević, D. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107540 · Cited by 217
The promise and challenges of multimodal learning analytics
Cukurova, M., Giannakos, M., & Martinez‐Maldonado, R. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.13015 · Cited by 169
Self-citationOuyang, F., Xu, W., & Cukurova, M. (2023) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-023-09387-z · Cited by 124
Artificial intelligence in multimodal learning analytics: a systematic literature review
Mohammadi, M., Tajik, E., Martinez-Maldonado, R., Sadiq, S., Tomaszewski, W., & Khosravi, H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100426 · Cited by 85
Gašević, D., Greiff, S., & Shaffer, D. W. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107304 · Cited by 82
Metadata
- Authors
- Sharma, K., & Giannakos, M.
- Venue
- British Journal of Educational Technology
- Publisher
- Wiley
- Published
- 2020-07-04
- Volume
- 51
- Issue
- 5
- Pages
- 1450-1484
- ISSN
- 0007-1013 (ISSN-L)
- 1467-8535
- OpenAlex
- W3038141818
- Cited by
- 237
- References
- 15
- FWCI
- 19.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.