Challenges and opportunities of multimodal data in human learning: The computer science students' perspective
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Multimodal data have the potential to explore emerging learning practices that extend human cognitive capacities. A critical issue stretching in many multimodal learning analytics (MLA) systems and studies is the current focus aimed at supporting researchers to model learner behaviours, rather than directly supporting learners. Moreover, many MLA systems are designed and deployed without learners' involvement. We argue that in order to create MLA interfaces that directly support learning, we need to gain an expanded understanding of how multimodal data can support learners' authentic needs. We present a qualitative study in which 40 computer science students were tracked in an authentic learning activity using wearable and static sensors. Our findings outline learners' curated representations about multimodal data and the non‐technical challenges in using these data in their learning practice. The paper discusses 10 dimensions that can serve as guidelines for researchers and designers to create effective and ethically aware student‐facing MLA innovations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Verification strategies for establishing reliability and validity in qualitative research
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100202 · Cited by 4,978
Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars.
Maguire, M., & Delahunt, B. (2017) · All Ireland Journal of Higher Education · doi:10.62707/aishej.v9i3.335 · Cited by 2,099
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“This is aligned with the prospective usefulness of eye-trackers and wrist-mounted sensors in educational technology research (Calvo & D'Mello, 2010; Was et al., 2016).”
BackgroundRoutledge International Handbook of Participatory Design
(2012) · doi:10.4324/9780203108543 · Cited by 1,606
Solove, D. J. (2006) · University of Pennsylvania Law Review · doi:10.2307/40041279 · Cited by 1,388
Learning Analytics: The Emergence of a Discipline
Siemens, G. (2013) · American Behavioral Scientist · doi:10.1177/0002764213498851 · Cited by 1,284 · cited 2×
“Concerns about data quality, sufficient scope of the data captured to reflect accurately the learning experience, privacy, and ethics of analytics are among the most significant concerns ’ (Siemens, 2013, p. 394).”
BackgroundLearning Analytics: Ethical Issues and Dilemmas
Slade, S., & Prinsloo, P. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479366 · Cited by 1,078
The current landscape of learning analytics in higher education
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.027 · Cited by 766 · cited 2×
“Fundamental to these competences is learners' ability to interpret, comprehend, and generate inference from analytics derived from multimodal data (Mangaroska & Giannakos, 2018; Viberg et al., 2018).”
BackgroundInterest, Prior Knowledge, and Learning
Tobias, S. (1994) · Review of Educational Research · doi:10.3102/00346543064001037 · Cited by 713 · cited 1×
“Therefore, neglecting prior knowledge can cause students to lose interest (Tobias, 1994), develop misconceptions (Mladenovic et al., 2016), or learn something opposed to the educator's intentions (Roschelle, 1997).”
Background
Cited by
Artificial intelligence in multimodal learning analytics: a systematic literature review
Mohammadi, M., Tajik, E., Martinez-Maldonado, R., Sadiq, S., Tomaszewski, W., & Khosravi, H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100426 · Cited by 85
AI-assisted automated scoring of picture-cued writing tasks for language assessment
Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P. L. H. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11473-y · Cited by 40
Personalisation methods in e-learning-A literature review
Mikić, V., Ilić, M., Kopanja, L., & Vesin, B. (2022) · Computer Applications in Engineering Education · doi:10.1002/cae.22566 · Cited by 39
“Previous literature reviews of personalised learning addressed the significance of the adaptive features of the employed e‐learning system [170] as well as the customisation features for learners to take control of the learning process [26, 181, 186].”
BackgroundSelf-citation
Metadata
- Authors
- Mangaroska, K., Martinez‐Maldonado, R., Vesin, B., & Gašević, D.
- Venue
- Journal of Computer Assisted Learning
- Publisher
- Wiley
- Published
- 2021-03-02
- Volume
- 37
- Issue
- 4
- Pages
- 1030-1047
- ISSN
- 0266-4909 (ISSN-L)
- 1365-2729
- OpenAlex
- W3135540143
- Cited by
- 54
- References
- 28
- FWCI
- 8.28
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.