Scalability, Sustainability, and Ethicality of Multimodal Learning Analytics
Published version of a preprint
Abstract
Multimodal Learning Analytics (MMLA) innovations are commonly aimed at supporting learners in physical learning spaces through state-of-the-art sensing technologies and analysis techniques. Although a growing body of MMLA research has demonstrated the potential benefits of sensor-based technologies in education, whether their use can be scalable, sustainable, and ethical remains questionable. Such uncertainty can limit future research and the potential adoption of MMLA by educational stakeholders in authentic learning situations. To address this, we systematically reviewed the methodological, operational, and ethical challenges faced by current MMLA works that can affect the scalability and sustainability of future MMLA innovations. A total of 96 peer-reviewed articles published after 2010 were included. The findings were summarised into three recommendations, including i) improving reporting standards by including sufficient details about sensors, analysis techniques, and the full disclosure of evaluation metrics, ii) fostering interdisciplinary collaborations among experts in learning analytics, software, and hardware engineering to develop affordable sensors and upgrade MMLA innovations that used discontinued technologies, and iii) developing ethical guidelines to address the potential risks of bias, privacy, and equality concerns with using MMLA innovations. Through these future research directions, MMLA can remain relevant and eventually have actual impacts on educational practices.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Similar replication issues have also been identified in other areas of educational technology research.
“Similar replication issues have also been identified in other areas of educational technology research (Yan et al., 2022).”
From Practical and ethical challenges of large language models in education: A systematic scoping review (2024) · cites this paper as Yan et al., 2022
The assessment tool adopted for technology readiness was the Australian Department of Defence's Technology Readiness Levels (TRL), which has been used to assess the maturity of educational technologies in prior systematic literature reviews.
“We adopted the assessment tool from the Australian government, namely the Australian Department of Defence's Technology Readiness Levels (TRL) (Defence Science and Technology Group, 2021), which has been used to assess the maturity of educational technologies in prior SLR(Yan et al., 2022).”
From Practical and ethical challenges of large language models in education: A systematic scoping review (2024) · cites this paper as Yan et al., 2022
Transformer models employ advanced attention mechanisms to achieve enhanced contextual processing, allowing for more subtle understanding and interaction.
“They employ advanced attention mechanisms to achieve enhanced contextual processing, allowing for more subtle understanding and interaction [142], [143], [144].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [143]
Cites
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748 · see citing works
The European Union general data protection regulation: what it is and what it means
Hoofnagle, C. J., van der Sloot, B., & Borgesius, F. Z. (2019) · Information & Communications Technology Law · doi:10.1080/13600834.2019.1573501 · Cited by 798 · see citing works
Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.10.002 · Cited by 725 · see citing works
Ethical and privacy principles for learning analytics
Pardo, A., & Siemens, G. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12152 · Cited by 577 · cited 1× · see citing works
“The ethicality of LA systems can be defined as the systematisa-tion of appropriate and inappropriate behaviour in physical spaces, as determined by all stakeholders [52].”
BackgroundLeveraging mobile technology for sustainable seamless learning: a research agenda
Looi, C., SEOW, P. S. K., Zhang, B., So, H., Chen, W., & Wong, L. H. (2009) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2008.00912.x · Cited by 499 · see citing works
Multimodal data as a means to understand the learning experience
Giannakos, M. N., Sharma, K., Pappas, I. O., Kostakos, V., & Velloso, E. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.02.003 · Cited by 210 · see citing works
The promise and challenges of multimodal learning analytics
Cukurova, M., Giannakos, M., & Martinez‐Maldonado, R. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.13015 · Cited by 169 · see citing works
Self-citationStages of Acceptance of A Learning Disability: The Impact of Labeling
Higgins, E. L., Raskind, M. H., Goldberg, R., et al. (2002) · Learning Disability Quarterly · doi:10.2307/1511187 · Cited by 127 · see citing works
Prieto, L. P., Sharma, K. K., Kidziński, Ł., Rodríguez‐Triana, M. J., & Dillenbourg, P. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12232 · Cited by 121 · see citing works
Multimodal Learning Analytics research with young children: A systematic review
Lanna, L. C. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12959 · Cited by 118 · see citing works
Automatically Measuring Question Authenticity in Real-World Classrooms
Kelly, S., Olney, A. M., Donnelly, P., Nystrand, M., & D’Mello, S. K. (2018) · Educational Researcher · doi:10.3102/0013189x18785613 · Cited by 114 · see citing works
Prieto, L. P., Sharma, K., Dillenbourg, P., & Jesús, M. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883927 · Cited by 113 · see citing works
Classroom sound can be used to classify teaching practices in college science courses
Owens, M. T., Seidel, S. B., Wong, M., Bejines, T. E., Lietz, S., Perez, J. R., Sit, S., Subedar, Z.-S., Acker, G. N., Akana, S., et al. (2017) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.1618693114 · Cited by 92 · see citing works
Towards Collaboration Translucence
Echeverría, V., Martínez‐Maldonado, R., & Shum, S. B. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300269 · Cited by 90 · see citing works
Guest Editorial: Ethics and Privacy in Learning Analytics
Ferguson, R., Hoel, T., Scheffel, M., & Drachsler, H. (2016) · Journal of Learning Analytics · doi:10.18608/jla.2016.31.2 · Cited by 88 · see citing works
Ochôa, X., Domínguez, F., Guamán, B., Maya, R., Falcones, G., & Castells, J. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170406 · Cited by 73 · see citing works
Papavlasopoulou, S., Sharma, K., & Giannakos, M. N. (2018) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2018.01.004 · Cited by 72 · see citing works
Automatic Teacher Modeling from Live Classroom Audio
Donnelly, P., Blanchard, N., Samei, B., Olney, A. M., Sun, X., Ward, B., Kelly, S., Nystran, M., & D’Mello, S. K. (2016) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/2930238.2930250 · Cited by 64 · see citing works
Implementing learning analytics for learning impact: Taking tools to task
Knight, S., Gibson, A., & Shibani, A. (2020) · The Internet and Higher Education · doi:10.1016/j.iheduc.2020.100729 · Cited by 64 · see citing works
Expertise estimation based on simple multimodal features
Ochôa, X., Chiluiza, K., Méndez, G. G., Luzardo, G., Guamán, B., & Castells, J. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2533789 · Cited by 61 · see citing works
Donnelly, P., Blanchard, N., Olney, A. M., Kelly, S., Nystrand, M., & D’Mello, S. K. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027417 · Cited by 61 · see citing works
A Multimodal Analysis of Making
Worsley, M. B., & Blikstein, P. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0160-1 · Cited by 61 · see citing works
Using multimodal learning analytics to study collaboration on discussion groups
Riquelme, F., Muñoz, R., Lean, R. M., Villarroel, R. H., Barcelos, T., & de Albuquerque, V. H. C. (2019) · Universal Access in the Information Society · doi:10.1007/s10209-019-00683-w · Cited by 52 · see citing works
Moodoo: Indoor Positioning Analytics for Characterising Classroom Teaching
Martínez‐Maldonado, R., Echeverría, V., Schulte, J. B., Shibani, A., Mangaroska, K., & Shum, S. B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-52237-7_29 · Cited by 42 · see citing works
Improving student engagement in and with digital learning technologies
D’Mello, S. K. (2021) · Digital education outlook · doi:10.1787/8a451974-en · Cited by 42 · see citing works
Privacy-Preserving Speech Analytics for Automatic Assessment of Student Collaboration
Bassiou, N., Tsiartas, A., Smith, J. G., Bratt, H., Richey, C., Shriberg, E. E., D’Angelo, C. M., & Alozie, N. M. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1569 · Cited by 36 · see citing works
Multimodal Analytics for Real-Time Feedback in Co-located Collaboration
Praharaj, S., Scheffel, M., Drachsler, H., & Specht, M. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98572-5_15 · Cited by 36 · see citing works
What is Sustainable Technology?: Perceptions, Paradoxes and Possibilities
Mulder, K., Ferrer, D., & van Lente, H. (2011) · Medical Entomology and Zoology · Cited by 35 · see citing works
Echeverría, V., Martínez‐Maldonado, R., Power, T., Hayes, C., & Shum, S. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93846-2_14 · Cited by 35 · see citing works
Estimation of Presentations Skills Based on Slides and Audio Features
Luzardo, G., Guamán, B., Chiluiza, K., et al. (2014) · doi:10.1145/2666633.2666639 · Cited by 33 · see citing works
Worsley, M. B. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170420 · Cited by 33 · see citing works
Multi-sensor modeling of teacher instructional segments in live classrooms
Donnelly, P., Blanchard, N., Samei, B., Olney, A. M., Sun, X., Ward, B., Kelly, S., Nystrand, M., & D’Mello, S. K. (2016) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2993148.2993158 · Cited by 28 · see citing works
Footprints at School: Modelling In-class Social Dynamics from Students’ Physical Positioning Traces
Yan, L., Martínez‐Maldonado, R., Cordoba, B. G., Deppeler, J., Corrigan, D., Fernandez-Nieto, G. M., & Gašević, D. (2021) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3448139.3448144 · Cited by 28 · see citing works
Self-citationEditorial: Best Practice Approaches for Mixed Methods Research in Psychological Science
Anguera, M. T., Blanco-Villaseñor, A., Jonsson, G. K., Losada, J. L., & Portell, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.590131 · Cited by 24 · see citing works
Problem solving, domain expertise and learning
Oviatt, S. L. (2013) · ACM International Conference on Multimodal Interaction (ICMI) · doi:10.1145/2522848.2533791 · Cited by 24 · see citing works
Enhancing consent forms to support participant decision making in multimodal learning data research
Beardsley, M., Moreno, J. M., Vujović, M., Santos, P., & Hernández‐Leo, D. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12983 · Cited by 24 · see citing works
I Know What You Know: What Hand Movements Reveal about Domain Expertise
Oviatt, S. L., Lin, J., & Sriramulu, A. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3423049 · Cited by 22 · see citing works
Kasepalu, R., Chejara, P., Prieto, L. P., & Ley, T. (2021) · Technology Knowledge and Learning · doi:10.1007/s10758-021-09522-5 · Cited by 22 · see citing works
Where are You? Exploring Micro-Location in Indoor Learning Environments
Riquelme, F., Noël, R., Cornide-Reyes, H., Geldes, G., Cechinel, C., Miranda, D., Villarroel, R. H., & Muñoz, R. (2020) · IEEE Access · doi:10.1109/access.2020.3008327 · Cited by 21 · see citing works
Designing An Automated Assessment of Public Speaking Skills Using Multimodal Cues
Chen, L., Feng, G., Leong, C. W., et al. (2016) · Journal of Learning Analytics · doi:10.18608/jla.2016.32.13 · Cited by 20 · see citing works
Blanchard, N., D’Mello, S. K., Olney, A. M., et al. (2015) · Educational Data Mining · Cited by 19 · see citing works
Scaling and Adopting a Multimodal Learning Analytics Application in an Institution-Wide Setting
Domínguez, F., Ochôa, X., Zambrano, D., Sellán, K. M. C., & Castells, J. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3100778 · Cited by 18 · see citing works
Domínguez, F., Chiluiza, K., Echeverría, V., & Ochôa, X. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2830606 · Cited by 16 · see citing works
Alwahaby, H., Cukurova, M., Papamitsiou, Z., & Giannakos, M. (2021) · doi:10.35542/osf.io/sd23y · Cited by 15 · see citing works
Social Intelligence Modeling using Wearable Devices
Mihoub, A., & Lefebvre, G. (2017) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3025171.3025195 · Cited by 14 · see citing works
EduBrowser: A Multimodal Automated Monitoring System for Co-located Collaborative Learning
Chua, Y. H. V., Rajalingam, P., Tan, S. C., & Dauwels, J. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-20798-4_12 · Cited by 12 · see citing works
Pelánek, R. (2020) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3375462.3375463 · Cited by 11 · see citing works
Nguyen, Q., Poquet, O., Brooks, C. A., et al. (2020) · Educational Data Mining · Cited by 9 · see citing works
Classroom size, activity and attendance
Brennan, A., Peace, C., & Munguia, P. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170401 · Cited by 7 · see citing works
Realizing a Mobile Multimodal Platform for Serious Games Analytics
Shoukry, L., & Göbel, S. P. (2019) · International Journal of Serious Games · doi:10.17083/ijsg.v6i4.323 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 935 · see citing works
“Similar replication issues have also been identified in other areas of educational technology research (Yan et al., 2022).”
BackgroundMethodInfluentialSelf-citationHuman-centred learning analytics and AI in education: A systematic literature review
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100215 · Cited by 281 · see citing works
Self-citationMcIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“Scalability issues, due to the extensive computational resources needed, are exacerbated by limited high-performance hardware availability [121], [122].”
BackgroundArtificial intelligence in multimodal learning analytics: a systematic literature review
Mohammadi, M., Tajik, E., Martinez-Maldonado, R., Sadiq, S., Tomaszewski, W., & Khosravi, H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100426 · Cited by 85 · see citing works
- Authors
- Yan, L., Zhao, L., Gasevic, D., & Martinez-Maldonado, R.
- Affiliations
- Monash University (AU)
- Monash University, Australia
- Venue
- LAK22: 12th International Learning Analytics and Knowledge Conference
- Publisher
- ACM
- Published
- 2022-03-21
- Pages
- 13-23
- OpenAlex
- W4214842915
- Cited by
- 83
- References
- 50
- Citations by year
- 2026: 11 · 2025: 32 · 2024: 18 · 2023: 19 · 2022: 3
- FWCI
- 23.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Learning Analytics and Knowledge (LAK) (publishedVersion)
- EdArXiv (OSF Preprints) (submittedVersion)
- HAL (submittedVersion)
- Funders
- Australian Research Council (DP210100060)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Educational Technology and Pedagogy
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- multimodal learning analytics, scalability, sustainability, ethics, physical learning environment, reporting guidelines, interdisciplinary collaboration, privacy, algorithmic bias, educational technology adoption
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
