Utilizing Multimodal Data Through fsQCA to Explain Engagement in Adaptive Learning
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Investigating and explaining the patterns of learners' engagement in adaptive learning conditions is a core issue towards improving the quality of personalized learning services. This article collects learner data from multiple sources during an adaptive learning activity, and employs a fuzzy set qualitative comparative analysis (fsQCA) approach to shed light to learners' engagement patterns, with respect to their learning performance. Specifically, this article measures and codes learners' engagement by fusing and compiling clickstreams (e.g., response time), physiological data (e.g., eye-tracking, electroencephalography, electrodermal activity), and survey data (e.g., goal-orientation) to determine what configurations of those data explain when learners can attain high or medium/low learning performance. For the evaluation of the approach, an empirical study with 32 undergraduates was conducted. The analysis revealed six configurations that explain learners' high performance and three that explain learners' medium/low performance, driven by engagement measures coming from the multimodal data. Since fsQCA explains the outcome of interest itself, rather than its variance, these findings advance our understanding on the combined effect of the multiple indicators of engagement on learners' performance. Limitations and potential implications of the findings are also discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
School Engagement: Potential of the Concept, State of the Evidence
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004) · Review of Educational Research · doi:10.3102/00346543074001059 · Cited by 12,581 · cited 4×
“S UCCESSFUL learning and important educational outcomes such as persistence in learning, confidence, and academic achievement, have been linked to learners’ engagement [1]–[3].”
BackgroundMethodInfluentialBuilding Better Causal Theories: A Fuzzy Set Approach to Typologies in Organization Research
Fiss, P. C. (2011) · Academy of Management Journal · doi:10.5465/amj.2011.60263120 · Cited by 5,930 · cited 3×
“Conditions that appear in the intermediate solution but not in the parsimonious are called “peripheral conditions” [88].”
BackgroundInfluentialNicol, D. J., & Macfarlane‐Dick, D. (2006) · Studies in Higher Education · doi:10.1080/03075070600572090 · Cited by 5,846
The Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 4×
“…to feedback usually depends on whether they consider it to convey information about their personal growth, or if it scaffolds their self-regulatory learning processes and understanding of the content [32], since adaptive self-regulated learning is integral to learners’ engagement [17].”
BackgroundInfluentialConfigurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques
Rihoux, B., & Ragin, C. (2009) · SAGE Publications · doi:10.4135/9781452226569 · Cited by 3,089
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867
On the measurement of achievement goals: Critique, illustration, and application.
Elliot, A. J., & Murayama, K. (2008) · Journal of Educational Psychology · doi:10.1037/0022-0663.100.3.613 · Cited by 1,281 · cited 1×
“Achievement goals are considered a facet of motivation given that they provide a purpose or focus for the task and, as such, influence students’ learning behaviors, emotions, thoughts, and performance [33].”
BackgroundMeasuring student engagement in technology-mediated learning: A review
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.09.005 · Cited by 1,010 · cited 3×
“We conceptualize engagement at the adaptive activity level using three dimensions: 1) thoughts; 2) feelings; and 3) behaviors [1], [8], [11].”
BackgroundMethodSun, J. C., & Rueda, R. (2012) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2010.01157.x · Cited by 785
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015) · Review of Educational Research · doi:10.3102/0034654314564881 · Cited by 714 · cited 1×
“Adaptation has a positive impact on learners’ selfregulation and engagement with the activities [19], [20], which is also reflected on learners’ performance [14], [90].”
BackgroundRethinking the Use of Tests: A Meta-Analysis of Practice Testing
Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017) · Review of Educational Research · doi:10.3102/0034654316689306 · Cited by 544 · cited 1×
“A recent meta-analysis examined the effects of practice tests versus nontesting learning conditions, and the results revealed that practice tests are more beneficial for learning than restudying and all other comparison conditions [77].”
BackgroundCue strength as a moderator of the testing effect: The benefits of elaborative retrieval.
Carpenter, S. K. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0017021 · Cited by 503
Chapter 3 The Two QCAs: From a Small-N to a Large-N Set Theoretic Approach
Greckhamer, T., Misangyi, V. F., & Fiss, P. C. (2013) · Research in the Sociology of Organizations · doi:10.1108/s0733-558x(2013)0000038007 · Cited by 435
Azevedo, R. (2015) · Educational Psychologist · doi:10.1080/00461520.2015.1004069 · Cited by 418 · cited 7×
“…hinder our understanding on the deployment of engagement: they lack unobtrusiveness and the potential to capture the complexity of learners’ continues involvement with the activities, failing to provide clues about what is needed for enriching instruction to motivate and engage learners [8], [47].”
BackgroundInfluentialMolinillo, S., Aguilar-Illescas, R., Anaya-Sánchez, R., & Vallespín-Arán, M. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.04.012 · Cited by 303
Duffy, M. C., & Azevedo, R. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.05.041 · Cited by 283
Testing Enhances the Transfer of Learning
Carpenter, S. K. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721412452728 · Cited by 230
Identification of personal traits in adaptive learning environment: Systematic literature review
Afini Normadhi, N. B., Shuib, L., Md Nasir, H. N., Bimba, A., Idris, N., & Balakrishnan, V. (2019) · Computers & Education · doi:10.1016/j.compedu.2018.11.005 · Cited by 204 · cited 4×
“As already stated in the introduction, adaptive learning settings are inherently supportive to learners’ motivation and adaptation positively impacts learners’ engagement [19], [20].”
BackgroundResultInfluentialInvestigating the effect of an adaptive learning intervention on students’ learning
Liu, M., McKelroy, E., Corliss, S. B., & Carrigan, J. (2017) · Educational Technology Research and Development · doi:10.1007/s11423-017-9542-1 · Cited by 138
Thorson, K. R., West, T. V., & Mendes, W. B. (2018) · Psychological Methods · doi:10.1037/met0000166 · Cited by 106
Affective learning: Improving engagement and enhancing learning with affect-aware feedback
Grawemeyer, B., Mavrikis, M., Holmes, W., Gutiérrez-Santos, S., Wiedmann, M., & Rummel, N. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9188-z · Cited by 105 · cited 1×
“Former studies in adaptive learning contexts explored mostly the behavioral (as observed actions/reactions) and affective aspects of engagement (e.g., [37]–[39]), or the relation between motivation and cognitive behaviors of engagement (e.g., [13], [32], [40]).”
Background
Research works printed but not linked: 70
- 10.1080/00405840802577544 · DOI · Paper page
- 10.1016/j.chb.2018.06.016 · DOI
- 10.1016/j.jbusres.2016.01.024 · DOI
- 10.1145/3036399 · DOI
- 10.1016/j.compedu.2019.103729 · DOI · Paper page
- 10.1080/02602938.2011.592934 · DOI · Paper page
- 10.1016/j.ijer.2011.05.001 · DOI
- 10.1007/978-3-319-98572-5_64 · DOI
- 10.1111/bjet.12747 · DOI
- 10.1145/2851581.2892329 · DOI
- 10.18608/hla17.011 · DOI · Paper page
- 10.1186/s41239-018-0113-2 · DOI
- 10.7158/D13-002.2015.20.1 · DOI
- 10.1007/978-3-540-73078-1_22 · DOI
- 10.1145/3107990.3107998 · DOI
- 10.1109/TAFFC.2016.2515084 · DOI · Paper page
- grafsgaard, 0, Automatically recognizing facial expression: Predicting engagement and frustration, Proc Educ Data Mining, 43
- ostrow, 0, Enhancing student motivation and learning within adaptive tutors, Proc Int Conf Educational Data Mining, 668
- 10.18608/jla.2016.32.14 · DOI
- 10.1007/978-3-642-39112-5_120 · DOI
- 10.1007/s40593-018-0165-4 · DOI · Paper page
- 10.1145/3079628.3079669 · DOI
- 10.1145/3170358.3170397 · DOI
- 10.1016/j.jbusres.2012.12.021 · DOI · Paper page
- 10.1016/j.chb.2017.10.010 · DOI · Paper page
- pappas, 0, Making Sense of Learning Analytics with a Configurational Approach, Proc LAK Workshop Smart Environ Analytics Video-Based Learn, 42
- 10.1109/TLT.2016.2639508 · DOI · Paper page
- paramythis, 0, Adaptive learning environments and e-learning standards, Proc 2nd Eur Conf e-Learn, 369
- becker, 2018, Horizon report 2018 higher education edition brought to you by educause
- 10.1177/0146621617697959 · DOI · Paper page
- 10.1007/s10462-015-9432-z · DOI
- towle, 2005, Designing Adaptive Learning Environments With Learning Design, 215
- 10.1145/3170358.3170409 · DOI · Paper page
- 10.1016/j.ijinfomgt.2019.02.003 · DOI · Paper page
- 10.1007/978-3-030-02631-8_5 · DOI
- 10.1080/02699931.2014.975188 · DOI · Paper page
- 10.3102/00346543065003245 · DOI · Paper page
- 10.1145/3027385.3027447 · DOI · Paper page
- 10.1007/978-3-540-89208-3_324 · DOI · Paper page
- 10.1145/3027385.3027431 · DOI
- 10.1016/j.chb.2018.11.008 · DOI
- 10.1109/TAFFC.2014.2316163 · DOI · Paper page
- 10.1007/s11257-015-9167-1 · DOI
- 10.1080/00461520.2014.989230 · DOI · Paper page
- 10.1016/j.compedu.2013.07.004 · DOI
- 10.1109/TLT.2016.2633268 · DOI
- 10.1145/2856767.2856784 · DOI · Paper page
- 10.1007/s10648-006-9012-5 · DOI · Paper page
- 10.1109/IC3e.2013.6735958 · DOI
- 10.1016/j.lindif.2017.03.013 · DOI · Paper page
- 10.1145/2567574.2567609 · DOI
- 10.1007/978-3-319-39583-8_43 · DOI
- 10.1007/s10648-010-9130-y · DOI · Paper page
- 10.1145/3012430.3012602 · DOI
- 10.3758/s13421-014-0462-6 · DOI
- 10.1016/j.jbusres.2015.10.134 · DOI · Paper page
- 10.3102/0034654313480891 · DOI · Paper page
- 10.1016/j.chb.2017.02.059 · DOI · Paper page
- 10.1016/j.cedpsych.2018.01.002 · DOI · Paper page
- 10.1111/bjet.12854 · DOI · Paper page
- joseph, 2005, Engagement tracing: using response times to model student disengagement, Artificial Intelligence in Education Supporting Learning Through Intelligent and Socially Informed Technology, 125, 88
- 10.1145/2460296.2460330 · DOI · Paper page
- 10.1108/EJM-10-2017-0707 · DOI · Paper page
- 10.1016/j.jbusres.2014.07.006 · DOI · Paper page
- 10.7208/chicago/9780226702797.001.0001 · DOI · Paper page
- 10.1016/j.compedu.2018.02.007 · DOI · Paper page
- 10.1109/ACII.2009.5349483 · DOI
- 10.1080/00461520.2014.999920 · DOI · Paper page
- wang, 0, Physiological measurement on students’ engagement in a distributed learning environment, Proc 2nd Int Conf Physiological Comput Syst, 149
- 10.1080/00461520.2015.1004068 · DOI · Paper page
Cited by
Pappas, I. O., & Woodside, A. G. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102310 · Cited by 2,254
Self-citationAI-enabled adaptive learning systems: A systematic mapping of the literature
Kabudi, T., Pappas, I., & Olsen, D. H. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100017 · Cited by 734
Self-citationCao, G., Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2021) · Technovation · doi:10.1016/j.technovation.2021.102312 · Cited by 571
du Plooy, E., Casteleijn, D., & Franzsen, D. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39630 · Cited by 310
Unpacking student engagement in higher education learning analytics: a systematic review
Bergdahl, N., Bond, M., Sjöberg, J., Dougherty, M., & Oxley, E. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00493-y · Cited by 138
Metadata
- Authors
- Papamitsiou, Z., Pappas, I. O., Sharma, K., & Giannakos, M. N.
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2020-10-01
- Volume
- 13
- Issue
- 4
- Pages
- 689-703
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W3082311223
- Cited by
- 86
- References
- 21
- FWCI
- 3.79
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.