Cited by 45Innovative Teaching and Learning MethodsOnline Learning and AnalyticsEducational Strategies and Epistemologies

Temporal Assessment of Self-Regulated Learning by Mining Students’ Think-Aloud Protocols

Open access
Lim, Lyn, Bannert, Maria, van der Graaf, Joep, Molenaar, Inge, Fan, Yizhou, Kilgour, Jonathan, Moore, Johanna, Gašević, Dragan·2021·Frontiers in Psychology

Abstract

It has been widely theorized and empirically proven that self-regulated learning (SRL) is related to more desired learning outcomes, e.g., higher performance in transfer tests. Research has shifted to understanding the role of SRL during learning, such as the strategies and learning activities, learners employ and engage in the different SRL phases, which contribute to learning achievement. From a methodological perspective, measuring SRL using think-aloud data has been shown to be more insightful than self-report surveys as it helps better in determining the link between SRL activities and learning achievements. Educational process mining on the basis of think-aloud data enables a deeper understanding and more fine-grained analyses of SRL processes. Although students’ SRL is highly contextualized, there are consistent findings of the link between SRL activities and learning outcomes pointing to some consistency of the processes that support learning. However, past studies have utilized differing approaches which make generalization of findings between studies investigating the unfolding of SRL processes during learning a challenge. In the present study with 29 university students, we measured SRL via concurrent think-aloud protocols in a pre-post design using a similar approach from a previous study in an online learning environment during a 45-min learning session, where students learned about three topics and wrote an essay. Results revealed significant learning gain and replication of links between SRL activities and transfer performance, similar to past research. Additionally, temporal structures of successful and less successful students indicated meaningful differences associated with both theoretical assumptions and past research findings. In conclusion, extending prior research by exploring SRL patterns in an online learning setting provides insights to the replicability of previous findings from online learning settings and new findings show that it is important not only to focus on the repertoire of SRL strategies but also on how and when they are used.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2021.749749

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Learners often struggle with effective learning regulation, particularly in digital settings, leading to missing learning outcome effects.

    “learners have been found to often struggle with effective learning regulation, particularly in digital settings, leading to missing learning outcome effects (e.g., Azevedo & Feyzi-Behnagh, 2011;Lim et al., 2021).”

    From How do students regulate their learning with a genAI chatbot? (2026) · cites this paper as Lim et al., 2021

    The Introduction states that students have difficulty regulating their learning in digital or online settings and that self-regulated learning is associated with better learning outcomes. It does not establish that those difficulties cause “missing learning outcome effects”; the Results instead report a significant learning gain in the study’s online setting. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 84 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. Gatta R. Vallati M. Lenkowicz J. Rojas E. Damiani A. Sacchi L.. (2017). “Generating and comparing knowledge graphs of medical processes using pMineR.” in Proceedings of the Knowledge Capture Conference; December 4–6, 2017; 1– 4.
  2. Günther C. W. Van Der Aalst W. M. (2007). “Fuzzy mining–adaptive process simplification based on multi-perspective metrics.” in International Conference on Business Process Management; September 24–28, 2007; 328–343.
  3. Kia F. S. Hatala M. Baker R. S. Teasley S. D. (2021). “Measuring students’ self-regulatory phases in LMS with behavior and real-time self report.” in LAK21: 11th International Learning Analytics and Knowledge Conference; April 12–16, 2021; 259–268.
  4. Matcha W. Gašević D. Uzir N. A. Jovanović J. Pardo A. (2019). “Analytics of learning strategies: associations with academic performance and feedback.” in Proceedings of the 9th International Conference on Learning Analytics & Knowledge; March 4–8, 2019; 461–470.
  5. Saint J. Fan Y. Singh S. Gasevic D. Pardo A. (2021). “Using process mining to analyse self-regulated learning: a systematic analysis of four algorithms.” in LAK21: 11th International Learning Analytics and Knowledge Conference; April 12–16, 2021; 333–343.
  6. Weijters A. van Der Aalst W. M. De Medeiros A. A. (2006). Process Mining With the HeuristicsMiner Algorithm. Vol. 166. Technische Universiteit Eindhoven, 1–34.

Cited by

Sorted by citations

Authors
Lim, L., Bannert, M., van der Graaf, J., Molenaar, I., Fan, Y., Kilgour, J., Moore, J., & Gašević, D.
Affiliations
  • Technical University of Munich (DE)
  • Radboud University Nijmegen (NL)
  • University of Edinburgh (GB)
  • Monash University (AU)
Corresponding author
Lim, Lyn
Venue
Frontiers in Psychology
Published
2021-11-04
Volume
12
Pages
749749-749749
ISSN
1664-1078 (ISSN-L)
OpenAlex
W3213710532
Cited by
44
References
84
Citations by year
2026: 10 · 2025: 13 · 2024: 7 · 2023: 10 · 2022: 4
FWCI
7.69
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Monash University Research Portal (Monash University) (submittedVersion, cc-by)
  • HAL (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion)
  • mediaTUM (Technical University of Munich) (submittedVersion, other-oa)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
Funders
  • NWO (NWO464.18.104)
  • ESRC (ES/S015701/1)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
Keywords
student self-regulation, think-aloud, process mining, educational process mining, online learning, transfer performance, learning gains, replication of findings, university students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review