The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index; 2 of the 30 sources its claims cite were checked for support.
Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study
Open accessAbstract
Background: Generative AI tools such as ChatGPT are increasingly integrated into higher education. Empirical evidence on how they shape learning dynamics over time remains limited. Previous studies have often examined single learning outcomes, neglecting the interplay between cognitive, motivational, and metacognitive processes that evolve throughout the semester. Objectives: This longitudinal study investigated how generative AI influences students' knowledge gain, motivation, cognitive load, critical thinking, and reflective use across an entire semester. It further explored whether prior knowledge and AI experience moderated these learning effects, addressing concerns about potential Matthew effects. Methods: A quasi‐experimental design was implemented with three parallel classes of business informatics students ( N = 87) assigned to different instructional conditions: tutor‐AI use, unguided AI, and a control group without AI. Data were collected at three measurement points (T1–T3). Results and Conclusion: Students showed significant knowledge gains across all groups, but no additional benefits of AI integration. Motivation remained largely stable; germane cognitive load increased and was positively associated with knowledge gain. Critical thinking remained stable, while reflective use was higher in the AI condition and positively associated with critical thinking. No evidence emerged for a Matthew effect, indicating that AI did not intensify performance inequalities. Overall, the findings did not support the view of generative AI as a consistent learning outcome. Its impact depended on pedagogical framing and learners' metacognitive regulation, underscoring the need for structured guidance and reflective engagement in AI‐supported education.
Overview
CoChat Summary
TL;DR
This longitudinal quasi-experiment with 87 business informatics students found significant knowledge gains across all conditions but no additional benefit of generative AI integration. AI’s impact depended on pedagogical framing and learners’ metacognitive regulation, with reflective use higher in AI conditions and germane cognitive load predicting knowledge gain.
Key findings
- Knowledge increased from T1 to T3 across all groups, F(1, 50) = 29.87, p < 0.001, η²p = 0.374; there was no significant Group or Time × Group effect (p = 0.682 and p = 0.469), and no moderation by prior achievement or prior AI experience (BF01 = 8.70), indicating no Matthew effect.
- Motivation remained stable (no Time effect, p = 0.467) and did not predict knowledge gain (R² = 0.010, p = 0.533). Germane cognitive load increased from T2 to T3, F(1, 37) = 10.86, p = 0.002, η²p = 0.227; GCL (β = 0.51, p = 0.001) and ECL (β = 0.36, p = 0.018) predicted knowledge gain.
- Critical thinking remained stable and did not predict knowledge gain (p = 0.204) or cognitive load. Reflective use was higher in the AI-supported condition than control, F(1, 40) = 20.21, p < 0.001, η²p = 0.336, and positively associated with critical thinking (β = 0.43, p < 0.001), but did not predict knowledge gain (p = 0.652).
- Of 16 hypotheses tested, six were supported and one partially supported.
Methods
- Quasi-experimental longitudinal design across one semester with three parallel classes of second-semester Business Informatics students (N = 87; 19 females, 68 males; age 18–29), assigned to tutor-AI, unguided AI, or control conditions; the same instructor and materials were used, and students completed weekly individual in-class question-generation tasks over a nine-week theory phase.
- Data were collected at T1 (first lecture), T2 (fourth lecture), and T3 (eighth lecture) via anonymised LimeSurvey questionnaires; knowledge was assessed with a 14-item multiple true-false test (maximum 70 points, converted to percentages), alongside self-report measures of motivation, cognitive load (ICL, ECL, GCL), critical thinking, and reflective use.
- Analyses in JASP included repeated-measures ANOVA/ANCOVA, linear regressions, and PROCESS moderation/mediation models; groups were compared either as three conditions or as AI versus control depending on focus and power, with α = 0.05 and complete-case handling.
Limitations
- The paper states that the quasi-experimental design assigned participants by pre-existing class groups rather than random allocation, so class-level confounds and causal caution remain; attrition was high and uneven (T3 missing relative to T1: Group C 38.1%, Group B 25.9%, Group A 12.1%), and complete-case analysis reduced statistical power.
- The paper states further limitations: small and uneven group sizes limited detection of interaction effects; the single business informatics, practice-oriented programme limits generalisability; the approximately three-month semester may be too short for critical thinking or durable AI literacy; no explicit scaffolding or structured prompting was included; novelty/Hawthorne effects may have influenced engagement; and self-report measures were used, with the motivation scale showing low internal consistency (α ≈ 0.50).
- Evident from the text: sample sizes vary widely by construct and measurement point (e.g., ECL T2 N = 33 vs. T3 N = 56; knowledge T1 N = 81 vs. T3 N = 62), and analyses use different complete-case Ns, which complicates cross-construct comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
1 Valid · 1 Not valid · 23 Not checked · 163 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While there are first attempts at longitudinal perspectives, they rarely link adoption patterns of AI to core educational outcomes such as knowledge gain, motivation, or critical thinking (Polyportis 2024).”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 1.7 | Research Gap and Aim
What the sources say
- Partly supportedcites Polyportis 2024 sourceExperimental — accuracy not yet measured
The Introduction describes this study as a longitudinal investigation of students’ ChatGPT usage, and the Results analyze changes in usage, trust, perceived behavioral control, and emotional creepiness. The Limitations section says the study did not examine details of how students used ChatGPT for academic purposes. This supports the paper as an example that does not link adoption to knowledge gain, motivation, or critical thinking, but it does not establish that longitudinal studies rarely make such links. (LLM, conf=0.86)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Few studies, however, track how cognitive load evolves across a semester (Sweller 1988;Leppink et al. 2014).”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 1.4 | Cognitive Load
What the sources say
- Not checked — no full text of the sourcecites Sweller 1988 source
- Could not verifycites Leppink et al. 2014 sourceExperimental — accuracy not yet measured
Study II (section 3) measured cognitive load after studying and after a subsequent task, while Study I’s Procedure (section 2.1.2) describes a questionnaire completed at the end of a class. Neither passage addresses whether few studies track cognitive load across a semester. Searches for semester-long and longitudinal tracking found no such statement in our copy. (LLM, conf=0.73)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reflective use was higher in the AI condition and positively associated with critical thinking”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > Abstract
plainstates an associationn = 87- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reflective use was higher in the AI condition”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > Abstract
plainn = 87- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“but no additional benefits of AI integration”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > Abstract
plainn = 87- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
103 more claims
Create a free account to see all 188 claims and to verify the 3 references still open.
- It may state two separate claims as one. 56
- Probably not a substantial claim. 38
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F(1, 39) = 0.82, p = 0.204”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 3.4 | RQ4: Critical Thinking
Reported p = .204 · recomputed p = .371
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F(1, 39) = 0.15, p = 0.693”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 3.3 | RQ3: Cognitive Load
Reported p = .693 · recomputed p = .701
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upF(1, 39) = 0.15reported p = .693 · recomputed p = .701
“F(1, 39) = 0.15, p = .693”
- Doesn’t add upF(1, 39) = 0.82reported p = .204 · recomputed p = .371
“F(1, 39) = 0.82, p = .204”
- Not confirmedF(1, 39) = 0.01reported p = .945 · recomputed p = .921
“F(1, 39) = 0.01, p = .945”
- ConsistentF(1, 50) = 29.87reported p < .001 · recomputed p < .001
“F(1, 50) = 29.87,p < .001”
- ConsistentF(2, 50) = 0.39reported p = .682 · recomputed p = .679
“F(2, 50) = 0.39, p = .682”
30 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 3.5 | RQ5: Reflective Use
“For ECL, the model was nonsignificant, F(2, 53) = 0.39, p = 0.678, R 2 = 0.02.”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 5 | Conclusion
“| Reflective use (T2) | 23 | 16 | 11 | 50 |”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 3.5 | RQ5: Reflective Use
“A linear regression examined whether reflective use (T3) was positively associated with critical thinking (T3).The model was significant, F(1, 58) = 12.95, p < 0.001, R 2 = 0.183.”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 5 | Conclusion
“| Critical thinking (T3) | 27 | 17 | 12 | 56 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 2.2 | Participants
“At T1, data were available for up to 81 students, at T2 for up to 52, and at T3 for up to 62.”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 5 | Conclusion
“| Motivation (T2) | 23 | 16 | 11 | 50 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 4.6 | Limitations and Future Research
“Moreover, the intervention did not include explicit scaffolding or structured prompting, which may have limited the pedagogical impact of AI support.”
§ Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study > 2.1 | Research Design
“Group A received explicit instructional introduction to the AI-as-tutor approach, including structured guidance on effective prompting (role-task-output framing) and explicit contrasts between appropriate and inappropriate forms of generative AI regarding the task.”
Citations
Cites
The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior
Deci, E. L., & Ryan, R. M. (2000) · Psychological Inquiry · doi:10.1207/s15327965pli1104_01 · Cited by 34,210
Intrinsic Motivation and Self-Determination in Human Behavior
Deci, E. L., & Ryan, R. M. (1985) · doi:10.1007/978-1-4899-2271-7 · Cited by 26,772
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006) · Educational Psychologist · doi:10.1207/s15326985ep4102_1 · Cited by 7,776
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · Elsevier eBooks · doi:10.1007/978-1-4419-1428-6_446 · Cited by 6,210
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · doi:10.1007/978-1-4419-8126-4 · Cited by 5,531
Stanovich, K. E. (1986) · Reading Research Quarterly · doi:10.1598/rrq.21.4.1 · Cited by 5,222
Research works printed but not linked: 5
- Hayes A. F., 2022, Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression‐Based Approach
- Liu Y. W.Cui S.Wu andW.Kek Tan.2025.“Generative AI in Education: A Review of Applications Impacts and Future Research Directions.”ICIS 2025 Proceedings (International Conference on Information Systems 2025).https://aisel.aisnet.org/icis2025/learn_curricula/learn_curricula/13.
- Melanou C., 2026, Transforming Education With AI
- Rigney D., 2010, The Matthew Effect: How Advantage Begets Further Advantage
- Wong A. H. L. W.Wong S. C.Hooi andS. S.Lee.2024.Student‐Generated Questions: A Novel Approach for Encouraging Cognitive EngagementHigher Education Conference in Singapore (HECS) 2024. National University of Singapore.https://blog.nus.edu.sg/hecs/hecs2024‐ahpwong‐et‐al‐2/.
Cited by
Artificial Intelligence and Human Cognitive Development: A Thematic Analysis
GUDIA, R. (2026) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2026.v08i06.91433
Wei, J. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.2029407
Metadata
- Authors
- Melanou, C., Beege, M., & Kimmig, M.
- Affiliations
- Baden-Wuerttemberg Cooperative State University (DE)
- University of Education Freiburg (DE)
- Corresponding author
- Melanou, Chrysanthi
- Venue
- Journal of Computer Assisted Learning
- Publisher
- Wiley
- Published
- 2026-09-10
- Volume
- 42
- Issue
- 5
- ISSN
- 0266-4909 (ISSN-L)
- 1365-2729
- OpenAlex
- W7212183669
- Semantic Scholar
- abb3784e5c6bb84a28c373e0942d9bf82992f846
- Cited by
- 2
- References
- 56
- Citations by year
- 2026: 2
- FWCI
- 9.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (vor)
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Locations
- Journal of Computer Assisted Learning (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- generative AI in education, ChatGPT, higher education, knowledge improvement, student motivation, cognitive load, critical thinking, Matthew effect, prior knowledge, AI experience, metacognitive regulation, business informatics education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study. https://app.cochat.ai/papers/10.1002/jcal.70322
- The paper
- Melanou, C., Beege, M., & Kimmig, M. (2026). Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study. Journal of Computer Assisted Learning. https://doi.org/10.1002/jcal.70322
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
