The DOI matches its Crossref record; 220 of the 220 references it cites were found in a registry or an index.
A Systematic Meta-analysis of the Reliability and Validity of Subjective Cognitive Load Questionnaires in Experimental Multimedia Learning Research
Abstract
For more than three decades, cognitive load theory has been addressing learning from a cognitive perspective. Based on this instructional theory, design recommendations and principles have been derived to manage the load on working memory while learning. The increasing attention paid to cognitive load theory in educational science quickly culminated in the need to measure its types of cognitive load — intrinsic, extraneous, and germane cognitive load which additively contribute to the overall load. In this meta-analysis, four frequently used cognitive load questionnaires were examined concerning their reliability (internal consistency) and validity (construct validity and criterion validity). Results revealed that the internal consistency of the subjective cognitive load questionnaires can be considered satisfactory across all four questionnaires. Moreover, moderator analyses showed that reliability estimates of the cognitive load questionnaires did not differ between educational settings, domains of the instructional materials, presentation modes, or number of scale points. Correlations among the cognitive load types partially contradict theory-based assumptions, whereas correlations with learning-related variables support assumptions derived from cognitive load theory. In particular, results seem to support the three-factor model consisting of intrinsic cognitive load, extraneous cognitive load, and germane cognitive load. Results are discussed in relation to current trends in cognitive load theory and recommendations for the future use of cognitive load questionnaires in experimental research are suggested.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cognitive load was assessed using an adapted version of the validated German-language multidimensional cognitive load questionnaire developed by Krieglstein et al. (2022), which is grounded in Cognitive Load Theory.”
From Generative AI and Learning Dynamics in Higher Education: A Longitudinal Empirical Study (2026) · cites this paper as Krieglstein et al. (2022)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Making sense of cronbach’s alpha.
Tavakol, M., & Dennick, R. (2011) · International Journal of Medical Education · doi:10.5116/ijme.4dfb.8dfd · Cited by 15,126
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048
The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education
Taber, K. S. (2018) · Research in Science Education · doi:10.1007/s11165-016-9602-2 · Cited by 10,679
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 1×
“The alpha value represents the average of all split-half reliabilities (Cortina, 1993; Ferketich, 1990).”
BackgroundPolit, D. F., & Beck, C. T. (2006) · Research in Nursing & Health · doi:10.1002/nur.20147 · Cited by 7,510 · cited 1×
“In this vein, the content validity index (CVI) is calculated based on item relevance ratings made by experts (Polit & Beck, 2006).”
MethodThe magical number 4 in short-term memory: A reconsideration of mental storage capacity
Cowan, N. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01003922 · Cited by 6,779
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
Nine Ways to Reduce Cognitive Load in Multimedia Learning
Mayer, R. E., & Moreno, R. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_6 · Cited by 4,207
Effect size, confidence interval and statistical significance: a practical guide for biologists
Nakagawa, S., & Cuthill, I. C. (2007) · Biological Reviews · doi:10.1111/j.1469-185x.2007.00027.x · Cited by 3,835 · cited 1×
“An effect size was confirmed as significant (p < 0.05) when the associated confidence interval (CI) does not include zero (Hedges et al., 1992; Nakagawa & Cuthill, 2007).”
BackgroundInfluentialStarting at the beginning: an introduction to coefficient alpha and internal consistency
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8001_18 · Cited by 3,794
Richardson, M., Abraham, C., & Bond, R. (2012) · Psychological Bulletin · doi:10.1037/a0026838 · Cited by 3,646 · cited 1×
“This procedure was also applied by several meta-analyses that have aggregated correlation coefficients (e.g., Capaldi et al., 2014; Edwards & Holtzman, 2017; Richardson et al., 2012).”
MethodDesign Experiments in Educational Research
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003) · Educational Researcher · doi:10.3102/0013189x032001009 · Cited by 3,548
Cognitive Load Theory and Instructional Design: Recent Developments
Paas, F., Renkl, A., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_1 · Cited by 3,041
Short-term retention of individual verbal items.
Peterson, L., & Peterson, M. J. (1959) · Journal of Experimental Psychology · doi:10.1037/h0049234 · Cited by 2,939
Effect size guidelines for individual differences researchers
Gignac, G. E., & Szodorai, E. T. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.06.069 · Cited by 2,860
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
The reliability of a two-item scale: Pearson, Cronbach, or spearman-brown?
Eisinga, R., Grotenhuis, M. T., & Pelzer, B. (2013) · International Journal of Public Health · doi:10.1007/s00038-012-0416-3 · Cited by 2,695
Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…
Hayes, A. F., & Coutts, J. J. (2020) · Communication Methods and Measures · doi:10.1080/19312458.2020.1718629 · Cited by 2,675
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 2×
“On the assumption that reliability estimates represent different populations, a random-effects model was preferred to a fixed-effects model for the meta-analysis (Higgins et al., 2009).”
MethodThanks coefficient alpha, we’ll take it from here.
McNeish, D. (2018) · Psychological Methods · doi:10.1037/met0000144 · Cited by 2,533
Cognitive Architecture and Instructional Design: 20 Years Later
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09465-5 · Cited by 2,337 · cited 9×
“On the one hand, several design recommendations have been made regarding how best to support learners to transfer new information into long-term memory and integrate it with pre-existing knowledge (Sweller et al., 2019).”
BackgroundInfluentialMayer, R. E. (2001) · doi:10.1017/cbo9781139164603 · Cited by 2,304
Paas, F. G. W. C. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.4.429 · Cited by 2,150
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load
Sweller, J. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9128-5 · Cited by 2,020
Cognitive Theory of Multimedia Learning
Mayer, R. E. (2014) · The Cambridge Handbook of Multimedia Learning · doi:10.1017/cbo9781139547369.005 · Cited by 1,976
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,946
Validity and reliability of measurement instruments used in research
Kimberlin, C. L., & Winterstein, A. G. (2008) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp070364 · Cited by 1,805
Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma
Revelle, W., & Zinbarg, R. E. (2008) · Psychometrika · doi:10.1007/s11336-008-9102-z · Cited by 1,793
Evaluation of methods used for estimating content validity
Almanasreh, E., Moles, R. J., & Chen, T. F. (2018) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2018.03.066 · Cited by 1,785
Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application
Cook, D. A., & Beckman, T. J. (2006) · The American Journal of Medicine · doi:10.1016/j.amjmed.2005.10.036 · Cited by 1,745
On the Use of Beta Coefficients in Meta-Analysis.
Peterson, R. A., & Brown, S. P. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.175 · Cited by 1,643
An Embedded-Processes Model of Working Memory
Cowan, N. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139174909.006 · Cited by 1,521
Validity and reliability in quantitative studies
Heale, R., & Twycross, A. (2015) · Evidence Based Nursing · doi:10.1136/eb-2015-102129 · Cited by 1,518
Why Some Material Is Difficult to Learn
Sweller, J., & Chandler, P. (1994) · Cognition and Instruction · doi:10.1207/s1532690xci1203_1 · Cited by 1,455
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 1×
“To compare the effect sizes (i.e., aggregated reliability and validity estimates) for significant differences, the 95% percent confidence intervals (CI) were consulted (Cumming & Finch, 2005).”
MethodChapter 20 What are the differences between long-term, short-term, and working memory?
Cowan, N. (2008) · Progress in Brain Research · doi:10.1016/s0079-6123(07)00020-9 · Cited by 1,394
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994) · Journal of Educational Psychology · doi:10.1037/0022-0663.86.1.122 · Cited by 1,198 · cited 2×
“This assessment is usually made after learning has taken place (Jiang & Kalyuga, 2020; Paas & Van Merriënboer, 1994).”
BackgroundMethodThe relationship between nature connectedness and happiness: a meta-analysis
Capaldi, C. A., Dopko, R. L., & Zelenski, J. M. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00976 · Cited by 1,183
Cognitive load theory, educational research, and instructional design: some food for thought
de Jong, T. (2010) · Instructional Science · doi:10.1007/s11251-009-9110-0 · Cited by 1,180 · cited 1×
“However, cognitive load measurement is still an ongoing challenge in educational research (e.g., Ayres, 2018; de Jong, 2010; Kirschner et al., 2011; Moreno, 2010).”
BackgroundField, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162
Development of an instrument for measuring different types of cognitive load
Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0334-1 · Cited by 1,066 · cited 3×
“Following the generally accepted definition of the ICL (e.g., Leppink et al., 2013), the domain-specific prior knowledge should correlate negatively with this cognitive load type.”
BackgroundMethodInfluentialExpertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 2×
“Accordingly, the interaction between the learner’s expertise and the instructional procedures can lead to a reversal effect indicating that novices benefit more from an instructional intervention (reduced ECL), whereas experts may not benefit due to redundancies and associated inferences (no change or even enhanced EC…”
BackgroundSystematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017 · cited 1×
“However, problems concerning the heterogeneity of the primary studies cannot be fully ruled out (e.g., Thompson, 1994).”
BackgroundValidity and Reliability in Social Science Research
Drost, E. A. (2011) · Education Research and Perspectives · doi:10.70953/erpv38.11005 · Cited by 982
Implications of Cognitive Load Theory for Multimedia Learning
PAAS, F. G. W. C., & Sweller, J. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139547369.004 · Cited by 924
Cognitive load theory and educational technology
Sweller, J. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-019-09701-3 · Cited by 911
Review of Educational Measurement.
Langmuir, C. R. (1952) · Journal of Educational Psychology · doi:10.1037/h0051392 · Cited by 906
Direct Measurement of Cognitive Load in Multimedia Learning
Brunken, R., Plass, J. L., & Leutner, D. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_7 · Cited by 896
Rethinking Internal Consistency in Cronbach’s Alpha
Vaske, J. J., Beaman, J., & Sponarski, C. C. (2017) · Leisure Sciences · doi:10.1080/01490400.2015.1127189 · Cited by 856
Current Concerns in Validity Theory
Kane, M. T. (2001) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2001.tb01130.x · Cited by 753
Cognitive Load Theory: How Many Types of Load Does It Really Need?
Kalyuga, S. (2011) · Educational Psychology Review · doi:10.1007/s10648-010-9150-7 · Cited by 725
Assimilating complex information
Pollock, E., Chandler, P., & Sweller, J. (2002) · Learning and Instruction · doi:10.1016/s0959-4752(01)00016-0 · Cited by 692 · cited 1×
“In addition, the isolated elements effect describes the learning-beneficial effect when information with a high element interactivity are learned in an isolated form in the first step (Pollock et al., 2002).”
BackgroundAveraging correlation coefficients: Should Fisher’s z transformation be used?
Silver, N. C., & Dunlap, W. P. (1987) · Journal of Applied Psychology · doi:10.1037/0021-9010.72.1.146 · Cited by 606
Klepsch, M., Schmitz, F., & Seufert, T. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01997 · Cited by 601
Nelson, T. O., & Dunlosky, J. (1991) · Psychological Science · doi:10.1111/j.1467-9280.1991.tb00147.x · Cited by 580
A Cognitive Load Approach to Collaborative Learning: United Brains for Complex Tasks
Kirschner, F., Paas, F., & Kirschner, P. A. (2009) · Educational Psychology Review · doi:10.1007/s10648-008-9095-2 · Cited by 547
Quantifying construct validity: Two simple measures.
Westen, D., & Rosenthal, R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.3.608 · Cited by 543
Limitations of Coefficient Alpha as an Index of Test Unidimensionality1
Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977) · Educational and Psychological Measurement · doi:10.1177/001316447703700403 · Cited by 522
Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. A. (2019) · Psychological Assessment · doi:10.1037/pas0000648 · Cited by 521
Effects of pairs of problems and examples on task performance and different types of cognitive load
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.12.001 · Cited by 520
Effect of the number of scale points on reliability: A Monte Carlo approach.
Lissitz, R. W., & Green, S. B. (1975) · Journal of Applied Psychology · doi:10.1037/h0076268 · Cited by 488
Instructional Efficiency: Revisiting the Original Construct in Educational Research
van Gog, T., & Paas, F. (2008) · Educational Psychologist · doi:10.1080/00461520701756248 · Cited by 473
Vacha‐Haase, T. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058001002 · Cited by 472
Robust estimation of Cronbach's alpha
Christmann, A., & Aelst, S. V. (2005) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2005.05.012 · Cited by 442
Mayer, R. E. (1996) · Educational Psychology Review · doi:10.1007/bf01463939 · Cited by 433
Effects of augmented reality on learning and cognitive load in university physics laboratory courses
Thees, M., Kapp, S., Strzys, M. P., Beil, F., Lukowicz, P., & Kühn, J. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106316 · Cited by 416
Testing the Difference Between Reliability Coefficients Alpha and Omega
Deng, L., & Chan, W. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164416658325 · Cited by 405
Four Ways of Considering Emotion in Cognitive Load Theory
Plass, J. L., & Kalyuga, S. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09473-5 · Cited by 404
Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects
Ginns, P. (2006) · Learning and Instruction · doi:10.1016/j.learninstruc.2006.10.001 · Cited by 403 · cited 1×
“To counter such additional cognitive processes, CLT recommends following the principles of spatial and temporal contiguity (Ginns, 2006).”
BackgroundSelf-citationSignaling in virtual reality influences learning outcome and cognitive load
Albus, P., Vogt, A., & Seufert, T. (2021) · Computers & Education · doi:10.1016/j.compedu.2021.104154 · Cited by 393
Using subjective measures to detect variations of intrinsic cognitive load within problems
Ayres, P. (2006) · Learning and Instruction · doi:10.1016/j.learninstruc.2006.09.001 · Cited by 381
Making Reliability Reliable: A Systematic Approach to Reliability Coefficients
Cho, E. (2016) · Organizational Research Methods · doi:10.1177/1094428116656239 · Cited by 378
Matell, M. S., & Jacoby, J. (1972) · Journal of Applied Psychology · doi:10.1037/h0033601 · Cited by 376
Survey questionnaire translation and assessment
Harkness, J., Pennell, B., & Schoua‐Glusberg, A. (2004) · Methods for Testing and Evaluating Survey Questionnaires · doi:10.1002/0471654728.ch22 · Cited by 370
Cognitive load in multimedia learning environments: A systematic review
Bayraktar, D. M., Çoşğun, V., & Altan, T. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103618 · Cited by 367
Psychometrics Is Datametrics: The Test Is Not Reliable
Thompson, B., & Vacha‐Haase, T. (2000) · Educational and Psychological Measurement · doi:10.1177/0013164400602002 · Cited by 357
Coefficient alpha and related internal consistency reliability coefficients.
Osburn, H. G. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.3.343 · Cited by 356
Adapting Tests for Use in Multiple Languages and Cultures
Hambleton, R. K., & Patsula, L. N. (1998) · Social Indicators Research · doi:10.1023/a:1006941729637 · Cited by 331
Is the Meta-Analysis of Correlation Coefficients Accurate When Population Correlations Vary?
Field, A. P. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.444 · Cited by 329
The Redundancy Principle in Multimedia Learning
Kalyuga, S., & Sweller, J. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139547369.013 · Cited by 310
The evolution of cognitive load theory and its application to medical education
Leppink, J., & Van den Heuvel, A. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0192-x · Cited by 299
Managing cognitive load—recent trends in cognitive load theory
Bannert, M. (2002) · Learning and Instruction · doi:10.1016/s0959-4752(01)00021-4 · Cited by 294
The reliability of relationship satisfaction: a reliability generalization meta-analysis.
Graham, J. M., Diebels, K. J., & Barnow, Z. B. (2011) · Journal of family psychology · doi:10.1037/a0022441 · Cited by 289
Coefficient Alpha: Interpret With Caution
Panayiotis Panayides, A. (2013) · Europe’s Journal of Psychology · doi:10.5964/ejop.v9i4.653 · Cited by 277
A meta-analysis of how signaling affects learning with media
Schneider, S., Beege, M., Nebel, S., & Rey, G. D. (2018) · Educational Research Review · doi:10.1016/j.edurev.2017.11.001 · Cited by 268
Self-citationKlepsch, M., & Seufert, T. (2020) · Instructional Science · doi:10.1007/s11251-020-09502-9 · Cited by 266
Liao, C.-W., Chen, C., & Shih, S.-J. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.01.013 · Cited by 265
Reliability Methods: A Note on the Frequency of Use of Various Types
Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970691 · Cited by 260
Processes of Working Memory in Mind and Brain
Jonides, J., Lacey, S. C., & Nee, D. E. (2005) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2005.00323.x · Cited by 257
Optimising worked example instruction: Different ways to increase germane cognitive load
Paas, F., & van Gog, T. (2006) · Learning and Instruction · doi:10.1016/j.learninstruc.2006.02.004 · Cited by 252
Altmeyer, K., Kapp, S., Thees, M., Malone, S., Kühn, J., & Brünken, R. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12900 · Cited by 245
Schmeck, A., Opfermann, M., van Gog, T., PAAS, F. G. W. C., & Leutner, D. (2014) · Instructional Science · doi:10.1007/s11251-014-9328-3 · Cited by 237
The Split-Attention Principle in Multimedia Learning
Ayres, P., & Sweller, J. (2014) · The Cambridge Handbook of Multimedia Learning · doi:10.1017/cbo9781139547369.011 · Cited by 234
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · Springer eBooks · doi:10.1007/978-1-4419-8126-4_6 · Cited by 230
A meta-analysis of correlations between depression and first person singular pronoun use
Edwards, T., & Holtzman, N. S. (2017) · Journal of Research in Personality · doi:10.1016/j.jrp.2017.02.005 · Cited by 224
Gilpin, A. R. (1993) · Educational and Psychological Measurement · doi:10.1177/0013164493053001007 · Cited by 223
Piqueras, J. A., Martín-Vivar, M., Sandín, B., Costas, C. S. L., & Pineda, D. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.022 · Cited by 220
Cognitive Load as Motivational Cost
Feldon, D. F., Callan, G., Juth, S., & Jeong, S. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09464-6 · Cited by 218
Mayer, R. E., Mathias, A., & Wetzell, K. (2002) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.8.3.147 · Cited by 214
Psychological Distance Between Categories in the Likert Scale
Wakita, T., Ueshima, N., & Noguchi, H. (2012) · Educational and Psychological Measurement · doi:10.1177/0013164411431162 · Cited by 212
Schroeder, N. L., & Cenkci, A. T. (2018) · Educational Psychology Review · doi:10.1007/s10648-018-9435-9 · Cited by 210
What does germane load mean? An empirical contribution to the cognitive load theory
Debue, N., & van de Leemput, C. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01099 · Cited by 209
Cognitive Load Theory: Historical Development and Relation to Other Theories
Moreno, R., & Park, B. (2010) · Cognitive Load Theory · doi:10.1017/cbo9780511844744.003 · Cited by 204
The Expertise Reversal Effect is a Variant of the More General Element Interactivity Effect
Chen, O., Kalyuga, S., & Sweller, J. (2017) · Educational Psychology Review · doi:10.1007/s10648-016-9359-1 · Cited by 200
Post-examination analysis of objective tests
Tavakol, M., & Dennick, R. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.564682 · Cited by 198
Keep it Coherent: A Meta-Analysis of the Seductive Details Effect
Sundararajan, N., & Adesope, O. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09522-4 · Cited by 197
Can we use Google Scholar to identify highly-cited documents?
Martín‐Martín, A., Orduña‐Malea, E., Harzing, A.-W. K., & López‐Cózar, E. D. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.11.008 · Cited by 195
Mayer, R. E., & Moreno, R. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511844744.009 · Cited by 194
Working Memory, Long-term Memory, and Instructional Design
Sweller, J. (2016) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2015.12.002 · Cited by 192
Cognitive Load and Working Memory in Multimedia Learning: Conceptual and Measurement Issues
Anmarkrud, Ø., Andresen, A., & Bråten, I. (2019) · Educational Psychologist · doi:10.1080/00461520.2018.1554484 · Cited by 188 · cited 1×
“For this purpose, CLT proposes design principles for instructional materials and procedures that aim at reducing unnecessary load on working memory and freeing up capacity for learning-related processing (Anmarkrud et al., 2019).”
Test design : developments in psychology and psychometrics
Embretson, S. E. (1985) · Cited by 188
Fletcher, J. D., & Tobias, S. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511816819.008 · Cited by 183
Internal consistency estimates of reliability
Ferketich, S. L. (1990) · Research in Nursing & Health · doi:10.1002/nur.4770130612 · Cited by 182
Limitations of subjective cognitive load measures in simulation‐based procedural training
Naismith, L. M., Cheung, J. J. H., Ringsted, C. V., & Cavalcanti, R. B. (2015) · Medical Education · doi:10.1111/medu.12732 · Cited by 181
Sungur, S. (2007) · Scandinavian Journal of Educational Research · doi:10.1080/00313830701356166 · Cited by 168
Using mobile devices to enhance inquiry-based learning processes
Becker, S., Klein, P., Gößling, A., & Kühn, J. (2020) · Learning and Instruction · doi:10.1016/j.learninstruc.2020.101350 · Cited by 168
Expertise reversal effect and its instructional implications: introduction to the special issue
Kalyuga, S., & Renkl, A. (2009) · Instructional Science · doi:10.1007/s11251-009-9102-0 · Cited by 167
Brom, C., Stárková, T., & D'Mello, S. K. (2018) · Educational Research Review · doi:10.1016/j.edurev.2018.09.004 · Cited by 163
The Implications of Research on Expertise for Curriculum and Pedagogy
Feldon, D. F. (2006) · Educational Psychology Review · doi:10.1007/s10648-006-9009-0 · Cited by 161
Eysink, T. H. S., de Jong, T., Berthold, K., Kolloffel, B., Opfermann, M., & Wouters, P. (2009) · American Educational Research Journal · doi:10.3102/0002831209340235 · Cited by 158
A meta-analysis of signaling principle in multimedia learning environments
Alpizar, D., Adesope, O. O., & Wong, R. M. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-020-09748-7 · Cited by 157
Differentiating Different Types of Cognitive Load: a Comparison of Different Measures
Korbach, A., Brünken, R., & Park, B. (2018) · Educational Psychology Review · doi:10.1007/s10648-017-9404-8 · Cited by 153
Sweller, J. (2018) · Perspectives on Medical Education · doi:10.1007/s40037-017-0395-4 · Cited by 152
Sweller, J. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09647-0 · Cited by 152
Development of metacognitive skillfulness: A longitudinal study
van der Stel, M., & Veenman, M. V. J. (2009) · Learning and Individual Differences · doi:10.1016/j.lindif.2009.11.005 · Cited by 151
Measurement error and person-specific reliability in multilevel autoregressive modeling.
Schuurman, N. K., & Hamaker, E. L. (2018) · Psychological Methods · doi:10.1037/met0000188 · Cited by 150
Extrinsic Convergent Validity Evidence to Prevent Jingle and Jangle Fallacies
González, O., MacKinnon, D. P., & Muniz, F. B. (2020) · Multivariate Behavioral Research · doi:10.1080/00273171.2019.1707061 · Cited by 148
Context Effects on Survey Responses to Questions About Abortion
Schuman, H., Presser, S., & Ludwig, J. (1981) · Public Opinion Quarterly · doi:10.1086/268652 · Cited by 145
Contemporary cognitive load theory research: The good, the bad and the ugly
Kirschner, P. A., Ayres, P., & Chandler, P. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2010.06.025 · Cited by 144 · cited 2×
“However, cognitive load measurement is still an ongoing challenge in educational research (e.g., Ayres, 2018; de Jong, 2010; Kirschner et al., 2011; Moreno, 2010).”
BackgroundSkulmowski, A., Pradel, S., Kühnert, T., Brunnett, G., & Rey, G. D. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.10.011 · Cited by 136
Self-citationCognitive load theory: more food for thought
Moreno, R. (2010) · Instructional Science · doi:10.1007/s11251-009-9122-9 · Cited by 132
A Meta-analysis of the Segmenting Effect
Rey, G. D., Beege, M., Nebel, S., Wirzberger, M., Schmitt, T. H., & Schneider, S. (2019) · Educational Psychology Review · doi:10.1007/s10648-018-9456-4 · Cited by 131
Self-citationGender Imbalance in Instructional Dynamic Versus Static Visualizations: a Meta-analysis
Castro-Alonso, J. C., Wong, M., Adesope, O., Ayres, P., & PAAS, F. G. W. C. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09469-1 · Cited by 123
Testing for moderator variables in meta‐analysis: Issues and methods
Hall, J. A., & Rosenthal, R. (1991) · Communication Monographs · doi:10.1080/03637759109376240 · Cited by 116
The Rhythm Method: A New Method for Measuring Cognitive Load—An Experimental Dual‐Task Study
Park, B., & Brünken, R. (2014) · Applied Cognitive Psychology · doi:10.1002/acp.3100 · Cited by 111
Learning from video: Factors influencing learners' preconceptions and invested mental effort
Cennamo, K. S. (1993) · Educational Technology Research and Development · doi:10.1007/bf02297356 · Cited by 107
Schweppe, J., & Rummer, R. (2013) · Educational Psychology Review · doi:10.1007/s10648-013-9242-2 · Cited by 106
Zheng, R. Z. (2018) · Cited by 106
Colliot, T., & Jamet, É. (2018) · Educational Technology Research and Development · doi:10.1007/s11423-018-9594-x · Cited by 102
Varying coefficient meta-analytic methods for alpha reliability.
Bonett, D. G. (2010) · Psychological Methods · doi:10.1037/a0020142 · Cited by 97
Glogger‐Frey, I., Gaus, K., & Renkl, A. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.11.002 · Cited by 97
From test validity to construct validity … and back?
Colliver, J. A., Conlee, M. J., & Verhulst, S. J. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04194.x · Cited by 96
Krell, M. (2017) · Cogent Education · doi:10.1080/2331186x.2017.1280256 · Cited by 91
Self-citationZheng, R. Z., & Gupta, U. (2020) · European Journal of STEM Education · doi:10.20897/ejsteme/9252 · Cited by 91
Measuring Cognitive Load: Are There More Valid Alternatives to Likert Rating Scales?
Ouwehand, K., Kroef, A. V. D., Wong, J., & Paas, F. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.702616 · Cited by 90
Beckmann, J. F. (2010) · Learning and Instruction · doi:10.1016/j.learninstruc.2009.02.024 · Cited by 89
Eitel, A., Bender, L., & Renkl, A. (2018) · Applied Cognitive Psychology · doi:10.1002/acp.3479 · Cited by 89
The reliability of romantic love: A reliability generalization meta‐analysis
Graham, J. M., & CHRISTIANSEN, K. (2009) · Personal Relationships · doi:10.1111/j.1475-6811.2009.01209.x · Cited by 88
Nebel, S., Schneider, S., & Rey, G. D. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.09.035 · Cited by 86
Self-citationCurrent Issues and Open Questions in Cognitive Load Research
Brünken, R., Plass, J. L., & Moreno, R. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511844744.014 · Cited by 84
Cook, D. A., Castillo, R. M., Gas, B. L., & Artino, A. R. (2017) · Medical Education · doi:10.1111/medu.13405 · Cited by 83
Anthropomorphism in decorative pictures: Benefit or harm for learning?
Schneider, S., Nebel, S., Beege, M., & Rey, G. D. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000207 · Cited by 82
Self-citationLiao, S., Kruger, J., & Doherty, S. (2020) · The Journal of Specialised Translation · doi:10.26034/cm.jostrans.2020.549 · Cited by 80
Making an Effort Versus Experiencing Load
Klepsch, M., & Seufert, T. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.645284 · Cited by 79
Park, B., Korbach, A., & Brünken, R. (2015) · Educational Technology & Society · Cited by 75
Learning-related emotions in multimedia learning: An application of control-value theory
Stark, L., Malkmus, E., Stark, R., Brünken, R., & Park, B. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.05.003 · Cited by 69
Reconsidering the Voice Principle with Non-native Language Speakers
Davis, R. O., Vincent, J., & Park, T. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103605 · Cited by 69
Goal-Setting in Educational Video Games: Comparing Goal-Setting Theory and the Goal-Free Effect
Nebel, S., Schneider, S., Schledjewski, J., & Rey, G. D. (2016) · Simulation & Gaming · doi:10.1177/1046878116680869 · Cited by 62
Self-citationShould learners use their hands for learning? Results from an eye‐tracking study
Korbach, A., Ginns, P., Brünken, R., & Park, B. (2019) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12396 · Cited by 59
Self-citationBeege, M., Schneider, S., Nebel, S., & Rey, G. D. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106483 · Cited by 59
Self-citationKhatib, S. A. A. (2010) · Cited by 57
Confirmatory Factor Analysis of Cognitive Load Ratings Supports a Two-Factor Model
Jiang, D., & Kalyuga, S. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.3.p216 · Cited by 57
Hafdahl, A. R., & Williams, M. A. (2009) · Psychological Methods · doi:10.1037/a0014697 · Cited by 56
Schneider, S., Dyrna, J., Meier, L., Beege, M., & Rey, G. D. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000209 · Cited by 56
Self-citationExperiments for Educational Evaluation and Improvement
Borman, G. D. (2002) · Peabody Journal of Education · doi:10.1207/s15327930pje7704_2 · Cited by 55
Schneider, S., Häßler, A., Habermeyer, T., Beege, M., & Rey, G. D. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000273 · Cited by 54
Self-citationAdapting tests for use in other languages and cultures.
Ercikan, K., & Lyons-Thomas, J. (2013) · American Psychological Association eBooks · doi:10.1037/14049-026 · Cited by 54
Social entities in educational videos: Combining the effects of addressing and professionalism
Beege, M., Nebel, S., Schneider, S., & Rey, G. D. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.11.051 · Cited by 52
Self-citationLehmann, J., Hamm, V., & Seufert, T. (2018) · Applied Cognitive Psychology · doi:10.1002/acp.3509 · Cited by 51
Skulmowski, A., & Rey, G. D. (2020) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.190 · Cited by 50
Self-citationEmotional design of multimedia learning using background images with motivational cues
Chung, S., & Cheon, J. (2020) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12450 · Cited by 49
On the Stability of Variability: Retest Reliability of Metatraits
Baumeister, R. F. (1991) · Personality and Social Psychology Bulletin · doi:10.1177/0146167291176005 · Cited by 48
Anthropomorphisms in multimedia learning: Attract attention but do not enhance learning?
Stárková, T., Lukavský, J., Javora, O., & Brom, C. (2019) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12359 · Cited by 48
Qualitätsanforderungen an Tests und Fragebogen („Gütekriterien“)
Moosbrugger, H., & Kelava, A. (2020) · Springer eBooks · doi:10.1007/978-3-662-61532-4_2 · Cited by 48
Subjective cognitive load surveys lead to divergent results for interactive learning media
Skulmowski, A., & Rey, G. D. (2020) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.184 · Cited by 47
Self-citationSequencing Tracing with Imagination
Wang, B., Ginns, P., & Mockler, N. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09625-6 · Cited by 47
Self-citationHedges, L. V., Cooper, H. M., & Bushman, B. J. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.1.188 · Cited by 46
Realistic details in visualizations require color cues to foster retention
Skulmowski, A., & Rey, G. D. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.03.012 · Cited by 46
Self-citationInternational Journal for Research in Education
(2023) · The International Journal for Research in Education · doi:10.36771/ijre · Cited by 46
Effect Sizes Based on Correlations
(2009) · doi:10.1002/9780470743386.ch6 · Cited by 45
Nebel, S., Schneider, S., Beege, M., Kolda, F., Mackiewicz, V., & Rey, G. D. (2017) · Educational Technology Research and Development · doi:10.1007/s11423-017-9511-8 · Cited by 45
Self-citationThe Moderating Effects of Group Work on the Relationship Between Motivation and Cognitive Load
Costley, J., & Lange, C. H. (2018) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v19i1.3325 · Cited by 45
The moderating role of arousal on the seductive detail effect in a multimedia learning setting
Schneider, S., Wirzberger, M., & Rey, G. D. (2018) · Applied Cognitive Psychology · doi:10.1002/acp.3473 · Cited by 45
Self-citationZu, T.-L., Hutson, J. P., Loschky, L. C., & Rebello, N. S. (2020) · Journal of Educational Psychology · doi:10.1037/edu0000441 · Cited by 45
Tracing Enhances Recall and Transfer of Knowledge of the Water Cycle
Tang, M. C. S., Ginns, P., & Jacobson, M. J. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09466-4 · Cited by 43
Self-citationThe effect of signaling in dependence on the extraneous cognitive load in learning environments
Beege, M., Nebel, S., Schneider, S., & Rey, G. D. (2020) · Cognitive Processing · doi:10.1007/s10339-020-01002-5 · Cited by 39
Self-citationDeveloping Norm-Referenced Standardized Tests
Miller, L. J. (2020) · Routledge eBooks · doi:10.4324/9781315859811 · Cited by 34
Middle Response Scale Options are Inappropriate for Ideal Point Scales
Dalal, D. K., Carter, N. T., & Lake, C. J. (2013) · Journal of Business and Psychology · doi:10.1007/s10869-013-9326-5 · Cited by 33
Pentapati, K. C., Yeturu, S. K., & Siddiq, H. (2020) · Journal of Oral Biology and Craniofacial Research · doi:10.1016/j.jobcr.2020.10.017 · Cited by 33
Assessment of the Maximal Split-Half Coefficient to Estimate Reliability
Thompson, B. L., Green, S. B., & Yang, Y. (2010) · Educational and Psychological Measurement · doi:10.1177/0013164409355688 · Cited by 32
Bender, L., Renkl, A., & Eitel, A. (2021) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12560 · Cited by 32
Thees, M., Kapp, S., Altmeyer, K., Malone, S., Brünken, R., & Kühn, J. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.705551 · Cited by 30
Research works printed but not linked: 9
- Artino, A. R. (2008). Cognitive load theory and the role of learner experience. An abbreviated review for educational practitioners. AACE Journal, 16, 425–439.
- Baddeley, A. (1986). Working memory. Oxford University Press.
- *Beege, M., Wirzberger, M., Nebel, S., Schneider, S., Schmidt, N., & Rey, G. D. (2019b). Spatial continuity effect vs. spatial contiguity failure. Revising the effects of spatial proximity between related and unrelated representations. Frontiers in Education, 4, 86. https://doi.org/10.3389/feduc.2019b.00086 · DOI
- Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Sage Publications.
- Leahy, W. (2018). Case studies in cognitive load measurement. In R. Z. Zheng (Ed.), Cognitive load measurement and application: A theoretical framework for meaningful research and practice (pp. 199–223). Routledge/Taylor & Francis Group.
- Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). American Council on Education and Macmillan.
- *Schneider, S., Nebel, S., Beege, M., & Rey, G. D. (2018d). The autonomy-enhancing effects of choice on cognitive load, motivation and learning with digital media. Learning and Instruction, 58, 161–172. https://doi.org/10.1016/j.learninstruc.2018d.06.006 · DOI
- Weidenmann, B. (2002). Multicodierung und Multimodalität im Lernprozess [Multicoding and multimodality in the learning process]. In L. J. Issing & P. Klimsa (Eds.), Information und Lernen mit Multimedia (3rd edition, pp. 45–62). Beltz PVU.
- Zavgorodniaia, A., Duran, R., Hellas, A., Seppala, O., & Sorva, J. (2020, September). Measuring the cognitive load of learning to program: A replication study. In J. Maguire, & Q. Cutts (Eds.), United Kingdom & Ireland Computing Education Research Conference (pp. 3–9). https://doi.org/10.1145/3416465.34164 · DOI
Cited by
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Gkintoni, E., Antonopoulou, H., Sortwell, A., & Halkiopoulos, C. (2025) · Brain Sciences · doi:10.3390/brainsci15020203 · Cited by 309
Cognitive load theory and its relationships with motivation: A self-determination theory perspective
Evans, P., Vansteenkiste, M., Parker, P., Kingsford-Smith, A., & Zhou, S. (2024) · Educational Psychology Review · doi:10.1007/s10648-023-09841-2 · Cited by 200
Krieglstein, F., Beege, M., Rey, G. D., Sanchez-Stockhammer, C., & Schneider, S. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09738-0 · Cited by 139
Self-citationEndres, T., Lovell, O., Morkunas, D., Rieß, W., & Renkl, A. (2023) · British Journal of Educational Psychology · doi:10.1111/bjep.12563 · Cited by 57
Metadata
- Authors
- Krieglstein, F., Beege, M., Rey, G. D., Ginns, P., Krell, M., & Schneider, S.
- ORCID
- Krieglstein, Felix
- Venue
- Educational Psychology Review
- Published
- 2022-05-20
- Volume
- 34
- Issue
- 4
- Pages
- 2485-2541
- ISSN
- 1040-726X
- 1573-336X
- Cited by
- 118
- References
- 220
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Technische Universität Chemnitz
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

