CoChat
Sign Up
Cited by 601Visual and Cognitive Learning ProcessesInnovative Teaching and Learning MethodsPsychometric Methodologies and Testing

Development and Validation of Two Instruments Measuring Intrinsic, Extraneous, and Germane Cognitive Load

Open access
Klepsch, Melina, Schmitz, Florian, Seufert, Tina·2017·Frontiers in Psychology
DOI 10.3389/fpsyg.2017.01997PMC PMC5696680
Open this paper in CoChat — free accountOpen PDF

Abstract

Cognitive Load Theory is one of the most powerful research frameworks in educational research. Beside theoretical discussions about the conceptual parts of cognitive load, the main challenge within this framework is that there is still no measurement instrument for the different aspects of cognitive load, namely intrinsic, extraneous, and germane cognitive load. Hence, the goal of this paper is to develop a differentiated measurement of cognitive load. In study 1 (N = 97) we developed and analyzed two strategies to measure cognitive load in a differentiated way: (1) Informed rating: We trained learners in differentiating the concepts of cognitive load, so that they could rate them in an informed way. They were asked then to rate 24 different learning situations or learning materials related to either high or low intrinsic, extraneous, or germane load. (2) Naïve rating: For this type of rating of cognitive load we developed a questionnaire with 2-3 items for each type of load. With this questionnaire, the same learning situations had to be rated. In the second study (N = between 65 and 95 for each task) we improved the instrument for the naïve rating. For each study, we analyzed whether the instruments are reliable and valid, for study 1, we also checked for comparability of the two measurement strategies. In study 2 we conducted a simultaneous scenario based factor analysis. The informed rating seems to be a promising strategy to assess the different aspects of cognitive load, but it seems not economic and feasible for larger studies and a standardized training would be necessary. The improved version of the naïve rating turned out to be a useful, feasible, and reliable instrument. Ongoing studies analyze the conceptual validity of this measurement with up to now promising results.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper develops and validates two self-report approaches for measuring intrinsic, extraneous, and germane cognitive load: an “informed rating” after CLT training and a “naïve rating” questionnaire usable without training. Across two studies, the informed rating was valid but not economic, while the revised naïve questionnaire was reliable and valid for ICL and ECL and, after item revision, sensitive for GCL.

Key findings

  • In Study 1 (N = 97), the informed rating showed low internal consistency across the three load items (aggregated Cronbach’s α = 0.25), as expected if ICL, ECL, and GCL are separable; the naïve rating showed α = 0.86 for ICL, α = 0.80 for ECL, and α = 0.80 for GCL, with α = 0.86 across all seven items.
  • Study 1 validity: both instruments differentiated low vs. high ICL and ECL, but the naïve rating failed to differentiate low vs. high GCL [GCL naïve: F(1,88) = 0.04, p = 0.84, η² < 0.001], whereas the informed rating did [F(1,88) = 105.84, p < 0.001, η² = 0.55].
  • Study 2 (N = 65–95 per task) revised the GCL items. Reliability was α = 0.81 for ICL, α = 0.86 for ECL, α = 0.67 for the three-item GCL scale, and α = 0.85 for the two-item GCL scale.
  • Study 2 validity: all scales differentiated low vs. high load as expected [ICL: t(76) = 7.85, p < 0.001, d = 0.73; ECL: t(89) = 7.28, p < 0.001, d = 0.94; GCL: t(76) = 3.39, p < 0.01, d = 0.35]. Confirmatory factor analysis favored the three-factor model (Model 2: TLI = 0.951, CFI = 0.970, RMSEA = 0.021, AIC = 845.106) over the one-factor model and slightly over the two-factor model (Model 4: TLI = 0.951, CFI = 0.974, RMSEA = 0.021, AIC = 861.891).

Methods

  • Study 1: 97 German computer science or psychology students in their first or third semester were randomly assigned to informed rating (n = 48) or naïve rating (n = 49). Participants rated 24 hypothetical learning tasks/scenarios across five domains (language learning, biology, mathematics, technology, didactics), designed to vary ICL, ECL, or GCL. Analyses included reliability generalization, mixed ANOVA, contrasts, and correlations with Paas’s mental-effort item.
  • Study 2: an online study with 65–95 participants per task, all German university students majoring in psychology or computer science in their first or second semester. A pilot study with 27 participants tested nine new GCL items; the final questionnaire used two ICL items, three ECL items, and three GCL items. Participants rated 17 tasks across five domains, with each task varying ICL, ECL, and GCL. Analyses included reliability, t-tests for low vs. high load, and a simultaneous scenario-based confirmatory factor analysis comparing competing models.

Limitations

  • The paper states that the informed rating is not economic or feasible for larger studies and would require standardized training; the naïve rating’s GCL scale initially failed validity and required revision.
  • The paper states that Study 1 used hypothetical tasks, could not qualify absolute rating accuracy, and recommends real learning tasks and larger samples; Study 2 used short, artificial tasks without learning outcomes, and the authors note self-report measures require learners to be aware of their cognitive states.
  • The text makes evident additional constraints: no demographic data beyond major and semester were assessed; in Study 2 only 65 of 95 participants completed all tasks; the three-item GCL scale had only moderate reliability (α = 0.67), and the authors recommend deciding whether to include the instructional-support GCL item based on the learning material and reliability checks.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2017.01997

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 73 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The improved version of the naïve rating is a reliable instrument.

    “The improved version of the naïve rating turned out to be a useful, feasible, and reliable instrument.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.
  • Findingreasoning

    Reliability generalization of Cronbach's α for the informed rating resulted in α = 0.25.

    “Reliability generalization of Cronbach’s α resulted in a low α = 0.25, as expected due to the different aspects of load which we measured with the three items.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For the naïve rating, internal consistency across all seven ICL, ECL, and GCL items was α = 0.86.

    “The internal consistency between all ICL, ECL, and GCL items (seven items) for each task in the naïve rating showed an aggregated α of 0.86, which was way higher than expected.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For ICL, there was a main effect of load level, with theoretically low ICL tasks rated significantly lower than theoretically high ICL tasks across both instruments.

    “For ICL (see Figure 1A and Table 2), we found a main effect for the amount of load [F(1,89) = 490.23, p < 0.001, η2 = 0.85]. Theoretically low ICL tasks (M = 1.80, SD = 0.64) have been rated significantly lower than theoretically high ICL tasks (M = 4.29, SD = 1.09) with both instruments.”
    Quote in the paperSubstantial claim
  • Findingown data

    For the informed rating, internal consistency of the three load items was low for all tasks.

    “As expected, reliability was low for all the tasks.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 73
  • May combine two claims. 44
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    In Study 1, the implementation of a variation of one type of load did not affect another type of load.

    “It should be mentioned that we quickly noticed that the implementation of a variation of one type of load did not affect another type of load.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.

  • May combine two claims.

    Findingown data

    For the naïve rating, aggregated Cronbach's alpha was 0.86 for ICL, 0.80 for ECL, and 0.80 for GCL.

    “The aggregated alphas that we obtained though reliability generalization have been as expected: For ICL α = 0.86, for ECL α = 0.80, and for GCL α = 0.80.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    The developed questionnaire directly measures the constitutional parts of cognitive load, i.e., ICL, ECL, and GCL.

    “using a questionnaire with several items that directly measures the constitutional parts of cognitive load, i.e., ICL, ECL, and GCL.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the confirmatory factor analysis, Model 2 had the best fit, Model 1 the worst, Models 3a-3c were inferior to Model 2, and Model 4 had a decent fit.

    “Overall, Model 2, representing the traditional view of three types of cognitive load, had the best fit. Conversely, Model 1 revealed the worst fit. Model 3a to 3c were inferior to Model 2. Model 4, with the recently discussed only two types of load, does reveal a decent fit.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For ECL, the difference between instruments was significant for high ECL tasks but not for low ECL tasks.

    “Contrasts revealed no significantly different ratings between the questionnaires for theoretically low ECL tasks [F(1,95) = 1.30, p = 0.26, η2 = 0.01] but different ratings for theoretically high ECL tasks [F(1,95) = 14.92, p < 0.001, η2 = 0.14].”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    For GCL, there was a main effect of load level, with theoretically low GCL tasks rated significantly lower than theoretically high GCL tasks across instruments.

    “For GCL (see Figure 1C and Table 2), we again found a main effect for the amount of load [F(1,88) = 53.24, p < 0.001, η2 = 0.38]. Again, in general, theoretically low GCL load tasks (M = 3.67, SD = 1.68) have been rated significantly lower than theoretically high GCL tasks (M = 4.88, SD = 1.13).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For GCL, the informed group rated low GCL tasks lower and high GCL tasks higher than the naïve group, whose ratings were high for both.

    “The informed group rated much more sophisticated (low GCL tasks: M = 2.57, SD = 1.29; high GCL tasks: M = 5.14, SD = 1.13), whereas the naïve ratings are on a high level for low GCL tasks (M = 4.68, SD = 1.32) and high GCL tasks (M = 4.88, SD = 1.10).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Study 2, there were significantly different ratings for low versus high load groups for each type of load: ICL t(76)=7.85, p<0.001, d=0.73; ECL t(89)=7.28, p<0.001, d=0.94; GCL t(76)=3.39, p<0.01, d=0.35.

    “Replicating results obtained in study 1, we found significantly different ratings for the low versus high load groups of tasks for each type of load [ICL: t(76) = 7.85, p < 0.001, d = 0.73; ECL: t(89) = 7.28, p < 0.001, d = 0.94; GCL: t(76) = 3.39, p < 0.01, d = 0.35].”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Study 2, reliability generalization showed mean Cronbach's α was 0.81 for ICL, 0.86 for ECL, and 0.67 for GCL with three items.

    “Reliability generalization showed that the mean Cronbach’s α was 0.81 for the ICL scale, α = 0.86 for the ECL scale, and α = 0.67 for the GCL scale with three items.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For GCL, the ratings of both low and high load tasks differed between questionnaires.

    “The rating of theoretically low load tasks differed between questionnaires [F(1,88) = 58.66, p < 0.001 η2 = 0.40] as did the rating for theoretically high load tasks [F(1,88) = 4.82, p < 0.05, η2 = 0.05].”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A competing model based on the current view on CLT with only two factors (and an additional item-specific method factor) followed closely and revealed decent fit.

    “a competing model based on the current view on CLT with only two factors (and an additional item-specific method factor) followed closely, and revealed decent fit as well.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both the informed and naïve instruments showed a significant difference between low and high ICL task ratings.

    “Concerning the validity of the two instruments, respectively, we also found a significant difference between low and high load ratings for each instrument [informed rating: F(1,89) = 293.32, p < 0.001, η2 = 0.77; naïve rating: F(1,89) = 199.96, p < 0.001, η2 = 0.69].”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For the naïve group, correlations between the specific load ratings and the mental-effort item ranged from r = 0.47 to 0.81 for ICL tasks, r = 0.36 to 0.68 for ECL tasks, and r = 0.51 to 0.77 for GCL tasks.

    “For the naïve rating concerning tasks where ICL was varied the correlations of the intrinsic-load rating with the mental-effort item resulted in r between 0.47 and 0.81. The correlations of the extraneous-load rating with the mental-effort item was r between 0.36 and 0.68 for tasks where ECL was varied. When GCL was varied, we found correlations of the germane-load rating and the mental-effort”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For the informed group, correlations between the specific load ratings and the mental-effort item ranged from r = 0.41 to 0.68 for ICL tasks, r = 0.32 to 0.60 for ECL tasks, and r = 0.32 to 0.41 for GCL tasks.

    “We found substantial correlations for almost all ratings of the informed group: For tasks where ICL was varied, we found correlations between the intrinsic-load rating and the mental-effort item of r between 0.41 and 0.68 for the respective tasks. If ECL was varied, we found correlations between the extraneous-load rating and the mental-effort item of r between 0.32 and 0.60. The germane-load rat”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both the informed and naïve instruments showed a significant difference between low and high ECL task ratings.

    “Contrasts reveal the same results for each questionnaire [informed rating: F(1,95) = 174.60, p < 0.001, η2 = 0.65; naïve rating: F(1,95) = 105.32, p < 0.001, η2 = 0.53].”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Study 1 showed that the ease of operationalization does not apply to the concept of GCL, mainly because of the different possibility of understanding the items.

    “Study 1 also showed us that this does not apply to the concept of GCL, mainly because of the different possibility of understanding the items.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Study 2, the three-load model (Model 2) and the two-load model with germane-load items as part of the intrinsic-load scale (Model 4) do not differ much, only Model 2 resulting in minimal better model fit.

    “The evaluated models in Study 2 revealed that the model with three types of load (Model 2) and the model with two types of load with the germane-load items as a part of the intrinsic-load scale (Model 4) do not differ that much, only with Model 2 resulting in minimal better model fit.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Germane load reflects the effort that contributes to the construction of schemas.

    “germane load, which “reflects the effort that contributes to the construction of schemas” (Sweller et al., 1998, p. 259).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sweller et al., 1998 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The learner's prior knowledge plays a fundamental role because new information can be linked with existing schemata.

    “The prior knowledge of the learner plays a fundamental role as new information can be linked with existing schemata (Gerjets et al., 2004).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gerjets et al., 2004 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Element interactivity corresponds to the number of elements that the learner must simultaneously process in working memory while dealing with the task.

    “Element interactivity (Chandler and Sweller, 1996) corresponds to the number of elements that the learner must simultaneously process in working memory while dealing with the task.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chandler and Sweller, 1996

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    Intrinsic load arises from the number of interrelated elements of the learning task.

    “While intrinsic load arises from the number of interrelated elements of the learning task”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some studies have used objective techniques such as dual-task measures or physiological parameters to assess cognitive load.

    “Some other studies used objective techniques, such as dual task measures (e.g., Brünken et al., 2004), or physiological parameters (e.g., heart rate: Paas and van Merriënboer, 1994; eye blink parameters: Goldstein et al., 1992).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Brünken et al., 2004 source

      Not checked in the free run.

    • cites Paas and van Merriënboer, 1994 source

      Not checked in the free run.

    • cites Goldstein et al., 1992 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Extraneous cognitive load is caused by the instructional design of the learning material.

    “Extraneous cognitive load is caused by the instructional design of the learning material.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The pretraining principle reduces intrinsic cognitive load by providing the learner with information about the content before starting with the learning material.

    “The pretraining principle (Mayer, 2005a) reduces ICL by providing the learner with information about the content before starting with the learning material.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mayer, 2005a

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Knowledge is organized and retained in the form of schemata.

    “according to schema theory, knowledge is organized and retained in the form of schemata (Rumelhart, 1981).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Rumelhart, 1981

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Cognitive Load Theory differentiated three independent sources of memory load: intrinsic, extraneous, and germane cognitive load.

    “For a long time, CLT differentiated three independent sources of memory load, namely ICL, ECL, and GCL (e.g., Sweller et al., 1998).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sweller et al., 1998 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Intrinsic cognitive load depends on the element interactivity of the task and the learner's prior knowledge.

    “This load type depends on two different factors (Moreno and Park, 2010), the element interactivity of the task and the learner’s prior knowledge.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moreno and Park, 2010

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The segmenting principle aims at reducing element interactivity by presenting information step by step.

    “The segmenting principle (Mayer and Moreno, 2010) aims at reducing element interactivity by presenting information step by step”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mayer and Moreno, 2010

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Long-term memory is virtually unlimited.

    “The second assumption of CLT is that long-term memory is virtually unlimited (Sweller et al., 1998)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sweller et al., 1998 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Working memory is limited in capacity when processing information and in time when holding information.

    “One of the main assumptions of CLT is that working memory is limited in capacity, when processing information, as well as in time, when it comes to holding information (Paas et al., 2010).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Paas et al., 2010 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    It is difficult to substantially reduce intrinsic load without altering the learning objectives.

    “In general, it is difficult to substantially reduce intrinsic load without altering the learning objectives.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    Extraneous load is caused by additional requirements produced by suboptimal instructional design that are not directly linked to the task.

    “extraneous load is caused by the additional requirements that are produced by suboptimal instructional design and are not directly linked to the task.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning

    Trying to change intrinsic cognitive load can easily end up in also changing extraneous cognitive load.

    “As one may notice, trying to change ICL can easily end up in also changing ECL.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning

    Extraneous cognitive load is reduced when necessary processes such as imagery or linking information are facilitated by design.

    “Consequently, ECL is reduced when necessary processes, such as imagery or linking information, is facilitated by design.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Learners perform better when learning from a combination of text and pictures rather than from pictures alone.

    “The multimedia principle (Mayer, 2001; Fletcher and Tobias, 2005): numerous empirical findings show that learners perform better when learning from a combination of text and pictures rather than from pictures alone.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Mayer, 2001 source

      Not checked in the free run.

    • cites Fletcher and Tobias, 2005

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Paas scale is a modified version of a scale by Bratfisch et al. that was constructed to measure task difficulty.

    “This scale is a modified version of a scale by Bratfisch et al. (1972), which was constructed to measure task difficulty.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bratfisch et al., 1972

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Dual-task measures are intrusive; they disturb the learning process and impose load by themselves.

    “However, the most obvious disadvantage is the intrusiveness of such techniques; they disturb the learning process and impose load by themselves.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Dual-task measures cannot identify the type of load that is measured.

    “Another disadvantage is, as already mentioned for the subjective ratings, that it is also not possible to identify the type of load that is measured.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    According to Sweller et al., Leppink et al. only approved the measurement of intrinsic and extraneous load as being meaningful.

    “As a conclusion, according to Sweller et al. (2011), they only approved the measurement of intrinsic and extraneous load as being meaningful.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sweller et al., 2011 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Split attention occurs when learners are forced to mentally integrate separated sources of information that could be better presented in an integrated way.

    “The split attention effect (Ayres and Sweller, 2005): Split attention occurs when learners are forced to mentally integrate separated sources of information that could be better presented in an integrated way.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ayres and Sweller, 2005

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisereasoning

    The designer of the learning material can manipulate extraneous cognitive load in a relatively easy way.

    “This means that the designer of the learning material can manipulate ECL in a relatively easy way.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Germane load refers to the working memory resources available to deal with the element interactivity associated with intrinsic cognitive load.

    “Hence, a new definition of germane load is that it “refers to the working memory resources available to deal with the element interactivity associated with ICL” (Sweller, 2010b, p. 126).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sweller, 2010b

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning

    Germane load is highly dependent on intrinsic load.

    “The given descriptions make it obvious that germane load is highly dependent on intrinsic load.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A standardized training would be necessary for the informed rating.

    “and a standardized training would be necessary.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: An introduction to Cognitive Load Theory enables informed raters to differentiate intrinsic, extraneous, and germane cognitive load.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The most popular scale for measuring cognitive load is a rating scale developed by Paas.

    “The most popular scale for measuring cognitive load is a rating scale developed by Paas (1992).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Paas, 1992 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Learners are only able to devote germane resources if the amount of extraneous load does not exceed their working memory capacity.

    “Moreover, learners are only able to devote germane resources if the amount of extraneous load is not exceeding their working memory capacity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    It is better to present a text auditorily, not visually, when combined with a picture.

    “The modality principle (Mayer and Moreno, 2003; Low and Sweller, 2005; Mayer, 2005a): According to this principle it is better to present a text auditorily, not visually, when combined with a picture.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Mayer and Moreno, 2003 source

      Not checked in the free run.

    • cites Low and Sweller, 2005

      Not checked in the free run.

    • cites Mayer, 2005a

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Measuring cognitive load is one of the persistent challenges in educational research.

    “Many researchers stated that measuring cognitive load is one of the persistent challenges in educational research (Mayer and Moreno, 2002; Brünken et al., 2003, 2010; Schnotz and Kürschner, 2007; de Jong, 2010; Moreno, 2010).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Mayer and Moreno, 2002 source

      Not checked in the free run.

    • cites Brünken et al., 2003, 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Moreno, 2010

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    In Study 1, each participant was randomly assigned to the informed rating group or the naïve rating group.

    “Each participant was randomly assigned to the informed rating group (n = 48) or the naïve rating group (n = 49).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The improved version of the naïve rating is a useful and feasible instrument.

    “The improved version of the naïve rating turned out to be a useful, feasible, and reliable instrument.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Paas scale consists of one item and uses a 9-point Likert scale ranging from very, very low mental effort to very, very high mental effort.

    “In fact, the scale used by Paas consists of one item, and participants are asked to use a 9-point Likert scale for their responses, ranging from very, very low mental effort (1) to very, very high mental effort (9).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Paas, 1992 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Learners who explain learning material to themselves achieve a higher learning outcome.

    “This effect indicates that learners who explain learning material to themselves achieve a higher learning outcome.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites VanLehn et al., 1992 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Introspective ratings are not highly reliable; overall retrospective measures of load are generally higher than the mean of several measures during learning.

    “The fact that introspective ratings are not highly reliable has been demonstrated by the study of van Gog et al. (2011), where overall retrospective measures of load were generally higher than the mean of several measures during learning.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites van Gog et al., 2011

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    High germane load indicates that learners are engaged and direct their mental resources to learning processes.

    “Therefore, high germane load indicates that learners are engaged and direct their mental resources to learning processes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The informed rating is not economic and feasible for larger studies.

    “but it seems not economic and feasible for larger studies”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.
    • Rests on an assumption: An introduction to Cognitive Load Theory enables informed raters to differentiate intrinsic, extraneous, and germane cognitive load.

  • May combine two claims.

    Premisecitation

    Leppink et al. were missing a positive correlation between items supposed to measure germane cognitive load and learning outcomes, as well as a substantial correlation between the old items and the new one.

    “In their studies, they were missing a positive correlation between items that are supposed to measure GCL and learning outcomes as along with a substantial correlation between the “old” items and the new one.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Leppink et al., 2014 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Eysink et al. found differences for all three types of cognitive load, but these were not theoretically linked or discussed regarding learning outcome.

    “They found differences for all three types of cognitive load, but they were not theoretically linked or discussed regarding the learning outcome, i.e., productive or unproductive aspects.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Eysink et al., 2009 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    Using hypothetical learning scenarios allowed cross-validation of the measures in various settings and comparison of the two instruments for several ratings.

    “However, one strength of this approach was that it allowed us to cross-validate our measures in various settings and to compare our two instruments for several ratings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.

  • Classified as probably not a substantial claim.

    Premisecitation

    Germane cognitive load results from activities required of a learner that facilitate learning and contribute to transfer performance, helping to build correct mental models.

    “Germane cognitive load, the third type of cognitive load, results from activities required of a learner that facilitate learning and contribute to transfer performance, helping to build correct mental models (Paas et al., 2003a).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Paas et al., 2003a

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Cierniak et al. found significant matches between performance data and measured cognitive load.

    “In their studies, they found significant matches between performance data and measured cognitive load.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Cierniak et al., 2009 source

      Not checked in the free run.

    • cites Cierniak, 2011

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Leppink et al. (2014) found that adding an item for invested mental effort considerably decreased internal consistency.

    “Leppink et al. (2014) discuss a similar problem, as their original four items focus on the influence of the carried-out activity. Adding one item, for invested mental effort considerably decreased internal consistency.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leppink et al. (2014) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The instructional support item is appealing because of its face validity for learning material including worked examples or prompts.

    “However, this specific item (“The learning task consisted of elements supporting my comprehension of the task.”) is appealing because of its face validity for learning material including worked examples or prompts.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    Overall, the questionnaire had considerably good item and scale characteristics.

    “Overall, the questionnaire had considerably good item and scale characteristics.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The naïve rating scales were valid for ICL and ECL, but not for GCL.

    “The scales were also valid for ICL and ECL, but not for GCL.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The last GCL item may not be fitting for each learning situation but is important if germane load is varied on purpose.

    “Consequently, this item may not be fitting for each learning situation but is rather important if germane load is varied on purpose.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The naïve rating approach does not require an introduction into CLT.

    “An advantage of this approach is that it does not require an introduction into CLT.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Model 2 was found to offer the best solution, supporting three inter-related factors of cognitive load.

    “In conclusion, Model 2 was found to offer the best solution, thereby supporting three inter-related factors of cognitive load.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The two scales on ICL and ECL again showed satisfying reliability and validity scores, as previously shown in Study 1.

    “The two scales on ICL and ECL again showed satisfying reliability and validity scores, as previously shown in Study 1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The questionnaire allowed differentiation between high and low load levels, albeit not as sensitively in the present form.

    “The questionnaire also allowed us to differentiate between high and low load levels, albeit it appeared not as sensitive in the present form.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    GCL, unlike ICL or ECL, is not imposed by the learning material.

    “It is argued that GCL, other than ICL or ECL, is not imposed by the learning material.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Ayres (2011)

      Not checked in the free run.

    • cites Kalyuga (2011) source

      Not checked in the free run.

    • cites Sweller et al. (2011) source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Germane resources in working memory need to be allocated to deal with the intrinsic load resulting from the learning material.

    “there are germane resources needed in working memory that need to be allocated to deal with the intrinsic load resulting from the learning material.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Ayres (2011)

      Not checked in the free run.

    • cites Kalyuga (2011) source

      Not checked in the free run.

    • cites Sweller et al. (2011) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Participants were able to rate the amount of different aspects of cognitive load as intended by the design and in line with theoretical predictions.

    “Participants were able to rate the amount of different aspects of cognitive load as intended by the design and in line with the theoretical predictions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The informed rating strategy is not very economic and difficult to put into practice in educational research.

    “However, this strategy is not very economic, and, therefore, difficult to put into practice in educational research.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Some participants did not conduct all tasks; 65 out of 95 participants completed all tasks.

    “Unfortunately, some participants did not conduct all tasks (65 out of 95 participants completed all tasks).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The three types of load are clearly distinguished in theory.

    “This is not surprising, given that the three types of load are clearly distinguished in theory (Moreno and Park, 2010).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moreno and Park, 2010

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The last GCL item, targeting supporting elements, is important for studies using worked examples, prompts for learning strategies, or similar elements that should enhance germane load.

    “The last item, which targeted supporting elements, is important for studies using worked examples, prompts for learning strategies, or similar elements that should enhance germane load.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The authors did not test a huge variety of items, as is common when designing a questionnaire.

    “we did not test a huge variety of items, as is common when designing a questionnaire.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Allocated germane resources can be used for schema acquisition and deeper understanding and additionally for elaboration and connections to prior knowledge.

    “These allocated germane resources can be used for schema acquisition and deeper understanding and additionally for elaboration and connections to prior knowledge.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sweller et al. (2011) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Seufert et al. found germane load, called engagement, increases steadily with increasing disfluency and remains high even on the highest disfluency level.

    “Germane load, which was called engagement in their study, on the other hand, increases steadily with increasing disfluency and remains high even on the highest disfluency level.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Seufert et al. (2016) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In Leppink et al.'s approach, those concepts are needed for rephrasing their items to fit the material and to analyze the load resulting from understanding these concepts.

    “In their approach, those concepts are needed for rephrasing their items to fit the material and to analyze the load resulting from understanding these concepts.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leppink et al. (2013, 2014)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Allocated germane resources are nothing other than GCL according to the traditional view of load.

    “When thinking to an end, they are nothing other than GCL according to the traditional view (Moreno and Park, 2010) of load.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moreno and Park, 2010

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Our questionnaire is not that specific to the learning content and therefore only needs adoption based on the material used.

    “Our questionnaire is not that specific to the learning content and, therefore, only needs adoption based on the material you use, e.g., text, video, or podcast.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    We developed our questionnaire to fit a wide variety of domains and studies.

    “We developed our questionnaire to fit a wide variety of domains and studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The items were developed and carefully worded based on given definitions and descriptions of types of load through various researchers.

    “Our items were developed and carefully worded based on given definitions and descriptions of types of load through various researchers (e.g., Moreno and Park, 2010; Sweller, 2010a).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Moreno and Park, 2010

      Not checked in the free run.

    • cites Sweller, 2010a

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The naïve rating is easier to utilize and does not need as much time and cognitive investment, as no introduction to cognitive load is needed.

    “The naïve rating’s advantage is that it is easier to utilize and does not need as much time and cognitive investment, as no introduction to cognitive load is needed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductnone given

    The tasks used to impose the different types of load for validation have been of short duration and were embedded in an artificial learning context.

    “the tasks we used to impose the different types of load for validation have been of short duration and were embedded in an artificial learning context.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Leppink et al.'s differentiated questionnaire is especially useful if one can clearly detect the most relevant concepts within the learning tasks.

    “Their differentiated questionnaire, in contrast to ours, is especially useful if you can clearly detect the most relevant concepts within your learning tasks, e.g., the main terms in a statistics course or a geography text.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leppink et al. (2013, 2014)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Rogers et al. found the novel roll notation resulted in significantly higher germane load than in the group with standard sheet notation.

    “The novel roll notation also resulted in significant higher germane load, than in the group with the standard sheet notation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rogers et al. (2014)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Seufert et al. (2016) found that slightly to medium levels of disfluency can foster learning compared to a fluent text, but learning outcome gets worse when disfluency gets too intense.

    “Their main result was that slightly to medium levels of disfluency can foster learning compared to a fluent text, but that learning outcome gets worse when disfluency gets too intense.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Seufert et al. (2016) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The informed rating strategy requires an introduction into CLT to be given to learners and study participants beforehand.

    “The downside of the informed rating strategy is that an introduction into CLT needs to be given to learners and study participants beforehand.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The general problem remains that cognitive-load ratings are self-reported measures.

    “The general problem remains that cognitive-load ratings are nevertheless self-reported measures.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The questionnaire measures all processes that could require working memory resources and lead to cognitive load as theoretically modeled in CLT.

    “we try to measure all processes that could require working memory resources and lead to cognitive load in some way as they are theoretically modeled in papers on CLT (e.g., Moreno and Park, 2010; Sweller, 2010a).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Moreno and Park, 2010

      Not checked in the free run.

    • cites Sweller, 2010a

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Rogers et al. (2014) found P.I.A.N.O. resulted in the most accurate performance while playing a piece of music compared to a group using standard sheet notation and a group using Synthesia.

    “It resulted in the most accurate performance while playing a piece of music compared to a group using a standard sheet notation and a group using the software Synthesia to learn the piece of music.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rogers et al. (2014)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Seufert et al. found extraneous load was on a medium level for all disfluency groups, except the highest disfluency group, which was rated as highly loading extraneously.

    “extraneous load was on a medium level for all disfluency groups, despite the group with the highest level of disfluency, which has been rated as highly loading extraneously.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Seufert et al. (2016) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Rogers et al. found the projected roll notation reduced perceived intrinsic load significantly and the avoided split attention also reduced extraneous load.

    “The projected roll notation also reduced perceived intrinsic load significantly and the therefore avoided split attention also reduced extraneous load.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rogers et al. (2014)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The positive relation between germane load and learning outcome is not per se to be expected; the interplay between different load types, the overall amount of load, and especially individual skills affect whether enhanced germane processes are successful.

    “In our view, this positive relation is not per se to be expected as the interplay between different load types; the overall amount of load and especially the individual skills affect whether enhanced germane processes really are successful.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The questionnaire is easy to apply and fits to each content, especially for short interventions.

    “Therefore, the questionnaire is easy to apply and fits to each content, especially for short interventions like they are, e.g., analyzed in Mayers widely used multimedia learning materials (e.g., Mayer and Anderson, 1991, 1992; Mayer, 1997).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Mayer and Anderson, 1991, 1992

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Mayer, 1997 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    All items reflect theoretical aspects of the different types of load.

    “All items reflect theoretical aspects of the different types of load.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    In all studies participation was voluntary, all subjects signed a written informed consent, and were aware that they could withdraw their data at any point.

    “In all studies participation was voluntary and all subjects signed a written informed consent and were aware that they had the chance to withdraw their data at any point of the study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Rogers et al. assume a motivational factor to be relevant based on GCL ratings and measured user experience.

    “Based on GCL ratings and the measured user experience, Rogers et al. (2014) assume a motivational factor to be relevant.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rogers et al. (2014)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Rogers et al. found P.I.A.N.O. ranked highest among their tested systems in a user experience test.

    “In a user experience test, they also found P.I.A.N.O. to be ranked highest among their tested systems.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rogers et al. (2014)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    This study was exempt from an ethics committee approval due to the recommendations of the German Research Association; all subjects were in no risk of physical or emotional pressure, were fully informed about the goals and process, and none were patients, minors, or persons with disabilities.

    “This study was exempt from an ethic committee approval due to the recommendations of the German Research Association: All subjects were in no risk out of physical or emotional pressure, we fully informed all subjects about the goals and process of this study and none of the subjects were patients, minors or persons with disabilities.”
    Quote in the paperNot a substantial claim
102 of 102

  • The paper relies on this without support.

    The described hypothetical learning scenarios are adequate proxies for real learning situations for measuring cognitive load.

    “Therefore, learning situations and load rating were hypothetical.”
    “we used descriptions of hypothetical learning situations (verbally or in pictorial form via screenshots).”

  • The paper relies on this without support.

    An introduction to Cognitive Load Theory enables informed raters to differentiate intrinsic, extraneous, and germane cognitive load.

    “The informed rating group, which got an introduction to CLT, as described later, to be able to rate the following tasks in an informed way”

  • The paper relies on this without support.

    Learners who know the three load types and their differences can rate them correctly.

    “If learners know what these load types are and in which way they differ, they should be able to rate the three types of cognitive load correctly.”

  • The paper relies on this without support.

    After a 30-min training, participants are able to rate cognitive load differentially in an informed way.

    “Afterward the training, the participants might not be experts, but they should be able to rate cognitive load differentially in an informed way.”

  • The paper relies on this without support.

    Mental effort is high whenever a task implies high load, irrespective of load type, and low whenever a task is designed for low load.

    “We predicted moderate to high correlations, as mental effort should be rated high whenever a task implies a high load—irrespective of which load type—and should be low whenever a task is designed to result in low load.”

  • The paper relies on this without support.

    In naturalistic learning settings, all three types of load can always be qualified as either high or low.

    “Moreover, in a naturalistic learning setting all three types of load can always be qualified as either high or low.”

  • The paper relies on this without support.

    Instructional means to support germane load are not always inherent in each learning environment.

    “the item for instructional support to enhance germane load within the learning task was taken out, as corresponding instructional means are not always inherent in each learning environment.”

  • The paper relies on this without support.

    The current extensively used method to measure cognitive load with only one item asking for invested mental effort is not sufficient.

    “The current extensively used method to measure cognitive load with only one item asking for “invested mental effort” (Paas, 1992) is from a methodological view not sufficient.”

  • The paper relies on this without support.

    The problem of not knowing which aspect of load was enhanced and afterward inferring the possible source of load in relation to learning outcomes is not satisfying.

    “the resulting problem of not knowing which aspect of load really was enhanced and afterward inferring the possible source of load in relation to learning outcomes is in our view also not satisfying.”

  • The paper relies on this without support.

    Learners with different prerequisites dealing with the same learning task may experience different levels of different types of load during learning.

    “When learners with different prerequisites, such as prior knowledge, memory capacity, learning strategies, etc., deal with the same learning task, this might lead to different levels of different types of load during learning.”

  • The paper relies on this without support.

    Difficulties can enhance learning even though they should lead to worse learning outcomes from a cognitive load point of view.

    “difficulties should lead to worse learning outcomes, but they actually can even enhance learning.”

  • The paper relies on this without support.

    Learners need to be aware of their current state of cognitive resources and task demands for self-reported cognitive load measures to be valid.

    “Learners need to be aware of their current state of cognitive resources and task demands.”
12 of 12

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(1, 89) = 490.23reported p < .001 · recomputed p < .001

    “F(1,89) = 490.23, p < .001”

  • ConsistentF(1, 89) = 293.32reported p < .001 · recomputed p < .001

    “F(1,89) = 293.32, p < .001”

  • ConsistentF(1, 89) = 199.96reported p < .001 · recomputed p < .001

    “F(1,89) = 199.96, p < .001”

  • ConsistentF(1, 89) = 1.18reported p = .28 · recomputed p = .280

    “F(1,89) = 1.18, p = .28”

  • ConsistentF(1, 89) = 0.79reported p = .38 · recomputed p = .376

    “F(1,89) = 0.79, p = .38”

  • ConsistentF(1, 89) = 3.81reported p = .05 · recomputed p = .054

    “F(1,89) = 3.81, p = .05”

  • ConsistentF(1, 89) = 6.13reported p < .05 · recomputed p = .015

    “F(1,89) = 6.13, p < .05”

  • ConsistentF(1, 95) = 275.91reported p < .001 · recomputed p < .001

    “F(1,95) = 275.91, p < .001”

  • ConsistentF(1, 95) = 174.6reported p < .001 · recomputed p < .001

    “F(1,95) = 174.60, p < .001”

  • ConsistentF(1, 95) = 105.32reported p < .001 · recomputed p < .001

    “F(1,95) = 105.32, p < .001”

  • ConsistentF(1, 95) = 8.32reported p < .05 · recomputed p = .005

    “F(1,95) = 8.32, p < .05”

  • ConsistentF(1, 95) = 1.3reported p = .26 · recomputed p = .257

    “F(1,95) = 1.30, p = .26”

  • ConsistentF(1, 95) = 14.92reported p < .001 · recomputed p < .001

    “F(1,95) = 14.92, p < .001”

  • ConsistentF(1, 95) = 4.72reported p < .05 · recomputed p = .032

    “F(1,95) = 4.72, p < .05”

  • ConsistentF(1, 88) = 53.24reported p < .001 · recomputed p < .001

    “F(1,88) = 53.24, p < .001”

  • ConsistentF(1, 88) = 105.84reported p < .001 · recomputed p < .001

    “F(1,88) = 105.84, p < .001”

  • ConsistentF(1, 88) = 0.04reported p = .84 · recomputed p = .842

    “F(1,88) = 0.04, p = .84”

  • ConsistentF(1, 88) = 18.02reported p < .001 · recomputed p < .001

    “F(1,88) = 18.02, p < .001”

  • ConsistentF(1, 88) = 58.66reported p < .001 · recomputed p < .001

    “F(1,88) = 58.66, p < .001”

  • ConsistentF(1, 88) = 4.82reported p < .05 · recomputed p = .031

    “F(1,88) = 4.82, p < .05”

  • ConsistentF(1, 88) = 57.35reported p < .001 · recomputed p < .001

    “F(1,88) = 57.35, p < .001”

  • Consistentt(26) = 4.75reported p < .001 · recomputed p < .001

    “t(26) = 4.75, p < .001”

  • Consistentt(26) = 5.31reported p < .001 · recomputed p < .001

    “t(26) = 5.31, p < .001”

  • Consistentt(26) = 6.17reported p < .001 · recomputed p < .001

    “t(26) = 6.17, p < .001”

  • Consistentt(76) = 7.85reported p < .001 · recomputed p < .001

    “t(76) = 7.85, p < .001”

  • Consistentt(89) = 7.28reported p < .001 · recomputed p < .001

    “t(89) = 7.28, p < .001”

  • Consistentt(76) = 3.39reported p < .01 · recomputed p = .001

    “t(76) = 3.39, p < .01”

27 of 27

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Informed raters' low-ECL mean differs between text and table

    Minor

    The text reports a mean of 3.03 for informed raters on low-ECL tasks, while Table 2 reports 3.02 for the same group and task category.

  • Informed raters' high-GCL mean differs between text and table

    Minor

    The text reports 5.14 for informed raters on high-GCL tasks, but Table 2 reports 5.15 for that same mean.

  • Overall high-GCL mean is below one subgroup's mean

    Minor

    The reported overall mean for high-GCL tasks is 4.88, identical to the naïve raters' mean, although the informed raters' mean is 5.15. An overall mean combining those groups must be greater than 4.88.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 13

  1. Ayres, 2011, “Rethinking germane cognitive load,” in, Proceedings of the EARLI Conference 2011 “Education for a Global Networked Society”: Book of Abstracts and Extended Summaries, 1768
  2. Bratfisch, 1972, Perceived Item-difficulty in Three Tests of Intellectual Performance Capacity.
  3. Cierniak, 2011, Facilitating and Inhibiting Learning by the Spatial Contiguity of Text and Graphic: How Does Cognitive Load Mediate the Split-Attention and Expertise Reversal Effect?
  4. Mayer, “Principles for managing essential processing in multimedia learning: segmenting, pretraining, and modality principles,” in, The Cambridge Handbook of Multimedia Learning, 169
  5. Mayer, The Cambridge Handbook of Multimedia Learning.
  6. Park, 2011, “The rhythm task - a new method for measuring cognitive load,” in, Proceedings of the EARLI Conference 2011 “Education for a Global Networked Society”: Book of Abstracts and Extended Summaries, 1059
  7. Rogers, 2014, “P.I.A.N.O,” in, ITS ’14 Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, 149
  8. Rumelhart, 1981, “Schemata: the building blocks of cognition,” in, Comprehension and Teaching: Research Reviews, 3
  9. Sweller, “Cognitive load theory: recent theoretical advances,” in, Cognitive Load Theory, 29
  10. van Gog, 2011, “When to measure cognitive load with mental effort rating scales,” in, Proceedings of the EARLI Conference 2011 “Education for a Global Networked Society”: Book of Abstracts and Extended Summaries, 1056
  11. Wilson, 1991, “Physiological measures of workload in multi-task environments,” in, Multiple-Task Performance, 329
  12. Zander, 2010, Motivationale Lernervoraussetzungen in der Cognitive Load Theory [Motivational Prerequisites of Learners in Cognitive Load Theory].
  13. Zimmerman, 1989, “Models of self-regulated learning and academic achievement,” in, Self-Regulated Learning and Academic Achievement, 1

Cited by

Sorted by citations

Authors
Klepsch, M., Schmitz, F., & Seufert, T.
Affiliations
Universität Ulm (DE)
Corresponding author
Klepsch, Melina
ORCID
Venue
Frontiers in Psychology
Publisher
Frontiers Media SA
Published
2017-11-16
Volume
8
Pages
1997-1997
ISSN
1664-1078 (ISSN-L)
DOI
10.3389/fpsyg.2017.01997
PMC
PMC5696680
OpenAlex
W2767740804
Cited by
601
References
61
Citations by year
2026: 140 · 2025: 122 · 2024: 104 · 2023: 72 · 2022: 68 · 2021: 48 · 2020: 24 · 2019: 16 · 2018: 7
FWCI
16.62
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • OPen Access Repositorium der Universität Ulm (OPARU) (Ulm University) (submittedVersion, other-oa)
  • Universität Ulm, Kommunikations- und Informationszentrum (kiz) (cc-by)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords
cognitive load, intrinsic cognitive load, extraneous cognitive load, germane cognitive load, cognitive load assessment, questionnaire development, instrument validation, reliability statistics, factor analysis, learning materials

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review