Effects of pairs of problems and examples on task performance and different types of cognitive load

May combine two claims.

Exam performance was positively correlated with the germane factor (r = .140, p = .084), but this correlation was not statistically significant at the .05 level.

“exam performance was positively correlated with the factor supposed to represent germane cognitive load during the lecture (r ¼ .140,p ¼ .084),although this correlation was not statistically significant at the conventional a ¼ .05level.”

May combine two claims.

The factor correlations resembled Leppink et al.'s: negative between extraneous and germane, positive between intrinsic and extraneous, and near zero between intrinsic and germane.

“The factor correlations are similar to those reported by Leppink, Paas et al. (2013): negative between the factors supposed to represent extraneous and germane cognitive load, positive between the factors supposed to represent intrinsic and extraneous cognitive load, and around zero between the factors supposed to intrinsic and germane cognitive load.”

• cites Leppink, Paas et al., 2013Not checked yet

  Not checked in the free run.

May combine two claims.

In both contexts, the germane items loaded on the first factor, intrinsic items on the second, and extraneous items on the third.

“In both contexts, the items supposed to represent germane cognitive load loaded on the first factor, the items supposed to represent intrinsic cognitive load loaded on the second factor, and the items supposed to represent extraneous cognitive load loaded on the third factor.”

May combine two claims.

Exam performance was negatively correlated with the intrinsic factor (r = -.210, p = .010) and the extraneous factor (r = -.320, p < .001).

“exam performance was negatively correlated with the factors supposed to represent intrinsic (r ¼ À.210, p ¼ .010)and extraneous (r ¼ À.320, p < .001)cognitive load during the lecture.”

May combine two claims.

The statistics and language versions produced similar factor loadings and internal consistency values.

“The two versions yield similar factor loadings and internal consistency values.”

May combine two claims.

No statistically significant interaction effects between format and time or between first and second format appeared in Tables 10-12.

“None of Tables 10e12 indicate statistically significant interaction effects between format and time or between first and second format.”

May combine two claims.

Adding counterbalanced training context to the analysis did not improve explanation of posttest performance; its effect was small and not statistically significant, F(1,72)=.891, p=.348, η2=.012.

“Expanding BS ANOVA to three-way to account for counterbalanced training context to our model did not contribute to the explanation of posttest performance; counterbalanced training context had only a small and not statistically significant effect on posttest performance, F(1, 72) ¼ .891,p ¼ .348,h 2 ¼ .012”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In Study II, posttest performance had a mean of 4.49, SD 2.34, and skewness of -1.195.

“Posttest performance was somewhat skewed to the left (skewness ¼ À1.195), with a mean score (M) of 4.49 and a standard deviation (SD) 2.34.”

May combine two claims.

Participants who studied a worked example in the second task averaged .472 points higher on the 0-6 posttest than those who solved a problem autonomously in the second task, but this difference was not statistically significant, F(1,70)=.805, p=.373, η2=.011, a small effect at best.

“Although participants who studied a worked example in the second task (i.e., problemeexample or exampleeexample) scored on average .472points (range: 0e6) higher than participants who solved a problem autonomously in the second task, this difference was not statistically significant, F(1, 70) ¼ .805,p ¼ .373,h 2 ¼ .011,and the effect size indicates that we are talking about a small effect at best.”

May combine two claims.

Counterbalanced training context did not contribute to explanation of any factor scores; η2 values ranged .005-.014 and p-values .309-.542.

“Finally, as was the case for posttest performance, including counterbalanced training context in the analysis did not contribute to the explanation of any of the factor scores; all h 2 -values were in the range of .005and .014(comparable to the h 2 -value of .012 in the case of posttest performance) and p-values ranged from .309 to .542.”

May combine two claims.

The three factors had similar Cronbach's alpha values for training and posttest, except the extraneous factor during training had a somewhat lower alpha, possibly due to restriction of range.

“Table 8 indicates that the three factors yielded comparable Cronbach's alpha values for training and posttest, only a somewhat lower Cronbach's alpha value for the factor supposed to represent extraneous cognitive load for the training which may reflect a restriction of range effect.”

May combine two claims.

The total number of correct responses had a Cronbach's alpha of .950, item p-values between .70 and .78, and corrected item-total correlations of .695 or higher.

“It is therefore of little surprise that the sum of correct responses (i.e., an integer on the scale of 0e6) yielded a very high Cronbach's alpha (a ¼ .950)with all item p-values (i.e., percentage of correct response) in the range of .70e.78 and corrected itemetotal correlations of .695and higher.”

May combine two claims.

Correlations between posttest performance and the three factors were similar across the two measurement occasions and, especially for intrinsic and extraneous factors, close to zero.

“Table 9 indicates that the correlations between posttest performance and the three factors supposed to represent intrinsic, extraneous, and germane cognitive load are similar for the two measurement occasions and (especially for the first two factors) close to zero.”

May combine two claims.

Participants who studied a worked example in the first task scored 1.417 points higher on the 0-6 posttest than those who solved a problem autonomously in the first task; this effect was statistically significant, F(1,70)=7.245, p=.009, η2=.094, medium to somewhat larger.

“The effect of the first task, however, was statistically significant, F(1, 70) ¼ 7.245, p ¼ .009,h 2 ¼ .094,and the effect size indicates a medium to somewhat larger effect; participants who studied a worked example in the first task performed much better on the posttest (1.417 points on the 0e6 scale) than participants who solved a problem autonomously in the first task.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Every student finished all six posttest questions within the allotted fifteen minutes.

“All students managed to complete all six questions within the fifteen minutes they were given for the posttest.”

May combine two claims.

Example-example and example-problem pairs produced better posttest performance than problem-problem and problem-example pairs (H4a), consistent with Van Gog et al. (2011).

“In line with the study by Van Gog et al. (2011), we found that exampleeexample and exampleeproblem pairs led to better posttest performance than problemeproblem and probleme example pairs (H4a), and that the second format did not influence posttest performance significantly (H4b).”

• cites Van Gog et al. (2011) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Only first format in Table 12 was statistically significant, with η2=.170 indicating a large effect.

“Only first format in Table 12 is statistically significant, and the h 2 -value of .170indicates a large effect.”

May combine two claims.

In Study II, posttest means were 3.50 (SD 2.78) for problem-problem, 4.11 (2.52) for problem-example, 5.06 (1.83) for example-problem, and 5.39 (1.65) for example-example.

“The problemeproblem condition performed worst (M ¼ 3.50, SD ¼ 2.78), followed by the problemeexample condition (M ¼ 4.11, SD ¼ 2.52), followed by the exampleeproblem condition (M ¼ 5.06, SD ¼ 1.83), and the exampleeexample condition performed best (M ¼ 5.39, SD ¼ 1.65).”

May combine two claims.

In Leppink et al.'s randomized experiment, the correlation between germane load and later task performance was lower than expected and not statistically significant.

“Firstly, the correlation between germane cognitive load and subsequent task performance in the randomized experiment was lower than expected and not statistically significant.”

• cites Leppink, Paas et al., 2013Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

If germane load is working memory resources allocated to intrinsic load, then germane and intrinsic load may be hard to distinguish.

“If germane cognitive load pertains to the working memory resources allocated to dealing with intrinsic cognitive load, as suggested recently by Sweller (2010) and Kalyuga (2011), it may be difficult to distinguish between germane cognitive load and intrinsic cognitive load.”

• cites Sweller, 2010 sourceNot checked yet

  Not checked in the free run.

• cites Kalyuga, 2011 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Intrinsic cognitive load depends on the number of interacting information elements contained in the learning task or material.

“Intrinsic cognitive load is determined by the intrinsic nature of the information to be learned, more specifically, by the number of interacting information elements that the learning task or the learning material comprises (Sweller, 1994;Sweller et al., 2011).”

• cites Sweller, 1994Not checked yet

  Not checked in the free run.

• cites Sweller et al., 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The new psychometric instrument showed a robust three-factor structure.

“Although this new psychometric instrument (Leppink, Paas et al. 2013) revealed a robust three-factor structure, for a number of reasons it is not yet clear whether these three factors indeed represent the three types of cognitive load.”

• cites Leppink, Paas et al., 2013Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S1 (asserted), S4 (based).

If the three factors represent the load types or stable related constructs, they should also appear in other contexts.

“If the three factors indeed represent the three types of cognitive load e or stable constructs related to these types of cognitive load e one would expect these factors to come to the surface in other contexts as well.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Cognitive load theory recommends optimizing intrinsic load by matching materials to prior knowledge, minimizing extraneous load, and challenging learners to use germane processes or resources.

“Cognitive load theory states that intrinsic cognitive load should be optimized in instructional design by selecting materials that match the learner's prior knowledge or proficiency, while extraneous cognitive load should be minimized, and learners should be challenged to engage in processes that evoke germane cognitive load (in the old conceptualization of cognitive load theory) or the use of (in”

• cites Sweller et al., 1998 sourceNot checked yet

  Not checked in the free run.

• cites Van Merriënboer & Sweller, 2005, 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Subjective rating scales such as Paas's nine-point mental effort scale have been widely used to measure overall cognitive load.

“Subjective rating scales like Paas' (1992) nine-point unidimensional mental effort rating scale have been used intensively for measuring the overall cognitive load experienced by learners (for reviews : Paas, Tuovinen, Tabbers, & Van Gerven, 2003;Van Gog & Paas, 2008).”

• cites Paas, Tuovinen, Tabbers, & Van Gerven, 2003 sourceNot checked yet

  Not checked in the free run.

• cites Van Gog & Paas, 2008Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Varying the order of the items does not significantly affect factor loadings or internal consistency values.

“Since Leppink, Paas et al. (2013) demonstrated that varying the order in which the items are asked does not significantly influence factor loadings or internal consistency values, we decided to present the ten items in the order presented in Table 1; note that the item order was the same for both questionnaires.”

• cites Leppink, Paas et al., 2013Not checked yet

  Not checked in the free run.

May combine two claims.

Human working memory can hold about seven plus or minus two chunks when only maintaining information, and about four when actively processing it.

“Working memory has a limited capacity of seven plus or minus two elements (or chunks) of information when merely holding information (Miller, 1956) and even fewer (circa four) when processing information (Cowan, 2001).”

• cites Miller, 1956 sourceNot checked yet

  Not checked in the free run.

• cites Cowan, 2001 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Mental effort is the cognitive capacity actually allocated to meet task demands and is taken to reflect actual cognitive load.

“Mental effort has been defined by Paas et al., as “the cognitive capacity that is actually allocated to accommodate the demands imposed by the task; thus, it can be considered to reflect the actual cognitive load” (Paas, Tuovinen et al., 2003, p. 64; see also Paas & Van Merriënboer, 1994b).”

• cites Paas, Tuovinen et al., 2003 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

If intrinsic, extraneous, and germane load were independent and additive, their correlations would be around zero.

“If intrinsic, extraneous, and germane cognitive load were indeed independent and additive types of cognitive load as Sweller et al. (1998) suggest, then one would expect correlations between these types of cognitive load to be (around) zero.”

• cites Sweller et al., 1998 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Measuring overall cognitive load is less useful for instructional design than measuring the different load types separately.

“While measuring overall experienced cognitive load by subjective or objective techniques can be informative e especially in relation to measures of learning outcomes (Van Gog & Paas, 2008) e it is less specific than measurement of different types of cognitive load separately when it comes to informing the design of instruction.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A recently developed psychometric instrument may partly solve the difficulty of distinguishing the different cognitive load types.

“Recently, a psychometric instrument was developed that took an alternative approach to the formulation of the questions for measuring different types of cognitive load (Leppink, Paas, Van der Vleuten, Van Gog, & Van Merriënboer, 2013), which may solve the problem of not being able to distinguish between different types of cognitive load at least to a certain extent.”

• cites Leppink, Paas, Van der Vleuten, Van Gog, & Van Merriënboer, 2013Not checked yet

  Not checked in the free run.

May combine two claims.

Worked example information is redundant for more knowledgeable learners, and processing redundant information adds extraneous load, the redundancy effect.

“The information presented in worked examples is redundant for more knowledgeable learners, who are able to solve the problem without instructional guidance, and processing redundant information contributes to extraneous cognitive load (i.e., redundancy effect; Chandler & Sweller, 1991).”

• cites Chandler & Sweller, 1991 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The language sample of 108 was too small for confirmatory factor analysis but large enough for exploratory factor analysis by the ten-participants-per-item rule.

“Furthermore, the total sample size of the language classes (n ¼ 108) was too small for confirmatory factor analysis, while following the rule of thumb that a number of participants as large as ten times the number of items (i.e., ten) in the instrument, the group was large enough for exploratory factor analysis.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Using several indicators for each load type may measure it more precisely and separate the types more clearly than using one indicator per scale.

“The use of multiple indicators for each of the separate types of cognitive load might yield a more precise measurement and might enable researchers to separate the types of cognitive load more clearly than the use of a single indicator for each scale.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A given instructional feature may be linked to germane load for one learner and extraneous load for another, as the expertise reversal effect shows.

“Further, when referring to one very specific instructional feature or cognitive process to measure extraneous cognitive load or germane cognitive load, a conceptual problem may arise, because the expertise reversal effect illustrates that a particular instructional feature may be associated with germane cognitive load for one learner and with extraneous cognitive load for another learner (Kalyuga”

• cites Kalyuga et al., 2001 sourceNot checked yet

  Not checked in the free run.

• cites Kalyuga et al., 2003 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Germane cognitive load involves relating relevant long-term memory or contextual information to new elements and concerns working memory resources devoted to intrinsic load.

“This type of load arises from relating relevant information from long-term memory or context to the new information elements (Sweller, 2010;Sweller et al., 2011) and as such pertains to the working memory resources allocated to dealing with intrinsic cognitive load (Kalyuga, 2011;Sweller, 2010).”

• cites Sweller, 2010 sourceNot checked yet

  Not checked in the free run.

• cites Sweller et al., 2011Not checked yet

  Not checked in the free run.

• cites Kalyuga, 2011 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Cognitive load is set by how many information elements must be handled at the same time within a given period.

“Working memory load (or cognitive load) is therefore determined by the number of information elements that need to be processed simultaneously within a certain amount of time (Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007).”

• cites Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007Not checked yet

  Not checked in the free run.

May combine two claims.

Several studies have tried to measure the three load types separately, but a weakness is that one or more types were measured by only one item.

“Therefore, several studies have attempted to develop instruments for measuring the three types of cognitive load separately (Ayres, 2006;Cierniak, Scheiter, & Gerjets, 2009;De Leeuw & Mayer, 2008;Eysink et al., 2009;Galy, Cariou, & Mélan, 2012).A drawback of those studies is that one or more types of cognitive load were represented by one single item.”

• cites Ayres, 2006 sourceNot checked yet

  Not checked in the free run.

• cites Cierniak, Scheiter, & Gerjets, 2009Not checked yet

  Not checked in the free run.

• cites De Leeuw & Mayer, 2008 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The findings may indicate that students can rate the 'knowledge and understanding' factor and judge it differently across conditions.

“It may indicate that students are able to rate this 'knowledge and understanding' factor and judge it differently between conditions.”

• Also stated in: § 4. General discussion · hedged

  “The findings of Study II may indicate that students can rate the 'knowledge and understanding' factor and judge it differently across conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Study II was a randomized experiment.

“In Study II, a randomized experiment was conducted.”

Probably not a substantial claim: may not pass S4 (based).

If the intrinsic and extraneous factors truly represent those load types, the instrument can help researchers differentiate intrinsic from extraneous load.

“If the two factors that are supposed to represent intrinsic and extraneous cognitive load indeed represent these two types of cognitive load, this would imply that we have developed a psychometric instrument that can help researchers to differentiate intrinsic and extraneous cognitive load.”

• AssumptionThe authors state this as an assumption: The questionnaire items intended to measure intrinsic, extraneous, and germane cognitive load actually measure those constructs.

Probably not a substantial claim: may not pass S3 (load-bearing).

The findings do not contradict that reconceptualization.

“At least, the findings do not point against such a reconceptualization.”

May combine two claims.

Van Gog and Kester (2012) replicated the immediate-test finding and showed that on a delayed test one week later, example-example outperformed example-problem.

“Van Gog and Kester (2012) replicated this finding regarding the immediate test and demonstrated that at a delayed test one week later, performance in an exampleeexample condition was even better than in an exampleeproblem condition.”

• cites Van Gog and Kester (2012)Not checked yet

  Not checked in the free run.

May combine two claims.

The correlations between the third factor and germane cognitive load are positive, but their size indicates the relation is limited at best.

“While these correlations are again positive, their magnitude appears to indicate that the relation between the third factor and germane cognitive load is limited at best.”

• AssumptionThe authors state this as an assumption: The third factor in the psychometric instrument represents or closely relates to germane cognitive load.

May combine two claims.

The items supposed to capture intrinsic cognitive load measure one latent construct, and the items supposed to capture extraneous cognitive load measure one latent construct.

“This appears to indicate that the items supposed to capture intrinsic cognitive load measure one and the same latent construct, and the items supposed to capture extraneous cognitive load measure one and the same latent construct.”

May combine two claims.

The problem-example and problem-problem conditions did not differ in test performance or studying-phase mental effort.

“Thirdly, the problemeexample and probleme problem conditions did not differ from each other in test performance or invested mental effort in the studying phase.”

• cites Van Gog et al. (2011) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

This finding is consistent with prior research on worked example effects (Cooper & Sweller, 1987; Paas, 1992; Paas & Van Merriënboer, 1994a; Sweller & Cooper, 1985; Van Gog et al., 2006).

“The finding that starting self-study of a new topic with a worked example instead of with autonomous problem solving has beneficial effects on task performance is in line with previous research on worked example effects (Cooper & Sweller, 1987;Paas, 1992;Paas & Van Merriënboer, 1994a;Sweller & Cooper, 1985;Van Gog et al., 2006) and with previous research on problemeexample and example problem pa”

• cites Cooper & Sweller, 1987 sourceNot checked yet

  Not checked in the free run.

• cites Paas, 1992 sourceNot checked yet

  Not checked in the free run.

• cites Paas & Van Merriënboer, 1994aNot checked yet

  Not checked in the free run.

May combine two claims.

Van Gog et al. found no significant difference in test performance between example-example and example-problem conditions, and no significant difference in studying-phase mental effort.

“First of all, their study was the first to compare an exampleeexample and exampleeproblem condition and to demonstrate that there was no significant difference in test performance in the exampleeexample and exampleeproblem conditions, and no significant difference in mental effort invested in the studying phase.”

• cites Van Gog et al. (2011) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Nine students canceled and one was excluded for noncompliance, leaving 20 in problem-problem, 18 in problem-example, 18 in example-problem, and 18 in example-example.

“Nine students canceled their participation last minute due to changes in their educational timetable, and another one other student did not comply with the instructions and was therefore excluded from the experiment.This resulted in the following situation: problemeproblem (n ¼ 20), problemeexample (n ¼ 18), exampleeproblem (n ¼ 18), and exampleeexample (n ¼ 18).”

May combine two claims.

This finding shows that not only the amount of guidance but also its timing matters.

“This finding is particularly interesting in the light of recent debates on instructional guidance (e.g., Clark, Kirschner, & Sweller, 2012), as it illustrates that it not only matters how much guidance is provided, but also when it is provided.”

• cites Clark, Kirschner, & Sweller, 2012Not checked yet

  Not checked in the free run.

May combine two claims.

Solving a problem first may create high extraneous load and little learning from that problem.

“When solving a problem first, there may be high extraneous cognitive load and little learning from solving that problem.”

May combine two claims.

Together, the findings provide some support for the assumption that intrinsic and extraneous cognitive load can be differentiated with a psychometric instrument, or at least that the factors supposed to represent them relate to those constructs.

“Together, the findings appear to provide some support for the assumption that intrinsic and extraneous cognitive load can be differentiated using a psychometric instrument, or at least that the factors that were supposed to represent intrinsic and extraneous cognitive load relate to intrinsic and extraneous cognitive load.”

• Also stated in: § 4. General discussion · hedged

  “Although these findings appear to provide support for the assumption that the two factors either represent or closely relate to intrinsic and extraneous cognitive load, respectively, further experimentation is needed to examine the validity of this assumption.”

• AssumptionThe authors state this as an assumption: Intrinsic and extraneous cognitive load can be differentiated using a psychometric instrument, or the factors supposed to represent them at least relate to those constructs.

Probably not a substantial claim: may not pass S3 (load-bearing).

This finding is consistent with prior research on problem-example and example-problem pairs (Reisslein et al., 2006; Van Gog et al., 2011).

“The finding that starting self-study of a new topic with a worked example instead of with autonomous problem solving has beneficial effects on task performance is in line with previous research on worked example effects (Cooper & Sweller, 1987;Paas, 1992;Paas & Van Merriënboer, 1994a;Sweller & Cooper, 1985;Van Gog et al., 2006) and with previous research on problemeexample and example problem pa”

• cites Reisslein et al., 2006 sourceNot checked yet

  Not checked in the free run.

• cites Van Gog et al., 2011 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The intrinsic and extraneous items may reflect estimated required load rather than actually invested effort.

“The items supposed to capture intrinsic and extraneous cognitive load could reflect an estimation of the required intrinsic and extraneous cognitive load activity rather than an indication of the actually invested effort in intrinsic and extraneous cognitive load, and a similar reasoning might hold for the items supposed to capture germane cognitive load.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The two constructs may be intrinsic and extraneous cognitive load or factors related to those two load types.

“The two constructs in question may be intrinsic and extraneous cognitive load or factors related to these two types of cognitive load.”

May combine two claims.

The advantage of example-problem over problem-example agrees with Reisslein et al. and suggests that studying an example first builds a schema that can be used when solving the problem.

“However, this finding is also in line with findings by Reisslein, Atkinson, Seeling, and Reisslein (2006), and suggests that condition matters because studying an example first allows for building a cognitive schema (i.e., germane cognitive load activity) that can subsequently be used when solving the problem (lowering the intrinsic cognitive load of the problem compared to a probleme problem”

• cites Reisslein, Atkinson, Seeling, and Reisslein (2006)Not checked yet

  Not checked in the free run.

May combine two claims.

In Van Gog et al. (2011), example-example and example-problem produced better test performance than problem-example and problem-problem, with lower studying-phase mental effort.

“Secondly, in the study by Van Gog et al. (2011), both exampleeexample and exampleeproblem resulted in better test performance than problemeexample and problemeproblem, which, moreover, was reached with lower invested mental effort in the studying phase in the exampleeexample and exampleeproblem conditions than in the problemeexample and problemeproblem conditions.”

• cites Van Gog et al. (2011) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

This produced a precise measure of a student's ability to apply Bayes' theorem and optimal statistical power for instructional-condition effects on posttest performance.

“This yielded a very precise measurement of the extent to which a student is able to apply Bayes' theorem, and thus optimal statistical power for treatment effects of instructional conditions on participants' posttest performance.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two problems in different contexts were designed and presented in counterbalanced order.

“To account for potential context effects, we designed two problems in a different context.The two contexts were presented in counterbalanced order, meaning that about half of the participants completed their first task in one context while the others completed their first task in the other context, and vice versa.”

Classified as probably not a substantial claim.

Eighty-four first-year bachelor students were randomly assigned to the four conditions.

“A total of eighty-four first-year bachelor students in the Social and Health Sciences were allocated randomly to the probleme problem, problemeexample, exampleeproblem, and examplee example condition.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Responses to the items may reflect perceived task complexity, perceived instructional ineffectiveness, and perceived understanding rather than actual invested effort or working memory activity.

“However, there is another possibility we did not really anticipate in Study I (and neither did Leppink, Paas et al. 2013): responses to the items may reflect an estimation or perception of (expected) task complexity, a reflection on (expected) sources of ineffectiveness in the instruction, and a reflection on understanding and knowledge acquisition independent of any actually invested effort in”

May combine two claims.

Table 12 findings support the hypothesis that the example-example and example-problem conditions report higher germane cognitive load in the studying phase (H8a) and on the posttest (H8b) than the problem-example and problem-problem conditions, with η2=.170 indicating a large effect.

“The findings presented in Table 12 provide support for the hypothesis that students in the exampleeexample and exampleeproblem condition would report higher germane cognitive load in the studying phase (H8a) and on the posttest (H8b) than students in the probleme example and problemeproblem condition (the h 2 -value of .170indicating a large effect).”

• Also stated in: § 3.3. Discussion · hedged

  “However, in this light it is perhaps even more intriguing that the findings presented in Table 12 do provide support for the hypothesis that students in the exampleeexample and exampleeproblem condition would report higher germane cognitive load in the studying phase and on the posttest than students in the problemeexample and problemeproblem condition.”

• AssumptionThe authors state this as an assumption: The third factor in the psychometric instrument represents or closely relates to germane cognitive load.

May combine two claims.

The added item's corrected item-total correlation is high but distinguishable from the other items, possibly because the other four items do not directly capture invested effort in germane cognitive load activity.

“While the corrected itemetotal correlation of the added item is still quite high, this value is distinguishable from the values of the other items, which may reflect that the other four items do not directly capture actually invested effort in germane cognitive load activity.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Validity of a story concerns a chain of evidence (Kane, 2006).

“Although law cases and empirical research often appear to have little in common, at least one thing they do have in common: validity of a story is about a chain of evidence (Kane, 2006).”

• cites Kane, 2006Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Validity of a measurement instrument is not established by one or two studies; it requires a chain of evidence.

“Like a suspect in a criminal case should not be convicted based on only one piece of evidence e unus testis nullus testis e validity of a measurement instrument is not established in one or two (sets of) studies; it is a journey in search for a chain of evidence, and to obtain that chain of evidence some elements in the instrument may need revision or adjustment.”