No negative Flynn effect in France: Why variations of intelligence should not be assessed using tests based on cultural knowledge

May combine two claims.

Both meta-analyses listed a small number of studies reporting negative Flynn effects, which suggests that observed intelligence decreases can reflect fluctuations around a stable or increasing average due to chance or methodological bias.

“Both meta-analyses listed a small number of studies reporting negative Flynn effects, which suggests that observed intelligence decreases can reflect fluctuations around a stable or increasing average (see Pietschnig & Voracek, 2015, Figure 2;Trahan et al., 2014, Figure 2), due either to chance or to methodological bias.”

• cites Pietschnig & Voracek, 2015 sourceNot checked — no full text of the source
• cites Trahan et al., 2014 sourcePartly supported

  The Results (Overall Model; page unavailable) report a positive mean Flynn effect and estimate that 82% of true effects were above zero. The Results (Moderator Analyses of the Modern Tests; page unavailable) describe lower-than-average effects in Figure 2 and a negative estimate after the first study in a cumulative analysis, but do not establish how many Figure 2 studies had negative effects. The paper discusses possible measurement artifacts, while its Results (Distribution of Effects; page unavailable) say the observed heterogeneity is not consistent with sampling error alone; it does not establish the proposed chance explanation for the decreases. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

May combine two claims.

The weight of declarative knowledge is substantially greater in VCI subtests reflecting Gc than in all other subtests that place little emphasis on declarative knowledge, as illustrated by their lesser decline with age and higher correlation with education and socio-economic levels.

“Still, the weight of declarative knowledge is substantially greater in VCI subtests reflecting Gc, as illustrated by their lesser decline with age (Baxendale, 2011, Grégoire, 1993;Kaufman et al., 1989, Ryan et al., 2000) and their higher correlation with education and socio-economic levels (Dori & Chelune, 2004;Heaton et al., 2003;Holdnack et al., 2013), than all other subtests that place little”

• cites Baxendale, 2011 sourceNot checked — no full text of the source
• cites Grégoire, 1993 sourcePartly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Kaufman et al., 1989 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Item parameter estimation used Stocking's method A.

“Item parameter estimation was performed using Stocking's method A (Stocking, 1988; see also Birnbaum, 1968;Ban et al., 2001).”

• cites Stocking, 1988 sourceSupported
• cites Birnbaum, 1968Not checked — no full text of the source
• cites Ban et al., 2001 sourceSupported

May combine two claims.

Stocking's method A works well with limited sample sizes, removes the necessity of picking anchor items to link ability scales, and estimates general ability based on all available information instead of only the items being tested for DIF.

“Stocking's method A works well with limited sample sizes, as was the case for the 2019 sample; it removes the necessity of picking anchor items to link the ability scales of the two groups on the same metric; and it estimates general ability based on all available information, instead of only the items being tested for DIF.”

• cites Stocking, 1988 sourcePartly supported

  The Method B section says Method A does not use anchor items and summarizes adaptive-test information in an ability estimate, which supports those aspects of the claim. But the paper studies on-line calibration, not linking two groups for DIF analysis, and the Discussion reports that Method A’s estimation error increases across rounds. It does not establish that Method A works well with the cited limited sample size. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Flieller (1988) reached the same conclusion in a French dataset over three decades ago; Brand and colleagues (1989) also found a similar result of decreasing scores due to changes of item difficulty.

“As an example, Flieller (1988) reached the same conclusion in a French dataset over three decades ago; Brand and colleagues (1989) also found a similar result of decreasing scores due to changes of items difficulty”

• cites Flieller, 1988 sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Brand et al., 1989 sourceNot checked — no full text of the source

May combine two claims.

At the index level in the corrected sample, only VCI and FSIQ showed a statistically significant difference between versions at the .05 level, and both survived correction for multiple comparisons.

“At the level of indices, only the VCI and FSIQ showed a statistically significant difference between versions at the .05level; both survived correction for multiple comparisons.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All Study 2 subjects were native French speakers, and none had a history of major neurologic or psychiatric disorder.

“All subjects were native French speakers, and none had a history of major neurologic or psychiatric disorder.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Study 2 included 81 subjects recruited by word of mouth in the community.

“A total of 81 subjects, recruited by word of mouth in the community, participated in the study.”

May combine two claims.

The Study 1 sample had an unbalanced range of ability, with no gifted subjects and a highest FSIQ of 123.

“yielded an unbalanced range of ability given that there were no gifted subjects at the other end of the scale (the highest FSIQ in this sample was 123).”

May combine two claims.

A Mantel-Haenszel DIF analysis found significant bias for vocabulary (11 biased items), similarities (3), information (8), comprehension (1), and arithmetic (3), mostly against the 2019 sample.

“Another analysis using the Mantel-Haenszel method, with all items recoded as binary and with ability computed as total score on a subtest, also found significant bias for vocabulary (11 biased items), similarities (3 biased items), information (8 biased items), comprehension (1 biased item), and arithmetic (3 biased items), mostly against the 2019 sample.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The general ability estimate correlated between .41 and .60 with the five biased subtests.

“This general ability estimate correlated between .41 and .60 with the five biased subtests.”

May combine two claims.

Subjects in the 2019 sample performed somewhat above the 1999 average, and there was a greater range of scores in the 1999 sample, especially for lower ability levels.

“Subjects in the 2019 sample performed somewhat above the 1999 average, and there was a greater range of scores in the 1999 sample, especially for lower ability levels.”

May combine two claims.

The full-scale IQ (FSIQ) was significantly lower on the WAIS-III than on the WAIS-IV (p < .001), compatible with a decrease of intelligence between the WAIS-III and WAIS-IV normative samples.

“The results confirmed that the full-scale IQ (FSIQ) was significantly lower on the WAIS-III than on the WAIS-IV (p < .001),compatible with a decrease of intelligence between the WAIS-III and WAIS-IV normative samples.”

• AssumptionRests on an assumption: The WAIS-III and WAIS-IV normative samples are valid and comparable reference points for estimating ability differences across time.
• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

May combine two claims.

The difference was mostly driven by Gc subtests: Vocabulary, Information and Comprehension subtests, along with the corresponding VCI, all showed medium-to-large performance decreases (dz between .54 and .62).

“Contrary to this hypothesis, however, the difference was mostly driven by Gc subtests: Vocabulary, Information and Comprehension subtests, along with the corresponding VCI, all showed medium-to-large (Cohen, 1988) performance decreases (dz between .54 and .62).”

• cites Cohen, 1988Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: The WAIS-III and WAIS-IV normative samples are valid and comparable reference points for estimating ability differences across time.
• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The median test-retest interval in Study 2 was 35 days (M = 35.92 days, SD = 14.92 days, range = 14 to 70 days), similar for the two orders.

“The median test-retest interval was 35 days (M = 35.92days, SD = 14.92 days, range = 14 to 70 days), similar for the two orders (median = 35 days for both).”

May combine two claims.

The Matrix Reasoning subtest, used as the best available index of Gf, had an even smaller effect size and showed a non-significant difference (p > .10) after correction for multiple comparisons.

“The Matrix Reasoning subtest, used as the best available index of Gf (Marshalek et al., 1983;Carpenter et al., 1990), had an even smaller effect size and showed a non-significant difference (p > .10)after correction for multiple comparisons.”

• cites Marshalek et al., 1983 sourceNot checked — no full text of the source
• cites Carpenter et al., 1990 sourceNot checked — no full text of the source

• AssumptionRests on an assumption: The WAIS-III and WAIS-IV normative samples are valid and comparable reference points for estimating ability differences across time.
• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

Probably not a substantial claim: may not pass S3 (load-bearing).

The descriptive statistics for age in the sample were M = 44.53 years, SD = 13.71 years, range = 30 to 63 years.

“The descriptive statistics for age, as stated in the test manual and in Dutton and Lynn (2015), were M = 44.53years, SD = 13.71years, range = 30 to 63 years.”

• cites Dutton and Lynn (2015) sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The authors corrected two errors prior to analysis: a misreporting for the Arithmetic subtest in the WAIS-IV manual (WAIS-IV performance reported as M = 10.1, SD = 3.0 instead of M = 10.63, SD = 3.24), and a mistranslation by Dutton and Lynn in their Table 4.

“We just corrected two errors prior to analysis.The first error was a misreporting for the Arithmetic subtest in the WAIS-IV manual (WAIS-IV performance reported as M = 10.1,SD = 3.0 instead of M = 10.63,SD = 3.24).The second error was a mistranslation on the part of Dutton and Lynn (in their Table 4, line 5; the numbers they report as”

• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

May combine two claims.

The Bayes factor was not informative for Symbol Search, diverging from the frequentist results, and did not support the existence of a large difference between versions for this subtest.

“The Bayes factor was not informative for Symbol Search, thus diverging from the frequentist results: this did not support the existence of a large difference between versions for this subtest.”

May combine two claims.

The prevalence of intellectual disability in the Study 1 sample was 6.3%, larger than the expected 2.5% in the general population.

“This represented a larger prevalence of intellectual disability than expected in the general population (6.3% of the sample instead of an expected 2.5%)”

May combine two claims.

The second finding of Study 1, after reanalyzing the data, was that the difference between WAIS-III and WAIS-IV was statistically significant with a medium effect size, but was almost entirely driven by three of the four subtests of the VCI, i.e., three subtests primarily depending on Gc.

“The second finding, after reanalyzing the data, was that the difference between WAIS-III and WAIS-IV was statistically significant with a medium effect size, but was almost entirely driven by three of the four subtests of the VCI -in other words, three subtests primarily depending on Gc.”

May combine two claims.

Bayesian analyses confirmed very strong evidence in favor of differences between WAIS-III and WAIS-IV for the same five Gc subtests, and only for these subtests.

“Bayesian analyses confirmed that there was very strong evidence in favor of differences between the WAIS-III and WAIS-IV for the same five subtests depending on Gc, and only for these subtests.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Socio-economic groups in Study 2 were representative of the general population.

“Socio-economic groups were representative of the general population (1 farmer, 3 artisans, 17 executives and other intellectual professions, 19 workers of intermediate level, 20 employees, 15 laborers, and 6 students or unemployed persons; retired persons counted as per their former profession).”

May combine two claims.

Of the five subtests demonstrating a difference between WAIS-III and WAIS-IV, all except Block Design involved a significant contribution of declarative knowledge, and the difference for Block Design only existed in one of four subgroups.

“In sum, of the five subtests demonstrating a difference between WAIS-III and WAIS-IV, all except Block Design involved a significant contribution of declarative knowledge, and the difference for the latter only existed in one of four subgroups.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The combined Study 1 and Study 2 ANOVA results were very similar to Study 1 and Study 2 considered separately.

“The results of ANOVAs for this complementary analysis are summarized in Table 4; they were very similar to Study 1 and Study 2 considered separately.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Arithmetic demonstrated a marginal two-way interaction between test version and test order, F(1, 70) = 3.30, p = .073, η2p = .05.

“and a marginal two-way interaction between test version and test order, F(1, 70) = 3.30, p = .073,η²p = .05.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Alternative DIF methods yielded different results and different lists of items identified with DIF, but this did not change the overall picture much.

“As is common with tests of DIF, alternative methods yielded different results and the list of items identified with DIF varied, but this did not change the overall picture much.”

May combine two claims.

A contrast analysis indicated the same pattern for Block Design and Arithmetic: only older adults who completed the WAIS-III first performed significantly higher on the WAIS-IV (p < .001); there was no difference between WAIS-III and WAIS-IV for younger adults in either order condition, or for older adults who performed the WAIS-IV first.

“A contrast analysis indicated the same pattern for both subtests: only older adults who completed the WAIS-III first performed significantly higher on the WAIS-IV (p < .001);there was no difference between WAIS-III and WAIS-IV for younger adults in either order condition, or for older adults who performed the WAIS-IV first (all other ps > .43 for Block Design and all other ps > .24for Arithmetic).”

May combine two claims.

No difference appeared for the sole Gf subtest Matrix Reasoning, or for working memory or processing speed.

“No difference appeared for the sole Gf subtest, Matrix Reasoning, or for working memory or processing speed.”

May combine two claims.

Four of the Gc subtest differences replicated from Study 1; the only change in Study 2 was for the Similarities subtest, which had not shown a significant difference in Study 1.

“Four of these replicated from Study 1; the only change in Study 2 was for the Similarities subtest, which had not shown a significant difference in Study 1.”

Classified as probably not a substantial claim. The quote only partly states this claim.

After excluding the five subjects with intellectual disability, the reanalysis sample size was N = 74.

“The results (N = 74) are displayed in Table 2 and in Figure 2a.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Study 2 sample included 42 women and 39 men, mean age = 38.44 years, SD = 11.94, range = 20 to 60 years, with a continuous distribution.

“The sample included 42 women and 39 men, mean age = 38.44 years, SD = 11.94,range = 20 to 60 years (with a continuous distribution).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For Digit Symbol Coding and Picture Completion, test version interacted with test order (p = .026 and p < .001 respectively).

“For Digit Symbol Coding and Picture Completion, test version interacted with test order (p = .026and p < .001respectively)”

May combine two claims.

A complementary linear regression indicated that the predicted difference of FSIQ between versions for a subject with a null difference in the VCI was small and not significantly different from zero, b = -0.12, t = -0.07, p = .947, confirming that Gc subtests accounted for most or all of the difference in FSIQ.

“A complementary analysis using linear regression indicated that the predicted difference of FSIQ between versions for a subject with a null difference in the VCI was small and not significantly different from zero, b = -0.12,t = -0.07,p = .947,confirming that Gc subtests accounted for most or all of the difference in FSIQ.”

• AssumptionRests on an assumption: The WAIS-III and WAIS-IV normative samples are valid and comparable reference points for estimating ability differences across time.
• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

May combine two claims.

There was substantial evidence in favor of the null hypothesis for the three subtests assessing Gf and Gv - Matrix Reasoning, Picture Completion and Block Design - and for the corresponding PRI, as well as for other subtests reflecting Gsm and Gs.

“Critically, there was substantial evidence in favor of the null hypothesis for the three subtests assessing Gf and Gv -Matrix Reasoning, Picture Completion and Block Design -and for the corresponding PRI, as well as for other subtests reflecting Gsm and Gs.”

May combine two claims.

After exclusion of subjects with intellectual disability, significant differences remained for the Arithmetic subtest, which includes a significant Gc loading, and for Block Design, a Gv subtest, but these differences were more unstable and only appeared in the subgroup of older adults who performed the WAIS-III first.

“After exclusion of subjects with intellectual disability, significant differences also remained for the Arithmetic subtest, which also includes a significant Gc loading, and for Block Design, a Gv subtest, but these differences were more unstable and only appeared in the subgroup of older adults who performed the WAIS-III first.”

May combine two claims.

In the combined datasets, there were no differences for the other subtests, apart from a slightly higher performance in the WAIS-IV version of Symbol Search, driven by the Study 2 sample.

“There were no differences for the other subtests, apart from a slightly higher performance in the WAIS-IV version of Symbol Search, driven by the Study 2 sample.”

May combine two claims.

The five subjects with intellectual disability were outliers on both Matrix Reasoning and Block Design, with a standard score of 1 only in the WAIS-III.

“these five subjects were outliers on both the Matrix Reasoning subtest and the Block Design subtest, with a standard score of 1 (the minimum possible, reflecting complete failure to meet task requirements), but only in the WAIS-III.”

May combine two claims.

Two Arithmetic items required ability more than one standard deviation higher in the 2019 sample, and both required mental division; out of the other five biased items, four also required mental division, either explicitly or in the context of computing a proportion.

“Two items required ability more than one standard deviation higher in the 2019 sample, and both required mental division; out of the other five biased items, four also required mental division, either explicitly or in the context of computing a proportion.”

May combine two claims.

Heritability and dysgenics both had non-significant correlations with performance decline.

“heritability and dysgenics both had non-significant correlations with performance decline”

May combine two claims.

Performance decline on a subtest correlated near unity with its cultural load: correlations were between .85 and .93 for the three separate measures and reached .95 for the composite cultural load, all significant at the .001 level.

“Performance decline on a subtest correlated near unity with its cultural load: the correlations were between .85 and .93 for the three measures of cultural load considered separately, and reached .95 for the composite cultural load.All correlations were significant at the .001level.”

May combine two claims.

The reported Jensen effect is both non-significant and highly unstable.

“It is clear from these results that the reported Jensen effect is both non-significant (the confidence interval includes zero) and highly unstable (with plausible values or r ranging between -0.5 and 2).”

May combine two claims.

There did not seem to be large differences in racial composition of the two WAIS samples, and the share of immigrants in the French population increased by about 2 percentage points between 1999 and 2019, which seems much too small to create the large differences reported.

“there did not seem to be large differences in racial composition of the two WAIS samples, and the share of immigrants in the French population has increased by about 2 percentage points between 1999 and 2019, which seems much too small to create the large differences reported here”

May combine two claims.

For the Information subtest, 11 out of 28 items were significantly more difficult in the 2019 sample, and none were significantly easier.

“For the Information subtest, 11 out of 28 items were significantly more difficult in the 2019 sample, and none were significantly easier.”

May combine two claims.

DIF was relatively limited for the Similarities and Comprehension subtests, where item parameters could not be estimated for many items which lacked discriminating power in the current sample; but it was clearly apparent for the three other subtests where more items were available.

“DIF was relatively limited for the Similarities and Comprehension subtests, where item parameters could not be estimated for many items which lacked discriminating power in the current sample; but it was clearly apparent for the three other subtests where more items were available.”

May combine two claims.

Out of the remaining six Comprehension items for which estimation was possible, two items were significantly more difficult in the 2019 sample, and a third item was marginally more difficult.

“Out of the remaining six items for which estimation was possible, two items were significantly more difficult in the 2019 sample, and a third item was marginally more difficult.”

May combine two claims.

The correction for restriction of range inflated the correlation by about 50%, was not entirely clearly justified, and was applied incorrectly.

“It is not entirely clear whether this correction, which inflated the correlation by about 50%, was justified in this case (this approach makes assumptions about how the data were selected, and what the performance decline would have been with a different version of the WAIS), and besides, it was applied incorrectly.”

May combine two claims.

For Vocabulary, the negative net effect of bias was partly compensated by some words being easier in the recent sample, more consistent with a change in language frequency patterns than with an absolute decrease in vocabulary skills.

“For Vocabulary, the negative net effect of bias was partly compensated by the fact that some words were easier in the recent sample, more consistent with a change in language frequency patterns than with an absolute decrease in vocabulary skills.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Correcting subtest-level estimates for their reliability changed the correlation estimates by less than .02.

“correcting subtest-level estimates for their reliability, as in the second analysis of Woodley of Menie & Dunkel, 2015, changed the correlation estimates by less than .02”

• cites Woodley of Menie & Dunkel, 2015 sourceNot checked — no full text of the source

May combine two claims.

The reported correlation of r = .72 with biological load is not statistically significant because the authors computed the correlation between performance decline and a composite of five variables including itself.

“It may seem surprising that the reported correlation of r = .72with biological load is not statistically significant: this is because the authors computed the correlation between performance decline and a composite of five variables including itself.”

May combine two claims.

Adjusting the confidence interval for the corrected correlation by the same correction factor gives r = .83, 95% CI [-0.60, 2.26], or with the appropriate correction for range restriction, r = .74, 95% CI [-0.53, 2.01].

“In the present case, this gives r = .83,95% CI [-0.60, 2.26], or with the appropriate correction for range restriction, r = .74,95% CI [-0.53, 2.01].”

May combine two claims.

For the Arithmetic subtest, 7 out of 20 items were significantly more difficult for the 2019 sample, and none were significantly easier.

“Lastly, for the Arithmetic subtest, 7 out of 20 items were significantly more difficult for the 2019 sample, and none were significantly easier.”

May combine two claims.

The authors performed statistical inference on the corrected correlation as if it had been uncorrected; inflating the correlation with a correction also inflates the risk of error, requiring an adjusted significance test.

“The third problem is that the authors performed statistical inference on the corrected correlation as if it had been uncorrected; but inflating the correlation with a correction also inflates the risk of error, which also requires an adjusted significance test.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For the Comprehension subtest, item parameter estimation was impossible for 12 out of 18 items due to too few failures in the 2019 sample.

“For the Comprehension subtest, item parameter estimation was impossible for 12 out of 18 items due to too few failures in the 2019 sample.”

May combine two claims.

The WAIS-III Information subtest includes 12 items asking about the identity of famous people (five of whom lived and died in the twentieth century), whereas the WAIS-IV version includes only 4 items about famous people, mostly at lower difficulties.

“the WAIS-III Information subtest includes 12 items asking about the identity of famous people (five of which lived and died in the twentieth century), whereas the WAIS-IV version includes only 4 items about famous people, mostly at lower difficulties.”

May combine two claims.

In the WAIS-III Comprehension subtest, 7 of the most difficult items concern civic education and economy, whereas the WAIS-IV version includes only 1 such item; conversely, the WAIS-IV version includes 5 items concerning ecology and development aid, topics absent from the WAIS-III.

“In the WAIS-III Comprehension subtest, 7 of the most difficult items concern civic education and economy, whereas the WAIS-IV version includes only 1 such item; conversely, the WAIS-IV version includes 5 items concerning ecology and development aid, topics which were absent from the WAIS-III.”

May combine two claims.

Almost all biased Information items concerned the identification of famous people; six out of the seven items that required ability at least one standard deviation higher in the 2019 sample involved famous people, with four involving famous writers.

“Almost all biased items concerned the identification of famous people; this was the case for six out of the seven items that required ability at least one standard deviation higher in the 2019 sample, with four of these items involving famous writers.”

May combine two claims.

The content of all five subtests was significantly modified between the WAIS-III and WAIS-IV: the French publisher changed 11 of 18 Comprehension items, 29 of 33 Vocabulary items, 21 of 28 Information items, and 12 of 19 Similarities items.

“the content of all five subtests was significantly modified between the WAIS-III and WAIS-IV: the French publisher changed 11 of the 18 Comprehension items, 29 of the 33 Vocabulary items, 21 of the 28 Information items, and 12 of the 19 Similarities items.”

May combine two claims.

Characteristic curves for Vocabulary, Information and Arithmetic showed substantial bias against the 2019 sample; bias was somewhat smaller for Similarities, and for Comprehension though this may be due to the very low number of items available for parameter estimation.

“Characteristic curves for Vocabulary, Information and Arithmetic showed substantial bias against the 2019 sample; bias was somewhat smaller for the Similarities subtest, and for the Comprehension subtest though this may be due to the very low number of items available for parameter estimation.”

May combine two claims.

Recalculating the Jensen effect using WAIS-IV g-loadings gives a raw Jensen effect of r = .29, p = .361.

“in this case, the raw Jensen effect was only r = .29,p = .361”

May combine two claims.

The correlations of performance decline with both biological load and g-loadings are substantial overestimates, and both are actually non-significant at the .05 level.

“First, the correlations of performance decline with both biological load and g-loadings are substantial overestimates, and both are actually non-significant at the .05level.”

May combine two claims.

The recent sample had lower total performance on the WAIS-III, and this difference between cohorts was exclusively driven by the five subtests involving Gc, acquired declarative knowledge tied to a specific cultural setting.

“We did find lower total performance on the WAIS-III for a recent sample, but contrary to the classic Flynn effect, this difference between cohorts was exclusively driven by the five subtests involving Gc -acquired declarative knowledge tied to a specific cultural setting.”

The quote only partly states this claim.

In Study 1, after excluding five subjects with intellectual disability, the WAIS-IV Full Scale IQ was significantly higher than the WAIS-III Full Scale IQ (M=102.39 vs 98.72, t=4.08, p<.001, dz=0.47).

“| Full scale IQ | 74 | 98.72 (14.16) | 102.39 (14.95) 4.08 | <.001 | 0.47 | *** | |”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The subsample of 39 subjects included 19 men and 20 women; socio-economic level appeared to match the composition of the general population.

“This subsample included 19 men and 20 women; socio-economic level was assessed based on the categories of the French national institute of statistics (INSEE), and appeared to match the composition of the general population”

Classified as probably not a substantial claim.

Prior research has shown that some items in intelligence tests tend to become significantly easier or harder over time (item drifts).

“In line with this idea, prior research has shown that some items in intelligence tests tend to become significantly easier or harder over time (item drifts; e.g. Brand et al., 1989).”

• cites Brand et al., 1989 sourceNot checked — no full text of the source

May combine two claims.

Dutton and Lynn could not perform any significance test because the publisher reported only averages and standard deviations for the whole sample, while the design was within-subjects.

“The second methodological reason to reserve judgment is the fact that the authors could not perform any significance test: the publisher reported only averages and standard deviations for the whole sample in the test manual, but the design was within-subjects (in which case inferential statistics require access to the raw data to compute the variance of difference scores).”

May combine two claims.

Subtests primarily reflecting Gc intrinsically depend on culture-based declarative knowledge, and culture as assessed by these tests changes over time, making them especially likely to develop DIF.

“On the other hand, subtests primarily reflecting Gc intrinsically depend on culture-based declarative knowledge, and culture as assessed by these tests changes over time (e.g.Pietschnig et al., 2013), making them especially likely to develop DIF (see Beaujean & Osterlind, 2008;Pietschnig et al., 2013;Wicherts et al., 2004).”

• cites Pietschnig et al., 2013 sourceNot checked — no full text of the source
• cites Beaujean & Osterlind, 2008 sourceNot checked — no full text of the source
• cites Wicherts et al., 2004 sourceNot checked — no full text of the source

May combine two claims.

All subtests of the WAIS do not equally assess intelligence: VCI subtests are largely reflections of culture-based declarative knowledge, whereas PRI subtests depend more on inductive reasoning and visuo-spatial ability.

“In other words, all subtests of the WAIS do not equally assess intelligence: subtests of the VCI (which require e.g.defining words or remembering trivia) are largely reflections of culture-based declarative knowledge, whereas subtests of the PRI depend more on inductive reasoning and visuo-spatial ability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Geographic regions in this subsample were somewhat unbalanced, with 28 of the 39 subjects coming from southern France.

“Geographic regions in this subsample were somewhat unbalanced, with 28 of the 39 subjects coming from southern France.”

May combine two claims.

Wicherts and colleagues (2004) found that measurement bias often affected subtests with a substantial weight of culture-based declarative knowledge; in their study with the WAIS, the two subtests with substantial measurement bias across cohorts were Similarities and Comprehension, both VCI subtests.

“Wicherts and colleagues (2004) found that this measurement bias often affected subtests with a substantial weight of culture-based declarative knowledge; in their study with the WAIS, the two subtests with substantial measurement bias across cohorts were Similarities and Comprehension, both subtests of the VCI.”

• cites Wicherts et al., 2004 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

There were missing values for the Digit Symbol Coding subtest, yielding unequal sample sizes across tests.

“There were missing values for the Digit Symbol Coding subtest, yielding unequal sample sizes across tests.”

May combine two claims.

Careful examination of the data reported by Dutton and Lynn (Table 3) suggests that the difference between WAIS-III and WAIS-IV was mainly driven by subtests depending on Gc (Vocabulary, Comprehension, Information), suggesting that it did not reflect a general decline of intelligence.

“Critically, careful examination of the data reported by Dutton and Lynn (Table 3) suggests that the difference between WAIS-III and WAIS-IV was mainly driven by subtests depending on Gc (Vocabulary, Comprehension, Information), suggesting that it did not reflect a general decline of intelligence.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Subtests primarily reflecting Gf (e.g., Matrix Reasoning) or Gv (e.g., Block Design) are based on abstract materials which should be relatively timeless; the same is true for subtests reflecting Gs and Gsm.

“Subtests primarily reflecting Gf, such as Matrix Reasoning, or Gv, such as Block Design, are based on abstract materials which should be relatively timeless; the same is true for subtests reflecting Gs and Gsm.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Woodley of Menie and Dunkel (2015) insisted that the decline was likely due to biological causes.

“A commentary of Woodley of Menie and Dunkel (2015) insisted that this decline was likely due to biological causes.”

• cites Woodley of Menie and Dunkel (2015) sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

To control for possible confounding, the DIF analysis equated the range of ability by removing subjects of the 1999 sample who performed lower or higher than all subjects of the 2019 sample; the final 1999 sample size was n = 767.

“To control for this possibility, a preliminary step for the DIF analysis was to equate the range of ability in the two samples by removing subjects of the 1999 sample who performed lower or higher than all subjects of the 2019 sample.The final sample size for the 1999 sample was n = 767.”

• AssumptionRests on an assumption: The 1999 WAIS-III normative sample is an appropriate reference for comparing item functioning with the 2019 sample.

May combine two claims.

These changes would not be sufficient, in and of themselves, to explain the difference reported by Dutton and Lynn, but they could inflate the difference between WAIS-III and WAIS-IV.

“These changes would not be sufficient, in and of themselves, to explain the difference reported by Dutton and Lynn 1 , but they could inflate the difference between WAIS-III and WAIS-IV”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

The easier Vocabulary items are compatible with evolving patterns of word use in the language.

“compatible with the idea of evolving patterns of word use in the language (e.g.Brand, 1989).”

• cites Brand, 1989 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Inferential statistics were obtained using the item parameter replication method with the correction proposed by Cervantes.

“Inferential statistics were obtained using the item parameter replication method (Oshima et al., 2006), with the correction proposed by Cervantes (2012Cervantes ( , 2017a; see also Clark & LaHuis, 2012).”

• cites Oshima et al., 2006 sourceNot checked — no full text of the source
• cites Cervantes (2012, 2017a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Clark & LaHuis, 2012 sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The latter property meant that the ability estimate was not confounded with measurement bias in the items being tested, and the same ability estimate was used to test DIF in the five subtests.

“The latter propriety meant that the ability estimate was not confounded with measurement bias in the items being tested, and that the same ability estimate was used to test DIF in the five subtests.”

Classified as probably not a substantial claim. May combine two claims.

The Flynn effect is supposed to be a latent increase at the first level of g, especially prevalent for the second level factor of Gf, and this increase is expected not to be caused by measurement bias at the third or fourth levels.

“The Flynn effect is supposed to be a latent increase at the first level of g, especially prevalent for the second level factor of Gf; and this increase is expected not to be caused by measurement bias at the third or fourth levels (see Flynn, 2009a).”

• cites Flynn, 2009aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The validity sample was collected by psychologists hired by the publisher, who received specific training, and protocols were checked for compliance and correct scoring.

“All psychologists received specific training from the publisher prior to data collection; each psychologist sent back protocols to the publisher to ensure that they complied with data collection instructions and that the test was scored correctly.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Bayes Factors were computed for the combined sample to represent the likelihood of the alternative hypothesis of a difference between WAIS-III and WAIS-IV over the null, with uniform priors.

“Taking advantage of the increased sample size for this combined sample, we also computed Bayes Factors representing the likelihood of the alternative hypothesis of a difference between WAIS-III and WAIS-IV over the null (with uniform priors).”

Classified as probably not a substantial claim.

Dutton and Lynn's (2015) interpretation considers all subtests of the WAIS as interchangeable measures of intelligence.

“Besides methodological issues, Dutton and Lynn's (2015) interpretation considers all subtests of the WAIS as interchangeable measures of intelligence.”

• cites Dutton and Lynn (2015) sourceNot checked — no full text of the source

May combine two claims.

Studies using IRT have shown that item parameters do vary across cohorts, with drifts of item difficulty for some items and loss of discriminating power for others, especially for tests involving culture-based knowledge such as vocabulary and mathematics.

“Studies using this approach have shown that item parameters do vary across cohorts, with drifts of item difficulty for some items and loss of discriminating power for others, especially for tests involving culture-based knowledge such as vocabulary and mathematics (Beaujean & Osterlind, 2008;Pietschnig et al., 2013).”

• cites Beaujean & Osterlind, 2008 sourceNot checked — no full text of the source
• cites Pietschnig et al., 2013 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Study 2 subjects were included using quotas on gender, age, and socio-economic level.

“As in Study 1, subjects were included using the method of quotas, with quotas on gender, age, and socio-economic level.”

May combine two claims.

If the WAIS-III items have become outdated, subjects performing the test now would perform lower than they would on the more recent WAIS-IV, and comparatively lower than the normative sample who completed the WAIS-III when it was first designed, creating the illusion of a score decline.

“If the WAIS-III items have become outdated, it is easy to understand why subjects performing the test now would perform lower than they would on the more recent WAIS-IV, and comparatively lower than the normative sample who completed the WAIS-III when it was first designed, creating the illusion of a score decline.”

Classified as probably not a substantial claim.

Performance can be represented as a hierarchical model with four levels: latent general intelligence g; latent broad dimensions such as Gf, Gc, Gq, Gs, Gsm; observed performance on subtests; and observed performance on items.

“Performance can be represented as a hierarchical model with four levels: on the top is latent general intelligence, g; on the second level are latent broad dimensions of intelligence such as Gf, Gc, Gq, Gs and Gsm; on the third level is observed performance on the various subtests that serve to estimate latent dimensions; and on the fourth level is observed performance on the specific items that”

• cites Wicherts et al., 2004 sourceNot checked — no full text of the source
• cites Wicherts, 2007Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Each experimenter collected 2 to 5 protocols, and each subject completed the WAIS-III and WAIS-IV with the same experimenter.

“Each experimenter collected 2 to 5 protocols, and each subject completed the WAIS-III and WAIS-IV with the same experimenter.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The counterbalancing with test order was performed correctly: 20 younger and 20 older subjects performed the WAIS-III first, whereas 21 younger and 18 older adults performed the WAIS-IV first.

“Note that the counterbalancing with test order was performed correctly: 20 younger and 20 older subjects performed the WAIS-III first, whereas 21 younger and 18 older adults performed the WAIS-IV first.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The conversion from raw scores to Wechsler standard scores is only a rough approximation because the conversion is nonlinear and varies as a function of age and level of ability.

“Note that this conversion is only a rough approximation, as the conversion from raw scores to standard scores is not linear (it uses a normalized scale, which means multiple raw scores yield the same standard score) and varies as a function of age and level of ability (we used the norms for 35 to 44 year-olds on the WAIS-III as a point of reference corresponding to the mean age of our sample).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Study 2 subjects completed both the WAIS-III and WAIS-IV in counterbalanced order (WAIS-III first: n = 42; WAIS-IV first: n = 39).

“Subjects completed both the WAIS-III and WAIS-IV (Wechsler 2000(Wechsler , 2011) ) in counterbalanced order (WAIS-III first: n = 42; WAIS-IV first: n = 39).”

Classified as probably not a substantial claim.

The lack of statistical tests in Dutton and Lynn's study made the point that the difference was driven by Gc subtests uncertain.

“Although the lack of statistical tests in their study made this point uncertain, such a result would severely limit interpretation of the findings in terms of a”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Data collection in Study 2 was performed by 24 young psychologists as part of their final graduate training, all of whom had received extensive training with Wechsler scales.

“Data collection was performed by 24 young psychologists as a part of their final graduate training (we reasoned that their level of expertise with the WAIS-IV would be similar to that of psychologists paid to collect the data during WAIS-IV development in Study 1); all had received extensive training with Wechsler scales.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Study 2 was designed as a replication of Study 1, with data collection planned for the same sample size.

“Study 2 was designed as a replication of Study 1, meaning data collection was planned for the same sample size.”

Classified as probably not a substantial claim.

The ability estimate used for DIF was an index of general intelligence computed as the average of standardized performance on all WAIS-III subtests, excluding the five subtests being tested for DIF, to avoid measurement bias influencing the ability estimate.

“We used as an estimate of subject ability an index of general intelligence, computed as the average of standardized performance on all subtests of the WAIS-III (with standardization performed conjointly over the two samples), excluding the five subtests being tested for DIF (to avoid the possibility of measurement bias influencing the ability estimate).”

May combine two claims.

The Symbol Search result suggests an increase in processing speed between the WAIS-III and WAIS-IV samples, even more incompatible with a negative Flynn effect.

“suggesting an increase in processing speed between the WAIS-III and WAIS-IV samples, even more incompatible with a negative Flynn effect.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A likely possibility for why differences appeared only in the older-adults-WAIS-III-first subgroup is random variation due to the small sample size (20 subjects in that condition).

“A likely possibility is random variation due to the small sample size (there were only 20 subjects in this particular condition)”

May combine two claims.

The corrected-sample results confirmed that the observed difference between WAIS-III and WAIS-IV was almost exclusively driven by Gc subtests, with no difference for most other subtests, including Matrix Reasoning, working memory, and processing speed.

“These results confirmed the prior conclusion that the observed difference between WAIS-III and WAIS-IV was almost exclusively driven by Gc subtests, whereas no difference appeared for most other subtests, including the Gf subtest of Matrix Reasoning, as well as working memory and processing speed subtests.”

May combine two claims.

In sum, there was a significant difference between WAIS-III and WAIS-IV in FSIQ, but it was mostly attributable to a difference in subtests with a large Gc loading: incompatible with a reversal of the Flynn effect classically observed to a greater extent on Gf subtests and also incompatible with a general decline of intelligence, but compatible with measurement bias on subtests involving culture-

“In sum, there was a significant difference between WAIS-III and WAIS-IV in FSIQ, but it was mostly attributable to a difference in subtests with a large Gc loading: incompatible with a reversal of the Flynn effect classically observed to a greater extent on Gf subtests and also incompatible with a general decline of intelligence, but compatible with measurement bias on subtests involving culture-”

• AssumptionRests on an assumption: The WAIS-III and WAIS-IV normative samples are valid and comparable reference points for estimating ability differences across time.
• AssumptionRests on an assumption: The raw data for the validity sample provided by the publisher are accurate and complete, except for the two errors the authors corrected.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Study 2 first series of analyses used within-subjects t-tests, with dz effect sizes and FDR correction for multiple comparisons.

“As in Study 1, the first series of analyses testing the difference between WAIS-III and WAIS-IV used within-subjects t-tests, with dz effect sizes and FDR correction for multiple comparisons.”

Classified as probably not a substantial claim. May combine two claims.

This finding questioned the meaning and stability of the difference between WAIS-III and WAIS-IV for Arithmetic and Block Design, and invited replication in Study 2.

“In any case, this finding questioned the meaning and the stability of the difference between WAIS-III and WAIS-IV for these two particular subtests, and invited replication in Study 2.”

May combine two claims.

The first finding of Study 1 was that the sample composition was poorly suited to address the specific question of Flynn effects, due to a design with two discrete age groups and the presence of five individuals with intellectual disability (6.3% of the sample) who completely failed some subtests of the WAIS-III.

“The first finding of Study 1 was that the sample composition was poorly suited to address the specific question of Flynn effects, due to a design with two discrete age groups (Figure 1), and the presence of five individuals with intellectual disability (6.3% of the sample) who completely failed some subtests of the WAIS-III.”

• AssumptionRests on an assumption: The publisher's validity sample was collected only to ensure that the WAIS-III correlated with the WAIS-IV, and was never intended to be analyzed for other purposes.

May combine two claims.

Intelligence tests, including the WAIS, do not demonstrate measurement invariance over time; the baseline difficulty of a given subtest can change over time, even when controlling for ability.

“In line with this idea, intelligence tests -including the WAIS -do not demonstrate measurement invariance over time: in particular, the baseline difficulty of a given subtest can change over time, even when controlling for ability (Wicherts et al., 2004).”

• cites Wicherts et al., 2004 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Dutton and Lynn (2015) published a study claiming that intelligence scores in France are decreasing.

“In 2015, Dutton and Lynn published a study claiming that intelligence scores in France are decreasing.”

• cites Dutton and Lynn (2015) sourceNot checked — no full text of the source

May combine two claims.

Two studies found that observed intelligence scores changed over several decades, while latent ability as estimated from IRT remained near-constant.

“When considering total performance, two studies found that observed intelligence scores changed over several decades, while latent ability as estimated from IRT remained near-constant (Beaujean & Osterlind, 2008;Beaujean & Sheng, 2010), confirming both that item properties can change without corresponding changes of ability, and that this can create the illusion of ability changes over time.”

• cites Beaujean & Osterlind, 2008 sourceNot checked — no full text of the source
• cites Beaujean & Sheng, 2010 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Dutton and Lynn concluded that intelligence had declined from 1999 to 2009, a reversal of the Flynn effect.

“Dutton and Lynn concluded that intelligence had declined from 1999 to 2009 -hence a reversal of the Flynn effect (Flynn, 1984;Rundquist, 1936).”

• cites Flynn, 1984 sourceNot checked — no full text of the source
• cites Rundquist, 1936Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Each subject completed both the WAIS-III and WAIS-IV in counterbalanced order (40 subjects performed the WAIS-III first and 39 performed the WAIS-IV first).

“Each subject completed both the WAIS-III and WAIS-IV in counterbalanced order (40 subjects performed the WAIS-III first and 39 performed the WAIS-IV first).”

May combine two claims.

In the present case, the discrepancy in discriminating power for low-performing subjects largely contributed to the observed difference between WAIS-III and WAIS-IV for Matrix Reasoning and Block Design, potentially biasing the results.

“in the present case, it largely contributed to the observed difference between WAIS-III and WAIS-IV for these two subtests (Table 1), potentially biasing the results.”

May combine two claims.

Item 15 involves customs that have practically disappeared in France in recent years due to the opening of borders within the European Union, and the French word for the other concept required by this item is now explicitly labeled as dated in many dictionaries.

“this makes particular sense because this item involves customs, which have practically disappeared in France in recent years due to the opening of borders within the European Union, and the French word for the other concept required by this item is now explicitly labeled as”

Probably not a substantial claim: may not pass S1 (asserted). May combine two claims.

The results of Dutton and Lynn suggest both that there is no decline for the first level of g, and a limited decline for the second level but only for Gc; and that this decline could be caused, not by an actual change of latent ability, but by measurement bias at the fourth level representing items.

“By contrast, the results of Dutton and Lynn suggest both that there is no decline for the first level of g, and a limited decline for the second level but only for Gc; and that this decline could be caused, not by an actual change of latent ability, but by measurement bias at the fourth level representing items.”

Classified as probably not a substantial claim.

Dutton and Lynn (2015) compared performance averages reported in the test manual for the validity sample and observed that these subjects had higher standardized scores on the WAIS-IV than on the WAIS-III.

“Dutton and Lynn (2015) compared performance averages reported in the test manual for this validity sample, and observed that these subjects had higher standardized scores on the WAIS-IV (in reference to its 2009 normative sample) than on the WAIS-III (in reference to its 1999 normative sample).”

• cites Dutton and Lynn (2015) sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Due to the low sample size in the 2019 sample, only difficulty parameters were examined, with the discrimination parameter fixed to 1; the one-parameter logistic model was used for items scored 0-1, and the partial credit model for items scored 0-1-2.

“Due to the low sample size in the 2019 sample, we only examined difficulty parameters, with the discrimination parameter fixed to 1: the one-parameter logistic model (1-PL; see Birnbaum, 1968;Rasch, 1960) was used for items scored 0-1, and the partial credit model (PCM with a slope fixed to 1; Masters, 1982) was used for items scored 0-1-2.”

• cites Birnbaum, 1968Not checked — no full text of the source
• cites Rasch, 1960Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Masters, 1982 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Subjects were recruited using quotas on age, gender, and socio-economic level.

“Subjects were recruited using the method of quotas, with quotas on age, gender, and socio-economic level.”

Classified as probably not a substantial claim. May combine two claims.

Given the limited sample size, the absence of statistical tests makes it unclear whether the difference between WAIS-III and WAIS-IV was significantly greater than chance, and if so, which subtests were affected.

“Given the limited sample size, this absence of statistical tests makes it unclear whether the difference between WAIS-III and WAIS-IV was in fact significantly greater than chance, and if so, which subtests of the WAIS were affected.”

May combine two claims.

The combined datasets of Study 1 and Study 2 were firmly incompatible with the possibility of a negative Flynn effect reflecting a general decrease of intelligence or a larger decrease for Gf subtests, and instead confirmed that performance decreased specifically for the subtests involving declarative knowledge.

“In sum, the combined datasets of Study 1 and Study 2 were firmly incompatible with the possibility of a negative Flynn effect reflecting a general decrease of intelligence or a larger decrease for Gf subtests, and instead confirmed that performance decreased specifically for the subtests involving declarative knowledge.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The WAIS is not ideal to test for DIF due in particular to sequential dependencies between items, which could especially bias item parameters for the final items of a subtest.

“This is all the more true that the WAIS is not ideal to test for DIF (due in particular to the presence of sequential dependencies between items, with a subtest being interrupted when subjects fail several items in a row; this could especially bias item parameters for the final items of a subtest).”

May combine two claims.

This was the case for all subtests except perhaps Similarities, where differential test functioning between the 1999 and 2019 sample accounted for about one third the size of the observed difference between WAIS-III and WAIS-IV.

“This was the case for all subtests except perhaps for Similarities, where differential test functioning between the 1999 and 2019 sample accounted for about one third the size of the observed difference between WAIS-III and WAIS-IV.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There is no obvious explanation for the Similarities discrepancy, though Similarities was also the subtest with the least DIF, which is more compatible with the nonsignificant difference found in Study 1.

“There does not seem to be an obvious explanation for this discrepancy, though we note that Similarities was also the subtest with the least DIF, which is more compatible with the nonsignificant difference found in Study 1.”

Classified as probably not a substantial claim.

Woodley of Menie and Dunkel (2015) argued that the results of Dutton and Lynn (2015) indicated a biological origin for the decline of performance between WAIS-III and WAIS-IV in French subjects.

“Woodley of Menie and Dunkel (2015) argued that the results of Dutton and Lynn (2015) indicated a biological origin for the decline of performance between WAIS-III and WAIS-IV in French subjects.”

• cites Woodley of Menie and Dunkel (2015) sourceNot checked — no full text of the source
• cites Dutton and Lynn (2015) sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Woodley of Menie and Dunkel reported r = .83 for the correlation between the amount of decline on a subtest and its g-loading.

“they reported r = .83for this effect.”

• cites Woodley of Menie and Dunkel (2015) sourceNot checked — no full text of the source

May combine two claims.

PISA surveys routinely point to significant inequalities in the academic skills of French pupils, and their average level of mathematics performance has declined since the early 2000s.

“results of the large-scale PISA surveys (performed on about 7.000 pupils) routinely point to significant inequalities in the academic skills of French pupils, and their average level of mathematics performance has declined since the early 2000s (e.g.OECD, 2019).”

• cites OECD, 2019 sourceNot checked — no full text of the source

May combine two claims.

The same was true when combining the two datasets, and Bayesian evidence was clearly in favor of the null for Gv and Gf subtests, in line with a difference on subtests assessing culture-dependent declarative knowledge but incompatible with the existence of a general decline of intelligence or a negative Flynn effect.

“The same was true when combining the two datasets, and Bayesian evidence was clearly in favor of the null for Gv and Gf subtests, in line with a difference on subtests assessing culture-dependent declarative knowledge but incompatible with the existence of a general decline of intelligence or a negative Flynn effect.”

Probably not a substantial claim: may not pass S2 (checkable).

It is not entirely meaningful to interpret the difference between WAIS-III and WAIS-IV based on differential test functioning for the WAIS-III, given that the WAIS-IV is based on very different materials.

“It is also not entirely meaningful to interpret the difference between WAIS-III and WAIS-IV based on differential test functioning for the WAIS-III, given that the WAIS-IV is based on very different materials.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Item 17 involves the process for obtaining rubber, which may be less obvious to subjects now than it was over 20 years ago.

“this item involves the process for obtaining rubber, which may be less obvious to subjects now than it was over 20 years ago.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The significant Block Design difference in the other dataset may have been a spurious effect partly caused by the particular sampling with discrete age groups.

“since this difference existed only for one of four subgroups in Study 1, the significant difference in the other dataset may have been a spurious effect partly caused by the particular sampling with discrete age groups.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The interpretation of heritability estimates poses well-known issues and has been criticized elsewhere.

“The interpretation of heritability estimates poses well-known issues and has been criticized elsewhere (e.g.Dickens & Flynn, 2001;Kan et al., 2013).”

• cites Dickens & Flynn, 2001 sourceNot checked — no full text of the source
• cites Kan et al., 2013 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

This principle generalizes to all comparisons between samples, not just intelligence fluctuations over time.

“In other words, this principle generalizes to all comparisons between samples, not just intelligence fluctuations over time”

Probably not a substantial claim: may not pass S3 (load-bearing).

These are not precise estimates, but a rough approximation based on extrapolating WAIS-III norms, computed just to get a sense of relative effect sizes.

“We reiterate that these are not precise estimates, but a rough approximation based on extrapolating WAIS-III norms, computed just to get a sense of relative effect sizes.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The low sample size for IRT analyses was attenuated by comparing item parameters to the much larger WAIS-III standardization sample (n = 767 after matching for ability) and by using scores on the rest of the WAIS-III as an index of subject ability, but estimates in Table 5 may be relatively unstable.

“This issue was attenuated as much as possible by comparing item parameters for DIF analyses to the much larger standardization sample for the WAIS-III retrieved from the publisher (n = 767 after matching for ability), and by using scores on the rest of the WAIS-III as an index of subject ability, which made parameter estimation considerably simpler; but estimates presented in Table 5 may be RelATI”

Probably not a substantial claim: may not pass S2 (checkable).

The hypothesis that a biological factor influenced culturally-loaded subtests due to a genetically-driven cultural decrease would be almost impossible to falsify.

“such a hypothesis would be almost impossible to falsify”

Probably not a substantial claim: may not pass S3 (load-bearing).

All three discrepancies between Studies 1 and 2 may represent random variation due to limited sample sizes.

“All three discrepancies between Studies 1 and 2 may represent random variation due to limited sample sizes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Sample size was still low for IRT-based analyses.

“However, sample size was still low for IRT-based analyses.”

May combine two claims.

The high average ability of the sample did not bias the analyses: there was no ceiling effect, scores were still in the normal range, had adequate dispersion, and were not skewed.

“Importantly, this high average ability did not bias the analyses: there was no ceiling effect, and scores were still in the normal range (the lowest FSIQ was 79), had adequate dispersion and were not skewed (all skewness coefficients were below |1|).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Combining the datasets for Study 1 and Study 2 partly answered the sample-size issue, although this was not a perfect strategy because the two samples were separated by a little under 10 years.

“Combining the datasets for Study 1 and Study 2 also partly answered this issue, although this was not a perfect strategy either (with the two samples being separated by a little under 10 years, even though their conclusions converged).”

May combine two claims.

The results of Study 2 replicated the conclusions of Study 1: there was a significant difference of FSIQ between WAIS-III and WAIS-IV, but it was exclusively driven by the five subtests depending on Gc; no difference appeared for Gv or Gf subtests.

“The results of Study 2 replicated the conclusions of Study 1: there was a significant difference of FSIQ between WAIS-III and WAIS-IV, but it was exclusively driven by the five subtests depending on Gc; no difference appeared for Gv or Gf subtests.”

May combine two claims.

The pattern of results is entirely distinct from the Flynn effect, which represents an increase in general intelligence, especially Gf performance, accompanied by much smaller changes on Gc.

“This pattern is entirely distinct from the Flynn effect, which represents an increase in general intelligence, and especially in Gf performance, accompanied by much smaller changes on Gc (Pietschnig & Voracek, 2015).”

• cites Pietschnig & Voracek, 2015 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

The pattern cannot reasonably be labeled a negative Flynn effect without extending the definition of the Flynn effect to the point where any difference between cohorts could be called a Flynn effect and where it would no longer be useful as a heuristic concept.

“Hence it is our conviction that this pattern reflects substantially different mechanisms and cannot reasonably be labeled a”

Probably not a substantial claim: may not pass S3 (load-bearing).

The limitation of this study was its relatively low sample size.

“The limitation of this study was its relatively low sample size.”

May combine two claims.

The g loading of a subtest can be viewed as an index of its cognitive complexity; a subtest with a higher g-loading is more complex, which means it may be affected to a greater extent not only by biological factors, but also by cultural factors.

“At its core, the g loading of a subtest can be viewed as an index of its cognitive complexity (Flynn, 2013; see also Gottfredson, 2016): a subtest with a higher g-loading is more complex, which means it may be affected to a greater extent not only by biological factors, but also by cultural factors (see especially Dickens & Flynn, 2001;Kan et al., 2013).”

• cites Flynn, 2013 sourceNot checked — no full text of the source
• cites Gottfredson, 2016 sourceNot checked — no full text of the source
• cites Dickens & Flynn, 2001 sourceNot checked — no full text of the source

May combine two claims.

The increase in Symbol Search performance is incompatible with a general decline of intellectual ability, and this result was unstable, with Bayesian analyses failing to find a substantial difference.

“The third discrepancy was the finding of an increase in performance for the Symbol Search subtest, a measure of Gs; although processing speed may not be the best index of general intelligence, this is even more incompatible with a general decline of intellectual ability.”

• Also stated in: § 3.3 Discussion · plain

  “this result was also more unstable, with Bayesian analyses failing to find a substantial difference.”

May combine two claims.

The high average ability was accounted for in analyses of DIF by trimming the normative sample to match the composition of the current sample and by using item response theory, which estimates item parameters using subjects of all available ability levels.

“This point was also accounted for in analyses of DIF, where the normative sample was trimmed to match the composition of the current sample (see section 3.1.3),and where we used item response theory which estimates item parameters using subjects of all available ability levels.”

Classified as probably not a substantial claim.

Woodley of Menie and Dunkel reported r = .72 for the correlation between the amount of decline on a subtest and its biological load composite.

“the reported correlation was r = .72.”

• cites Woodley of Menie and Dunkel (2015) sourceNot checked — no full text of the source

May combine two claims.

The current results constitute a reminder that intelligence scores are not pure reflections of intelligence and have multiple determinants, some of which can be affected by cultural factors that do not reflect intelligence itself.

“Beyond the specific case of average intelligence in France, the current results constitute a reminder that intelligence scores are not pure reflections of intelligence and have multiple determinants, some of which can be affected by cultural factors that do not reflect intelligence itself.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It is impossible to adjudicate between the two possibilities of a restricted/compensated decline versus a real decline without having the 1999 sample perform the WAIS-IV.

“It is impossible to adjudicate between these two possibilities (which would require having the 1999 sample perform the WAIS-IV)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For the first analysis testing which subtests demonstrated a difference between WAIS-III and WAIS-IV, low sample size was not a major problem, given that Study 2 corroborated Study 1.

“For the first analysis testing which subtests demonstrated a difference between WAIS-III and WAIS-IV, this was not a major problem, given that Study 2 corroborated Study 1.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The conclusion is in line with a tradition of studies attributing fluctuations of intelligence scores to methodological biases, especially as they relate to cultural item content.

“This conclusion is in line with a tradition of studies attributing fluctuations of intelligence scores to methodological biases, especially as they relate to [cultural] item content”

• cites Beaujean & Osterlind, 2008 sourceNot checked — no full text of the source
• cites Beaujean & Sheng, 2010 sourceNot checked — no full text of the source
• cites Kaufman, 2010 sourceNot checked — no full text of the source

May combine two claims.

There is no indication that the Jensen effect is biological: if something correlates with g-loadings, it does not mean that this something is biological.

“In the case of the Jensen effect, there is no indication that it is biological: if something correlates with gloadings, it does not mean that this something is biological.”

May combine two claims.

There was no negative Flynn effect in France, in the sense of a general decrease of intelligence or a decrease in the ability to perform logical reasoning.

“The results of both Study 1 and Study 2 unambiguously indicated that there was no negative Flynn effect in France, in the sense of a general decrease of intelligence or a decrease in the ability to perform logical reasoning”

• Also stated in: § 5. General Discussion · plain

  “In short, the conclusion that can be drawn from a comparison of WAIS-III and WAIS-IV is that over the last two decades, there has been no decline of reasoning abilities in the French population”