The Flynn effect: A meta-analysis.

May combine two claims.

The Tau, or estimated standard deviation of the true effects, was 0.25, resulting in a credibility interval of −0.26 to +0.72.

“The Tau, or estimated standard deviation of the true effects, was τ = 0.25, resulting in a credibility interval of −0.26 to +0.72.”

May combine two claims.

In the modern tests, the mixed effects meta-regression slope of effect size on ability level was b = −.01, 95% CI [−.016, −.004], z = −3.37, p < .0007, with Q = 11.38, accounting for 15.8% of the total variability.

“The significant mixed effects meta-regression slope of effect size on ability level was b = −.01, 95% CI [−.016, −.004), z = −3.37, p < .0007. The Q for the model in this analysis was 11.38, accounting for 15.8% of the total variability as estimated by the Unrestricted Likelihood method.”

May combine two claims.

The regression of effect size on age was nonsignificant, accounting for less than one percent of the variance in effect sizes.

“The regression of effect size on age was nonsignificant, accounting for less than one percent of the variance in effect sizes.”

May combine two claims.

Eliminating the three lowest ability effects results in a mean estimate of the remaining 53 effects (n = 3,951) of 0.293 points per year, 95% CI [0.23, 0.35], and the regression of effect on ability is no longer significant.

“Eliminating the three lowest ability effects results in a mean estimate of the remaining 53 effects (n = 3,951) of 0.293 points per year, 95% CI [0.23, 0.35], and the regression of effect on ability is no longer significant.”

May combine two claims.

The funnel plot showed no apparent publication bias, with no gap on the lower left or lower right side of the plot.

“There is no apparent publication bias, which would be represented by a gap on the lower left side of the plot. A similar absence of a gap is seen on the lower right side of the plot.”

May combine two claims.

The mean effect over 285 total studies (n = 14,031) in the random effects model was 0.231 IQ points per year, 95% CI [0.20, 0.26], z = 14.10, p < .0001, indicating that the Flynn effect is different from zero.

“The mean effect over 285 total studies (n = 14,031) in the random effects model was 0.231 IQ points per year, 95% CI [0.20, 0.26], z = 14.10, p < .0001, with a confidence interval and p-value indicating that the Flynn effect is different from zero1.”

• AssumptionRests on an assumption: The 27 multifactorial tests chosen represent the major IQ tests in use over the past few decades.

May combine two claims.

There was a significant between-test group effect, Q(5) = 231, p < .0001, with test group explaining 5.2% of the explainable variance in effects.

“There was a significant between-test group effect, Q(5) = 231, p < .0001, with test group explaining 5.2% of the explainable variance in effects.”

May combine two claims.

For screening tests, the mean effect (k = 17, n = 1,325) was 0.02 IQ points per year, 95% CI [−0.15, 0.19], z = 0.21, p < .84, not significantly different from zero.

“The mean effect (k = 17, n = 1,325) in the random effects model was 0.02 IQ points per year, 95% CI [−0.15, 0.19], z = 0.21, p < .84.”

May combine two claims.

For KABC tests, the mean effect (k = 34, n = 1,611) was 0.02 IQ points per year, 95% CI [−0.16, 0.19], z = 0.19, p = .85, not significantly different from zero.

“The mean effect (k = 34, n = 1,611) in the random effects model was 0.02 IQ points per year, 95% CI [−0.16, 0.19], z = 0.19, p = .85.”

May combine two claims.

In the overall model, the intercept for the regression of effects on ability level was significant (a = 0.38, z = 2.58, p < .01), but the slope was not (b = −.002, z = −1.08, p < .28), indicating that the effect did not change significantly over the range of ability levels represented.

“The intercept was significant (a = 0.38, z = 2.58, p < .01), but the slope was not (b = −.002, z = −1.08, p < .28), indicating that the effect did not change significantly over the range of ability levels represented in this set of effects.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Tests were frequently counterbalanced so that approximately half of the sample got each test first, but in a substantial number of studies one test was uniformly given first.

“Tests were frequently counterbalanced so that approximately half of the sample got each test first. However, in a substantial number of the studies, one test was uniformly given first.”

May combine two claims.

The mean effects for the counterbalanced group (k = 30, n = 2,912; M = 0.29, 95% CI [0.23, 0.36]) and the group where the old test was given second (k = 8, n = 505; M = 0.54, 95% CI [0.16, 0.91]) were significantly different from zero.

“The mean effects for the counterbalanced group (k = 30, n = 2,912) (M = 0.29, 95% CI [0.23, 0.36]) and the group of effects where the old test was given second (k = 8, n = 505) (M = 0.54, 95% CI [0.16, 0.91]) were significantly different from zero.”

May combine two claims.

The results show that the IQ increase is pervasive, not only with respect to geography and time, but also with respect to the tests used to measure IQ.

“The results show that the IQ increase is pervasive, not only with respect to geography and time, but also with respect to the tests used to measure IQ.”

May combine two claims.

The standard error of this estimate is 0.35 (resulting in a 95% CI that extends about .7, rounded to 1 point, on either side of 3 points per decade).

“and the standard error of this estimate is 0.35 (resulting in a 95% CI that extends about .7, rounded to 1 point, on either side of 3 points per decade).”

May combine two claims.

Examining just the modern tests administered in a counterbalanced order and excluding the three atypical studies showed that the estimates for pairings of Wechsler/Wechsler, Binet/Binet, and Wechsler/Binet tests (all about 0.29) were remarkably similar to the overall estimate of 0.293 per year.

“Examining just the modern tests administered in a counterbalanced order and excluding the three atypical studies showed that the estimates for pairings of Wechsler/Wechsler, Binet/Binet, and Wechsler/Binet tests (all about 0.29) were remarkably similar to the overall estimate of 0.293 per year.”

May combine two claims.

When only the modern Wechsler/Binet tests were considered in isolation, the magnitude of the effect appears to be close to 3 points per decade and showed no evidence of reducing in magnitude.

“when only the modern Wechsler/Binet tests were considered in isolation, the magnitude of the effect appears to be close to 3 points per decade and showed no evidence of reducing in magnitude.”

May combine two claims.

Using average correlations instead of sample-specific correlations changed mean effects by no more than 0.03 points per year, and all significance tests and heterogeneity tests resulted in the same conclusions.

“For the overall analyses and within the test groups, mean effects differed by no more than 0.03 points per year. All significance tests and tests of heterogeneity resulted in the same conclusions reached above.”

May combine two claims.

If the new test was given first, the Flynn effect estimate was 0.54, which was larger than the overall estimate.

“However, if the new test was given first, the estimate (0.54) was larger”

May combine two claims.

The Flynn effect is robust, manifested across various tests in nearly 300 studies.

“Our support for a robust Flynn effect, manifested across various tests in nearly 300 studies”

May combine two claims.

If the old test was given first, the Flynn effect estimate was 0.14, which was smaller than the overall estimate.

“If the old test was given first, the estimate (0.14) was smaller”

May combine two claims.

The present findings demonstrate the pervasiveness and stability of the Flynn effect across multiple tests and many decades.

“The present findings, which demonstrate the pervasiveness and stability of the Flynn effect across multiple tests and many decades”

May combine two claims.

Floor differences between the SB-4 and WAIS-R produce an apparent standard score difference of 9 points at the lowest possible scores, and as scores approach the mean of 100 the differences in scales and the effects become smaller.

“The lowest possible score on the SB-4 is 36, and the lowest possible score on the WAIS-R is 45. Individuals who obtain the lowest possible score on both tests will still have an apparent difference in their standard scores of 9 points. Consistent with the plot, as the scores get closer to the mean of 100, the differences in the scales become smaller, and the effects become smaller.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Flynn effect is less well known and often not taught in behavioral science training programs.

“The Flynn effect is less well known and often not taught in behavioral science training programs (Hagen, Drogin, & Guilmette, 2008).”

• cites Hagen, Drogin, & Guilmette, 2008Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Agreement between the first author and the volunteers on each variable ranged from 90.5-99.1% per block, with an average agreement of 95.8% per block.

“These estimates ranged from 90.5–99.1% per block, with an average agreement of 95.8% per block.”

May combine two claims.

The Flynn effect is robust across different age groups, measures, samples, and levels of performance.

“These results supported previous estimates of the Flynn effect and its robustness across different age groups, measures, samples, and levels of performance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There is considerable contention over factors that may cause the Flynn effect.

“Further, there is considerable contention over factors that may cause the Flynn effect (Flynn, 2007, 2012; Neisser, 1998).”

• cites Flynn, 2007, 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Neisser, 1998Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Calculations of IQ score gains between 1972 and 2006 for different normative versions of the Stanford-Binet, WAIS, and WISC showed an average increase of 0.31 IQ points per year, consistent with Flynn's earlier findings.

“More recently, the Flynn effect was supported by calculations of IQ score gains between 1972 and 2006 for different normative versions of the Stanford-Binet (SB), Wechsler Adult Intelligence Scale (WAIS), and Wechsler Intelligence Scale for Children (WISC) (Flynn, 2009a). The average increase in IQ scores per year was 0.31, which was consistent with Flynn’s (1984a) earlier findings.”

• cites Flynn, 2009aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Flynn, 1984aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent evidence suggests that middle and lower ability groups (IQ = 79-109) demonstrate the customary 0.31-0.37-point increase per year.

“Recent evidence has suggested that middle and lower ability groups (IQ = 79–109) demonstrate the customary 0.31–0.37-point increase per year, whereas higher ability groups (IQ = 110+) demonstrate a minimal increase of 0.06–0.15 points per year (Zhou, Zhu, Weiss, & Pearson, 2010).”

• cites Zhou, Zhu, Weiss, & Pearson, 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent evidence suggests that higher ability groups (IQ = 110+) demonstrate a minimal increase of 0.06-0.15 points per year.

“Recent evidence has suggested that middle and lower ability groups (IQ = 79–109) demonstrate the customary 0.31–0.37-point increase per year, whereas higher ability groups (IQ = 110+) demonstrate a minimal increase of 0.06–0.15 points per year (Zhou, Zhu, Weiss, & Pearson, 2010).”

• cites Zhou, Zhu, Weiss, & Pearson, 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors did not expect to find evidence for publication bias because the descriptive data are not typically subject to significance tests and many effects came from test manuals where no publication bias is expected.

“We did not expect to find evidence for publication bias in this meta-analysis. The descriptive data collected from each study in the form of sample sizes, means, and correlations between tests is not typically the type of data that is subject to tests of significance and thus would not be a direct cause of failure to publish due to non-significance. Additionally, many of the effects were gleaned”

• AssumptionThe authors state this as an assumption: The results of test validation studies provide important information irrespective of the findings and would therefore constitute publishable data.

May combine two claims.

The bimodal pattern indicates that some lower ability samples had higher than average Flynn effects whereas others had lower than average Flynn effects.

“This pattern indicates that some of the lower ability samples had higher than average Flynn effects whereas others had lower than average Flynn effects.”

May combine two claims.

Flynn's (1984a) study revealed a 13.8-point increase in IQ scores between 1932 and 1978, amounting to a 0.3-point increase per year, or approximately 3 points per decade.

“Flynn’s study revealed a 13.8-point increase in IQ scores between 1932 and 1978, amounting to a 0.3-point increase per year, or approximately 3 points per decade.”

• cites Flynn (1984a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Unawareness of the Flynn effect on the part of test examiners can compound this problem.

“Unawareness of the Flynn effect on the part of test examiners can compound this problem.”

May combine two claims.

Our finding that the Flynn effect has not diminished over time and may be larger for modern than older tests is not consistent with Sundet et al.'s (2008) hypothesis relating increasing IQ scores and decreasing family size.

“Our finding that the Flynn effect has not diminished over time and may be larger for modern than older tests is not consistent with Sundet et al.’s (2008) hypothesis relating increasing IQ scores and decreasing family size”

• cites Sundet et al.’s (2008)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The finding of no age difference is consistent with previous research (Flynn, 1984, 1987).

“This finding is consistent with previous research (Flynn, 1984, 1987).”

• cites Flynn, 1984, 1987Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

This child may be more likely to receive a diagnosis of a learning disability during this second assessment than a recommendation of special education services.

“This child may be more likely to receive a diagnosis of a learning disability during this second assessment than a recommendation of special education services.”

May combine two claims.

Given the variety of subtests underlying the IQ scores included in the meta-analyses, and the convergence of Flynn effect estimates around 0.29 for the modern tests, the order effect tends to be transitive with a mean magnitude of approximately ± .20.

“Given the variety of subtests underlying the IQ scores included in our meta-analyses, and the convergence of Flynn effect estimates around 0.29 for the modern tests, the order effect tends to be transitive with a mean magnitude of approximately ± .20.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The number of comparisons included in this review appears more than sufficient to assess the magnitude of the Flynn effect and the precision of the obtained value, and to address the additional research questions under consideration.

“However, the number of comparisons included in this review appears more than sufficient to assess the magnitude of the Flynn effect and the precision of the obtained value, and to address the additional research questions under consideration.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Several recent studies indicate that the Flynn effect is not limited to intelligence tests but may be measured in tests of memory and object naming, as well as certain commonly used neuropsychological tests.

“Several recent studies indicate that the Flynn effect is not limited to intelligence tests but may be measured in tests of memory (Baxendale, 2010; Rönnlund & Nilsson, 2008, 2009) and object naming (Connor, Spiro, Obler, & Martin, 2004), as well as certain commonly used neuropsychological tests (Dickinson & Hiscock, 2011).”

• cites Baxendale, 2010 sourceNot checked — no full text of the source
• cites Rönnlund & Nilsson, 2008, 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Connor, Spiro, Obler, & Martin, 2004 sourceNot checked — no full text of the source

May combine two claims.

Results revealed no difference in the Flynn effect based on participant age, suggesting that the Flynn effect is consistent across age cohorts.

“Results revealed no difference in the Flynn effect based on participant age, suggesting that the Flynn effect is consistent across age cohorts.”

May combine two claims.

The overall Flynn effect of 2.31 produced by this meta-analysis was lower than Flynn's (2009a) value of 3.11, Fletcher et al.'s (2010) value of 2.80, and Dickinson and Hiscock's (2010) estimate of 2.60.

“The overall Flynn effect of 2.31 produced by this meta-analysis was lower than Flynn’s (2009a) value of 3.11 and Fletcher et al.’s (2010) value of 2.80. It also fell below Dickinson and Hiscock’s (2010) estimate of 2.60, which was the average of separate calculations for each of the 11 Wechsler subtests.”

• cites Flynn’s (2009a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Fletcher et al.’s (2010) sourceNot checked — no full text of the source
• cites Dickinson and Hiscock’s (2010) sourceNot checked — no full text of the source

May combine two claims.

These findings suggest that the mean magnitude of the Flynn effect may not change significantly with level of ability and that the correction can be applied to scores across the spectrum of ability level.

“These findings suggest that the mean magnitude of the Flynn effect may not change significantly with level of ability and that the correction can be applied to scores across the spectrum of ability level.”

Classified as probably not a substantial claim.

We did not address the issue of differences in rates at which scores from various Wechsler subtests have risen over time in this metaanalysis.

“We did not address this issue in this metaanalysis”

Probably not a substantial claim: may not pass S3 (load-bearing).

The larger effect for modern than older tests could be regarded as consistent with Lynn's (2009) hypothesis pertaining to pre- and early postnatal nutrition.

“The larger effect for modern than older tests could be regarded as consistent with Lynn’s (2009) hypothesis pertaining to pre- and early postnatal nutrition.”

• cites Lynn’s (2009)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Our data do not address Kaufman's (2010) more specific concern about asymmetric order effects such that taking the newer test first increased subsequent performance on the older test more than taking the older test first increases subsequent performance on the newer test.

“Our data do not address Kaufman’s (2010) more specific concern about asymmetric order effects such that taking the newer test first increased subsequent performance on the older test more than taking the older test first increases subsequent performance on the newer test.”

• cites Kaufman (2010) sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The exact size of the Flynn effect may vary from one sample to another.

“Moreover, the exact size of the Flynn effect may vary from one sample to another.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

There may be significant order effects.

“there may be significant order effects”

May combine two claims.

The large number of studies included in our meta-analysis provides a strong empirical basis for concluding that comparable IQ increases are evident in samples ranging from preschool children to elderly adults.

“Additionally, the large number of studies included in our meta-analysis provides a strong empirical basis for concluding that comparable IQ increases are evident in samples ranging from preschool children to elderly adults.”

Classified as probably not a substantial claim.

The data available from tests other than IQ tests are not likely to be sufficient in quality or quantity to yield precise Flynn effect estimates.

“The data available from tests other than IQ tests are not likely to be sufficient in quality or quantity to yield precise Flynn effect estimates”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The usefulness of a meta-analysis depends to a great extent on the accessibility of studies meeting inclusion criteria.

“The usefulness of a meta-analysis depends to a great extent on the accessibility of studies meeting inclusion criteria.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The data collected from Scandinavian young men are more representative of the Scandinavian population than potential conscripts in the United States who self-select into the armed services are of the American population.

“the data collected from Scandinavian young men, most of whom are evaluated for suitability for the armed services, are more representative of the Scandinavian population than potential conscripts in the United States who self-select into the armed services are of the American population.”

Classified as probably not a substantial claim.

In capital punishment cases, life and death may reside on a 3-point difference of 76 versus 73, or 71 versus 68.

“However, in capital punishment cases, life and death may reside on a 3-point difference of 76 versus 73, or 71 versus 68.”

Probably not a substantial claim: may not pass S2 (checkable).

Our method has the advantage of being applicable to a very large number of informative samples.

“On the other hand, our method has the advantage of being applicable to a very large number of informative samples.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

For the study of a cohort phenomenon like the Flynn effect, the intergroup approach is appropriate.

“For the study of a cohort phenomenon like the Flynn effect, this approach is appropriate.”

• AssumptionRests on an assumption: The Flynn effect is a cohort phenomenon.

May combine two claims.

There was no dearth of effect sizes at the lower end of the distribution of effect sizes, which suggests there was no oversampling of studies producing higher Flynn effects.

“Further, there was no dearth of effect sizes at the lower end of the distribution of effect sizes (Figure 1), which suggests there was no oversampling of studies producing higher Flynn effects.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Our method is not as direct as Flynn's tracking of raw scores on Raven's Matrices, nor does it provide the detailed information that can be obtained by comparing old and new versions of the Wechsler and Stanford-Binet batteries in the same individuals.

“Our method is not as direct as Flynn’s tracking of raw scores on Raven’s Matrices, nor does it provide the detailed information that can be obtained by comparing old and new versions of the Wechsler and Stanford-Binet batteries in the same individuals.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Mean values can be greatly influenced by the number and magnitude of extreme values such that the resulting value may not be an adequate measure of central tendency nor an effective illustration of the relation between IQ score and the moderators assessed.

“For example, mean values can be greatly influenced by the number and magnitude of extreme values such that the resulting value may not be an adequate measure of central tendency nor an effective illustration of the relation between IQ score and the moderators assessed.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Precise estimates for IQ tests will provide a reliable standard against which data from other tests can be evaluated.

“but precise estimates for IQ tests will provide a reliable standard against which data from other tests can be evaluated.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A precise correction cannot be assured in all circumstances because the Flynn effect, as it applies to a given test, may strengthen or weaken at any time in the future.

“A precise correction, however, cannot be assured in all circumstances because the Flynn effect, as it applies to a given test, may strengthen or weaken at any time in the future.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Relying on one numerical value to represent a continuous variable, including IQ score and age, results in a significant loss of information.

“Relying on one numerical value to represent a continuous variable, including IQ score and age, results in a significant loss of information.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Many tests are never re-normed and therefore lie beyond the reach of this method.

“many tests are never re-normed and therefore lie beyond the reach of this method”

Probably not a substantial claim: may not pass S3 (load-bearing).

Although a thorough review was conducted on PsycINFO® and in test manuals, possibly there were studies meeting inclusion criteria that were not accessed.

“Although a thorough review was conducted on PsycINFO® and in test manuals, possibly there were studies meeting inclusion criteria that were not accessed.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

These findings are consistent with previous research.

“These findings are consistent with previous research”

Probably not a substantial claim: may not pass S3 (load-bearing).

In cases where an individual is assessed at two different sites, the IQ score produced by the second assessment may be particularly inflated due to both the Flynn effect and prior exposure.

“In this case, the IQ score produced by the second assessment may be particularly inflated due to both the Flynn effect and prior exposure.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Validation studies rely on group-level data and presuppose a representative normative basis for the derivation of a standardized IQ score.

“In addition, validation studies rely on group-level data and presuppose a representative normative basis for the derivation of a standardized IQ score.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

These findings are consistent with the argument that it is feasible and advisable to correct IQ scores for the Flynn effect in high stakes decisions.

“and with the argument that it is feasible and advisable to correct IQ scores for the Flynn effect in high stakes decisions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Direct within-examinee comparisons have not been made for many tests even if the tests have been re-normed.

“direct within-examinee comparisons have not been made for many tests even if the tests have been re-normed.”