Using item response theory to assess the Flynn effect in the National Longitudinal Study of Youth 79 Children and Young Adults data

Studies using IRT have shown that item parameters do vary across cohorts, with drifts of item difficulty for some items and loss of discriminating power for others, especially for tests involving culture-based knowledge such as vocabulary and mathematics.

“Studies using this approach have shown that item parameters do vary across cohorts, with drifts of item difficulty for some items and loss of discriminating power for others, especially for tests involving culture-based knowledge such as vocabulary and mathematics (Beaujean & Osterlind, 2008;Pietschnig et al., 2013).”

Subtests primarily reflecting Gc intrinsically depend on culture-based declarative knowledge, and culture as assessed by these tests changes over time, making them especially likely to develop DIF.

“On the other hand, subtests primarily reflecting Gc intrinsically depend on culture-based declarative knowledge, and culture as assessed by these tests changes over time (e.g.Pietschnig et al., 2013), making them especially likely to develop DIF (see Beaujean & Osterlind, 2008;Pietschnig et al., 2013;Wicherts et al., 2004).”

Two studies found that observed intelligence scores changed over several decades, while latent ability as estimated from IRT remained near-constant.

“When considering total performance, two studies found that observed intelligence scores changed over several decades, while latent ability as estimated from IRT remained near-constant (Beaujean & Osterlind, 2008;Beaujean & Sheng, 2010), confirming both that item properties can change without corresponding changes of ability, and that this can create the illusion of ability changes over time.”

Performance can differ between groups of subjects without representing a true difference of ability.

“this is an illustration of the principle that performance can differ between groups of subjects without representing a true difference of ability (Beaujean & Osterlind, 2008;Beaujean & Sheng, 2010).”

The conclusion is in line with a tradition of studies attributing fluctuations of intelligence scores to methodological biases, especially as they relate to cultural item content.

“This conclusion is in line with a tradition of studies attributing fluctuations of intelligence scores to methodological biases, especially as they relate to [cultural] item content”