Item bias and item response theory

“Measurement invariance (MI) as defined by Mellenbergh (1989) and Meredith (1993) implies that, given a certain score on the underlying factor, an observed score does not depend on subpopulation membership.”

“This approach is consistent with general definitions of measurement invariance that rely on conditional probability (Mellenbergh, 1989; Meredith, 1993; Meredith & Millsap, 1992).”

“In psychometrics, a significant array of theoretical models and associated techniques has been developed to get some grip on this problem (Mellenbergh, 1989; Meredith, 1993; Millsap & Everson, 1993).”

“In other words, the distribution of observed scores conditional on the factor scores is the same for all groups (Mellenbergh, 1989).”

“As proposed by Mellenbergh (1989), “measurement invariance” (MI) requires that the association between the items (or test scores) and the latent factors (or latent traits) of individuals should not depend on group membership or measurement occasion (i.e., time).”

“Statistically, this means that the mathematical function that relates latent variables to the observations must be the same in each of the groups involved in the comparison.(7,8) This idea has become known as the requirement of measurement invariance.”

“Swaminathan and Rogers combined the statistical test of uniform and nonuniform DIF detection into a single step.(2) Swaminathan and Rogers fit 3 logistic models to the data, and compared the difference in the 2 log likelihoods of the first and third models to a 2 From the Departments of *Internal Medicine, ‡Biostatist…”

“…differing intercept parameters (corresponding to the difficulty parameters in IRT; e.g., Lee et al., 2018) across groups, referred to as uniform DIF (e.g., Mellenbergh, 1989), Muth ´ en (1988, 1989) specified the MIMIC DIF model by further regressing the manifest indicators Y on the covariates X .”

“The measurement invariance can be formally defined as conditional independence of ob- served scores, X , given the underlying latent variable, W (Mellenbergh, 1989): f .”

“…method, unbalanced sample sizes Measurement invariance is defined as a condition in which the conditional probability of measured scores given one’s standing on the latent construct being measured is independent of one’s group membership (Mellenbergh, 1989; Millsap, 2011; Yoon & Millsap, 2007).”

“Measurement invariance (MI) implies that (for continuous observed variables), conditional on the latent trait scores, the covariances and the intercepts are equal across groups (cf. Mellenbergh, 1989).”

“This is interpreted to mean that the intelligence tests display uniform measurement bias (e.g., Mellenbergh, 1989 ) with respect to cohort.”

“For each studied item, an analysis begins with a general test that is sensitive to both uniform and nonuniform DIF (Camilli & Shepard, 1994, p. 59; Mellenbergh, 1989).”

“ORIGINAL RESEARCH ARTICLE published: 17 September 2014 doi: 10.3389/fpsyg.2014.00980 The consequences of ignoring measurement invariance for path coefficients in structural equation models”

“One’s position on the latent trait determines one’s probability of endorsing the items through the ICCs in the figure directly below the normal density: e.g. the more depressed, the higher the probability of endorsing the item ‘suicidal ideation’.”

“That is, violation of the PCI may not always imply change in 1 3 the meaning of one’s self-evaluation.”

“For each studied item, an analysis begins with a general test that is designed to identify both uniform and nonuniform DIF (Camilli & Shepard, 1994; Mellenbergh, 1989).”

“DIF may be uniform or nonuniform (Mellenbergh, 1989), where the group difference in endorsement probability (or item difficulty, if applicable) is constant over the latent continuum for uniform DIF but not for nonuniform DIF.”

“In comparing groups with respect to the latent trait, one needs to establish that the test used to measure the trait is not biased, i.e., that the instrument is ‘measurement invariant’ (MI) with respect to group”

“Mellenbergh (1989) defined measurement bias as a violation of measurement invariance.”

“, latent ability) of a person should not depend on group membership (Mellenbergh, 1989; Millsap & Everson, 1993).”

“We first look at the formal definition of measurement invariance (Mellenbergh, 1989), which is expressed in terms of the conditional distribution of manifest test scores Y [denoted byf(Y | )].”

“Mellenbergh [19] introduced the PCI to define item bias (or differential item functioning), with X representing an item response, A a latent trait, and V some group membership.”

“If measurement invariance does not hold it means that one or several items of the scale “functions” differently in the groups to be compared (resulting in the Rasch model in different item parameter, termed difficulty, in the two groups) and that group comparisons of the total scale score may be inaccurate; this pheno…”

“The definition of MI (or absence of bias) with respect to the selection variable V has been given by Mellenbergh (1989) as f ðY j h; sÞ ¼ f ðY j hÞ; ð7Þ where, as before, Y and h are observed scores and factor scores, respectively.”

“The psychometric issue of measurement invariance (Mellenbergh, 1989; Millsap & Everson, 1993) is crucial to the comparability of test scores across cultural groups in terms of latent variables, such as g.”

“The concept of measurement equivalence across a moderator (e.g. gender and age) is referred to as measurement invariance and can be examined with different statistical methods (Mellenbergh, 1989; Meredith, 1993; Millsap, 2012).”

“An item shows DIF if people from two or more distinct groups who have equivalent levels of the underlying construct have different probabilities of endorsing each response category for an item (Mellenbergh 1989).”

“for all X, W, V, where P( A|B) is the conditional probability function for A given B (Lord, 1980; Mellenbergh, 1989; Meredith & Millsap, 1992).”

“Full scalar equivalence is called strong measurement invariance in the psychometric literature (Meredith, 1993) and implies that the relationship between the observed score and the unobserved score on the latent factor of a person does not depend on group membership (Mellenbergh, 1989).”

“MI is said to exist if the distribution of possible item responses for an individual depends only on the person ’ s value for the latent variable and not also on other characteristics of the individual (Mellenbergh, 1989).”

“We also consider the psychometric issue of measurement invariance (Mellenbergh, 1989; Millsap & Everson, 1993), which is crucial (e.g., Byrne et al., 2009) to the comparability of test scores across cultural groups in terms of latent variables, such as general intelligence.”

“As mentioned, this finding suggests that the tests are unbiased with respect to group (Millsap, 1997a, 1997b; Mellenbergh, 1989; Meredith, 1993).”

“…observed outcomes given the common factor irrespective of the levels of a variable other than the common factor (Barendse, Oort, & Garst, 2010; Mellenbergh, 1989; Meredith, 1993; Millsap & Yun-Tein, 2004): P Y y G P Y yij i ij i( | , ) ( | ).= = =η η (1) Given the factor score (hi), the…”

“A measurement invariant test across different groups implies that group differences in the test scores can be attributed to group differences on the underlying variable (Mellenbergh, 1989): if women have a higher score on a measurement invariant depression test than men, the group difference in test score is attributa…”

“Mellenbergh (1989) used this notion to define measurement invariance more generally in statistical terms.”

“A necessary condition for this is that the questionnaire displays measurement invariance with respect to the groups under consideration (Mellenbergh 1989; Meredith 1993).”

“In the literature on structural equation modeling (SEM), this observation is known as the requirement of “measurement equivalence” or “invariance” (Meredith, 1993); psychometricians use the terms “item bias”, also known as “differential item functioning” or DIF (Mellenbergh, 1989); and in the measurement error literat…”

“…variable (Y) that reflect the underlying latent construct (Z) are independent of group membership (n), as expressed in the following equation (Mellenbergh, 1989): FðY|Z, nÞ=FðY|ZÞ for all Y ,Z, n: In other words, the scores on a behavior problem measure should reflect only the underlying…”

“A general formal definition of MI in the latent variable paradigm was given by Mellenbergh (1989) designed to measure a given latent variable Z , and suppose we have a variable X .”

“A disadvantage of MIMIC models is that they test for uniform, but not nonuniform1, DIF (see definitions in Mellenbergh 1989 and Camilli and Shepard 1994, p. 59).”

“Measurement invariance (MI; Meredith, 1993; Mellenbergh, 1989) refers to the notion that the item parameters from a measurement model are invariant across populations.”

“, subtest) of case i (yi) depends only on his or her value on the latent variable (gi) and not on sex: f [ yi|gi, sex]= f [ yi|gi] (Mellenbergh, 1989; Meredith, 1993).”

“Note that given this formal definition, we can distinguish two kinds of bias (Mellenbergh, 1989).”

“While acknowledging these limitations, it is important to note that the literature on measurement bias (Mellenbergh, 1989) suggests that while sample differences (or heterogeneity) might be a source of bias, this diversity is not typically considered a reason for ME, as found in the current study.”

“In modern surveys, we often use statistical models of the response process to represent the measurement, translating the old adage into a hypothesis that the process, or certain relevant model parameters that describe it, does not differ across groups (Mellenbergh 1989).”

“In its most broad term, measurement equivalence has been defined by Mellenbergh (1989) as:”

“First, it allows one to formulate and test measurement invariance in a rigorous manner (Lubke et al., 2003; Mellenbergh, 1989; Meredith, 1993; Widaman & Reise, 1997).”

“Under measurement invariance, the probabilities of attaining observed scores are conditional on the latent construct and are not affected by group membership (Mellenbergh, 1989).”

“This ssumption of equal measurement models is referred to as easurement invariance (Mellenbergh, 1989).”

“In psychometrics, an item is considered biased when respondents from different groups but with the same ability show different probabilities for answering an item correctly (Mellenbergh, 1989).”

“A set of psychometric scales X are defined to be measurement invariant with respect to an auxiliary variable V if (Mellenbergh 1989) f(xi|ti, vi, .”

“As an illustration, when groups differ in latent means but not in latent variances, one way to make a test unbiased in the predictive invariance sense is to add items that are uniformly biased (Mellenbergh, 1989) with respect to the lower scoring group (Millsap, 2007).”

“…these two kinds of effects are commonly referred to, respectively, as impact and differential item functioning (DIF; e.g., Holland & Wainer, 1993; Mellenbergh, 1989; Thissen, Steinberg, & Wainer, 1988, 1993), whereas within the FA tradition, the equivalence of item parameters is referred to as…”

“…for all indices used in this article, no significant indication of model misfit was found.12 11 Measurement invariance across countries implies that the measurement model meets a conditional independence property with respect to country membership; see e.g. Mellenbergh (1989) and Meredith (1993).”

“Absence of measurement bias with respect to a selection variable V has been de®ned by Mellenbergh (1989) in terms of the equality f (Y | h , v) = f (Y | h ), (1) where f ( ´ ) denotes the probability distribution function, h is a q-dimensional vector of scores on latent variables or factors…”

“In phenotypic factor modeling, this hypothesis can be formulated in terms of measurement invariance: all external variables that produce covariation in item responses exert their influence via the latent variable (Mellenbergh 1989; Meredith 1993).”

“For this purpose, we use the item response theory (IRT) approach in analyzing measurement equivalence, namely, the analysis of differential item functioning (DIF).”

“In what follows, we present the MG-EFA models, subject to target rotation, along with the requirements associated with measurement invariance, as defined by Mellenbergh (1989), and discussed in detail in the context of the common factor model by Meredith (1993).”

“Measurement invariance has gained attention in different social science disciplines since the breakthrough of Mellenbergh (1989) and Meredith (1993) and the systematic review on this topic by Vandenberg and Lance (2000) .”

“Measurement Invariance Across Groups Testing for measurement invariance is important to evaluate whether items measure the same attributes for different (groups of) respondents (Mellenbergh, 1989; Meredith, 1993).”

“Therefore, a significant effect of culture as a main effect or an interaction effect between culture and overall score implies that the item is biased (Mellenbergh, 1989; Van de Vijver & Leung, 1997).”

“…of MI at the level of individual items have received considerable attention in the literature, where a variety of methods for detecting differential item functioning (DIF; Mellenbergh, 1989) across observed groups has been proposed (Ackerman, 1992; Bock & Zimowski, 1997; Holland & Wainer, 1993).”