Item bias and item response theory

“To establish this, we adopt the formal definition of unbiasedness as presented by Mellenbergh (1989).”

“This potential measurement bias refers to the possibility that individuals equal in their true levels of depression but from different groups, that is, males and females, do not have identical probabilities of observed scores (Mellenbergh, 1989).”

“The total score of the (sub)test is often used as an internal criterion to match students with the same ability (Mellenbergh, 1989; Camilli and Shepard, 1994).”

“Whenever measurement invariance is justifiable, it is preferable to analyze data in a combined PCA across all observation, because this naturally allows for mean comparisons of the factor scores (Beauducel & Hilger, 2018; Mellenbergh, 1989; Meredith, 1993; Nesselroade, 2007; Van De Schoot et al., 2015).”

“Mellenbergh (1989) introduced the principle of conditional independence to define item bias (or DIF).”

“If a measurement scale is to be used in different subpopulations, it is necessary to ensure that the observed variables or scales used are invariant across populations (Mellenbergh, 1989; Meredith, 1993).”

“participants with equal values on the latent trait, group membership still affects the response to a particular item (Mellenbergh, 1989).”

“Essentially, such variations in difficulty across persons can be considered to constitute differential item functioning (DIF; see, e.g. Mellenbergh, 1989).”

“Note that these are the usual definitions of measurement invariance and bias (Mellenbergh, 1989; Millsap & Everson, 1993).”

“2Intercept and slope DIF are also referred to as uniform and non-uniform DIF in IRT literature (Mellenbergh, 1989).”

“…initial examinations of relevant scales’ reliability and validity per country groups, as well as either establishing greater measurement invariance such as scalar invariance, or detecting potential measurement of non-invariance such as differential item functioning (DIF) (Mellenbergh, 1989).”

“This concept has been labeled ‘‘measurement invariance’’ (e.g., Meredith, 1993), ‘‘measurement equivalence’’ (e.g., Byrne, Shavelson, & Muthén, 1989), ‘‘lack of item bias’’ (e.g., Mellenbergh, 1989), and ‘‘absence of differential item functioning’’ (DIF; for example, Swaminathan & Rogers, 1990).”

“Measurement invariance with regard to sex implies that the distribution of the observed scores on a subtest (yi) given a fixed level of the latent factor (η), depends on the score on the latent factor η, and not sex, i.e., f [ yi|η,sex]= f [ yi|η] (Mellenbergh, 1989).”

“…properties make these measurement models extremely useful for a wide range of applications, e.g., for investigating differential item functioning (Mellenbergh 1989; Meredith 1993), for testing group differences (J¨oreskog 1971), for the construction of tests and questionnaires (Kline 1986), for…”

“Formally, MI holds when the conditional distribution of the observed items is the same across subgroups, such as gender and ethnicity (Mellenbergh, 1989; Meredith, 1993).”

“, observed item or test scores) are a function of only the latent ability variables and are conditionally independent from scores on an external variable (Mellenbergh, 1989; Meredith, 1993).”

“If a measurement scale is to be used in different subsets of a sampling universe, as in this case, the observed variables or scales must be invariant across the whole research population (Mellenbergh, 1989; Meredith, 1993) at one of three levels labeled by Meredith as “weak,” “strong,” or “strict.””

“DIF exists if individuals from two groups with equivalent levels of an underlying construct have different probabilities of selecting specific response categories for an item intended to measure the construct (Mellenbergh 1989).”

“(Van De Schoot, Schmidt, De Beuckelaer, Lek, & Zondervan-Zwijnenburg, 2015) rely on the assumption of factorial structural validity for valid between groups comparisons (Mellenbergh, 1989).”

“The measurement invariance tests whether the function of the distribution of the observed scores conditional on the underlying space is independent from the group variable (Mellenbergh, 1989; Meredith, 1993).”

“In more simple terms, this means that the instrument is measuring the same construct in boys and girls (Mellenbergh 1989; Meredith 1993).”

“This has been referred to as nonuniform DIF (Camilli & Shepard, 1994, p. 59; Mellenbergh, 1989) because the nature of the DIF varies over the range of y. DIF in a (DIF-a) indicates that the IRFs for the two groups cross at some value of y.”

“When studying difference across groups, an equally important concept is measurement invariance (MI) or equivalence (Mellenbergh, 1989; Meredith, 1993; Millsap, 2011; Kim et al., 2012), which entails that measurements in different groups are comparable.”

“A central feature of all current detection methods is the “matching principle”: systematic group differences in scores on a scale or item are considered evidence of measurement bias only if group differences in scores remain among individuals who are all matched on the construct or latent variable being measured by th…”

“Even though there were different DIF detection techniques proposed, it was not possible to clearly explain the reason for DIF in item content (Mellenbergh, 1989).”

“From this perspective, two items will be equivalent in the source language (L1) and in the final language (L2) when their item characteristics curves are equivalent (Mellenbergh, 1989).”

“Measurement bias (or item bias, or differential item functioning) is formally defined as a violation of measurement invariance (cf. Mellenbergh, 1989): f1.”

“Mellenbergh (1989) depicted non-invariance (he called it item bias) by some form of DAG and speaks of causal influences as well as conditional independencies between observed variables (items), latent variables (traits), and groups.”

“If this is not established, group means cannot be compared, as groups respond differently to the items (Mellenbergh, 1989; Meredith, 1993; Millsap & Meredith, 2007).”

“In more technical terms, this implies that the measurement model meets a conditional independence property with respect to a set of subpopulations within the parent population (e.g. countries, gender, time) (Horn & Mcardle, 1992; Mellenbergh, 1989; Meredith, 1993).”

“A general formulation of DIF in the context of measurement invariance (Mellenbergh, 1989; Meredith, 1964, 1993; Meredith & Teresi, 2006) is that the conditional distribution of observed score, X, given the latent trait (θ ) is independent of the group G, and can be expressed as: f (X |θ,G) = f (X…”

“The comparison of patient-reported outcome scores is based on a belief that the meaning of concepts and measurement scales remains stable in individuals’ minds over time and is similar between groups [1–4].”

“Measurement invariance means that the probability of selecting a given item response category is comparable across groups, given similar levels of the latent construct being measured [59].”

“…present in an item will be uniform if the localization parameters of the adjacent m – 1 categories are different across the groups, and nonuniform (Mellenbergh, 1989) when the discrimination parameters differ across the groups, being equal (symmetrical DIF) or different (asymmetrical DIF)…”

“Definition and Model Notations Formally, measurement invariance (Mellenbergh, 1989) holds when the conditional probability distribution of the observed item score variable, X, given the latent variable to be measured, ξ, does not depend on the group membership variable.”

“If it is violated, the parameter estimates of the LC model can be severely biased (Asparouhov & Muthén, 2014; Masyn, 2017; Mellenbergh, 1989).”

“Measurement Invariance (MI), defined as how well a scale can measure a latent construct equivalently for different background populations (Mellenbergh, 1989) is a necessary property for a scale to produce the same trait scores for individuals who have the same latent trait level but differ on…”

“To establish measurement invariance, we fitted several increasingly restrictive models derived from approaches described in previous studies (Dolan, 2000; Mellenbergh, 1989; Meredith, 1993; Millsap & Yun-Tein, 2004; Muthen & Asparouhov, 2002; Muthén & Muthén, 2005), cumulating in this highly constrained model.”

“This definition of measurement bias was first proposed by Mellenbergh (1989) and has explicit and testable consequences in the CFA model (Meredith, 1993).”

“MI thus implies that a measurement instrument measures the same underlying factors or latent traits across groups (Mellenbergh, 1989; Meredith, 1993).”

“…are estimated for each group; (c) items with binary, ordinal, or nominal response scales may be tested; (d) both uniform and nonuniform DIF (defined by Camilli & Shepard, 1994, and Mellenbergh, 1989) may be evaluated; and (e) effect sizes are in readily comprehensible item–parameter units.”

“There is, however, only limited evidence on whether symptoms and cognitive deficits bias ratings at an item level [28, 29], that is, little is known as to whether items function differently as indicators of SQOL in patients with different levels of symptoms and deficits.”

“However, previously conducted research, comparing different groups by gender, ethnicity, or culture, has not provided guarantees for this measurement invariance, which is a prior step to assessing whether the items measure this attribute equally, without bias [22,23], because if the measurement invariance between the…”

“For instance, continuing with the aforementioned notation from the repeated measures cross-classified model (2), full invariance across raters supports P(Yiotr = 1|R, θ) = P(Yiotr = 1|θ) (3) (Mellenbergh, 1989).”

“The definition, as introduced by Mellenbergh (1989) is very general as it defines measurement invariance as statistical independence (not just linear independence), and variables X, T , and V can be of any measurement level, continuous or discrete, observed or latent.”

“Mellenbergh [3] introduced the PCI and emphasized its generality: X, T and V may be measured on the nominal, ordinal, interval, or ratio level, they may be latent or manifest, and their interrelationships may be linear or nonlinear.”

“As a consequence, the item score of a given person may depend not only on the latent dimensions of interest but will also depend on individual characteristics (Mellenbergh, 1989; Meredith, 1993).”

“DIF is parallel to measurement invariance detection techniques used in multiple-group CFA (MGCFA) and has been detailed in other sources (see, for example, Mellenbergh, 1989; Meredith & Millsap, 1992; B. Muthén & Lehman, 1985; Stark, Chernyshenko, & Drasgow, 2006).”

“In particular, it is recognized that nonequivalence in latent trait (IRT) models should be defined as differences in the item characteristic curves between groups (see, e.g., Lord, 1980, and Mellenbergh, 1989, 1992).”

“Meredith attributes this definition to Mellenbergh (1989).”

“To be able to meaningfully compare male and female scores, the PCL-R should adhere to measurement invariance across gender (Mellenbergh, 1989; Meredith, 1993).”

“Mellenbergh [16] gave a formal definition of”

“Mellenbergh (1989) developed a mathematical expression of measurement invariance concepts using conditional probability: P Y ij ¼ yjηi;G ¼ P Y ij ¼ yjηi ; ð1Þ where ηi is the factor score for ith examinee, Yij is the ith examinee’s response score for jth item, andG is a group membership.”

“An item shows DIF if two participants from two different sub-groups who have equivalent levels of the underlying construct being measured also have different probabilities of endorsing each response category for that item (Mellenbergh, 1989).”

“Several authors (Candell & Drasgow, 1988; Lord, 1980; Mellenbergh, 1989; Van Der Flier et al., 1984) have, however, argued that item bias can be interpreted as an extra dimension and therefore as a violation of the assumption of unidimensionality.”

“Stated differently, the instru- ment should measure the same construct, i.e., latent variable of interest, in boys and girls (Mellenbergh, 1989; Meredith, 1993).”

“…𝑃(𝑦𝑖 𝑗 |η𝑖;𝝑) with parameters 𝝑. Formally, measurement invariance holds when the conditional distribution of the observed items is the same across subgroups (Mellenbergh, 1989; Meredith, 1993), such that for every subgroup 𝑘 (𝑘 = 1, 2, . . . , 𝐾), like gender and ethnicity, 𝑃(𝑦𝑖 𝑗…”