An Updated Guideline for Assessing Discriminant Validity
Open accessThe DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Discriminant validity was originally presented as a set of empirical criteria that can be assessed from multitrait-multimethod (MTMM) matrices. Because datasets used by applied researchers rarely lend themselves to MTMM analysis, the need to assess discriminant validity in empirical research has led to the introduction of numerous techniques, some of which have been introduced in an ad hoc manner and without rigorous methodological support. We review various definitions of and techniques for assessing discriminant validity and provide a generalized definition of discriminant validity based on the correlation between two measures after measurement error has been considered. We then review techniques that have been proposed for discriminant validity assessment, demonstrating some problems and equivalencies of these techniques that have gone unnoticed by prior research. After conducting Monte Carlo simulations that compare the techniques, we present techniques called CI CFA (sys) and [Formula: see text](sys) that applied researchers can use to assess discriminant validity.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“those simulations defined discriminant validity solely based on the correlation between the two constructs but, importantly, failed to consider the AVE of the two constructs in the population.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Rönkkö & Cho, 2022
In the citing paper
“While several simulation studies (e.g., Henseler et al., 2015;Rönkkö & Cho, 2022) have shown the high false-negative and high false-positive rates for this approach”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Rönkkö & Cho, 2022
In the citing paper
“For example, in their simulation study, Rönkkö and Cho (2022, Table 9) define a correlation at 0.8 or higher as having discriminant validity issues.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Rönkkö and Cho, 2022
In the citing paper
“For example, Rönkkö and Cho (2022, Table 8) defined the correlation between two constructs at 0.7, 0.8, and 0.9 as evidence of discriminant validity while using AVE at 0.64 and 0.42 for the population parameters.Hence, applying the AVE-SV approach, they rejected discriminant validity in the population but considered it false-positive in their simulation.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Rönkkö and Cho (2022)
In the citing paper
“Against this, we agree with Rönkkö and Cho (2022) and common practice that such adjustment is unnecessary because each discriminant validity test for a pair of constructs is independent of the discriminant validity tests applied to other pairs of constructs.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Rönkkö & Cho, 2022
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Evaluating Structural Equation Models with Unobservable Variables and Measurement Error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.2307/3151312 · Cited by 109,314 · cited 5×
“…and, consequently, is poorly understood.10 One source of confusion is the similarity between the formula for AVE and that of congeneric reliability (Fornell & Larcker, 1981a): AVE ¼ Xk i¼1l 2 iXk i¼1l 2 i þ Xk i¼1s 2 ei ð5Þ CR ¼ ð Xk i¼1liÞ 2 ð Xk i¼1liÞ 2 þ Xk i¼1s 2 ei ð6Þ The meaning of AVE…”
BackgroundMethodInfluentialEvaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941
Principles and Practice of Structural Equation Modeling
Tabri, N., & Elliott, C. M. (2012) · Canadian Graduate Journal of Sociology and Criminology · doi:10.15353/cgjsc-rcessc.v1i1.25 · Cited by 38,891
A new criterion for assessing discriminant validity in variance-based structural equation modeling
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-014-0403-8 · Cited by 38,459 · cited 27×
“…a comprehensive analysis of discriminant validity assessment that focuses on the typical case of single-method and one-time measurements, updating or challenging some of the recent recommendations on discriminant validity (Henseler et al., 2015; J. A. Shaffer et al., 2016; Voorhees et al., 2016).”
BackgroundMethodResultInfluentiallavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955 · cited 1×
“We estimated the factor models with the lavaan package (Rosseel, 2012) and used semTools to calculate the reliability indices (Jorgensen et al., 2020).”
MethodConvergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · cited 10×
“First, validity is a feature of a test or a measure or its interpretation (Campbell & Fiske, 1959), not of any particular statistical analysis.”
BackgroundMethodResultInfluentialEvaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280
Introductory Econometrics: A Modern Approach
Wooldridge, J. M. (1999) · Cited by 14,442
Structural equation models with unobservable variables and measurement error: Algebra and statistics
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800313 · Cited by 13,287 · cited 5×
“…and, consequently, is poorly understood.10 One source of confusion is the similarity between the formula for AVE and that of congeneric reliability (Fornell & Larcker, 1981a): AVE ¼ Xk i¼1l 2 iXk i¼1l 2 i þ Xk i¼1s 2 ei ð5Þ CR ¼ ð Xk i¼1liÞ 2 ð Xk i¼1liÞ 2 þ Xk i¼1s 2 ei ð6Þ The meaning of AVE…”
BackgroundMethodInfluentialEvaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
Assessing construct validity in organizational research
Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991) · Administrative Science Quarterly · doi:10.2307/2393203 · Cited by 6,621
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900
Validation Guidelines for IS Positivist Research
Straub, D., & Gefen, D. (2004) · Communications of the Association for Information Systems · doi:10.17705/1cais.01324 · Cited by 2,797 · cited 1×
“…measure of the correlation at best.16 Third, while the various guidelines differ in how loadings should be interpreted (Henseler et al., 2015; Straub et al., 2004; Thompson, 1997), they all share the features of relying mostly on authors’ intuition instead of theoretical reasoning or…”
BackgroundUses and abuses of coefficient alpha
Schmitt, N. (1996) · Psychological Assessment · doi:10.1037/1040-3590.8.4.350 · Cited by 2,588 · cited 1×
“Thus, while marketed as a new technique, the HTMT index has actually been used for decades; parallel reliability is the oldest reliability coefficient (Brown, 1910), and disattenuated correlations have been used to assess discriminant validity for decades (Schmitt, 1996).”
MethodThanks coefficient alpha, we’ll take it from here.
McNeish, D. (2018) · Psychological Methods · doi:10.1037/met0000144 · Cited by 2,533 · cited 2×
“Recent research suggests that the congeneric reliability coefficient is a safer choice because of the less stringent assumption (Cho, 2016; Cho & Kim, 2015; McNeish, 2017).”
BackgroundHenson, R. K., & Roberts, J. K. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282485 · Cited by 2,454 · cited 2×
“If the factors are rotated orthogonally (e.g., Varimax) or are otherwise constrained to be uncorrelated, the pattern coefficients and structure coefficients are identical (Henson & Roberts, 2006).”
BackgroundMethodA constant error in psychological ratings.
Thorndike, E. L. (1920) · Journal of Applied Psychology · doi:10.1037/h0071663 · Cited by 2,163 · cited 2×
“The term “discriminant validity” was coined by Campbell and Fiske (1959), who presented a validation technique based on the long-standing idea that tests can be invalidated by too high correlations with unrelated tests (Campbell, 1960; Thorndike, 1920).”
BackgroundHandbook of Research Methods in Social and Personality Psychology
(2024) · Cambridge University Press eBooks · doi:10.1017/9781009170123 · Cited by 2,121
Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies
Voorhees, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-015-0455-4 · Cited by 2,111 · cited 14×
“The second issue is that some articles suggest that the significance level of the w2 difference test should be adjusted for multiple comparisons (Anderson & Gerbing, 1988; Voorhees et al., 2016).”
BackgroundMethodInfluentialRepresenting and Testing Organizational Theories: A Holistic Construal
Bagozzi, R. P., & Phillips, L. W. (1982) · Administrative Science Quarterly · doi:10.2307/2392322 · Cited by 2,064 · cited 1×
“Moreover, there is no consensus on what kind of attribute discriminant validity is; some studies (Bagozzi & Phillips, 1982; Hamann et al., 2013; Reichardt & Coleman, 1995; J. A. Shaffer et al., 2016) define discriminant validity as a matter of degree, while others (Schmitt & Stults, 1986; Werts &…”
BackgroundShaffer, J. P. (1995) · Annual Review of Psychology · doi:10.1146/annurev.ps.46.020195.003021 · Cited by 1,960 · cited 1×
“Šidák and the related Bonferroni corrections make the universal null hypothesis that all individual null hypotheses are true (Hancock & Klockars, 1996; Perneger, 1998; J. P. Shaffer, 1995).”
BackgroundMarsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032813-153700 · Cited by 1,801 · cited 10×
“A plausible expectation is that studies that do not use SEMs report scale score correlations and that in studies that use SEMs, the presented correlations are factor correlations.”
BackgroundMethodInfluentialDiscriminant validity of well-being measures.
Lucas, R. E., Diener, E., & Suh, E. M. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.71.3.616 · Cited by 1,747 · cited 1×
“Lucas et al. (1996) acknowledged discriminant validity between self-esteem and optimism based on rDTR of .83 (rSS ¼ :72), but Credé et al. (2017) criticized the conceptual redundancy between grit and conscientiousness based on a disattenuated correlation of .84 (rSS ¼ :66).”
BackgroundBorsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,674 · cited 1×
“This assumption was also present in the original article by Campbell and Fiske (1959) that assumed a construct to be a source of variation in the items thus closely corresponding to the definition of validity by Borsboom et al. (2004).”
ResultPower and sensitivity of alternative fit indices in tests of measurement invariance.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.3.568 · Cited by 1,668 · cited 4×
“The idea behind using DCFI in measurement invariance assessment is that the degrees of freedom of the invariance hypothesis depend on the model complexity, and the CFI index and consequently DCFI are less affected by this than the w(2) (Meade et al., 2008).”
MethodInfluentialMuch ado about grit: A meta-analytic synthesis of the grit literature
Credé, M., Tynan, M. C., & Harms, P. D. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000102 · Cited by 1,653 · cited 1×
“Lucas et al. (1996) acknowledged discriminant validity between self-esteem and optimism based on rDTR of .83 (rSS ¼ :72), but Credé et al. (2017) criticized the conceptual redundancy between grit and conscientiousness based on a disattenuated correlation of .84 (rSS ¼ :66).”
BackgroundCommunications of the Association for Information Systems
(2017) · Communications of the Association for Information Systems · doi:10.17705/1cais · Cited by 1,603
Evaluating Bifactor Models: Calculating and Interpreting Statistical Indices
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016) · Psychological Methods · doi:10.1037/met0000045 · Cited by 1,599 · cited 7×
“…rather supports any statistical technique including more complex factor structures (Asparouhov et al., 2015; Marsh et al., 2014; Morin et al., 2017; Rodriguez et al., 2016) and nonlinear models (Foster et al., 2017; Reise & Revicki, 2014) as long as these techniques can estimate correlations that…”
BackgroundMethodInfluentialZinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005) · Psychometrika · doi:10.1007/s11336-003-0974-7 · Cited by 1,439 · cited 1×
“This more general formulation seems to open the option of using hierarchical omega (Cho, 2016; Zinbarg et al., 2005), which assumes that the scale measures one main construct (main factor) but may also contain a number of minor factors that are assumed to be uncorrelated with the main factor.”
BackgroundGuidelines for psychological practice with transgender and gender nonconforming people.
American Psychological Association (2015) · American Psychologist · doi:10.1037/a0039906 · Cited by 1,361
Thompson, B., & Daniel, L. G. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056002001 · Cited by 918 · cited 1×
“Pattern coefficients, on the other hand, are analogous to (standardized) coefficients in regression analysis and are directional (Thompson & Daniel, 1996).”
MethodInsufficient discriminant validity: a comment on Bove, Pervan, Beatty, and Shiu (2009)
Farrell, A. M. (2010) · Journal of Business Research · doi:10.1016/j.jbusres.2009.05.003 · Cited by 914 · cited 1×
“Other misuses are that only one of the two AVE values or the average of the two AVE values should be greater than the SV (Farrell, 2010; Henseler et al., 2015).”
BackgroundOrganizational behavior: the state of the science
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4589 · Cited by 807
Construct Validity: Advances in Theory and Methodology
Strauss, M. E., & Smith, G. T. (2009) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.032408.153639 · Cited by 803
Evaluating Structural Equation Models with Unobservable Variables and Measurement Error: A Comment
Bagozzi, R. P. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800312 · Cited by 778 · cited 1×
“Notably, Bagozzi (1981) wrote a critical commentary, to which Fornell and Larcker (1981b) published a rejoinder, but neither of these articles addressed the issues that we raise in this article.”
BackgroundCronbach’s Coefficient Alpha: Well Known but Poorly Understood
Cho, E., & Kim, S. (2015) · Organizational Research Methods · doi:10.1177/1094428114555994 · Cited by 764
Self-citationPodsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016) · Organizational Research Methods · doi:10.1177/1094428115624965 · Cited by 712 · cited 1×
“We suggest starting by following the guidelines by J. A. Shaffer et al. (2016) and Podsakoff et al. (2016) for assessing the conceptual distinctiveness of the constructs (see also M. S. Krause, 2012).”
MethodRecommendations for APA test standards regarding construct, trait, or discriminant validity.
Campbell, D. T. (1960) · American Psychologist · doi:10.1037/h0048255 · Cited by 560 · cited 1×
“The term “discriminant validity” was coined by Campbell and Fiske (1959), who presented a validation technique based on the long-standing idea that tests can be invalidated by too high correlations with unrelated tests (Campbell, 1960; Thorndike, 1920).”
BackgroundMathieu, J. E., & Farr, J. L. (1991) · Journal of Applied Psychology · doi:10.1037//0021-9010.76.1.127 · Cited by 560 · cited 1×
“For example, Le et al. (2010) diagnosed a discriminant validity problem between job satisfaction and organizational commitment based on a correlation of .91, and Mathieu and Farr (1991) declared no problem of discriminant validity between the same variables on the basis of a correlation of .78.”
BackgroundSOME EXPERIMENTAL RESULTS IN THE CORRELATION OF MENTAL ABILITIES 1
BROWN, W. (1910) · British Journal of Psychology, 1904-1920 · doi:10.1111/j.2044-8295.1910.tb00207.x · Cited by 559
Factor Analysis and Related Methods
McDonald, R. P. (2014) · Psychology Press eBooks · doi:10.4324/9781315802510 · Cited by 533 · cited 3×
“Discriminant validity is sometimes presented as the property of a construct (Reichardt & Coleman, 1995) and other times as the property of its measures or empirical representations constructed from those measures (McDonald, 1985).”
BackgroundResearch Methods for Organizational Studies
Schwab, D. P. (2013) · Psychology Press eBooks · doi:10.4324/9781410611284 · Cited by 455
Measurement error masks bipolarity in affect ratings.
GREEN, D. P., Goldman, S. L., & Salovey, P. (1993) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.64.6.1029 · Cited by 437 · cited 1×
“…be out of the range of one of the thermometers that would consequently display zero centigrade.18 In the social sciences, a well-known example is the measurement of happiness and sadness, two constructs that can be thought of as opposite poles of mood (D. P. Green et al., 1993; Tay & Jebb, 2018).”
BackgroundA Critical Examination of Common Beliefs About Partial Least Squares Path Modeling
Rönkkö, M., & Evermann, J. (2013) · Organizational Research Methods · doi:10.1177/1094428112474693 · Cited by 434 · cited 1×
“…original criterion is that both AVE values must be greater than the SV. Finally, the AVE statistic is sometimes calculated from a partial least squares analysis (AVEPLS), which overestimates indicator reliabilities and thus cannot detect even the most serious problems (Rönkkö & Evermann, 2013).”
MethodSelf-citationShaffer, J. A., DeGeest, D., & Li, A. (2016) · Organizational Research Methods · doi:10.1177/1094428115598239 · Cited by 393 · cited 9×
“…a comprehensive analysis of discriminant validity assessment that focuses on the typical case of single-method and one-time measurements, updating or challenging some of the recent recommendations on discriminant validity (Henseler et al., 2015; J. A. Shaffer et al., 2016; Voorhees et al., 2016).”
BackgroundMethodInfluentialMaking Reliability Reliable: A Systematic Approach to Reliability Coefficients
Cho, E. (2016) · Organizational Research Methods · doi:10.1177/1094428116656239 · Cited by 378
Self-citationSchmidt, F. L., Le, H., & Ilies, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.206 · Cited by 362
Factor Analysis at 100:Historical Developments and Future Directions
(2003) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3803_06 · Cited by 356 · cited 1×
“Moreover, a linear model where factors, error terms, and observed variables are all continuous (Bartholomew, 2007) is not always realistic.”
BackgroundAsparouhov, T., Muthén, B., & Morin, A. J. S. (2015) · Journal of Management · doi:10.1177/0149206315591075 · Cited by 340 · cited 3×
“Second, a factor model where each item loads on only one factor may be too constraining for applied research (Asparouhov et al., 2015; Marsh et al., 2014; Morin et al., 2017; Rodriguez et al., 2016).”
BackgroundMethodMethodology Review: Analysis of Multitrait-Multimethod Matrices
Schmitt, N., & Stults, D. M. (1986) · Applied Psychological Measurement · doi:10.1177/014662168601000101 · Cited by 311 · cited 2×
“First, it clearly states that discriminant validity is a feature of measures and not constructs and that it is not tied to any particular statistical test or cutoff (Schmitt, 1978; Schmitt & Stults, 1986).”
BackgroundHu, J., & Liden, R. C. (2015) · Academy of Management Journal · doi:10.5465/amj.2012.1142 · Cited by 284 · cited 1×
“For this group of researchers, the term referred to “whether the two variables . . . are distinct from each other” (Hu & Liden, 2015, p. 1110).”
BackgroundLe, H., Schmidt, F. L., Harter, J. K., & Lauver, K. J. (2010) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2010.02.003 · Cited by 272 · cited 3×
“To address the issue that the w(2)ð1Þ test can flag correlations that differ from 1 by trivial amounts as significant, some recent articles (Le et al., 2010; J. A. Shaffer et al., 2016) have suggested comparing models by calculating the difference between the comparative fit indices (CFIs) of two models (DCFI), which…”
BackgroundThe Correction for Attenuation
Muchinsky, P. M. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056001004 · Cited by 255
Morin, A. J. S., Boudrias, J.-S., Marsh, H. W., McInerney, D. M., Dagenais-Desmarais, V., Madore, I., & Litalien, D. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-016-9448-7 · Cited by 226
The Routledge companion to philosophy of science
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0251 · Cited by 225
New Standards For Test Evaluation
Guilford, J. P. (1946) · Educational and Psychological Measurement · doi:10.1177/001316444600600401 · Cited by 215 · cited 2×
“Importantly, factorial validity is an attribute of “a test” (Guilford, 1946), whereas only pairs of measures can exhibit discriminant validity.”
BackgroundHandbook of Item Response Theory Modeling: Applications to Typical Performance Assessment
(2014) · Routledge eBooks · doi:10.4324/9781315736013 · Cited by 198
McKenny, A. F., Short, J. C., & Payne, G. T. (2013) · Organizational Research Methods · doi:10.1177/1094428112459910 · Cited by 193
Bobko, P., Roth, P. L., & Buster, M. A. (2007) · Organizational Research Methods · doi:10.1177/1094428106294734 · Cited by 191 · cited 1×
“In this approach, the observed variables are first standardized before taking a sum or a mean; alternatively, a weighted sum or mean with 1=sxi is taken as the weights (i.e., X ¼ P i Xi=sxi ) (Bobko et al., 2007).”
BackgroundPath analysis: Psychological examples.
Werts, C. E., & Linn, R. L. (1970) · Psychological Bulletin · doi:10.1037/h0029778 · Cited by 187 · cited 1×
“…validity is; some studies (Bagozzi & Phillips, 1982; Hamann et al., 2013; Reichardt & Coleman, 1995; J. A. Shaffer et al., 2016) define discriminant validity as a matter of degree, while others (Schmitt & Stults, 1986; Werts & Linn, 1970) define discriminant validity as a dichotomous attribute.”
BackgroundExploring the Dimensions of Organizational Performance: A Construct Validity Study
Hamann, P. M., Schiemann, F., Bellora, L., & Guenther, T. W. (2013) · Organizational Research Methods · doi:10.1177/1094428112470007 · Cited by 181 · cited 1×
“Moreover, there is no consensus on what kind of attribute discriminant validity is; some studies (Bagozzi & Phillips, 1982; Hamann et al., 2013; Reichardt & Coleman, 1995; J. A. Shaffer et al., 2016) define discriminant validity as a matter of degree, while others (Schmitt & Stults, 1986; Werts &…”
BackgroundThompson, B. (1997) · Educational and Psychological Measurement · doi:10.1177/0013164497057001001 · Cited by 179 · cited 1×
“…of the correlation at best.16 Third, while the various guidelines differ in how loadings should be interpreted (Henseler et al., 2015; Straub et al., 2004; Thompson, 1997), they all share the features of relying mostly on authors’ intuition instead of theoretical reasoning or empirical evidence.”
BackgroundTesting parameters in structural equation modeling: Every "one" matters.
Gonzalez, R., & Griffin, D. (2001) · Psychological Methods · doi:10.1037//1082-989x.6.3.258 · Cited by 160 · cited 1×
“Both techniques produce the same estimate, although the standard errors (and CIs) can be different (Gonzalez & Griffin, 2001).”
BackgroundAn empirical application of confirmatory factor analysis to the multitrait-multimethod matrix
Kenny, D. A. (1976) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(76)90055-x · Cited by 159
Charles, E. P. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.2.206 · Cited by 154 · cited 3×
“First, CFA correlations estimated with maximum likelihood (ML) can be expected to be more efficient than multistep techniques that rely on corrections for attenuation (Charles, 2005; Muchinsky, 1996).”
BackgroundMethodThe Quest for α: Developments in Multiple Comparison Procedures in the Quarter Century Since
Hancock, G. R., & Klockars, A. J. (1996) · Review of Educational Research · doi:10.3102/00346543066003269 · Cited by 143 · cited 1×
“Šidák and the related Bonferroni corrections make the universal null hypothesis that all individual null hypotheses are true (Hancock & Klockars, 1996; Perneger, 1998; J. P. Shaffer, 1995).”
BackgroundGetting Through the Gate: Statistical and Methodological Issues Raised in the Reviewing Process
Green, J. P., Tonidandel, S., & Cortina, J. M. (2016) · Organizational Research Methods · doi:10.1177/1094428116631417 · Cited by 141
Le, H., Schmidt, F. L., & Putka, D. J. (2007) · Organizational Research Methods · doi:10.1177/1094428107302900 · Cited by 111 · cited 7×
“However, several articles point out that this may be a simplistic view of measurement error that only considers random and item-specific factor errors, ignoring time-specific transient errors (Le et al., 2009; Woehr et al., 2012).”
BackgroundMethodInfluentialPower to the Principals! An Experimental Look at Shareholder Say-On-Pay Voting
Krause, R. A., Whitler, K. A., & Semadeni, M. (2013) · Academy of Management Journal · doi:10.5465/amj.2012.0035 · Cited by 97 · cited 1×
“For these researchers, discriminant validity means that “two measures are tapping separate constructs” (R. Krause et al., 2014, p. 102) or that the measured “scores are not (or only weakly) associated with potential confounding factors” (De Vries et al., 2014, p. 1343).”
BackgroundNimon, K., Zientek, L. R., & Henson, R. K. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00102 · Cited by 93 · cited 2×
“A common criticism is that the correction can produce inadmissible correlations (i.e., greater than 1 or less than –1) (Charles, 2005; Nimon et al., 2012), but this issue is by no means a unique problem because the same can occur with a CFA.”
BackgroundAntecedents of Individuals' Interteam Coordination: Broad Functional Experiences as a Mixed Blessing
Vries, T. A. D., Walter, F. H., van der Vegt, G. S., & Essens, P. J. M. D. (2013) · Academy of Management Journal · doi:10.5465/amj.2012.0360 · Cited by 84 · cited 1×
“For these researchers, discriminant validity means that “two measures are tapping separate constructs” (R. Krause et al., 2014, p. 102) or that the measured “scores are not (or only weakly) associated with potential confounding factors” (De Vries et al., 2014, p. 1343).”
BackgroundSmall Sample Solutions for Structural Equation Modeling
Rosseel, Y. (2020) · Routledge eBooks · doi:10.4324/9780429273872-19 · Cited by 83
Path Analysis of Multitrait-Multimethod Matrices
Schmitt, N. (1978) · Applied Psychological Measurement · doi:10.1177/014662167800200201 · Cited by 75 · cited 1×
“First, it clearly states that discriminant validity is a feature of measures and not constructs and that it is not tied to any particular statistical test or cutoff (Schmitt, 1978; Schmitt & Stults, 1986).”
BackgroundMaraun, M. D., & Gabriel, S. M. (2011) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2011.02.006 · Cited by 67 · cited 1×
“Many articles in this category are ambiguous on whether discriminant validity is a property of a construct or a property of a scale from which construct correlation is estimated. whereas a common factor is part of a statistical model estimated from data (Maraun & Gabriel, 2013).”
BackgroundEstablishing Construct Continua in Construct Validation: The Process of Continuum Specification
Tay, L., & Jebb, A. T. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918775707 · Cited by 59
Methods for Estimating Item-Score Reliability
Zijlmans, E. A. O., van der Ark, L. A., Tijmstra, J., & Sijtsma, K. (2018) · Applied Psychological Measurement · doi:10.1177/0146621618758290 · Cited by 56 · cited 1×
“We use the term “single-admission reliability” (Cho, 2016; Zijlmans et al., 2018) instead of the more commonly used “internal consistency reliability” because the former is more descriptive and less likely to be misunderstood than the latter (Cho & Kim, 2015).”
MethodAn Examination of G-Theory Methods for Modeling Multitrait–Multimethod Data
Woehr, D. J., Putka, D. J., & Bowler, M. C. (2011) · Organizational Research Methods · doi:10.1177/1094428111408616 · Cited by 54 · cited 3×
“However, several articles point out that this may be a simplistic view of measurement error that only considers random and item-specific factor errors, ignoring time-specific transient errors (Le et al., 2009; Woehr et al., 2012).”
BackgroundMethodMeasurement Error Models With Uncertainty About the Error Variance
Oberski, D. L., & Satorra, A. (2013) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2013.797820 · Cited by 51 · cited 1×
“Third, calculating the CIs for a disattenuated correlation is complicated (Oberski & Satorra, 2013).”
BackgroundReview of Item Response Theory Practices in Organizational Research
Foster, G. C., Min, H., & Zickar, M. J. (2017) · Organizational Research Methods · doi:10.1177/1094428116689708 · Cited by 51 · cited 7×
“…more complex factor structures (Asparouhov et al., 2015; Marsh et al., 2014; Morin et al., 2017; Rodriguez et al., 2016) and nonlinear models (Foster et al., 2017; Reise & Revicki, 2014) as long as these techniques can estimate correlations that are properly corrected for measurement error…”
BackgroundMethodInfluentialThe Criteria for Convergent and Discriminant Validity in a Multitrait-Multimethod Matrix
Reichardt, C. S., & Coleman, S. C. (1995) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3004_3 · Cited by 45 · cited 5×
“However, the term was introduced without a clear definition of the concept (Reichardt & Coleman, 1995); instead, the article focuses on discriminant validation or how discriminant validity can be shown empirically using multitrait-multimethod (MTMM) matrices.”
BackgroundInfluentialZimmerman, D. W. (2007) · Educational and Psychological Measurement · doi:10.1177/0013164406299132 · Cited by 41 · cited 1×
“While the basic disattenuation formula has been extended to cases where its assumptions are violated in known ways (Wetcher-Hendricks, 2006; Zimmerman, 2007), the complexities of modeling the same set of violations in both the reliability estimates and the disattenuation equation do not seem…”
BackgroundUsing Planned Comparisons in Management Research: A Case for the Bonferroni Procedure
Castaiieda, M. B., Levin, J. R., & Dunham, R. B. (1993) · Journal of Management · doi:10.1177/014920639301900311 · Cited by 40 · cited 2×
“While multiple comparison techniques can be useful (Castañeda et al., 1993), there are scenarios where they should not be applied (Perneger, 1998), and the literature on discriminant validity has failed to provide compelling reasons to do so.”
BackgroundCorrelation Attenuation Due to Measurement Error: A New Approach Using the Bootstrap Procedure
Padilla, M. A., & Veprinsky, A. (2012) · Educational and Psychological Measurement · doi:10.1177/0013164412443963 · Cited by 38 · cited 1×
“However, this final concern can be alleviated to some extent through the use of bootstrap CIs (Henseler et al., 2015); in particular, the bias-corrected and accelerated (BCa) technique has been shown to work well for this particular problem (Padilla & Veprinsky, 2012, 2014).”
MethodMeasurement Validity is Fundamentally a Matter of Definition, Not Correlation
Krause, M. S. (2012) · Review of General Psychology · doi:10.1037/a0027701 · Cited by 37 · cited 1×
“We suggest starting by following the guidelines by J. A. Shaffer et al. (2016) and Podsakoff et al. (2016) for assessing the conceptual distinctiveness of the constructs (see also M. S. Krause, 2012).”
MethodA cautionary note on the finite sample behavior of maximal reliability.
Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2018) · Psychological Methods · doi:10.1037/met0000176 · Cited by 33 · cited 1×
“Using factor scores in this context is not a good idea because the reliability will be positively biased (Aguirre-Urreta et al., 2019), and, consequently, the correlation will be undercorrected.”
BackgroundSelf-citationSurvey Design and Measure Development
Spector, P. E. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199934874.013.0009 · Cited by 24 · cited 1×
“The desirable pattern of correlations in a factorial validity assessment is similar to the pattern in discri- minant validity assessment in an MTMM study (Spector, 2013), so in practice the difference between discriminant validity and factorial validity is not as clear-cut.”
ResultToward Application and Testing of Measurement Scales and an Example
Kuppelwieser, V. G., Putinas, A.-C., & Bastounis, M. (2017) · Sociological Methods & Research · doi:10.1177/0049124117701486 · Cited by 22 · cited 1×
“The heterotrait-monotrait (HTMT) ratio was recently introduced in marketing (Henseler et al., 2015) and is being adopted in other disciplines as well (Kuppelwieser et al., 2019).”
Using planned comparisons in management research: A case for the bonferroni procedure
Castañeda, M. (1993) · Journal of Management · doi:10.1016/0149-2063(93)90012-c · Cited by 22
Hancock, G. R., & Klockars, A. J. (1996) · Review of Educational Research · doi:10.2307/1170524 · Cited by 21
Adjustments to the correction for attenuation.
Wetcher-Hendricks, D. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.207 · Cited by 15 · cited 1×
“While the basic disattenuation formula has been extended to cases where its assumptions are violated in known ways (Wetcher-Hendricks, 2006; Zimmerman, 2007), the complexities of modeling the same set of violations in both the reliability estimates and the disattenuation equation do not seem…”
BackgroundBootstrapped Deattenuated Correlation: Nonnormal Distributions
Padilla, M. A., & Veprinsky, A. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164414531780 · Cited by 11
Recent developments in maximum likelihood estimation of MTMM models for categorical data
Jeon, M., & Rijmen, F. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00269 · Cited by 11 · cited 1×
“Indeed, a number of item response theory models have been introduced (Foster et al., 2017; Reise & Revicki, 2014) to address these scenarios and have been applied to assess discriminant validity using MTMM data (Jeon & Rijmen, 2014).”
Method
Research works printed but not linked: 7
- 10.2307/3150979 · doi:10.2307/3150979 · Paper page
- 10.2307/3150980 · doi:10.2307/3150980 · Paper page
- 10.1037/1082-989X.6.3.258 · doi:10.1037/1082-989x.6.3.258 · Paper page
- 10.1037/0022-3514.64.6.1029 · doi:10.1037/0022-3514.64.6.1029 · Paper page
- 10.1037/0022-3514.71.3.616 · DOI · Paper page
- 10.1037/0021-9010.76.1.127 · doi:10.1037/0021-9010.76.1.127 · Paper page
- Nunnally J. C., 1994, Psychometric theory
Cited by
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“Against this, we agree with Rönkkö and Cho (2022) and common practice that such adjustment is unnecessary because each discriminant validity test for a pair of constructs is independent of the discriminant validity tests applied to other pairs of constructs.”
InfluentialA perspective on using partial least squares structural equation modelling in data articles
Ringle, C. M., Sarstedt, M., Sinkovics, N., & Sinkovics, R. R. (2023) · Data in Brief · doi:10.1016/j.dib.2023.109074 · Cited by 767
semTools: Useful Tools for Structural Equation Modeling
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2012) · CRAN: Contributed Packages · doi:10.32614/cran.package.semtools · Cited by 603
HTMT2–an improved criterion for assessing discriminant validity in structural equation modeling
Roemer, E., Schuberth, F., & Henseler, J. (2021) · Industrial Management & Data Systems · doi:10.1108/imds-02-2021-0082 · Cited by 630
“Simulation results IMDS 121,12 2646 In line with prior research on the traditional HTMT, we recommend using statistical inference for theHTMT2 to detect discriminant validity problems (Franke and Sarstedt, 2019; R€onkk€o and Cho, 2020).”
BackgroundMethodInfluentialGoing beyond the untold facts in PLS–SEM and moving forward
Hair, J. F., Sarstedt, M., Ringle, C. M., Sharma, P. N., & Liengaard, B. D. (2024) · European Journal of Marketing · doi:10.1108/ejm-08-2023-0645 · Cited by 237
Metadata
- Authors
- Rönkkö, M., & Cho, E.
- Affiliations
- University of Jyväskylä (FI)
- Kwangwoon University (KR)
- Corresponding author
- Cho, Eunseong
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2020-11-23
- Volume
- 25
- Issue
- 1
- Pages
- 6-14
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W3108789059
- Cited by
- 1,309
- References
- 93
- Citations by year
- 2026: 346 · 2025: 326 · 2024: 287 · 2023: 183 · 2022: 130 · 2021: 35 · 2020: 1
- FWCI
- 55.50
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Organizational Research Methods (publishedVersion, cc-by-nc)
- Jyväskylä University Digital Archive (University of Jyväskylä) (submittedVersion, cc-by-nc)
- Funders
- Kwangwoon University (the Research Grant of Kwangwoon University in 2019)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- discriminant validity, measurement error, construct validity, Monte Carlo comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.