Understanding the Impact of Convergent Validity on Research Results
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findings, creating challenges for the accumulation and interpretation of research. Using data from published research, the authors find that substantial differences in findings between studies using desired and proxy variables occur even at levels of convergent validity as high as r = .85. Implications of using measures with less-than-ideal convergent validity for the interpretability of research results are examined. Convergent validities above r = .70 are recommended, whereas those below r = .50 should be avoided. Researchers are encouraged to develop and report convergent validity data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“With regard to ECL, the correlation showed satisfactory convergent validity considering the recommendations of Carlson and Herdman (2012).”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carlson and Herdman (2012)
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Research in Organizational Behavior.
Grimes, A. J., Cummings, L. L., & Staw, B. M. (1986) · Administrative Science Quarterly · doi:10.2307/2392975 · Cited by 10,899
Journal of personality and social psychology
Armstrong, J. S. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90126-4 · Cited by 10,485
KRISTOF‐BROWN, A. L., ZIMMERMAN, R. D., & JOHNSON, E. C. (2005) · Personnel Psychology · doi:10.1111/j.1744-6570.2005.00672.x · Cited by 4,082
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Podsakoff, N. P., Whiting, S. W., Podsakoff, P. M., & Blume, B. D. (2009) · Journal of Applied Psychology · doi:10.1037/a0013079 · Cited by 2,356
Watson, D., Wiese, D., Vaidya, J., & Tellegen, A. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.76.5.820 · Cited by 2,103
On the nature and direction of relationships between constructs and measures.
Edwards, J. R., & Bagozzi, R. P. (2000) · Psychological Methods · doi:10.1037//1082-989x.5.2.155 · Cited by 1,745
HIGH-INVOLVEMENT WORK PRACTICES, TURNOVER, AND PRODUCTIVITY: EVIDENCE FROM NEW ZEALAND.
Guthrie, J. P. (2001) · Academy of Management Journal · doi:10.2307/3069345 · Cited by 1,737
RESEARCH METHODOLOGY IN MANAGEMENT: CURRENT PRACTICES, TRENDS, AND IMPLICATIONS FOR FUTURE RESEARCH.
Scandura, T. A., & Williams, E. A. (2000) · Academy of Management Journal · doi:10.2307/1556348 · Cited by 1,186
On the validity of subjective measures of company performance
WALL, T. D., MICHIE, J., PATTERSON, M., WOOD, S. J., SHEEHAN, M., CLEGG, C. W., & WEST, M. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.tb02485.x · Cited by 1,121
Kale, P., & Singh, H. (2007) · Strategic Management Journal · doi:10.1002/smj.616 · Cited by 1,070
Risk propensity differences between entrepreneurs and managers: A meta-analytic review.
Stewart, W. H., & Roth, P. L. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.1.145 · Cited by 939
Why Summaries of Research on Psychological Theories are Often Uninterpretable
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 909
Self-efficacy and work-related performance: The integral role of individual differences.
Judge, T. A., Jackson, C. L., Shaw, J. C., Scott, B. A., & Rich, B. L. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.1.107 · Cited by 876
Short, J. C., Ketchen, D. J., Shook, C. L., & Ireland, R. D. (2010) · Journal of Management · doi:10.1177/0149206309342746 · Cited by 816
Acquaah, M. (2007) · Strategic Management Journal · doi:10.1002/smj.632 · Cited by 743
Situational and institutional determinants of firms' R&D search intensity
Chen, W., & Miller, K. D. (2007) · Strategic Management Journal · doi:10.1002/smj.594 · Cited by 708
BOMMER, W. H., JOHNSON, J. L., RICH, G. A., PODSAKOFF, P. M., & MACKENZIE, S. B. (1995) · Personnel Psychology · doi:10.1111/j.1744-6570.1995.tb01772.x · Cited by 633
Validity of questionnaire and TAT measures of need for achievement: Two meta-analyses.
Spangler, W. D. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.140 · Cited by 621
Combs, J. G., Crook, T. R., & Shook, C. L. (2005) · Research Methodology in Strategy and Management · doi:10.1016/s1479-8387(05)02011-4 · Cited by 542
The axioms and principal results of classical test theory
Novick, M. R. (1966) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(66)90002-2 · Cited by 509
Gong, Y., Law, K. S., Chang, S., & Xin, K. R. (2009) · Journal of Applied Psychology · doi:10.1037/a0013116 · Cited by 445
Bakker, A. B., Demerouti, E., & Dollard, M. F. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.4.901 · Cited by 415
A comparison of Likert scale and traditional measures of self-efficacy.
Maurer, T. J., & Pierce, H. R. (1998) · Journal of Applied Psychology · doi:10.1037//0021-9010.83.2.324 · Cited by 345
Miner, J. B., & Raju, N. S. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.1.3 · Cited by 268
Reliability for the Social Sciences: Theory and Applications
Traub, R. E. (1994) · Medical Entomology and Zoology · Cited by 259
Disentangling compensation and employment risks using the behavioral agency model
Larraza‐Kintana, M., Wiseman, R. M., Gomez‐Mejia, L. R., & Welbourne, T. M. (2007) · Strategic Management Journal · doi:10.1002/smj.624 · Cited by 222
Home Country Networks and Foreign Expansion: Evidence from the Venture Capital Industry
Guler, I., & Guillén, M. F. (2010) · Academy of Management Journal · doi:10.5465/amj.2010.49389027 · Cited by 220
Top management incentive compensation and knowledge sharing in multinational corporations
Fey, C. F., & Furu, P. (2008) · Strategic Management Journal · doi:10.1002/smj.712 · Cited by 200
Lee, G. K. (2007) · Strategic Management Journal · doi:10.1002/smj.566 · Cited by 197
Human resource configurations: Investigating fit with the organizational context.
Toh, S. M., Morgeson, F. P., & Campion, M. A. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.4.864 · Cited by 193
INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS
MURPHY, K. R., & DESHON, R. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb02421.x · Cited by 183
Bercovitz, J., & Mitchell, W. (2007) · Strategic Management Journal · doi:10.1002/smj.568 · Cited by 174
Stewart, W. H., & Roth, P. L. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.1.14 · Cited by 159
Does Prevalence Mitigate Relevance? The Moderating Effect of Group-Level OCB on Employee Performance
Bommer, W. H., Dierdorff, E. C., & Rubin, R. S. (2007) · Academy of Management Journal · doi:10.5465/amj.2007.28226149 · Cited by 138
McDonald, M. L., & Westphal, J. D. (2010) · Academy of Management Journal · doi:10.5465/amj.2010.49389006 · Cited by 130
Positive and Negative Affectivity and Facet Satisfaction: A Meta-analysis
Bowling, N. A., Hendricks, E. A., & Wagner, S. H. (2008) · Journal of Business and Psychology · doi:10.1007/s10869-008-9082-0 · Cited by 120
Consequences of measurement problems in strategic management research: the case of Amihud and Lev
Boyd, B., Gove, S., & Hitt, M. A. (2004) · Strategic Management Journal · doi:10.1002/smj.445 · Cited by 118
The Normalization of Deviant Organizational Practices: Wage Arrears in Russia, 1991–98
Earle, J. S., Spicer, A. H., & Peter, K. S. (2010) · Academy of Management Journal · doi:10.5465/amj.2010.49387426 · Cited by 88
Traditional, Likert, and Simplified Measures of Self-Efficacy
Maurer, T. J., & Andrews, K. D. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970899 · Cited by 82
Citing and Building on Meta-Analytic Findings: A Review and Recommendations
Carlson, K. D., & Ji, F. X. (2010) · Organizational Research Methods · doi:10.1177/1094428110384272 · Cited by 65
Self-citationRestrictions on the Possible Values of r12, Given r13 and r23
Stanley, J. C., & Wang, M. D. (1969) · Educational and Psychological Measurement · doi:10.1177/001316446902900304 · Cited by 31
Geometric Proof of the Restriction On the Possible Values of rxy When r xz and ryz Are Fixed
Glass, G. V., & Collins, J. R. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000103 · Cited by 28
Decimal Dust, Significant Digits, and the Search for Stars
Bedeian, A. G., Sturman, M. C., & Streiner, D. L. (2008) · Organizational Research Methods · doi:10.1177/1094428108321153 · Cited by 26
Research works printed but not linked: 6
- Budden M., 2007, Applied Mathematics E-notes, 7, 53
- 10.1037/1082-989X.5.2.155 · doi:10.1037/1082-989x.5.2.155 · Paper page
- Jackson D. N., 1976, Personality inventory manual
- 10.1037/0021-9010.83.2.324 · doi:10.1037/0021-9010.83.2.324 · Paper page
- Nunnally J. C., 1967, Psychometric theory
- 10.1037/0033-2909.112.1.140 · DOI · Paper page
Cited by
Partial least squares structural equation modeling
Sarstedt, M., Ringle, C. M., & Hair, J. F. (2017) · Handbook of Market Research · doi:10.1007/978-3-319-05542-8_15-1 · Cited by 3,285
Partial Least Squares Structural Equation Modeling
Sarstedt, M., Ringle, C. M., & Hair, J. F. (2022) · Handbook of Market Research · doi:10.1007/978-3-319-57413-4_15 · Cited by 1,483
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965
Partial Least Squares Structural Equation Modeling
Sarstedt, M., Ringle, C. M., & Hair, J. F. (2021) · Handbook of Market Research · doi:10.1007/978-3-319-05542-8_15-2 · Cited by 876
Becker, T. E., Atinc, G., Breaugh, J. A., Carlson, K. D., Edwards, J. R., & Spector, P. E. (2016) · Journal of Organizational Behavior · doi:10.1002/job.2053 · Cited by 989
“Hence, researchers testing hypotheses that include proxies can produce different results than would have occurred if a more meaningful CV had been used (Carlson & Herdman, 2012).”
BackgroundSelf-citation
Metadata
- Authors
- Carlson, K. D., & Herdman, A. O.
- Affiliations
- Virginia Tech (US)
- East Carolina University (US)
- Corresponding author
- Carlson, Kevin D.
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2010-12-30
- Volume
- 15
- Issue
- 1
- Pages
- 17-32
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W1975398860
- Cited by
- 642
- References
- 45
- Citations by year
- 2026: 56 · 2025: 83 · 2024: 81 · 2023: 91 · 2022: 78 · 2021: 71 · 2020: 50 · 2019: 48 · 2018: 27 · 2017: 23 · 2016: 14 · 2015: 8 · 2014: 9 · 2013: 2
- FWCI
- 1.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- convergent validity, proxy variables, measurement validity, construct measurement, measurement error
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.