Tackling the Problem of Construct Proliferation: A Guide to Assessing the Discriminant Validity of Conceptually Related Constructs
Open accessAbstract
Construct proliferation—the accumulation of ostensibly different but potentially identical constructs representing organizational phenomena—is a salient problem in contemporary research. While a number of construct validation procedures exist, relatively few validation studies conduct comprehensive assessments of the discriminant validity of theoretically distinct constructs. In this article, we outline the key considerations a researcher must take into account when attempting to establish the empirical distinctness of new or existing constructs and provide a step-by-step guide on how to assess the discriminant validity of constructs while accounting for three major sources of measurement error: random error, specific factor error, and transient error. Using a number of popular measures from the leadership literature, we provide an illustrative example of how to conduct a study of discriminant validity. We include several analytic strategies in our study and discuss the similarities and differences between the results they yield. We also discuss several additional issues related to this type of research and make recommendations for conducting discriminant validity analyses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparing the correlation against 1.0 as a test of discriminant validity has two shortcomings: the possible outcomes are whether two constructs are perfectly correlated or not, but perfect correlation is rare in the population; and with a large enough sample size, even a correlation at 0.95 is statistically significantly different from 1.0.
“However, Shaffer et al. (2016) summarized two shortcomings of comparing the correlation against 1.0 as a test of discriminant validity.First, the possible outcomes are whether two constructs are perfectly correlated or not, but it is rare for them to be perfectly correlated in the population.Second, with a large enough sample size (e.g., 1,000), even a correlation at 0.95 is statistically signific”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Shaffer et al., 2016
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredShaffer et al. (2016) suggested examining the correlation disattenuated for measurement errors to evaluate discriminant validity.
“Recently, Shaffer et al. (2016) suggested examining the correlation disattenuated for measurement errors to evaluate discriminant validity.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Shaffer et al. (2016)
The available passages identify the paper as a guide to assessing discriminant validity and list measurement error as a keyword, but they do not state that it recommends examining correlations disattenuated for measurement error. Searches for “disattenuated” and for correlations corrected for measurement error or unreliability found no passage establishing that recommendation. (LLM, conf=0.69)
Experimental — accuracy not yet measuredComparing the disattenuated correlation or HTMT ratio with a threshold value of 0.85 disregards sampling error.
“Finally, comparing the disattenuated correlation or HTMT ratio with a threshold value of 0.85 disregards sampling error (Shaffer et al., 2016).”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Shaffer et al., 2016
The supplied title and Abstract-labeled metadata contain no discussion of disattenuated correlations, HTMT, a 0.85 threshold, or sampling error. Searches for those terms and for confidence intervals returned no substantive passage, so our copy cannot establish whether the paper makes the attributed point. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190 · see citing works
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
Confirmatory factor analysis for applied research
Brown, T. A. (2007) · Choice Reviews Online · doi:10.5860/choice.44-2769 · Cited by 15,613 · see citing works
Research in Organizational Behavior.
Grimes, A. J., Cummings, L. L., & Staw, B. M. (1986) · Administrative Science Quarterly · doi:10.2307/2392975 · Cited by 11,303 · see citing works
Power analysis and determination of sample size for covariance structure modeling.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.130 · Cited by 10,418 · see citing works
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Graen, G. B., & Uhl-Bien, M. (1995) · The Leadership Quarterly · doi:10.1016/1048-9843(95)90036-5 · Cited by 7,299 · see citing works
Assessing construct validity in organizational research
Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991) · Administrative Science Quarterly · doi:10.2307/2393203 · Cited by 6,621 · see citing works
Handbook of industrial and organizational psychology
(1993) · Choice Reviews Online · doi:10.5860/choice.30-5278 · Cited by 5,483 · see citing works
Authentic leadership development: Getting to the root of positive forms of leadership
Avolio, B. J., & Gardner, W. L. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.03.001 · Cited by 5,478 · cited 1× · see citing works
“Sometimes scale items across measures are similar because the constructs explicitly share conceptual overlap (e.g., Avolio & Gardner, 2005; Conger & Kanungo, 1994).”
BackgroundEthical leadership: A social learning perspective for construct development and testing
Brown, M. E., Treviño, L. K., & Harrison, D. A. (2005) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2005.03.002 · Cited by 5,435 · see citing works
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,354 · see citing works
From transactional to transformational leadership: Learning to share the vision
Bass, B. M. (1990) · Organizational Dynamics · doi:10.1016/0090-2616(90)90061-s · Cited by 4,697 · see citing works
What Constitutes a Theoretical Contribution?
Whetten, D. A. (1989) · Academy of Management Review · doi:10.5465/amr.1989.4308371 · Cited by 4,296 · see citing works
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095 · see citing works
Consequences of abusive supervision.
Tepper, B. J. (2000) · Academy of Management Journal · doi:10.2307/1556375 · Cited by 3,810 · see citing works
Avolio, B. J., Bass, B. M., & Jung, D. I. (1999) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317999166789 · Cited by 3,034 · see citing works
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023 · see citing works
Internet, mail, and mixed-mode surveys: The tailored design method *
Wetzel, A. P. (2010) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.20083 · Cited by 2,572 · see citing works
Douglas, O. B., & Holland, B. F. (2015) · MacMillan Co eBooks · doi:10.1037/14910-020 · Cited by 2,283 · see citing works
THE BIG FIVE PERSONALITY TRAITS, GENERAL MENTAL ABILITY, AND CAREER SUCCESS ACROSS THE LIFE SPAN
Judge, T. A., Higgins, C. A., Thoresen, C. J., & Barrick, M. R. (1999) · Personnel Psychology · doi:10.1111/j.1744-6570.1999.tb00174.x · Cited by 2,184 · see citing works
Servant leadership: Development of a multidimensional measure and multi-level assessment
Liden, R. C., Wayne, S. J., Zhao, H., & Henderson, D. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2008.01.006 · Cited by 2,058 · see citing works
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 1,925 · see citing works
Power and sensitivity of alternative fit indices in tests of measurement invariance.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.3.568 · Cited by 1,668 · see citing works
Longitudinal Research: The Theory, Design, and Analysis of Change
Ployhart, R. E., & Vandenberg, R. J. (2010) · Journal of Management · doi:10.1177/0149206309352110 · Cited by 1,496 · see citing works
Publication Manual of the American Psychological Association
Yao, R. (2011) · Family and Consumer Sciences Research Journal · doi:10.1111/j.1552-3934.2011.02081.x · Cited by 1,459 · see citing works
Why g matters: The complexity of everyday life
Gottfredson, L. S. (1997) · Intelligence · doi:10.1016/s0160-2896(97)90014-3 · Cited by 1,408 · see citing works
Barriers to the Advance of Organizational Science: Paradigm Development as a Dependent Variable
Pfeffer, J. (1993) · Academy of Management Review · doi:10.5465/amr.1993.9402210152 · Cited by 1,404 · see citing works
Five-factor model of personality and transformational leadership
Judge, T. A., & Bono, J. E. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.5.751 · Cited by 1,382 · see citing works
Wang, H., Law, K. S., Hackett, R. D., Wang, D., & Chen, Z. X. (2005) · Academy of Management Journal · doi:10.5465/amj.2005.17407908 · Cited by 1,373 · see citing works
DERUE, D. S., NAHRGANG, J. D., WELLMAN, N., & HUMPHREY, S. E. (2011) · Personnel Psychology · doi:10.1111/j.1744-6570.2010.01201.x · Cited by 1,356 · see citing works
EHRHART, M. G. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.tb02484.x · Cited by 1,322 · see citing works
Causal Analysis: Assumptions, Models, and Data
Schriesheim, C. A., James, L. R., Muliak, S. A., & Brett, J. M. (1984) · Academy of Management Review · doi:10.2307/258244 · Cited by 1,240 · see citing works
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090 · see citing works
A Short Measure of Transformational Leadership
Carless, S. A., Wearing, A. J., & Mann, L. (2000) · Journal of Business and Psychology · doi:10.1023/a:1022991115523 · Cited by 979 · see citing works
Statistical Procedures and the Justification of Knowledge in Psychological Science
Rosnow, R., & Rosenthal, R. (1989) · American Psychologist · doi:10.1037/0003-066x.44.10.1276 · Cited by 951 · see citing works
A Hierarchical Taxonomy of Leadership Behavior: Integrating a Half Century of Behavior Research
Yukl, G., Gordon, A., & Taber, T. (2002) · Journal of Leadership & Organizational Studies · doi:10.1177/107179190200900102 · Cited by 951 · see citing works
The Handbook of Research Synthesis
Ziegel, E. R., Cooper, H., & Hedges, L. V. (1995) · Technometrics · doi:10.2307/1269651 · Cited by 936 · see citing works
Kneale, W., & Kneale, M. (1962) · Medical Entomology and Zoology · Cited by 922 · see citing works
Review of Educational Measurement.
Langmuir, C. R. (1952) · Journal of Educational Psychology · doi:10.1037/h0051392 · Cited by 906 · see citing works
Hershcovis, M. S. (2010) · Journal of Organizational Behavior · doi:10.1002/job.689 · Cited by 898 · see citing works
Interpretation of Educational Measurements
Freeman, F. S., & Kelley, T. L. (1928) · The American Journal of Psychology · doi:10.2307/1414476 · Cited by 865 · see citing works
The European Health Psychologist
Todorova, I. L. G., Araújo‐Soares, V., & Sniehotta, F. F. (2005) · European Health Psychologist · Cited by 777 · see citing works
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 738 · see citing works
Pfeffer, J. (1977) · Academy of Management Review · doi:10.5465/amr.1977.4409175 · Cited by 722 · see citing works
The Dispositional Approach To Job Attitudes: A Lifetime Longitudinal Test
Staw, B. M., Bell, N. E., & Clausen, J. A. (1986) · Administrative Science Quarterly · doi:10.2307/2392766 · Cited by 720 · see citing works
Charismatic leadership and follower effects
Conger, J. A., KANUNGO, R. N., & Menon, S. T. (2000) · Journal of Organizational Behavior · doi:10.1002/1099-1379(200011)21:7 · Cited by 716 · see citing works
The description of supervisory behavior.
Fleishman, E. A. (1953) · Journal of Applied Psychology · doi:10.1037/h0056314 · Cited by 677 · see citing works
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1992) · American Psychological Association eBooks · doi:10.1037/10109-027 · Cited by 653 · see citing works
Short, J. C., Broberg, J. C., Cogliser, C. C., & Brigham, K. H. (2010) · Organizational Research Methods · doi:10.1177/1094428109335949 · Cited by 644 · see citing works
Handbook of Research Methods. in Social and Personality Psychology
Lai, H. L. T. (2000) · Cited by 627 · see citing works
Taxonomic efforts in the description of leader behavior: A synthesis and functional interpretation
Fleishman, E. A., Mumford, M. D., Zaccaro, S. J., Levin, K. Y., Korotkin, A. L., & Hein, M. B. (1991) · The Leadership Quarterly · doi:10.1016/1048-9843(91)90016-u · Cited by 580 · see citing works
Manual for the leader behavior description questionnaire - Form XII
Stogdill, R. M. (1963) · Bureau of Business Research, College of Commerce and Administration, Ohio State University eBooks · Cited by 575 · see citing works
Measurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567 · see citing works
Charismatic leadership in organizations: Perceived behavioral attributes and their measurement
Conger, J. A., & KANUNGO, R. N. (1994) · Journal of Organizational Behavior · doi:10.1002/job.4030150508 · Cited by 554 · see citing works
The MLQ revisited: psychometric properties and recommendations
Tejeda, M. J., Scandura, T. A., & Pillai, R. (2001) · The Leadership Quarterly · doi:10.1016/s1048-9843(01)00063-7 · Cited by 400 · see citing works
Job Burnout and Employee Engagement
Cole, M. S., Walter, F. H., Bedeian, A. G., & O’Boyle, E. H. (2011) · Journal of Management · doi:10.1177/0149206311415252 · Cited by 392 · see citing works
(2000) · Cited by 375 · see citing works
Testing differences between nested covariance structure models: Power analysis and null hypotheses.
Maccallum, R., Browne, M., & Cai, L. (2006) · Psychological methods · doi:10.1037/1082-989x.11.1.19 · Cited by 374 · see citing works
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371 · see citing works
Schmidt, F. L., Le, H., & Ilies, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.206 · Cited by 362 · see citing works
A role set analysis of managerial reputation
Tsui, A. S. (1984) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(84)90037-0 · Cited by 351 · see citing works
Hoyle, R. H. (2000) · Elsevier eBooks · doi:10.1016/b978-012691360-6/50017-3 · Cited by 346 · see citing works
Rowold, J., & Heinitz, K. (2007) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.01.003 · Cited by 277 · see citing works
Le, H., Schmidt, F. L., Harter, J. K., & Lauver, K. J. (2010) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2010.02.003 · Cited by 272 · see citing works
Assessing the discriminant validity of transformational leader behaviour as measured by the MLQ 1
Carless, S. A. (1998) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1998.tb00681.x · Cited by 271 · see citing works
Theory testing and measurement error
Schmidt, F. L., & Hunter, J. E. (1999) · Intelligence · doi:10.1016/s0160-2896(99)00024-0 · Cited by 254 · see citing works
Causal Impact of Employee Work Perceptions on the Bottom Line of Organizations
Harter, J. K., Schmidt, F. L., Asplund, J. W., Killham, E. A., & Agrawal, S. (2010) · Perspectives on Psychological Science · doi:10.1177/1745691610374589 · Cited by 240 · see citing works
Citation Analysis and Discourse Analysis
Swales, J. M. (1986) · Applied Linguistics · doi:10.1093/applin/7.1.39 · Cited by 240 · see citing works
Composing Group-Level Constructs From Individual-Level Survey Data
van Mierlo, H., Vermunt, J. K., & Rutte, C. G. (2008) · Organizational Research Methods · doi:10.1177/1094428107309322 · Cited by 240 · see citing works
SCIENTIFIC COMMUNICATION AND LITERATURE RETRIEVAL
White, H. D. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.7 · Cited by 221 · see citing works
Tepper, B. J., & Henle, C. A. (2010) · Journal of Organizational Behavior · doi:10.1002/job.688 · Cited by 192 · see citing works
Conger, J. A., KANUNGO, R. N., Menon, S. T., & Mathur, P. (1997) · Canadian Journal of Administrative Sciences / Revue Canadienne des Sciences de l Administration · doi:10.1111/j.1936-4490.1997.tb00136.x · Cited by 179 · see citing works
Hinkin, T. R., & Schriesheim, C. A. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2008.07.001 · Cited by 178 · see citing works
Detecting and Correcting the Lies That Data Tell
Schmidt, F. (2010) · Perspectives on Psychological Science · doi:10.1177/1745691610369339 · Cited by 163 · see citing works
CONFIRMATORY FACTOR ANALYSIS OF MULTITRAIT‐MULTIMETHOD MATRICES
Marsh, H. W., & Hocevar, D. (1983) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1983.tb00202.x · Cited by 160 · see citing works
Charles, E. P. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.2.206 · Cited by 154 · see citing works
The Human Factor in Shaping the Course of History and Development
Adjibolosoo, S. (2000) · Medical Entomology and Zoology · Cited by 128 · see citing works
Watson, D., & Humrichouse, J. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.91.5.959 · Cited by 116 · see citing works
Rousseau, D. M. (2007) · Academy of Management Journal · doi:10.5465/amj.2007.27155539 · Cited by 108 · see citing works
Barrick, M. R., Swider, B. W., & Stewart, G. L. (2010) · Journal of Applied Psychology · doi:10.1037/a0019918 · Cited by 104 · see citing works
Le, H., Schmidt, F. L., & Putka, D. J. (2007) · Organizational Research Methods · doi:10.1177/1094428107302900 · Cited by 102 · see citing works
Conceptual Versus Empirical Distinctions Among Constructs: Implications for Discriminant Validity
Harter, J. K., & Schmidt, F. L. (2008) · Industrial and Organizational Psychology · doi:10.1111/j.1754-9434.2007.00004.x · Cited by 95 · see citing works
A proposed procedure for construct definition in marketing
Gilliam, D. A., & Voss, K. E. (2013) · European Journal of Marketing · doi:10.1108/03090561311285439 · Cited by 88 · see citing works
How important is transient error in estimating reliability? Going beyond simulation studies.
Becker, G. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.3.370 · Cited by 84 · see citing works
Examining the Factor Structure of the MLQ
Heinitz, K., Liepmann, D., & Felfe, J. (2005) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.21.3.182 · Cited by 81 · see citing works
Validity and the Multitrait-Multimethod Matrix
Althauser, R. P., & Heberlein, T. A. (1970) · Sociological Methodology · doi:10.2307/270788 · Cited by 59 · see citing works
Systematically Retrieving Research: A Case Study Evaluating Seven Databases
Taylor, B. J., Wylie, E. C., Dempster, M., & Donnelly, M. (2007) · Research on Social Work Practice · doi:10.1177/1049731507304402 · Cited by 56 · see citing works
Determining relative importance of predictors with the observational design.
Cortina, J. M., & DeShon, R. P. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.5.798 · Cited by 34 · see citing works
Reeve, C. L. L., & Bonaccio, S. (2011) · Intelligence · doi:10.1016/j.intell.2011.06.009 · Cited by 26 · see citing works
Estimation and Uses of the Proportion of Method Variance for Multitrait-Multimethod Data
Conway, J. M. (1998) · Organizational Research Methods · doi:10.1177/109442819812003 · Cited by 20 · see citing works
Greenberg, J., Solomon, S., Pyszczynski, T., & Steinberg, L. (1988) · Psychological Review · doi:10.1037/0033-295x.95.4.566 · Cited by 16 · see citing works
Schucan Bird, K., & Tripney, J. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.48 · Cited by 13 · see citing works
Estimation of Transient Error in Cognitive Ability Scales
Reeve, C. L. L., Heggestad, E. D., & George, E. (2005) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2005.00328.x · Cited by 13 · see citing works
WITHDRAWNPreprint
Crutzen, R. (2024) · doi:10.31234/osf.io/a8tkc · Cited by 10 · see citing works
Schmidt, F. L., Le, H., & Oh, I. (2013) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12044 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262 · see citing works
“Recently, Shaffer et al. (2016) suggested examining the correlation disattenuated for measurement errors to evaluate discriminant validity.”
InfluentialAn Updated Guideline for Assessing Discriminant Validity
Rönkkö, M., & Cho, E. (2022) · Organizational Research Methods · doi:10.1177/1094428120968614 · Cited by 1,309 · see citing works
Leadership, creativity, and innovation: A critical review and practical recommendations
Hughes, D. J., Lee, A., Tian, A. W., Newman, A., & Legood, A. (2018) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.03.001 · Cited by 1,080 · see citing works
A Review of Key Likert Scale Development Advances: 1995–2019
Jebb, A. T., Ng, V., & Tay, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.637547 · Cited by 932 · see citing works
Leadership, creativity and innovation: a meta-analytic review
Lee, A., Legood, A., Hughes, D., Tian, A. W., Newman, A., & Knight, C. (2020) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2019.1661837 · Cited by 425 · see citing works
Langer, M., & Landers, R. N. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106878 · Cited by 338 · see citing works
Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000406 · Cited by 331 · see citing works
Mackey, J. D., Parker Ellen, B., McAllister, C. P., & Alexander, K. C. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.10.037 · Cited by 320 · see citing works
Abusive supervision: A systematic review and fundamental rethink
Fischer, T., Tian, A. W., Lee, A., & Hughes, D. J. (2021) · The Leadership Quarterly · doi:10.1016/j.leaqua.2021.101540 · Cited by 318 · see citing works
Partial least squares path modeling: Time for some serious second thoughts
Rönkkö, M., McIntosh, C. N., Antonakis, J., & Edwards, J. R. (2016) · Journal of Operations Management · doi:10.1016/j.jom.2016.05.002 · Cited by 312 · see citing works
Qing, M., Asif, M., Hussain, A., & Jameel, A. (2019) · Review of Managerial Science · doi:10.1007/s11846-019-00340-9 · Cited by 332 · see citing works
Construct redundancy in leader behaviors: A review and agenda for the future
Banks, G. C., Gooty, J., Ross, R. L., Williams, C. E., & Harrington, N. T. (2018) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.12.005 · Cited by 275 · see citing works
Lambert, L. S., & Newman, D. A. (2022) · Organizational Research Methods · doi:10.1177/10944281221115374 · Cited by 301 · see citing works
Working on my own: Measuring the challenges of gig work
Caza, B. B., Reid, E. M., Ashford, S. J., & Granger, S. (2022) · Human Relations · doi:10.1177/00187267211030098 · Cited by 249 · see citing works
Hammer, J. H., Parent, M. C., & Spiker, D. A. (2018) · Journal of Counseling Psychology · doi:10.1037/cou0000248 · Cited by 207 · see citing works
A critique of the Leader-Member Exchange construct: Back to square one
Gottfredson, R. K., Wright, S. L., & Heaphy, E. D. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101385 · Cited by 205 · see citing works
Asif, M., Qing, M., Hwang, J., & Shi, H. (2019) · Sustainability · doi:10.3390/su11164489 · Cited by 196 · see citing works
A Review of Best Practice Recommendations for Text Analysis in R (and a User-Friendly App)
Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-017-9528-3 · Cited by 194 · see citing works
Integration of Industry 4.0 technologies into Total Productive Maintenance practices
Tortorella, G. L., Fogliatto, F. S., Cauchick-Miguel, P. A., Kurnia, S., & Jurburg, D. (2021) · International Journal of Production Economics · doi:10.1016/j.ijpe.2021.108224 · Cited by 180 · see citing works
Asif, M., Jameel, A., Hussain, A., Hwang, J., & Sahito, N. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16132381 · Cited by 184 · see citing works
A Look Back and a Leap Forward: A Review and Synthesis of the Individual Work Performance Literature
Carpini, J. A., Parker, S. K., & Griffin, M. A. (2017) · Academy of Management Annals · doi:10.5465/annals.2015.0151 · Cited by 190 · see citing works
Smart Maintenance: an empirically grounded conceptualization
Bokrantz, J., Skoogh, A., Berlin, C., Wuest, T., & Stahre, J. (2019) · International Journal of Production Economics · doi:10.1016/j.ijpe.2019.107534 · Cited by 179 · see citing works
Mental health literacy as theory: current challenges and future directions
Spiker, D. A., & Hammer, J. H. (2018) · Journal of Mental Health · doi:10.1080/09638237.2018.1437613 · Cited by 167 · see citing works
Yao, J., Lim, S., Guo, C. Y., Ou, A. Y., & Ng, J. W. X. (2021) · Journal of Applied Psychology · doi:10.1037/apl0000870 · Cited by 152 · see citing works
What makes for future entrepreneurs? The role of digital competencies for entrepreneurial intention
Bachmann, N., Rose, R., Maul, V., & Hölzle, K. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2023.114481 · Cited by 196 · see citing works
Nohe, C., & Hertel, G. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01364 · Cited by 144 · see citing works
Hiller, N. J., Sin, H.-P., Ponnapalli, A. R., & Ozgen, S. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.11.003 · Cited by 144 · see citing works
A systematic review and critique of research on “healthy leadership”
Rudolph, C. W., Murphy, L. D., & Zacher, H. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2019.101335 · Cited by 146 · see citing works
Schöggl, J., Stumpf, L., & Baumgartner, R. J. (2023) · Business Strategy and the Environment · doi:10.1002/bse.3593 · Cited by 133 · see citing works
A Cross-Level Investigation of Informal Field-Based Learning and Performance Improvements
Wolfson, M., Tannenbaum, S., Mathieu, J. E., & Maynard, M. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000267 · Cited by 111 · see citing works
Evaluating Evidence for Conceptually Related Constructs Using Bivariate Correlations
Swank, J. M., & Mullen, P. R. (2017) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2017.1339562 · Cited by 129 · see citing works
Mo, S., Lupoli, M. J., Newman, A., & Umphress, E. E. (2022) · Journal of Organizational Behavior · doi:10.1002/job.2617 · Cited by 138 · see citing works
Ghost in the Machine: On Organizational Theory in the Age of Machine Learning
Leavitt, K., Schabram, K., Hariharan, P., & Barnes, C. M. (2020) · Academy of Management Review · doi:10.5465/amr.2019.0247 · Cited by 114 · see citing works
How and when does abusive supervision affect hospitality employees’ service sabotage?
Park, J., & Kim, H. J. (2018) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2018.10.014 · Cited by 110 · see citing works
Passion for work: Determinants and outcomes
Vallerand, R. J., Houlfort, N., & Forest, J. (2014) · Research Bank (Australian Catholic University) · Cited by 89 · see citing works
Warsinsky, S., Schmidt-Kraepelin, M., Rank, S., Thiebes, S., & Sunyaev, A. (2021) · Journal of Medical Internet Research · doi:10.2196/30390 · Cited by 100 · see citing works
Rosenbusch, H., Wanders, F., & Pit, I. L. (2019) · Psychological Methods · doi:10.1037/met0000244 · Cited by 84 · see citing works
Psychological Determinants of Health Behavior
Hagger, M. S. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-020124-114222 · Cited by 88 · see citing works
Hughes, D. J., & Evans, T. R. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02155 · Cited by 91 · see citing works
Hussain, A., Sial, M. S., Usman, S. M., Hwang, J., Jiang, Y., & Shafiq, A. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16060994 · Cited by 97 · see citing works
Hofstede's individual-level indulgence dimension: Scale development and validation
Heydari, A., Laroche, M., Paulin, M., & Richard, M. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102640 · Cited by 81 · see citing works
Pinto, J. (2019) · International Journal of Management Reviews · doi:10.1111/ijmr.12199 · Cited by 84 · see citing works
The validation crisis in the L2 motivational self system tradition
Al‐Hoorie, A. H., Hiver, P. V., & In’nami, Y. (2023) · Studies in Second Language Acquisition · doi:10.1017/s0272263123000487 · Cited by 85 · see citing works
Engaging leadership: Enhancing work engagement through intrinsic values and need satisfaction
van Tuin, L., Schaufeli, W. B., & den Broeck, A. V. (2021) · Human Resource Development Quarterly · doi:10.1002/hrdq.21430 · Cited by 92 · see citing works
Hughes, D. J. (2018) · Wiley eBooks · doi:10.1002/9781118489772.ch24 · Cited by 79 · see citing works
Worker Well-Being: What it Is, and how it Should Be Measured
Wijngaards, I., King, O. C., Burger, M., & van Exel, J. N. A. (2021) · Applied Research in Quality of Life · doi:10.1007/s11482-021-09930-w · Cited by 88 · see citing works
Relational Dynamics of Leadership: Problems and Prospects
Scandura, T. A., & Meuser, J. D. (2021) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-012420-091249 · Cited by 78 · see citing works
Measurement Development and Evaluation
Zickar, M. J. (2019) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-012119-044957 · Cited by 72 · see citing works
Paul Irwing, B., & Hughes, D. J. (2018) · Wiley eBooks · doi:10.1002/9781118489772.ch1 · Cited by 73 · see citing works
Ethical leadership as a substitute for justice enactment: An information-processing perspective.
Koopman, J., Scott, B. A., Matta, F. K., Conlon, D. E., & Dennerlein, T. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000403 · Cited by 71 · see citing works
Chen, L., Luo, F., Xiaomei, Z., Huang, X., & Liu, Y. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.560594 · Cited by 78 · see citing works
Southwick, D. A., Tsay, C., & Duckworth, A. L. (2019) · Research in Organizational Behavior · doi:10.1016/j.riob.2020.100126 · Cited by 74 · see citing works
Jolly, P. M., McDowell, C., Dawson, M., & Abbott, J. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.102921 · Cited by 68 · see citing works
Amarnani, R. K., Lajom, J. A. L., Restubog, S. L. D., & Capezio, A. (2019) · Human Relations · doi:10.1177/0018726719844812 · Cited by 69 · see citing works
Aceves, P., & Evans, J. A. (2023) · Organization Science · doi:10.1287/orsc.2023.1686 · Cited by 64 · see citing works
Mascareño, J., Rietzschel, E. F., & Wisse, B. M. (2021) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2021.1872544 · Cited by 62 · see citing works
Construct proliferation in leadership style research
Bormann, K. C., & Rowold, J. (2018) · Organizational Psychology Review · doi:10.1177/2041386618794821 · Cited by 67 · see citing works
Walsh, B. M., Lee, J., Jensen, J. M., McGonagle, A. K., & Samnani, A. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-017-9505-x · Cited by 62 · see citing works
Hu, X., Park, Y., Day, A., & Barber, L. K. (2021) · Occupational Health Science · doi:10.1007/s41542-021-00085-6 · Cited by 64 · see citing works
The dark side of leadership: A systematic review of creativity and innovation
Mehraein, V., Visintin, F., & Pittino, D. (2023) · International Journal of Management Reviews · doi:10.1111/ijmr.12334 · Cited by 74 · see citing works
The impact of eudaimonic well-being on experience and loyalty: a tourism context
Al-okaily, N. S., Alzboun, N., Alrawadieh, Z., & Slehat, M. (2023) · Journal of Services Marketing · doi:10.1108/jsm-08-2021-0282 · Cited by 59 · see citing works
The consequences of humility for leaders: A double-edged sword
Muir, C. P., & Hayes-Jones, L. C. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2019.01.006 · Cited by 56 · see citing works
Heimann, A. L., Ingold, P. V., & Kleinmann, M. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2019.101364 · Cited by 56 · see citing works
Impact of Nurse Manager Leadership Styles on Work Engagement: A Systematic Literature Review
Alluhaybi, A., Wilson, A., Usher, K. J., & Durkin, J. (2023) · Journal of Nursing Management · doi:10.1155/2023/5090276 · Cited by 68 · see citing works
Tortorella, G. L., Saurin, T. A., Fogliatto, F. S., Tlapa, D., Moyano‐Fuentes, J., Gaiardelli, P., Seyedghorban, Z., Vassolo, R. S., Vergara, A., Vijaya Sunder, M., et al. (2022) · Production Planning & Control · doi:10.1080/09537287.2022.2083996 · Cited by 54 · see citing works
Transformational and entrepreneurial leadership: A review of distinction and overlap
Ravet‐Brown, T., Furtner, M. R., & Kallmuenzer, A. (2023) · Review of Managerial Science · doi:10.1007/s11846-023-00649-6 · Cited by 54 · see citing works
Measuring leadership an assessment of the Multifactor Leadership Questionnaire
Batista‐Foguet, J. M., Estève, M., & van Witteloostuijn, A. C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0254329 · Cited by 73 · see citing works
Leadership in virtual work settings: what we know, what we do not know, and what we need to do
Höddinghaus, M., Nohe, C., & Hertel, G. (2023) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2023.2250079 · Cited by 62 · see citing works
Donaldson, S. I., & Donaldson, S. I. (2020) · Journal of Well-Being Assessment · doi:10.1007/s41543-020-00033-1 · Cited by 56 · see citing works
Matsuo, M. (2017) · Personnel Review · doi:10.1108/pr-06-2016-0132 · Cited by 52 · see citing works
Cloud-based Meta-analysis to Bridge Science and Practice: Welcome to metaBUS
Bosco, F. A., Steel, P., Oswald, F. L., Uggerslev, K. L., & Field, J. G. (2015) · Personnel Assessment and Decisions · doi:10.25035/pad.2015.002 · Cited by 51 · see citing works
Work-family conflict, perceived control, and health, family, and wealth: A 20-year study
Li, A., Shaffer, J., Wang, Z., & Huang, J. L. (2021) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2021.103562 · Cited by 44 · see citing works
Self-citationZhou, S., & Muscente, K. K. (2022) · Nonprofit and Voluntary Sector Quarterly · doi:10.1177/08997640221129540 · Cited by 49 · see citing works
Towards a taxonomy of personality facets
Paul Irwing, B., Hughes, D. J., Tokarev, A., & Booth, T. (2023) · European Journal of Personality · doi:10.1177/08902070231200919 · Cited by 58 · see citing works
Assessing the validity of inferences from scores on the cognitive reflection test
Blacksmith, N., Yang, Y., Behrend, T. S., & Ruark, G. A. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2133 · Cited by 48 · see citing works
Chan, K.-Y., Ho, M. R., Kennedy, J. C., Uy, M. A., Kang, B. N. Y., Chernyshenko, O. S., & Yu, K. Y. T. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02041 · Cited by 47 · see citing works
Lee, J., Cho, J., Baek, Y., Pillai, R., & Oh, S. H. (2019) · Asia Pacific Journal of Management · doi:10.1007/s10490-018-9596-6 · Cited by 52 · see citing works
Smart maintenance: instrument development, content validation and an empirical pilot
Bokrantz, J., Skoogh, A., Berlin, C., & Stahre, J. (2020) · International Journal of Operations & Production Management · doi:10.1108/ijopm-11-2019-0746 · Cited by 46 · see citing works
Cheng, J. T., Gerpott, F. H., Benson, A. J., Bucker, B., Foulsham, T., Lansu, T. A. M., Schülke, O., & Tsuchiya, K. (2022) · The Leadership Quarterly · doi:10.1016/j.leaqua.2022.101654 · Cited by 47 · see citing works
Canivez, G. L., Watkins, M. W., & McGill, R. J. (2018) · British Journal of Educational Psychology · doi:10.1111/bjep.12230 · Cited by 46 · see citing works
Pletzer, J. L., Breevaart, K., & Bakker, A. B. (2023) · Organizational Psychology Review · doi:10.1177/20413866231197519 · Cited by 56 · see citing works
Wang, G., Liu, X., & Liu, Y. (2018) · The International Journal of Human Resource Management · doi:10.1080/09585192.2018.1494027 · Cited by 44 · see citing works
Thought experiments: Review and recommendations
Aguinis, H., Beltran, J. R., Archibold, E. E., Jean, E. L., & Rice, D. B. (2022) · Journal of Organizational Behavior · doi:10.1002/job.2658 · Cited by 45 · see citing works
Han, H., Sahito, N., Nguyen, T. V. T., Hwang, J., & Asif, M. (2019) · Sustainability · doi:10.3390/su11174798 · Cited by 44 · see citing works
Exploring local food consumption in restaurants through the lens of locavorism
Choi, J., Park, J., Jeon, H., & Asperin, A. E. (2021) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2021.1923608 · Cited by 48 · see citing works
Kern, M., Heissler, C., & Zapf, D. (2020) · Journal of Business and Psychology · doi:10.1007/s10869-020-09702-7 · Cited by 43 · see citing works
Cultural intelligence: An identity lens on the influence of cross-cultural experience
Pidduck, R. J., Shaffer, M. A., Zhang, Y., Cheung, S. S. Y., & Yunlu, D. G. (2022) · Journal of International Management · doi:10.1016/j.intman.2022.100928 · Cited by 47 · see citing works
A Meta-analytic Systematic Review and Theory of the Effects of Perceived Listening on Work Outcomes
Kluger, A. N., Lehmann, M., Aguinis, H., Itzchakov, G., Gordoni, G., Zyberaj, J., & Bakaç, C. (2023) · Journal of Business and Psychology · doi:10.1007/s10869-023-09897-5 · Cited by 50 · see citing works
Sturm, R. E., Jolly, P. M., & Williams, S. D. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-021-09786-9 · Cited by 42 · see citing works
Letting go of self: the creation of the nonattachment to self scale
Whitehead, R., Bates, G., Elphinstone, B., Yang, Y., & Murray, G. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02544 · Cited by 41 · see citing works
Zhao, L., Jolly, P. M., & Zhao, S. (2022) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-03-2022-0357 · Cited by 43 · see citing works
Pfefferbaum, B. J., Nitiéma, P., & Newman, E. (2019) · Child & Youth Care Forum · doi:10.1007/s10566-019-09494-9 · Cited by 42 · see citing works
A systematic and critical review of research on respect in leadership
Rudolph, C. W., Katz, I. M., Ruppel, R., & Zacher, H. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101492 · Cited by 41 · see citing works
Business Angel Investment as an Informal Learning Process: Does Experience Matter?
Botelho, T., Harrison, R., & Mason, C. M. (2021) · British Journal of Management · doi:10.1111/1467-8551.12583 · Cited by 40 · see citing works
Farquharson, B., Dixon, D. L., Williams, B., Torrens, C., Philpott, M., Laidlaw, H., & McDermott, S. (2023) · BMC Cardiovascular Disorders · doi:10.1186/s12872-022-02904-2 · Cited by 40 · see citing works
Defrag and reboot? Consolidating information and communication technology research in I-O psychology
Hu, X.-Y., Barber, L. K., Park, Y., & Day, A. (2021) · Industrial and Organizational Psychology · doi:10.1017/iop.2021.3 · Cited by 32 · see citing works
Jolly, P. M., Gordon, S. E., & Self, T. T. (2021) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-02-2021-0280 · Cited by 40 · see citing works
Sun, Y. (2022) · Science Communication · doi:10.1177/10755470221088927 · Cited by 39 · see citing works
Theories of Performance: A Review and Integration
Marshall, J. D., Aguinis, H., & Beltran, J. R. (2024) · Academy of Management Annals · doi:10.5465/annals.2022.0049 · Cited by 49 · see citing works
Burnout, Depression, and Borderline Personality: A 1,163-Participant Study
Bianchi, R., Rolland, J., & Salgado, J. F. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02336 · Cited by 37 · see citing works
- Authors
- Shaffer, J. A., DeGeest, D., & Li, A.
- Affiliations
- West Texas A&M University (US)
- University of Groningen (NL)
- Corresponding author
- Shaffer, Jonathan A.
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2015-08-18
- Volume
- 19
- Issue
- 1
- Pages
- 80-110
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2337454271
- Cited by
- 393
- References
- 98
- Citations by year
- 2026: 56 · 2025: 48 · 2024: 53 · 2023: 44 · 2022: 39 · 2021: 44 · 2020: 29 · 2019: 36 · 2018: 22 · 2017: 15 · 2016: 3 · 2015: 3 · 2014: 1
- FWCI
- 17.90
- Type
- journal article
- Language
- en
- Open access
- green
- research.rug.nl
- Locations
- Organizational Research Methods (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, other-oa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Business Strategies and Management Research
- Social Sciences › Psychology › Social Psychology › Cognitive and psychological constructs research
- Keywords
- construct proliferation, discriminant validity, construct validation, measurement error, random error, transient faults
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review