Do we measure leadership effectively? Articulating and evaluating scale development psychometrics for best practice
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Abstract Leadership assessment has been a particular point of difficulty for contemporary scholarship, with many practitioners rejecting academically-driven leadership instruments and scales and preferring their own, less rigorous, scales. We believe that current conceptualizations and measurements of leadership are problematic, indicated by contemporary challenges that can be widely understood as failures of leadership (e.g. the Australian Banking Royal Commission and Volkswagen's ‘Dieselgate’). Also, how effective leadership is measured needs to change. This paper presents a systematic review of 17 leadership scales developed in the new millennium. The majority of scales lack some degree of rigor. Our response has been to conduct eighteen critical checks over four stages of scale development: theory generation, item development, content validity, and empirical evaluation. On the premise that understanding past practices, with their limitations, can be used to drive forward a suite of more effective organizational tools, we provide best practice recommendations using contemporary psychometric research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Making sense of cronbach’s alpha.
Tavakol, M., & Dennick, R. (2011) · International Journal of Medical Education · doi:10.5116/ijme.4dfb.8dfd · Cited by 15,126
Power analysis and determination of sample size for covariance structure modeling.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.130 · Cited by 10,418
Costello, A. B., & Osborne, J. (2005) · University of Massachusetts Amherst · doi:10.7275/jyj1-4868 · Cited by 10,297
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810
Structural Equation Modelling: Guidelines for Determining Model Fit
Hooper, D., Coughlan, J., & Mullen, M. R. (2008) · Dublin Institute of Technology · doi:10.21427/d7cf7r · Cited by 8,746
Structural model evaluation and modification: An interval estimation approach
Steiger, J. H. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_4 · Cited by 7,408
A QUANTITATIVE APPROACH TO CONTENT VALIDITY1
LAWSHE, C. H. (1975) · Personnel Psychology · doi:10.1111/j.1744-6570.1975.tb01393.x · Cited by 7,313
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1103_2 · Cited by 6,691
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567
Knowledge management: An organizational capabilities perspective
Gold, A. H., Malhotra, A., & Segars, A. H. (2001) · Journal of Management Information Systems · doi:10.1080/07421222.2001.11045669 · Cited by 6,233
Curran, P. J., West, S. G., & Finch, J. F. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.16 · Cited by 5,512
Sample size in factor analysis.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.1.84 · Cited by 5,497
Required sample size to detect the mediated effect
Fritz, M. S., & MacKinnon, D. P. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01882.x · Cited by 4,810
Principles and practice in reporting structural equation analyses.
McDonald, R. P., & Ho, M.-H. R. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.64 · Cited by 4,734
A First Course in Factor Analysis
Comrey, A. L., & Lee, H. B. (2013) · doi:10.4324/9781315827506 · Cited by 4,546
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013) · Educational and Psychological Measurement · doi:10.1177/0013164413495237 · Cited by 3,931
Starting at the beginning: an introduction to coefficient alpha and internal consistency
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8001_18 · Cited by 3,794
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608
Authentic Leadership: Development and Validation of a Theory-Based Measure†
Walumbwa, F. O., Avolio, B. J., Gardner, W. L., Wernsing, T. S., & Peterson, S. J. (2008) · Journal of Management · doi:10.1177/0149206307308913 · Cited by 3,471
A Review of Scale Development Practices in the Study of Organizations
Hinkin, T. R. (1995) · Journal of Management · doi:10.1177/014920639502100509 · Cited by 3,193
Relation of sample size to the stability of component patterns.
Guadagnoli, E., & Velicer, W. F. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.2.265 · Cited by 2,880
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
Organizational Citizenship Behavior: It's Construct Clean-Up Time
Organ, D. W. (1997) · Human Performance · doi:10.1207/s15327043hup1002_2 · Cited by 2,807
Uses and abuses of coefficient alpha
Schmitt, N. (1996) · Psychological Assessment · doi:10.1037/1040-3590.8.4.350 · Cited by 2,588
Content validity in psychological assessment: A functional approach to concepts and methods.
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995) · Psychological Assessment · doi:10.1037//1040-3590.7.3.238 · Cited by 2,387
Understanding the limitations of global fit assessment in structural equation modeling
Steiger, J. H. (2007) · Personality and Individual Differences · doi:10.1016/j.paid.2006.09.017 · Cited by 2,360
Organizational Theories: Some Criteria for Evaluation
Bacharach, S. B. (1989) · The Academy of Management Review · doi:10.5465/amr.1989.4308374 · Cited by 2,305
Structural equations modeling: Fit Indices, sample size, and advanced topics
Iacobucci, D. (2010) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2009.09.003 · Cited by 2,263
Servant leadership: Development of a multidimensional measure and multi-level assessment
Liden, R. C., Wayne, S. J., Zhao, H., & Henderson, D. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2008.01.006 · Cited by 2,058
Minimum Sample Size Recommendations for Conducting Factor Analyses
Mundfrom, D. J., Shaw, D. G., & Ke, T. L. (2005) · International Journal of Testing · doi:10.1207/s15327574ijt0502_4 · Cited by 1,704
Bayesian structural equation modeling: A more flexible representation of substantive theory
Muthén, B., & Asparouhov, T. (2012) · Psychological Methods · doi:10.1037/a0026802 · Cited by 1,562
Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., & Paxton, P. (2008) · Sociological Methods & Research · doi:10.1177/0049124108314720 · Cited by 1,538
Dinh, J. E., Lord, R. G., Gardner, W. L., Meuser, J. D., Liden, R. C., & Hu, J. (2014) · The Leadership Quarterly · doi:10.1016/j.leaqua.2013.11.005 · Cited by 1,426
Individual Trust in Online Firms: Scale Development and Initial Test
(2002) · Journal of Management Information Systems · doi:10.1080/07421222.2002.11045715 · Cited by 1,347
Scale development and construct clarification of servant leadership
Barbuto, J. E., & Wheeler, D. W. (2006) · Group & Organization Management · doi:10.1177/1059601106287091 · Cited by 1,288
Multidimensional constructs in organizational behavior research: an integrative analytical framework
Edwards, J. R. (2001) · Organizational Research Methods · doi:10.1177/109442810142004 · Cited by 1,128
The Servant Leadership Survey: Development and Validation of a Multidimensional Measure
van Dierendonck, D., & Nuijten, I. (2011) · Journal of Business and Psychology · doi:10.1007/s10869-010-9194-1 · Cited by 1,096
EDITOR'S COMMENTS: CONSTRUCT CLARITY IN THEORIES OF MANAGEMENT AND ORGANIZATION.
Suddaby, R. (2010) · Academy of Management Review · doi:10.5465/amr.2010.51141319 · Cited by 984
Interrater agreement and interrater reliability: Key concepts, approaches, and applications
Gisev, N., Bell, J. S., & Chen, T. F. (2013) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2012.04.004 · Cited by 924
Defining and Measuring Servant Leadership Behaviour in Organizations
Sendjaya, S., Sarros, J. C., & Santora, J. C. (2008) · Journal of Management Studies · doi:10.1111/j.1467-6486.2007.00761.x · Cited by 844
Kalshoven, K., Den Hartog, D. N., & De Hoogh, A. H. B. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2010.12.007 · Cited by 824
Bishara, A. J., & Hittner, J. B. (2012) · Psychological Methods · doi:10.1037/a0028087 · Cited by 766
Servant leadership: Validation of a short form of the SL-28
Liden, R. C., Wayne, S. J., Meuser, J. D., Hu, J., Wu, J., & Liao, C. (2015) · The Leadership Quarterly · doi:10.1016/j.leaqua.2014.12.002 · Cited by 712
The Authentic Leadership Inventory (ALI): Development and empirical tests
Neider, L. L., & Schriesheim, C. A. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.09.008 · Cited by 486
Asparouhov, T., Muthén, B., & Morin, A. J. S. (2015) · Journal of Management · doi:10.1177/0149206315591075 · Cited by 340
Sideridis, G., Simos, P., Papanicolaou, A., & Fletcher, J. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164414525397 · Cited by 246
Ethical character and virtue of organizations: an empirical assessment and strategic implications
Chun, R. (2005) · Journal of Business Ethics · doi:10.1007/s10551-004-6591-2 · Cited by 234
Why study expert decision making? Some historical perspectives and comments
Shanteau, J., & Stewart, T. R. (1992) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(92)90057-e · Cited by 173
Cited by
Crawford, J., Allen, K.-A., Pani, B., & Cowling, M. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2326956 · Cited by 224
“The con fi rmatory fi t index (CFI) and the Tucker-Lewis Index (TLI) were targeted to be below .9 and the Chi-Squared/Degree of Freedom (X 2 / df ) below fi ve as per Crawford and Kelder (2019).”
MethodSelf-citationFrick, N. R. J., Mirbabaie, M., Stieglitz, S., & Salomon, J. (2021) · Journal of Decision Systems · doi:10.1080/12460125.2020.1870065 · Cited by 138
“Likewise, as leadership is known to be perceived subjectively (Crawford & Kelder, 2019), and as environmental events are said to be perceived subjectively, it is necessary to focus on individual differences with regard to the attitude towards a change.”
BackgroundCrawford, J., Cui, Z.-Y. A., & Kewley, D. (2024) · Journal of Small Business Management · doi:10.1080/00472778.2023.2246061 · Cited by 60
“Likewise, those that were not easily quantifiable (corporate ethics, governance, and leadership: Crawford & Kelder, 2019) were rarely discussed in the literature.”
Self-citation
Metadata
- Authors
- Crawford, J. A., & Kelder, J.-A.
- Venue
- The Leadership Quarterly
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 30
- Issue
- 1
- Pages
- 133-144
- ISSN
- 1048-9843 (ISSN-L)
- OpenAlex
- W2884744537
- Cited by
- 111
- References
- 49
- FWCI
- 8.39
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.