Scale Development Research: A Content Analysis and Recommendations for Best Practices
The DOI matches its Crossref record; 93 of the 95 references it cites were found in a registry or an index.
Abstract
The authors conducted a content analysis on new scale development articles appearing in the Journal of Counseling Psychology during 10 years (1995 to 2004). The authors analyze and discuss characteristics of the exploratory and confirmatory factor analysis procedures in these scale development studies with respect to sample characteristics, factorability, extraction methods, rotation methods, item deletion or retention, factor retention, and model fit indexes. The authors uncovered a variety of specific practices that were at variance with the current literature on factor analysis or structural equation modeling. They make recommendations for best practices in scale development research in counseling psychology using exploratory and confirmatory factor analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because extant tested measures for legal abuse are limited, EFA is an appropriate analytic approach for the present study as it determines construct validity during initial scale development (Worthington & Whittaker, 2006).”
From Coercive Control in the Courtroom: the Legal Abuse Scale (LAS) (2023) · cites this paper as Worthington & Whittaker, 2006
In the citing paper
“Communalities were in the desirable range (0.54-0.80) (Worthington & Whittaker, 2006).”
From Coercive Control in the Courtroom: the Legal Abuse Scale (LAS) (2023) · cites this paper as Worthington & Whittaker, 2006
In the citing paper
“Parcelling, an approach whereby items are combined together into parcels, is the source of much debate.”
From Psychometric properties of acceptance measures: A systematic review (2019) · cites this paper as Worthington & Whittaker, 2006
In the citing paper
“especially in scale development where true relationships amongst scale items might be hidden (Little, Rhemtulla, Gibson, & Schoemann, 2013; Worthington & Whittaker, 2006).”
From Psychometric properties of acceptance measures: A systematic review (2019) · cites this paper as Worthington & Whittaker, 2006
In the citing paper
“Different methods of parcelling exist and each have benefits and drawbacks”
From Psychometric properties of acceptance measures: A systematic review (2019) · cites this paper as Worthington & Whittaker, 2006
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,032
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Principles and Practice of Structural Equation Modeling
Tabri, N., & Elliott, C. M. (2012) · Canadian Graduate Journal of Sociology and Criminology · doi:10.15353/cgjsc-rcessc.v1i1.25 · Cited by 38,891
Alternative Ways of Assessing Model Fit
Browne, M. W., & Cudeck, R. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002005 · Cited by 27,200
Comparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023
(1991) · Cited by 15,554
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
Hu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298
EQS : structural equations program manual
Bentler, P. M. (1989) · Medical Entomology and Zoology · Cited by 9,669
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,714
The Varimax Criterion for Analytic Rotation in Factor Analysis
Kaiser, H. F. (1958) · Psychometrika · doi:10.1007/bf02289233 · Cited by 7,580
A Reliability Coefficient for Maximum Likelihood Factor Analysis
Tucker, L. R., & Lewis, C. (1973) · Psychometrika · doi:10.1007/bf02291170 · Cited by 7,330
Structural Equation Modeling: Concepts, Issues, and Applications.
Smith, M. D., & Hoyle, R. H. (1996) · The Statistician · doi:10.2307/2988418 · Cited by 6,814
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1103_2 · Cited by 6,691
(2005) · Cited by 6,097
Practical issues in structural modeling
BENTLER, P. M., & CHOU, C.-P. (1987) · Sociological Methods & Research · doi:10.1177/0049124187016001004 · Cited by 5,804
Sample size in factor analysis.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.1.84 · Cited by 5,497
Akaike, H. (1987) · Psychometrika · doi:10.1007/bf02294359 · Cited by 5,264
Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming, Third Edition
Byrne, B. M. (2016) · doi:10.4324/9781315757421 · Cited by 4,653
A First Course in Factor Analysis
Comrey, A. L., & Lee, H. B. (2013) · doi:10.4324/9781315827506 · Cited by 4,546
Bozdogan, H. (1987) · Psychometrika · doi:10.1007/bf02294361 · Cited by 4,468
LISREL 7 : a guide to the program and applications
Jöreskog, K. G., & Sörbom, D. (1989) · Medical Entomology and Zoology · Cited by 4,299
(1904) · The American Journal of Psychology · doi:10.2307/1412635 · Cited by 4,187
O’connor, B. P. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200807 · Cited by 4,116
Structural Equation Modeling: Concepts, Issues, and Applications
Rigdon, E. E., & Hoyle, R. H. (1997) · Journal of Marketing Research · doi:10.2307/3151904 · Cited by 4,060
Akaike, H. (1987) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_29 · Cited by 4,027
Factor analysis in the development and refinement of clinical assessment instruments.
Floyd, F. J., & Widaman, K. F. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.286 · Cited by 3,970
Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size.
Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988) · Psychological Bulletin · doi:10.1037//0033-2909.103.3.391 · Cited by 3,824
Scale Development: Theory and Applications.
Xie, Y., & DeVellis, R. F. (1992) · Contemporary Sociology · doi:10.2307/2075704 · Cited by 3,635
A First Course in Factor Analysis
Price, B. (1993) · Technometrics · doi:10.1080/00401706.1993.10485363 · Cited by 3,512
Exploratory and confirmatory factor analysis: Understanding concepts and applications.
Thompson, B. (2004) · doi:10.1037/10694-000 · Cited by 3,469
Comparison of five rules for determining the number of components to retain.
Zwick, W. R., & Velicer, W. F. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.432 · Cited by 3,129
A New Incremental Fit Index for General Structural Equation Models
BOLLEN, K. A. (1989) · Sociological Methods & Research · doi:10.1177/0049124189017003004 · Cited by 3,002
Evaluation of goodness-of-fit indices for structural equation models.
Mulaik, S. A., James, L. R., Van Alstine, J., & Bennett, N. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.3.430 · Cited by 2,923
Relation of sample size to the stability of component patterns.
Guadagnoli, E., & Velicer, W. F. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.2.265 · Cited by 2,880
A Brief Guide to Structural Equation Modeling
Weston, R., & Gore, P. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286345 · Cited by 2,718
Statistically based tests for the number of common factors
Steiger, J. H. (1980) · Medical Entomology and Zoology · Cited by 2,413
Reading and understanding multivariate statistics
Grimm, L. G., & Yarnold, P. R. (1995) · Cited by 2,358
TESTS OF SIGNIFICANCE IN FACTOR ANALYSIS
BARTLETT, M. S. (1950) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1950.tb00285.x · Cited by 2,337
Latent variable models: An introduction to factor, path, and structural analysis
John, C. (1987) · Medical Entomology and Zoology · Cited by 2,120
Fan, X., Thompson, B., & Wang, L. (1999) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519909540119 · Cited by 1,663
On the fit of models to covariances and methodology to the Bulletin.
Bentler, P. M. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.3.400 · Cited by 1,657
Model modifications in covariance structure analysis: The problem of capitalization on chance.
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.111.3.490 · Cited by 1,632
Advanced Structural Equation Modeling: Issues and Techniques
Rigdon, E. E. (1998) · Applied Psychological Measurement · doi:10.1177/01466216980221008 · Cited by 1,631
The Analysis of Covariance Structures: Goodness-of-Fit Indices
HOELTER, J. W. (1983) · Sociological Methods & Research · doi:10.1177/0049124183011003003 · Cited by 1,624
Zinkus, P. W. (1987) · Textbook of Developmental Pediatrics · doi:10.1007/978-1-4613-1797-5_8 · Cited by 1,500
Choosing a multivariate model: Noncentrality and goodness of fit.
McDonald, R. P., & Marsh, H. W. (1990) · Psychological Bulletin · doi:10.1037//0033-2909.107.2.247 · Cited by 1,438
A First Course in Factor Analysis 2nd Ed
Comrey, A. L., & Lee, H. B. (1973) · Cited by 1,367
New Developments and Techniques in Structural Equation Modeling
(2001) · Psychology Press eBooks · doi:10.4324/9781410601858 · Cited by 1,338
Advanced Structural Equation Modeling
(2013) · Psychology Press eBooks · doi:10.4324/9781315827414 · Cited by 1,331
Uses of factor analysis in counseling psychology research.
Tinsley, H. E. A., & Tinsley, D. J. (1987) · Journal of Counseling Psychology · doi:10.1037//0022-0167.34.4.414 · Cited by 1,324
Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis
Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3302_1 · Cited by 1,307
Effect of the number of variables on measures of fit in structural equation modeling
Kenny, D. A., & McCoach, D. B. (2003) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1003_1 · Cited by 1,231
Exploratory Factor Analysis: Its Role in Item Analysis
Gorsuch, R. L. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6803_5 · Cited by 1,050
Affects of variable and subject sampling on factor pattern recovery.
Velicer, W. F., & Fava, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.2.231 · Cited by 998
The problem of equivalent models in applications of covariance structure analysis.
MacCallum, R. C., Wegener, D. T., Uchino, B. N., & Fabrigar, L. R. (1993) · Psychological Bulletin · doi:10.1037//0033-2909.114.1.185 · Cited by 901
Factor analysis and scale revision.
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.3.287 · Cited by 855
Specification searches in covariance structure modeling.
MacCallum, R. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.100.1.107 · Cited by 854
Survey Questions: Handcrafting the Standardized Questionnaire
Quarm, D., Converse, J. M., & Presser, S. (1988) · Teaching Sociology · doi:10.2307/1317544 · Cited by 766
Component Analysis versus Common Factor Analysis: Some issues in Selecting an Appropriate Procedure
Velicer, W. F., & Jackson, D. N. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2501_1 · Cited by 738
Viability of exploratory factor analysis as a precursor to confirmatory factor analysis
Gerbing, D. W., & Hamilton, J. G. (1996) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519609540030 · Cited by 732
Item Parceling Issues in Structural Equation Modeling
(2001) · New Developments and Techniques in Structural Equation Modeling · doi:10.4324/9781410601858-15 · Cited by 730
Latent Variable Models: An Introduction to Factor, Path, and Structural Analysis
Wardrop, J. L., & Loehlin, J. C. (1987) · Journal of Educational Statistics · doi:10.2307/1165058 · Cited by 726
Kahn, J. H. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286347 · Cited by 695
Factor Analysis and Related Methods
Widaman, K. F., & McDonald, R. P. (1987) · Journal of Educational Statistics · doi:10.2307/1164692 · Cited by 664
Converse, J., & Presser, S. (1986) · Sage eBooks · doi:10.4135/9781412986045 · Cited by 586
An evaluation of incremental fit indices: A clarification of mathematical and empirical properties.
Marsh, H. W., Balla, J. R., & Hau, K. (1996) · Dialnet (Universidad de la Rioja) · Cited by 558
Factor Analysis and Related Methods
McDonald, R. P. (2014) · Psychology Press eBooks · doi:10.4324/9781315802510 · Cited by 533
Implications of Recent Developments in Structural Equation Modeling for Counseling Psychology
Quintana, S. M., & Maxwell, S. E. (1999) · The Counseling Psychologist · doi:10.1177/0011000099274002 · Cited by 516
Widaman, K. F. (1993) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2803_1 · Cited by 513
Common Factor Analysis versus Component Analysis: Some Well and Little Known Facts
Gorsuch, R. L. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2501_3 · Cited by 418
An Investigation of the Parallel Analysis Criterion for Determining the Number of Common Factors
Humphreys, L. G., & Montanelli Jr., R. G. (1975) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1002_5 · Cited by 392
Principles of educational and psychological testing
BROWN, F. G. (1970) · Cited by 322
The Use of Structural Equation Modeling in Counseling Psychology Research
Martens, M. P. (2005) · The Counseling Psychologist · doi:10.1177/0011000004272260 · Cited by 320
An index of goodness-of-fit based on noncentrality
McDonald, R. P. (1989) · Journal of Classification · doi:10.1007/bf01908590 · Cited by 306
Evaluation of Factor Analytic Research Procedures by means of Simulated Correlation Matrices
Tucker, L. R., Koopman, R. F., & Linn, R. L. (1969) · Psychometrika · doi:10.1007/bf02290601 · Cited by 283
Treating Cronbach’s Alpha Reliability Coefficients as Data in Counseling Research
Helms, J. E., Henze, K. T., Sass, T. L., & Mifsud, V. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288308 · Cited by 276
Advanced Applications of Structural Equation Modeling in Counseling Psychology Research
Martens, M. P., & Haase, R. F. (2006) · The Counseling Psychologist · doi:10.1177/0011000005283395 · Cited by 245
Dawis, R. V. (1987) · Journal of Counseling Psychology · doi:10.1037/0022-0167.34.4.481 · Cited by 229
Use of structural equation modeling in counseling psychology research.
Fassinger, R. E. (1987) · Journal of Counseling Psychology · doi:10.1037//0022-0167.34.4.425 · Cited by 216
Effect-Size Measures and Meta-Analytic Thinking in Counseling Psychology Research
Henson, R. K. (2006) · The Counseling Psychologist · doi:10.1177/0011000005283558 · Cited by 199
A Comparison Of Component And Factor Patterns: A Monte Carlo Approach
Velicer, W. F., Peacock, A. C., & Jackson, D. N. (1982) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1703_5 · Cited by 169
Gorsuch, R. L. (2003) · Wiley eBooks · doi:10.1002/0471264385.wei0206 · Cited by 145
Construct Validation in Counseling Psychology Research
Hoyt, W. T., Warbasse, R. E., & Chu, E. Y. (2006) · The Counseling Psychologist · doi:10.1177/0011000006287389 · Cited by 144
Psychological Testing: Design, Analysis, and Use
Friedenberg, L. (1995) · Cited by 137
Discriminant Analysis in Counseling Psychology Research
Sherry, A. R. (2006) · The Counseling Psychologist · doi:10.1177/0011000006287103 · Cited by 133
Guidelines for Meta-Analyses of Counseling Psychology Research
Quintana, S. M., & Minami, T. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286991 · Cited by 83
Model modification in structural equation modeling by imposing constraints
Chou, C., & Bentler, P. M. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(02)00097-x · Cited by 64
Review of Principles of Educational and Psychological Testing 3rd ed.
Hays, W. L. (1984) · Contemporary Psychology · doi:10.1037/022838 · Cited by 57
Cattell, R. B. (1974) · Australian Journal of Psychology · doi:10.1080/00049537408255223 · Cited by 43
Pathways to the Future: Analyzing the Contents of a Content Analysis
Worthington, R. L., & Navarro, R. L. (2003) · The Counseling Psychologist · doi:10.1177/0011000002239402 · Cited by 16
Self-citation
Research works printed but not linked: 15
- 10.1037/0033-2909.112.3.400 · doi:10.1037/0033-2909.112.3.400 · Paper page
- 10.1037/0033-2909.88.3.588 · DOI · Paper page
- 10.1037/0022-0167.34.4.425 · doi:10.1037/0022-0167.34.4.425 · Paper page
- Gorsuch, R. L., 1983, Factor analysis
- Jöreskog, K. G., 1981, LISREL V: Analysis of linear structural relations by the method of maximum likelihood
- MacCallum, R. C., 1986, Psychological Bulletin, 107, 247
- 10.1037/0033-2909.111.3.490 · DOI · Paper page
- 10.1037/0033-2909.114.1.185 · doi:10.1037/0033-2909.114.1.185 · Paper page
- 10.1037/0033-2909.103.3.391 · doi:10.1037/0033-2909.103.3.391 · Paper page
- 10.1037/0033-2909.107.2.247 · doi:10.1037/0033-2909.107.2.247 · Paper page
- 10.1037/0033-2909.105.3.430 · DOI · Paper page
- Steiger, J. H., 1989, EzPATH: A supplementary module for SYSTAT and SYGRAPH
- Steiger, J. H., annual meeting of the Psychometric Society
- 10.1037/0022-0167.34.4.414 · doi:10.1037/0022-0167.34.4.414 · Paper page
- 10.1037/0033-2909.99.3.432 · DOI · Paper page
Cited by
Reporting practices in confirmatory factor analysis: an overview and some recommendations
Jackson, D. L., Gillaspy, J. A., & Purc-Stephenson, R. (2009) · Psychological Methods · doi:10.1037/a0014694 · Cited by 2,242
The Body Appreciation Scale-2: Item refinement and psychometric evaluation
Tylka, T. L., & Wood-Barcalow, N. L. (2015) · Body Image · doi:10.1016/j.bodyim.2014.09.006 · Cited by 1,298
Ten Steps in Scale Development and Reporting: A Guide for Researchers
Carpenter, S. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2017.1396583 · Cited by 1,141
“…of factors because the number of factors above one is greatly related to the total number of variables included a model, which researchers argue leads to the extraction of too many or too few factors (Fabrigar et al., 1999; Goldberg & Velicer, 2006; Kline, 2013; Worthington & Whittaker, 2006).”
BackgroundMethodInfluentialScience Education International
(2019) · Science education international · doi:10.33828/sei · Cited by 1,069
Current Methodological Considerations in Exploratory and Confirmatory Factor Analysis
Schmitt, T. A. (2011) · Journal of Psychoeducational Assessment · doi:10.1177/0734282911406653 · Cited by 978
Metadata
- Authors
- Worthington, R. L., & Whittaker, T. A.
- Affiliations
- University of Missouri (US)
- The University of Texas at Austin (US)
- Corresponding author
- Worthington, Roger L.
- Venue
- The Counseling Psychologist
- Publisher
- SAGE Publications
- Published
- 2006-11
- Volume
- 34
- Issue
- 6
- Pages
- 806-838
- ISSN
- 0011-0000 (ISSN-L)
- 1552-3861
- OpenAlex
- W2159466611
- Cited by
- 4,087
- References
- 95
- Citations by year
- 2026: 609 · 2025: 482 · 2024: 356 · 2023: 317 · 2022: 304 · 2021: 349 · 2020: 278 · 2019: 208 · 2018: 193 · 2017: 170 · 2016: 146 · 2015: 158 · 2014: 150 · 2013: 96 · 2012: 74
- FWCI
- 29.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- The Counseling Psychologist (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- scale development, factor analysis, exploratory factor analysis, confirmatory factor analysis, best practices, sample characteristics, extraction methods, rotation method, factor retention, counseling psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.