The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Ten Steps in Scale Development and Reporting: A Guide for Researchers
Abstract
Scale development involves numerous theoretical, methodological, and statistical competencies. Despite the central role that scales play in our predictions, scholars often apply measurement building procedures that are inconsistent with best practices. The defaults in statistical programs, inadequate training, and numerous evaluation points can lead to improper practices. Based on a quantitative content analysis of communication journal articles, scholars have improved very little in the communication of their scale development decisions and practices. To address these reoccurring issues, this article breaks down and recommends 10 steps to follow in the scale development process for researchers unfamiliar with the process. Furthermore, the present research makes a unique contribution by overviewing procedures scholars should employ to develop their dimensions and corresponding items. The overarching objective is to encourage the adoption of scale development best practices that yield stronger concepts, and in the long run, a more stable foundation of knowledge.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As pointed out by Carpenter (2018), it is common in scale development to remove items.”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carpenter (2018)
In the citing paper
“Besides, it is important to meet the requirement of an optimal scale length to ensure participants' motivation (Carpenter, 2018).”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carpenter, 2018
In the citing paper
“The proposed questionnaire was developed following recommendations from various methodologists (e.g., Carpenter, 2018;DeVellis and Thorpe, 2021;Worthington and Whittaker, 2006).”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carpenter, 2018
In the citing paper
“Measuring an abstract construct such as cognitive load with a small number of items is problematic (Carpenter, 2018).”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carpenter, 2018
In the citing paper
“These items were not deleted to maintain the theoretical convergence of the construct (e.g., Carpenter, 2018).”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Carpenter, 2018
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 79,882
(1991) · Cited by 15,554
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Costello, A. B., & Osborne, J. (2005) · University of Massachusetts Amherst · doi:10.7275/jyj1-4868 · Cited by 10,297
(2007) · Cited by 10,092
Multivariate Data Analysis, 6th ed
Tatham, R. L., Anderson, R. E., Babin, B. J., Black, W. C., & Hair, J. F. (2006) · Cited by 9,807
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,965
Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006) · The Journal of Educational Research · doi:10.3200/joer.99.6.323-338 · Cited by 6,937 · cited 1×
“Several content analyses have been done outside the field of communication evaluating such practices (Jackson, Gillaspy, & PureStephenson, 2009; Schreiber et al., 2006; Worthington &Whittaker, 2006).”
BackgroundConstructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567
Pallant, J. (2020) · Routledge eBooks · doi:10.4324/9781003117407 · Cited by 6,034
A Second Generation Little Jiffy
Kaiser, H. F. (1970) · Psychometrika · doi:10.1007/bf02291817 · Cited by 4,855
O’connor, B. P. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200807 · Cited by 4,116
Scale Development Research: A Content Analysis and Recommendations for Best Practices
Worthington, R. L., & Whittaker, T. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288127 · Cited by 4,087
Factor analysis in the development and refinement of clinical assessment instruments.
Floyd, F. J., & Widaman, K. F. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.286 · Cited by 3,970
Practical Assessment, Research & Evaluation, 2000-2001.
Rudner, L. M., & Schafer, W. D. (2001) · Practical assessment, research & evaluation · Cited by 3,905
Conventional wisdom on measurement: A structural equation perspective.
Bollen, K., & Lennox, R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.305 · Cited by 3,618
A First Course in Factor Analysis
Price, B., Comrey, A. L., & Lee, H. B. (1993) · Technometrics · doi:10.1080/00401706.1993.10485363 · Cited by 3,512
Exploratory and confirmatory factor analysis: Understanding concepts and applications.
Thompson, B. (2004) · doi:10.1037/10694-000 · Cited by 3,469
Applied Multivariate Research: Design and Interpretation
Meyers, L. S., Gamst, G. C., & GUARINO, A. J. (2017) · Sage eBooks · doi:10.4135/9781071802687 · Cited by 2,883
Relation of sample size to the stability of component patterns.
Guadagnoli, E., & Velicer, W. F. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.2.265 · Cited by 2,880
(2007) · Sage eBooks · doi:10.4135/9781412952644.n300 · Cited by 2,589
Henson, R. K., & Roberts, J. K. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282485 · Cited by 2,454
Determining the number of components from the matrix of partial correlations
Velicer, W. F. (1976) · Psychometrika · doi:10.1007/bf02293557 · Cited by 2,438
Critchley, F., Murtagh, F. D., & Heck, A. (1988) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983024 · Cited by 2,332
Reporting practices in confirmatory factor analysis: an overview and some recommendations
Jackson, D. L., Gillaspy, J. A., & Purc-Stephenson, R. (2009) · Psychological Methods · doi:10.1037/a0014694 · Cited by 2,242
Making Sense of Factor Analysis
Pett, M., Lackey, N., & Sullivan, J. (2003) · doi:10.4135/9781412984898 · Cited by 2,168
THE APPLICATION OF EXPLORATORY FACTOR ANALYSIS IN APPLIED PSYCHOLOGY: A CRITICAL REVIEW AND ANALYSIS
Ford, J. K., MacCallum, R. C., & Tait, M. (1986) · Personnel Psychology · doi:10.1111/j.1744-6570.1986.tb00583.x · Cited by 1,735
Minimum Sample Size Recommendations for Conducting Factor Analyses
Mundfrom, D. J., Shaw, D. G., & Ke, T. L. (2005) · International Journal of Testing · doi:10.1207/s15327574ijt0502_4 · Cited by 1,704
Sullivan, J., Pett, M. A., & Lackey, N. R. (2003) · Medical Entomology and Zoology · Cited by 1,692
Horst, P. (1948) · Psychological Bulletin · doi:10.1037/h0052785 · Cited by 1,555
Objective Tests as Instruments of Psychological Theory
Loevinger, J. (1957) · Psychological Reports · doi:10.2466/pr0.1957.3.3.635 · Cited by 1,476
Advanced and Multivariate Statistical Methods: Practical Application and Interpretation
Mertler, C. A., & Reinhart, R. V. (2001) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,469
Uses of factor analysis in counseling psychology research.
Tinsley, H. E. A., & Tinsley, D. J. (1987) · Journal of Counseling Psychology · doi:10.1037/0022-0167.34.4.414 · Cited by 1,286
Practical Considerations for Using Exploratory Factor Analysis in Educational Research
Beavers, A. S., Lounsbury, J. W., Richards, J., Huck, S. W., Skolits, G., & Esquivel, S. L. (2020) · doi:10.7275/qv2q-rk76 · Cited by 1,232
Exploratory Factor Analysis: Its Role in Item Analysis
Gorsuch, R. L. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6803_5 · Cited by 1,062
A Review and Evaluation of Exploratory Factor Analysis Practices in Organizational Research
Conway, J. M., & Huffcutt, A. I. (2003) · Organizational Research Methods · doi:10.1177/1094428103251541 · Cited by 1,025
PROMAX: A QUICK METHOD FOR ROTATION TO OBLIQUE SIMPLE STRUCTURE
Hendrickson, A. E., & White, P. O. (1964) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1964.tb00244.x · Cited by 1,009
Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000) · Springer eBooks · doi:10.1007/978-1-4615-4397-8_3 · Cited by 886
Best Practices in Exploratory Factor Analysis
Osborne, J. W., Costello, A. B., & Kellow, J. T. (2008) · Sage eBooks · doi:10.4135/9781412995627.d8 · Cited by 872
Repairing Tom Swift's Electric Factor Analysis Machine
Preacher, K. J., & MacCallum, R. C. (2003) · Understanding Statistics · doi:10.1207/s15328031us0201_02 · Cited by 860
Methods for Testing and Evaluating Survey Questionnaires
(2004) · doi:10.1002/0471654728 · Cited by 855
Factor analysis and scale revision.
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.3.287 · Cited by 855
Figuring Out Factors: The Use and Misuse of Factor Analysis
Streiner, D. L. (1994) · The Canadian Journal of Psychiatry · doi:10.1177/070674379403900303 · Cited by 835
Cognitive interviewing: verbal data in the design and pretesting of questionnaires
Drennan, J. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02579.x · Cited by 769
Differentiating Normal and Abnormal Personality
Strack, S., & Lorr, M. (1994) · Cited by 734
Factors Influencing Four Rules For Determining The Number Of Components To Retain
Zwick, W. R., & Velicer, W. F. (1982) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1702_5 · Cited by 519
A methodology for construct development in MIS research
Lewis, B. R., Templeton, G. F., & Byrd, T. A. (2005) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000552 · Cited by 500
Evaluating the Use of Exploratory Factor Analysis in Developmental Disability Psychological Research
Norris, M., & Lecavalier, L. (2009) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-009-0816-2 · Cited by 493
Riffe, D., Lacy, S. R., Watson, B. R., & Fico, F. G. (2019) · Routledge eBooks · doi:10.4324/9780429464287 · Cited by 469
Riff, D. (2014) · Routledge eBooks · doi:10.4324/9780203551691 · Cited by 464
Myers, T. A. (2011) · Communication Methods and Measures · doi:10.1080/19312458.2011.624490 · Cited by 459
Exploratory and Confirmatory Factor Analysis
(2012) · Routledge eBooks · doi:10.4324/9780203843130-15 · Cited by 442
An Investigation of the Parallel Analysis Criterion for Determining the Number of Common Factors
Humphreys, L. G., & Montanelli, R. G. (1975) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1002_5 · Cited by 391
Component analysis versus common factor analysis: A Monte Carlo study.
Snook, S. C., & Gorsuch, R. L. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.1.148 · Cited by 354
Statistical Analysis: An Interdisciplinary Introduction to Univariate & Multivariate Methods
Kachigan, S. K. (1986) · Medical Entomology and Zoology · Cited by 352
The Use of Exploratory Factor Analysis and Principal Components Analysis in Communication Research
Park, H. S., Dailey, R. M., & Lemus, D. R. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00824.x · Cited by 286
Applied Quantitative Analysis in Education and the Social Sciences
(2013) · Routledge eBooks · doi:10.4324/9780203108550 · Cited by 259
Pre-Testing in Questionnaire Design: A Review of the Literature and Suggestions for Further Research
Reynolds, N. L., Diamantopoulos, A., & Schlegelmilch, B. B. (1993) · Market Research Society Journal · doi:10.1177/147078539303500202 · Cited by 247
THE USE AND ABUSE OF FACTOR ANALYSIS IN COMMUNICATION RESEARCH
McCroskey, J. C., & Young, T. J. (1979) · Human Communication Research · doi:10.1111/j.1468-2958.1979.tb00651.x · Cited by 242
Determining Parallel Analysis Criteria
Watkins, M. W. (2005) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1162354020 · Cited by 224
The observation to variable ratio in factor analysis.
Barrett, P. T., & Kline, P. (1981) · Cited by 211
Exploratory Factor Analysis, Theory Generation, and Scientific Method
Haig, B. D. (2005) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr4003_2 · Cited by 206
Classical and Modern Methods of Psychological Scale Construction
Simms, L. J. (2007) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2007.00044.x · Cited by 206
Communication Research Statistics
C.Reinard, J. (2006) · Sage eBooks · doi:10.4135/9781412983693 · Cited by 205
Principles of Exploratory Factor Analysis.
Goldberg, L. R., & Velicer, W. F. (2006) · Cited by 145
Statistical Analysis: An Interdisciplinary Introduction to Univariate and Multivariate Methods
Williamson, J. B., & Kachigan, S. K. (1988) · Teaching Sociology · doi:10.2307/1317431 · Cited by 140
OBJECTIVE TESTS AS INSTRUMENTS OF PSYCHOLOGICAL THEORY: Monograph Supplement 9
Loevinger, J. (1957) · Psychological Reports · doi:10.2466/pr0.3.7.635-694 · Cited by 130
A Content Analysis of the Media Effects Literature
Potter, W. J., & Riddle, K. E. (2007) · Journalism & Mass Communication Quarterly · doi:10.1177/107769900708400107 · Cited by 125
Mass Communication Research Trends from 1980 to 1999
Kamhawi, R., & Weaver, D. B. (2003) · Journalism & Mass Communication Quarterly · doi:10.1177/107769900308000102 · Cited by 121
The Status of Online Public Relations Research: An Analysis of Published Articles in 1992–2009
Ye, L., & Ki, E. (2012) · Journal of Public Relations Research · doi:10.1080/1062726x.2012.723277 · Cited by 116
Ki, E., & Shin, J. (2006) · Public Relations Review · doi:10.1016/j.pubrev.2006.02.019 · Cited by 95
The Desirability of Using Confirmatory Factor Analysis on Published Scales
Levine, T. R., Hullett, C. R., Turner, M. M., & Lapinski, M. K. (2006) · Communication Research Reports · doi:10.1080/08824090600962698 · Cited by 77
The Relation of Sample Size to the Number of Variables in Using Factor Analysis Techniques
Aleamoni, L. M. (1976) · Educational and Psychological Measurement · doi:10.1177/001316447603600410 · Cited by 62
Evaluating Factor Analysis Decisions for Scale Design in Communication Research
Morrison, J. T. (2009) · Communication Methods and Measures · doi:10.1080/19312450903378917 · Cited by 59
An Open-System Approach to Building Theory in Public Relations
Broom, G. M. (2006) · Journal of Public Relations Research · doi:10.1207/s1532754xjprr1802_4 · Cited by 58
A Status Report on Methods Used in Mass Communication Research
Cooper, R., Potter, W. J., & Dupagne, M. (1993) · Journalism & Mass Communication Educator · doi:10.1177/107769589304800408 · Cited by 55
Research Methods in Mass Communication Research: A Census of Eight Journals 1990–2000
Trumbo, C. W. (2004) · Journalism & Mass Communication Quarterly · doi:10.1177/107769900408100212 · Cited by 50
Carpenter, S., Makhadmeh, N. M., & Thornton, L.-J. (2015) · Communication Education · doi:10.1080/03634523.2015.1041997 · Cited by 46
Self-citationA Cautionary Note on the Use of Principal Components Analysis
Borgatta, E. F., Kercher, K. A., & Stull, D. E. (1986) · Sociological Methods & Research · doi:10.1177/0049124186015001011 · Cited by 42
Carpenter, S., Grant, A. E., & Hoag, A. M. (2014) · Journalism & Mass Communication Educator · doi:10.1177/1077695814551835 · Cited by 42
Self-citationThe Practice of Survey Research
Hillygus, D. S. (2019) · Routledge eBooks · doi:10.4324/9781351054621-2 · Cited by 35
Schein, J. D., & Sivanich, G. (2001) · CRC Press eBooks · doi:10.3109/9780203212950-20 · Cited by 27
Statistical analyses in thejournal of broadcasting, 1970‐76
Wimmer, R. D., & Haynes, R. B. (1978) · Journal of Broadcasting · doi:10.1080/08838157809363881 · Cited by 8
Understanding the Intangibles of Measurement in the Social Sciences1
Viswanathan, M. (2010) · SAGE Publications Ltd eBooks · doi:10.4135/9781446268230.n15 · Cited by 7
Research works printed but not linked: 8
- Barrett P. T., 1981, Personality Study and Group Behavior, 1, 23
- Beavers A. S., 2013, Practical Assessment, Research & Evaluation, 18, 1
- Bollen K., 1991, MIS Quarterly, 28, 305
- Chaffee S. H., 1991, Explication
- Costello A. B., 2005, Practical Assessment, Research & Evaluation, 10, 1
- Gorsuch R. L., 1983, Factor analysis
- Nunnally J. C., 1994, Psychometric theory
- Tabachnick B. G., 2007, Using multivariate statistics
Cited by
Sorted by citations
Chatterjee, S., & Bhattacharjee, K. K. (2020) · Education and Information Technologies · doi:10.1007/s10639-020-10159-7 · Cited by 747
Chatterjee, S., & Kar, A. K. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102103 · Cited by 613
Rana, N. P., Chatterjee, S., Dwivedi, Y. K., & Akter, S. (2021) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1955628 · Cited by 500
Abeele, V. V., Spiel, K., Nacke, L., Johnson, D., & Gerling, K. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.102370 · Cited by 411
Adoption of artificial intelligence-integrated CRM systems in agile organizations in India
Chatterjee, S., Chaudhuri, R., Vrontis, D., Thrassou, A., & Ghosh, S. K. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120783 · Cited by 277
Metadata
- Authors
- Carpenter, S.
- Affiliations
- Michigan State University (US)
- Corresponding author
- Carpenter, Serena
- ORCID
- Carpenter, Serena
- Venue
- Communication Methods and Measures
- Publisher
- Informa UK Limited
- Published
- 2017-11-22
- Volume
- 12
- Issue
- 1
- Pages
- 25-44
- ISSN
- 1931-2458 (ISSN-L)
- 1931-2466
- OpenAlex
- W2768349557
- Cited by
- 1,141
- References
- 86
- Citations by year
- 2026: 208 · 2025: 240 · 2024: 209 · 2023: 130 · 2022: 142 · 2021: 108 · 2020: 67 · 2019: 32 · 2018: 5
- FWCI
- 25.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Communication Methods and Measures (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Keywords
- scale development, measurement, best practices, item development, measurement reliability, measurement validity, communication research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
