The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index.
A Review of Key Likert Scale Development Advances: 1995–2019
Open accessAbstract
Developing self-report Likert scales is an essential part of modern psychology. However, it is hard for psychologists to remain apprised of best practices as methodological developments accumulate. To address this, this current paper offers a selective review of advances in Likert scale development that have occurred over the past 25 years. We reviewed six major measurement journals (e.g., Psychological Methods, Educational, and Psychological Measurement) between the years 1995–2019 and identified key advances, ultimately including 40 papers and offering written summaries of each. We supplemented this review with an in-depth discussion of five particular advances: (1) conceptions of construct validity, (2) creating better construct definitions, (3) readability tests for generating items, (4) alternative measures of precision [e.g., coefficient omega and item response theory (IRT) information], and (5) ant colony optimization (ACO) for creating short forms. The Supplementary Material provides further technical details on these advances and offers guidance on software implementation. This paper is intended to be a resource for psychological researchers to be informed about more recent psychometric progress in Likert scale creation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to a 5-point scale, a 7-point scale provides greater variability and precision in capturing the intensity of responses while still being easy for participants to understand (Croasmun & Ostrom, 2011). Research suggests that 7-point scales improve reliability and validity by allowing respondents to express more varied opinions without overwhelming them with too many options (Jebb et al., 20”
From The Role of False Self-Presentation and Social Comparison in Excessive Social Media Use (2025) · cites this paper as Jebb et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · cited 1×
“(Cronbach and Meehl, 1955), which is the perspective endorsed by Clark and Watson (1995, 2019) and standards set forth by governing agencies for the North American educational and psychological measurement supracommunity (Newton and Shaw, 2013).”
Background(1932) · Medical Entomology and Zoology · Cited by 9,568
Statistical Theories of Mental Test Scores
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,022
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,076
Dunn, T. J., Baguley, T., & Brunsden, V. (2014) · British Journal of Psychology · doi:10.1111/bjop.12046 · Cited by 3,596
Dorigo, M., & Stützle, T. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/1290.001.0001 · Cited by 3,330
Optimization, Learning and Natural Algorithms
Dorigo, M. (1992) · Medical Entomology and Zoology · Cited by 3,307
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
Thanks coefficient alpha, we’ll take it from here.
McNeish, D. (2018) · Psychological Methods · doi:10.1037/met0000144 · Cited by 2,533
Henson, R. K., & Roberts, J. K. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282485 · Cited by 2,454
Content validity in psychological assessment: A functional approach to concepts and methods.
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995) · Psychological Assessment · doi:10.1037//1040-3590.7.3.238 · Cited by 2,387
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,674
Much ado about grit: A meta-analytic synthesis of the grit literature
Credé, M., Tynan, M. C., & Harms, P. D. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000102 · Cited by 1,653
Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2007.00051.x · Cited by 1,629
My Current Thoughts on Coefficient Alpha and Successor Procedures
Cronbach, L. J., & Shavelson, R. J. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164404266386 · Cited by 1,606
Constructing validity: New developments in creating objective measuring instruments.
Clark, L. A., & Watson, D. (2019) · Psychological Assessment · doi:10.1037/pas0000626 · Cited by 1,470
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005) · Psychometrika · doi:10.1007/s11336-003-0974-7 · Cited by 1,439
Anderson, J. C., & Gerbing, D. W. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.5.732 · Cited by 1,038
Initial Scale Development: Sample Size for Pilot Studies
Johanson, G. A., & Brooks, G. P. (2010) · Educational and Psychological Measurement · doi:10.1177/0013164409355692 · Cited by 1,012
Interpretation of Educational Measurements
Freeman, F. S., & Kelley, T. L. (1928) · The American Journal of Psychology · doi:10.2307/1414476 · Cited by 877
Test validity and the ethics of assessment.
Messick, S. (1980) · American Psychologist · doi:10.1037/0003-066x.35.11.1012 · Cited by 862
Factor analysis and scale revision.
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.3.287 · Cited by 855
Edelen, M. O., & Reeve, B. B. (2007) · Quality of Life Research · doi:10.1007/s11136-007-9198-0 · Cited by 824
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016) · Organizational Research Methods · doi:10.1177/1094428115624965 · Cited by 712
Graham, J. M. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164406288165 · Cited by 687
Ferrando, P. J., & Lorenzo-Seva, U. (2018) · Educational and Psychological Measurement · doi:10.1177/0013164417719308 · Cited by 581
Hunsley, J., & Meyer, G. J. (2003) · Psychological Assessment · doi:10.1037/1040-3590.15.4.446 · Cited by 561
Argument-Based Approach to Validity
(2022) · The SAGE Encyclopedia of Research Design · doi:10.4135/9781071812082.n28 · Cited by 556
Quantifying construct validity: Two simple measures.
Westen, D., & Rosenthal, R. (2003) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.84.3.608 · Cited by 542
Reliability from α to ω: A tutorial.
Revelle, W., & Condon, D. M. (2019) · Psychological Assessment · doi:10.1037/pas0000754 · Cited by 506
DeMars, C. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195377033.001.0001 · Cited by 497
Raykov, T. (1997) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3204_2 · Cited by 481
Measuring the Mind: Conceptual Issues in Contemporary Psychometrics
Borsboom, D. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490026 · Cited by 477
Coefficient Alpha and the Reliability of Composite Measurements
Novick, M. R., & Lewis, C. (1967) · Psychometrika · doi:10.1007/bf02289400 · Cited by 466
COEFFICIENT ALPHA AND THE RELIABILITY OF COMPOSITE MEASUREMENTS
Novick, M. R., & Lewis, C. (1966) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1966.tb00356.x · Cited by 446
Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019) · Psychological Assessment · doi:10.1037/pas0000648 · Cited by 441
The dangers of poor construct conceptualization
MacKenzie, S. B. (2003) · Journal of the Academy of Marketing Science · doi:10.1177/0092070303031003011 · Cited by 426 · cited 2×
“The advice in this table was taken from Mackenzie (2003), Locke (2012), and Podsakoff et al. (2016). the full scale measures each region of the latent trait.”
BackgroundWeng, L.-J. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164404268674 · Cited by 410
An Analysis of Variance Approach to Content Validation
Hinkin, T. R., & Tracey, J. B. (1999) · Organizational Research Methods · doi:10.1177/109442819922004 · Cited by 403
Shaffer, J. A., DeGeest, D., & Li, A. (2016) · Organizational Research Methods · doi:10.1177/1094428115598239 · Cited by 393
Vogt, D. S., King, D. W., & King, L. A. (2004) · Psychological Assessment · doi:10.1037/1040-3590.16.3.231 · Cited by 383
Schmidt, F. L., Le, H., & Ilies, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.206 · Cited by 362
Factor analysis in psychological assessment research: Common pitfalls and recommendations.
Sellbom, M., & Tellegen, A. (2019) · Psychological Assessment · doi:10.1037/pas0000623 · Cited by 346
Incremental Validity : a Recommendation
Sechrest, L. (1963) · Educational and Psychological Measurement · doi:10.1177/001316446302300113 · Cited by 345
Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases.
Binning, J. F., & Barrett, G. V. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.3.478 · Cited by 344
Probabilistic behaviour in ants: A strategy of errors?
Deneubourg, J., Pasteéls, J. M., & Verhaeghe, J. C. (1983) · Journal of Theoretical Biology · doi:10.1016/s0022-5193(83)80007-1 · Cited by 335
Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000406 · Cited by 328
Incremental Validity of New Clinical Assessment Measures.
Haynes, S. N., & Lench, H. C. (2003) · Psychological Assessment · doi:10.1037/1040-3590.15.4.456 · Cited by 324
Improving comprehension for cancer patients with low literacy skills: strategies for clinicians
Doak, C. C., Doak, L. G., Friedell, G. H., & Meade, C. D. (1998) · CA: A Cancer Journal for Clinicians · doi:10.3322/canjclin.48.3.151 · Cited by 306
On Construct Validity: Issues of Method and Measurement.
Smith, G. T. (2005) · Psychological Assessment · doi:10.1037/1040-3590.17.4.396 · Cited by 290
Benjamin, R. G. (2012) · Educational Psychology Review · doi:10.1007/s10648-011-9181-8 · Cited by 281
Stamp collecting versus science: Validation as hypothesis testing.
Landy, F. J. (1986) · American Psychologist · doi:10.1037//0003-066x.41.11.1183 · Cited by 263
Kelley, K., & Pornprasertmanit, S. (2016) · Psychological Methods · doi:10.1037/a0040086 · Cited by 259
CAPTION-ing the situation: A lexically-derived taxonomy of psychological situation characteristics.
Parrigon, S., Woo, S. E., Tay, L., & Wang, T. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000111 · Cited by 250
Self-citationTheory of mental and social measurements.
Thorndike, E. L. (1904) · doi:10.1037/13283-000 · Cited by 243
Item Response Theory: Fundamentals, Applications, and Promise in Psychological Research
Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2005.00342.x · Cited by 232
Olderbak, S., Wilhelm, O., Olaru, G., Geiger, M., Brenneman, M. W., & Roberts, R. D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01503 · Cited by 228
Incremental Validity Principles in Test Construction.
Smith, G. T., Fischer, S., & Fister, S. M. (2003) · Psychological Assessment · doi:10.1037/1040-3590.15.4.467 · Cited by 205
On the Shortcomings of Shortened Tests: A Literature Review
Kruyen, P. M., Emons, W. H. M., & Sijtsma, K. (2013) · International Journal of Testing · doi:10.1080/15305058.2012.703734 · Cited by 203
The attenuation paradox in test theory.
Loevinger, J. (1954) · Psychological Bulletin · doi:10.1037/h0058543 · Cited by 198
Zhang, Z., & Yuan, K.-H. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164415594658 · Cited by 186
Measurement precision in test score and item response models.
Mellenbergh, G. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.293 · Cited by 165
Improving Measurement Precision of Test Batteries Using Multidimensional Item Response Models.
Wang, W.-C., Chen, P.-H., & Cheng, Y.-Y. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.1.116 · Cited by 159
Meta-Heuristics in Short Scale Construction: Ant Colony Optimization and Genetic Algorithm
Schroeders, U., Wilhelm, O., & Olaru, G. (2016) · PLoS ONE · doi:10.1371/journal.pone.0167110 · Cited by 145
Leite, W. L., Huang, I., & Marcoulides, G. A. (2008) · Multivariate Behavioral Research · doi:10.1080/00273170802285743 · Cited by 143
Methods matter: Testing competing models for designing short-scale Big-Five assessments
Olaru, G., Witthöft, M., & Wilhelm, O. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2015.09.001 · Cited by 137
Ravens‐Sieberer, U., Devine, J., Bevans, K. B., Riley, A. W., Moon, J., Salsman, J. M., & Forrest, C. B. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.018 · Cited by 125
Conceptual Versus Empirical Distinctions Among Constructs: Implications for Discriminant Validity
Harter, J. K., & Schmidt, F. L. (2008) · Industrial and Organizational Psychology · doi:10.1111/j.1754-9434.2007.00004.x · Cited by 122
Cizek, G. (2012) · Psychological methods · doi:10.1037/a0026975 · Cited by 118
Why the resistance to statistical innovations? Bridging the communication gap.
Sharpe, D. (2013) · Psychological Methods · doi:10.1037/a0034177 · Cited by 118
Standards for talking and thinking about validity.
Newton, P. E., & Shaw, S. D. (2013) · Psychological Methods · doi:10.1037/a0032969 · Cited by 105
Variation in the Readability of Items Within Surveys
Calderón, J. L. P., Morales, L. S., Liu, H., & Hays, R. D. (2006) · American Journal of Medical Quality · doi:10.1177/1062860605283572 · Cited by 103
Hardy, B., & Ford, L. R. (2014) · Organizational Research Methods · doi:10.1177/1094428113520185 · Cited by 88
McPherson, J., & Mohr, P. B. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.120 · Cited by 81
Model Specification Searches Using Ant Colony Optimization Algorithms
Marcoulides, G. A., & Drezner, Z. (2003) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1001_8 · Cited by 79
A coefficient alpha for test-retest data.
Green, S. B. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.1.88 · Cited by 79
Zhang, X., & Savalei, V. (2015) · Educational and Psychological Measurement · doi:10.1177/0013164415596421 · Cited by 79
Cooksey, R., & Soutar, G. N. (2005) · Organizational Research Methods · doi:10.1177/1094428105283939 · Cited by 69
Does the rose still smell as sweet? Item variability across test forms and revisions.
Knowles, E. S., & Condon, C. A. (2000) · Psychological Assessment · doi:10.1037//1040-3590.12.3.245 · Cited by 66
ICLUST: A cluster analytic approach to exploratory and confirmatory scale construction
Revelle, W. (1978) · Behavior Research Methods · doi:10.3758/bf03205389 · Cited by 62
Personality measurement and the ethics of assessment.
Messick, S. (1965) · American Psychologist · doi:10.1037/h0021712 · Cited by 59
Establishing Construct Continua in Construct Validation: The Process of Continuum Specification
Tay, L., & Jebb, A. T. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918775707 · Cited by 59
Self-citationValidity in Psychological Testing and Scientific Realism
Hood, S. B. (2009) · Theory & Psychology · doi:10.1177/0959354309336320 · Cited by 56
Methods for Estimating Item-Score Reliability
Zijlmans, E. A. O., van der Ark, L. A., Tijmstra, J., & Sijtsma, K. (2018) · Applied Psychological Measurement · doi:10.1177/0146621618758290 · Cited by 56
Construct validity vs. concept validity
Locke, E. A. (2011) · Human Resource Management Review · doi:10.1016/j.hrmr.2011.11.008 · Cited by 52
Janssen, A. B., Schultze, M., & Grötsch, A. (2015) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000299 · Cited by 47
TaskMaster: A Tool for Determining When Subjects Are on Task
Permut, S., Fisher, M., & Oppenheimer, D. M. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919838479 · Cited by 44
Children with alcohol misusing parents
Zeitlin, H. (1994) · British Medical Bulletin · doi:10.1093/oxfordjournals.bmb.a072872 · Cited by 43
Barchard, K. A. (2011) · Psychological Methods · doi:10.1037/a0023351 · Cited by 39
Inferences from personnel tests and their validity.
Lawshe, C. H. (1985) · Journal of Applied Psychology · doi:10.1037//0021-9010.70.1.237 · Cited by 37
Weijters, B., Beuckelaer, A. D., & Baumgartner, H. (2014) · Applied Psychological Measurement · doi:10.1177/0146621614531850 · Cited by 29
A New Method for Analyzing Content Validity Data Using Multidimensional Scaling
Li, X., & Sireci, S. G. (2013) · Educational and Psychological Measurement · doi:10.1177/0013164412473825 · Cited by 21
An External Validity Approach for Assessing Essential Unidimensionality in Correlated-Factor Models
Ferrando, P. J., & Lorenzo‐Seva, U. (2019) · Educational and Psychological Measurement · doi:10.1177/0013164418824755 · Cited by 20
Peter, S. C., Whelan, J. P., Pfund, R. A., & Meyers, A. W. (2018) · Psychological Assessment · doi:10.1037/pas0000610 · Cited by 18
On Studying Common Factor Variance in Multiple-Component Measuring Instruments
Raykov, T., & Pohl, S. (2012) · Educational and Psychological Measurement · doi:10.1177/0013164412458673 · Cited by 16
An argument-based approach to validity in evaluation
Wallace, T. L. (2011) · Evaluation · doi:10.1177/1356389011410522 · Cited by 12
Raykov, T., Marcoulides, G. A., & Tong, B. (2015) · Educational and Psychological Measurement · doi:10.1177/0013164415604705 · Cited by 11
Krabbe, P. F. M. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-801504-9.00010-6 · Cited by 8
Cited by
The Role of False Self-Presentation and Social Comparison in Excessive Social Media Use
Nor, N., Iqbal, N., & Shaari, A. (2025) · Behavioral Sciences · doi:10.3390/bs15050675 · Cited by 85
Pedro, A. R., Dias, M. B., Laranjo, L., Cunha, A. S., & Cordeiro, J. V. (2023) · PLOS ONE · doi:10.1371/journal.pone.0290613 · Cited by 74
“With the purpose of evaluating the self-perceived command of digital technologies and knowledge about AI, we questioned respondents to estimate their literacy levels according to a 6-level Likert scale [63, 64] (Fig 2).”
MethodBeyari, H., & Hashem, T. (2025) · Behavioral Sciences · doi:10.3390/bs15050700 · Cited by 53
Airways Type-2 Related Disorders: Multiorgan, Systemic or Syndemic Disease?
Giombi, F., Pace, G. M., Pirola, F., Cerasuolo, M., Ferreli, F., Mercante, G., Spriano, G., Canonica, G. W., Heffler, E., Ferri, S., et al. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25020730 · Cited by 25
The Impact of Fintech on Financial Institutions: The Case of Georgia
Lashkhi, M., Charaia, V., Boyarchuk, A., & Ebralidze, L. (2022) · TalTech Journal of European Studies · doi:10.2478/bjes-2022-0010 · Cited by 9
Metadata
- Authors
- Jebb, A. T., Ng, V., & Tay, L.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2021-05-04
- Volume
- 12
- Pages
- 637547-637547
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W3157897384
- Cited by
- 770
- References
- 102
- FWCI
- 60.43
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.