Likert scales, levels of measurement, and the “laws” of statistics
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Reviewers of research reports frequently criticize the choice of statistical methods. While some of these criticisms are well-founded, frequently the use of various parametric methods such as analysis of variance, regression, correlation are faulted because: (a) the sample size is too small, (b) the data may not be normally distributed, or (c) The data are from Likert scales, which are ordinal, so parametric statistics cannot be used. In this paper, I dissect these arguments, and show that many studies, dating back to the 1930s consistently show that parametric statistics are robust with respect to violations of these assumptions. Hence, challenges like those above are unfounded, and parametric methods can be utilized without concern for "getting the wrong answer".
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A6-step Likert scale (1 = Never/Strongly Disagree to 6 = Always/Strongly Agree) was used for most items, providing ordinal data suitable for parametric statistical analysis [34].”
From AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking (2025) · cites this paper as [34]
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 1×
“Cohen (1968) later generalized the formulation to ‘‘weighted kappa’’, to be used with ordinal data such as Likert scales, where the data would be displayed as agreement in a 7 9 7 matrix.”
MethodFleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354 · cited 1×
“Surprisingly, if one proceeds to calculate an intraclass correlation with the same 7-point scale data, the results are mathematically identical, as proven by Fleiss and Cohen (1973).”
MethodThe two disciplines of scientific psychology.
Cronbach, L. J. (1957) · American Psychologist · doi:10.1037/h0043943 · Cited by 2,523
Likert scales: how to (ab)use them
Jamieson, S. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.02012.x · Cited by 1,602 · cited 3×
“To quote a recent article in Medical Education (Jamieson 2004) ‘‘the response categories have a rank order but the intervals between values cannot be presumed equal’’.”
Resolving the 50‐year debate around using and misusing Likert scales
Carifio, J., & Perla, R. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03172.x · Cited by 1,043 · cited 1×
“The first, perhaps the least radical, is that expounded by Carifio and Perla (2008) in their response to Jamieson (2004).”
The effects of violations of assumptions underlying the t test.
Boneau, C. A. (1960) · Psychological Bulletin · doi:10.1037/h0041412 · Cited by 790 · cited 1×
“Boneau (1960) looked at normal, rectangular and exponential distributions and sample sizes of 5 and 15, and showed that 17 of the 20 calculated P -values were between .04 and .07 for a nominal 0.05.”
InfluentialDichotomization of continuous variables: The implications for meta-analysis.
Hunter, J. E., & Schmidt, F. L. (1990) · Journal of Applied Psychology · doi:10.1037//0021-9010.75.3.334 · Cited by 376 · cited 1×
“The finding is neither new nor publishable; other authors have shown similar effects (Suissa 1991; Hunter and Schmidt 1990).”
Measurement scales and statistics: Resurgence of an old misconception.
Gaito, J. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.87.3.564 · Cited by 369
THE ANALYSIS OF VARIANCE IN CASES OF NON-NORMAL VARIATION
PEARSON, E. S. (1931) · Biometrika · doi:10.1093/biomet/23.1-2.114 · Cited by 263 · cited 1×
“Empirical studies of robustness of ANOVA date all the way back to Pearson (1931) who found ANOVA was robust for highly skewed non-normal distributions and sample sizes of 4, 5 and 10.”
Method
Research works printed but not linked: 1
- Pearson, E. S. (1932b). The test of signficance for the correlation coefficient: Some further results. Journal of the American Statistical Association, 27, 424–426. · DOI
Cited by
Analyzing and Interpreting Data From Likert-Type Scales
Sullivan, G. M., & Artino, A. R. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-5-4-18 · Cited by 3,044
A minimal common outcome measure set for COVID-19 clinical research
Marshall, J. C., Murthy, S., Diaz, J., Adhikari, N. K., Angus, D. C., Arabi, Y. M., Baillie, K., Bauer, M., Berry, S., Blackwood, B., et al. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30483-7 · Cited by 1,641
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · Societies · doi:10.3390/soc15010006 · Cited by 1,248
“A 6-step Likert scale (1 = Never/Strongly Disagree to 6 = Always/Strongly Agree) was used for most items, providing ordinal data suitable for parametric statistical analysis [34].”
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · doi:10.2139/ssrn.5082524 · Cited by 1,238
How to analyze Likert and other rating scale data
Harpe, S. E. (2015) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2015.08.001 · Cited by 874
“Likert scales are a common measurement method in educational contexts.(23) His approach involved the presentation”
BackgroundMethod
Metadata
- Authors
- Norman, G.
- Affiliations
- McMaster University (CA)
- Corresponding author
- Norman, Geoff
- ORCID
- Norman, Geoff
- Venue
- Advances in Health Sciences Education
- Published
- 2010-02-10
- Volume
- 15
- Issue
- 5
- Pages
- 625-632
- ISSN
- 1382-4996 (ISSN-L)
- 1573-1677
- OpenAlex
- W2061141931
- Cited by
- 4,850
- References
- 19
- Citations by year
- 2026: 731 · 2025: 496 · 2024: 377 · 2023: 376 · 2022: 418 · 2021: 440 · 2020: 363 · 2019: 308 · 2018: 309 · 2017: 240 · 2016: 187 · 2015: 200 · 2014: 168 · 2013: 135 · 2012: 65
- FWCI
- 100.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Advances in Health Sciences Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- Likert scale, levels of measurement, ordinal data, parametric statistics, robustness to non-normality, sample size, analysis of variance, regression analysis, correlation, statistical assumptions
- MeSH terms
- Analysis of Variance, Educational Measurement, Factor Analysis, Statistical, Humans, Regression Analysis, Statistics as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Likert scales, levels of measurement, and the “laws” of statistics. https://app.cochat.ai/papers/10.1007/s10459-010-9222-y
- The paper
- Norman, G. (2010). Likert scales, levels of measurement, and the “laws” of statistics. Advances in Health Sciences Education. https://doi.org/10.1007/s10459-010-9222-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.