The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Number of Response Options, Reliability, Validity, and Potential Bias in the Use of the Likert Scale Education and Social Science Research: A Literature Review
Open accessAbstract
This study reviews 60 papers using a Likert scale and published between 2012 – 2021. Screening for literature review uses the PRISMA method. The data analysis technique was carried out through data extraction, then synthesized in a structured manner using the narrative method. To achieve credible research results at the stage of the data collection and data analysis process, a group discussion forum (FGD) was conducted. The findings show that only 10% of studies use a measurement scale with an even answer choice category (4, 6, 8, or 10 choices). In general, (90%) of research uses a measurement instrument that involves a Likert scale with odd response choices (5, 7, 9, or 11) and the most popular researchers use a Likert scale with a total response of 5 points. The use of a rating scale with an odd number of responses of more than five points (especially on a seven-point scale) is the most effective in terms of reliability and validity coefficients, but if the researcher wants to direct respondents to one side, then a scale with an even number of responses (six points) is possible. more suitable. The presence of response bias and central tendency bias can affect the validity and reliability of the use of the Likert scale instrument.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Methods for the thematic synthesis of qualitative research in systematic reviews
Thomas, J., & Harden, A. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-45 · Cited by 9,312
Likert Scale: Explored and Explained
Joshi, A., Kale, S., Chandel, S., & Pal, D. (2015) · British Journal of Applied Science & Technology · doi:10.9734/bjast/2015/14975 · Cited by 3,870
Analyzing and Interpreting Data From Likert-Type Scales
Sullivan, G. M., & Artino, A. R. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-5-4-18 · Cited by 3,044
Dawes, J. (2008) · International Journal of Market Research · doi:10.1177/147078530805000106 · Cited by 2,107
Preston, C. C., & Colman, A. M. (2000) · Acta Psychologica · doi:10.1016/s0001-6918(99)00050-5 · Cited by 2,106
Taherdoost, H. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.3205040 · Cited by 2,047
Likert scales: how to (ab)use them
Jamieson, S. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.02012.x · Cited by 1,602
Boone, H., & Boone, D. (2012) · Journal of Extension · doi:10.34068/joe.50.02.48 · Cited by 1,407
Kokolakis, S. (2017) · Computers & Security · doi:10.1016/j.cose.2015.07.002 · Cited by 1,229
Bolarinwa, O. (2015) · Nigerian Postgraduate Medical Journal · doi:10.4103/1117-1936.173959 · Cited by 1,148
Effect of the Number of Response Categories on the Reliability and Validity of Rating Scales
Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008) · Methodology · doi:10.1027/1614-2241.4.2.73 · Cited by 843
A Guide for Systematic Reviews: PRISMA
Department of Otorhinolaryngology, H. S. U., & Selcuk, A. A. (2019) · Turkish Archives of Otorhinolaryngology · doi:10.5152/tao.2019.4058 · Cited by 649
Introduction to systematic review and meta-analysis
Ahn, E., & Kang, H. (2018) · Korean Journal of Anesthesiology · doi:10.4097/kjae.2018.71.2.103 · Cited by 628
A measure of second language writing anxiety: Scale development and preliminary validation
Cheng, Y.-S. (2004) · Journal of Second Language Writing · doi:10.1016/j.jslw.2004.07.001 · Cited by 614
The Impact of “No Opinion” Response Options on Data Quality
Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W. M., Kopp, R. J., Mitchell, R. C., Presser, S., Ruud, P. A., Smith, V. K., et al. (2001) · Public Opinion Quarterly · doi:10.1086/341394 · Cited by 546
Qualitative Analysis Techniques for the Review of the Literature
Onwuegbuzie, A., Leech, N., & Collins, K. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2012.1754 · Cited by 527
JBI's Systematic Reviews: Data Extraction and Synthesis
Munn, Z., Tufanaru, C., & Aromataris, E. (2014) · AJN, American Journal of Nursing · doi:10.1097/01.naj.0000451683.66447.89 · Cited by 510
Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019) · Psychological Assessment · doi:10.1037/pas0000648 · Cited by 441
Stuck in the middle: The use and interpretation of mid-points in items on questionnaires
Nadler, J. T., Weston, R., & Voyles, E. C. (2015) · The Journal of General Psychology · doi:10.1080/00221309.2014.994590 · Cited by 396
Use and Misuse of the Likert Item Responses and Other Ordinal Measures
Bishop, P. A., & Herron, R. L. (2015) · International Journal of Exercise Science · doi:10.70252/lanz1453 · Cited by 367
Automating data extraction in systematic reviews: a systematic review
Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0066-7 · Cited by 287
A Comparison of Parametric and Non-Parametric Methods Applied to a Likert Scale
Mircioiu, C., & Atkinson, J. (2017) · Pharmacy · doi:10.3390/pharmacy5020026 · Cited by 286
Kreitchmann, R. S., Abad, F. J., Ponsoda, V., Nieto, M. D., & Morillo, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02309 · Cited by 269
A classification of response scale characteristics that affect data quality: a literature review
DeCastellarnau, A. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0533-4 · Cited by 259
Applied Psychometrics: The Steps of Scale Development and Standardization Process
Kyriazos, T. A., & Stalikas, A. (2018) · Psychology · doi:10.4236/psych.2018.911145 · Cited by 247
Mathes, T., Klaßen, P., & Pieper, D. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0431-4 · Cited by 240 · cited 4×
“Then, the researcher changed the data obtained into new data by filtering the data into several categories (Mathes et al., 2017; Munn et al., 2014; Pedder et al., 2016; Schmidt et al., 2021).”
InfluentialTraditional and Inquiry-Based Learning Pedagogy: A Systematic Critical Review
Language Academy, U., & Khalaf, B. K. (2018) · International Journal of Instruction · doi:10.12973/iji.2018.11434a · Cited by 240
Some thoughts on Likert-type scales
Hartley, J. (2014) · International Journal of Clinical and Health Psychology · doi:10.1016/s1697-2600(14)70040-7 · Cited by 233
The Modified Abbreviated Math Anxiety Scale: A Valid and Reliable Instrument for Use with Children
Carey, E., Hill, F., Devine, A., & Szűcs, D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00011 · Cited by 193
Reporting and Interpreting Scores Derived from Likert-type Scales
Warmbrod, J. R. (2014) · Journal of Agricultural Education · doi:10.5032/jae.2014.05030 · Cited by 174
Some Biases in Likert Scaling Usage and its Correction
Pimentel, J. L. (2019) · International Journal of Sciences: Basic and Applied Research · Cited by 174
An application of item response theory to psychological test development
Zanon, C., Hutz, C. S., Yoo, H., & Hambleton, R. K. (2016) · Psicologia: Reflexão e Crítica · doi:10.1186/s41155-016-0040-x · Cited by 160
Zumsteg, J. M., Cooper, J. S., & Noon, M. S. (2012) · Journal of Industrial Ecology · doi:10.1111/j.1530-9290.2012.00476.x · Cited by 159
Data extraction for complex meta-analysis (DECiMAL) guide
Pedder, H., Sarri, G., Keeney, E., Nunes, V. D., & Dias, S. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0368-4 · Cited by 129
Baka, A., Figgou, L., & Triga, V. (2012) · International Journal of Electronic Governance · doi:10.1504/ijeg.2012.051306 · Cited by 124
The Effect of Labeling and Numbering of Response Scales on the Likelihood of Response Bias
Moors, G., Kieruj, N. D., & Vermunt, J. K. (2014) · Sociological Methodology · doi:10.1177/0081175013516114 · Cited by 123
Requirements engineering for safety-critical systems: A systematic literature review
Martins, L. E. G., & Gorschek, T. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.04.002 · Cited by 107
Data extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106
Jeong, J. S., González‐Gómez, D., Cañada, F. C., Gallego-Picó, A., & Bravo, J. C. (2019) · Journal of Technology and Science Education · doi:10.3926/jotse.530 · Cited by 93
A General Factor-Analytic Procedure for Assessing Response Bias in Questionnaire Measures
Ferrando, P. J., Lorenzo‐Seva, U., & Chico, E. (2009) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510902751374 · Cited by 68
A practical guide to data analysis in general literature reviews
Popenoe, R., Langius‐Eklöf, A., Stenwall, E., & Jervaeus, A. (2021) · Nordic journal of nursing research · doi:10.1177/2057158521991949 · Cited by 68
Reversed items in Likert scales: Filtering out invalid responders
Józsa, K., & Morgan, G. A. (2017) · SZTE Publicatio Repozitórium (University of Szeged) · Cited by 59
Assessing Quality in Systematic Literature Reviews: A Study of Novice Rater Training
Acosta, S. T., Garza, T., Hsu, H.-Y. M., & Goodson, P. (2020) · SAGE Open · doi:10.1177/2158244020939530 · Cited by 58
Altuna, O. K., & Arslan, F. M. (2016) · Siyasal Journal of Political Sciences · doi:10.17124/iusiyasal.320009 · Cited by 57
Tijmstra, J., Bolsinova, M. A., & Jeon, M. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0997-0 · Cited by 55
Lionello, M., Aletta, F., Mitchell, A., & Kang, J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.602831 · Cited by 50
Sangwan, A., Sangwan, A., & Punia, P. (2021) · TechTrends · doi:10.1007/s11528-020-00561-w · Cited by 40
Awareness and minimisation of systematic bias in research
Malone, H., Nicholl, H. M., & Tracey, C. (2014) · British Journal of Nursing · doi:10.12968/bjon.2014.23.5.279 · Cited by 40
Measuring User Experience With 3, 5, 7, or 11 Points: Does It Matter?
Lewis, J. R. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819881312 · Cited by 40
Development of STEM Attitude Scale for Secondary School Students: Validity and Reliability Study
Benek, İ., & Akçay, B. (2019) · International Journal of Education in Mathematics Science and Technology · doi:10.18404/ijemst.509258 · Cited by 38
Xiong, C., Ceja, C. R., Ludwig, C. J. H., et al. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934400 · Cited by 37
A validity and reliability study of the Attitude Scale of Computer Programming Learning (ASCOPL)
Korkmaz, Ö., & Altun, H. (2013) · Mevlana International Journal of Education · doi:10.13054/mije.13.73.4.1 · Cited by 33
Mathematical disposition of students’, teachers, and parents in distance learning: A survey
Ulia, N., & Kusmaryono, I. (2021) · Premiere Educandum Jurnal Pendidikan Dasar dan Pembelajaran · doi:10.25273/pe.v11i1.8869 · Cited by 32
Self-citationMethods to Analyze Likert-Type Data in Educational Technology Research
Chen, L., & Liu, L. (2020) · Journal of Educational Technology Development and Exchange · doi:10.18785/jetde.1302.04 · Cited by 22
The relationship of scale reliability and validity to respondent inconsistency
Biderman, M. D., & Reddock, C. M. (2012) · Personality and Individual Differences · doi:10.1016/j.paid.2011.12.012 · Cited by 21
Guerra, A. L., Gidel, T., & Vezzetti, E. (2016) · HAL · Cited by 15
Zhang, Y., Xu, Q., Lao, J., & Shen, Y. (2021) · Sustainability · doi:10.3390/su132212661 · Cited by 9
Dogan, E. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/1cf3-aq56 · Cited by 3
(2019) · HAL (Le Centre pour la Communication Scientifique Directe)
Cited by
Scientific evidence for clinical text summarization using large language models: scoping review
Bednarczyk, L., Reichenpfader, D., Gaudet-Blavignac, C., Ette, A. K., Zaghir, J., Zheng, Y., Bensahla, A., Bjelogrlic, M., & Lovis, C. (2025) · Journal of Medical Internet Research · doi:10.2196/68998 · Cited by 83
From Topic to Thesis: Critical Engagement with Generative AI in Thesis Preparation
Virkki, O. T., Naeem, U., & Styve, A. (2026) · 2026 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon67543.2026.11574181
Metadata
- Authors
- Kusmaryono, I., Wijayanti, D., & Maharani, H. R.
- Venue
- International Journal of Educational Methodology
- Published
- 2022-11-15
- Volume
- 8
- Issue
- 4
- Pages
- 625-637
- ISSN
- 2469-9632 (ISSN-L)
- OpenAlex
- W4296239696
- Cited by
- 283
- References
- 59
- FWCI
- 25.67
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.