An Empirical Evaluation of the System Usability Scale
The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Abstract
This article presents nearly 10 year's worth of System Usability Scale (SUS) data collected on numerous products in all phases of the development lifecycle. The SUS, developed by CitationBrooke (1996), reflected a strong need in the usability community for a tool that could quickly and easily collect a user's subjective rating of a product's usability. The data in this study indicate that the SUS fulfills that need. Results from the analysis of this large number of SUS scores show that the SUS is a highly robust and versatile tool for usability professionals. The article presents these results and discusses their implications, describes nontraditional uses of the SUS, explains a proposed modification to the SUS to provide an adjective rating that correlates with a given score, and provides details of what constitutes an acceptable SUS score.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Program usability is assessed using an adapted and abbreviated version of the System Usability Scale [58].”
In the citing paper
“Due in part to these issues, almost all higher education institutions have existing mechanisms to help freshmen with the transition to college, which can provide a framework for implementing a new prevention program.32”
From Feasibility of a Prototype Web-Based Acceptance and Commitment Therapy Prevention Program for College Students. (2014) · cites this paper as [32]
In the citing paper
“The SUS has been shown to be applicable across many domains and technologies [17, 18].”
From Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study (2019) · cites this paper as [17]
In the citing paper
“Among the most common measures for evaluating a technology interface are questionnaires like the System Usability Scale (Bangor et al., 2008).”
From Designing for Motivation, Engagement and Wellbeing in Digital Experience (2018) · cites this paper as Bangor et al., 2008
In the citing paper
“The Berger and Sandoval studies included evidence-based instruments for end user acceptance using Client Satisfaction Questionnaire (CSQ-8) [61] and System Usability Scale (SUS) [62].”
From Patient Engagement with Conversational Agents in Health Applications 2016–2022: A Systematic Review and Meta-Analysis (2024) · cites this paper as [62]
17 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429
Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting
Kluge, A., Badura, B., Urbas, L., & Burkolter, D. (2010) · Cited by 1,127
Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?
Frøkjær, E., Hertzum, M., & Hornbæk, K. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332455 · Cited by 650
SUMI: the Software Usability Measurement Inventory
Kirakowski, J., & Corbett, M. (1993) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.1993.tb00076.x · Cited by 625
Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies
Lewis, J. R. (2002) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2002.9669130 · Cited by 621
(2020) · The BNC Handbook · doi:10.1515/9780748628889-004 · Cited by 436
A method to standardize usability metrics into a single score
Sauro, J., & Kindlund, E. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055028 · Cited by 294
The system usability scale and non-native English speakers
Finstad, K. (2006) · Journal of Usability Studies archive · doi:10.5555/2835531.2835535 · Cited by 256
Research works printed but not linked: 4
- ISO . 1998 . Ergonomic requirements for office work with visual display terminal (VDT's)–Part 11: Guidance on usability , Geneva, Switzerland : Author . ISO 9241-11(E)
- Kirakowski , J. , Claridge , N. and Whitehand , R. . Human centered measures of success in Web design . Conference Proceedings of the 4th Conference on Human Factors and the Web . Retrieved from
- Lewis , J. . Binomial confidence intervals for small sample usability studies . Advances in applied ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics . Edited by: Ozok , A. F. and Salvendy , G. pp. 732 – 737 . West Lafayette, IN : USA Publishing .
- Lund , A. 2001 . Measuring usability with the USE Questionnaire . Usability Interface: The usability SIG newsletter of the Society for Technical Communications , 8 ( 2 ) Retrieved from
Cited by
Gamification in theory and action: a survey
Seaborn, K., & Fels, D. I. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.09.006 · Cited by 2,515
The System Usability Scale: Past, Present, and Future
Lewis, J. R. (2018) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2018.1455307 · Cited by 1,958
“The evolution of norms for the SUS The paper “ An empirical evaluation of the System Usability Scale ” (Bangor, Kortum, & Miller, 2008) was a seminal publication in the history of SUS research.”
BackgroundMethodResultInfluential(2013) · Journal of Usability Studies archive · Cited by 1,357
The Factor Structure of the System Usability Scale
Lewis, J. R., & Sauro, J. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-02806-9_12 · Cited by 1,278
“[8] conducted a factor analysis of their 2324 SUS questionnaires and concluded, on the basis of examining the eigenvalues and factor loadings for a one-factor solution, that there was only one significant factor, consistent with prevailing practitioner belief and practice.”
BackgroundMethodInfluentialDesigning for Motivation, Engagement and Wellbeing in Digital Experience
Peters, D., Calvo, R. A., & Ryan, R. M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00797 · Cited by 888
“Measures Relating to the Interface Among the most common measures for evaluating a technology interface are questionnaires like the System Usability Scale (Bangor et al., 2008).”
Method
Metadata
- Authors
- Bangor, A., Kortum, P. T., & Miller, J. T.
- Affiliations
- AT&T (United States) (US)
- Rice University (US)
- Corresponding author
- Kortum, Philip T.
- Venue
- International Journal of Human–Computer Interaction
- Publisher
- Informa UK Limited
- Published
- 2008-07-29
- Volume
- 24
- Issue
- 6
- Pages
- 574-594
- ISSN
- 1044-7318 (ISSN-L)
- 1532-7590
- OpenAlex
- W2032191478
- Cited by
- 5,181
- References
- 18
- Citations by year
- 2026: 598 · 2025: 685 · 2024: 568 · 2023: 529 · 2022: 555 · 2021: 468 · 2020: 370 · 2019: 287 · 2018: 248 · 2017: 157 · 2016: 162 · 2015: 161 · 2014: 148 · 2013: 98 · 2012: 68
- FWCI
- 63.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- System Usability Scale, usability evaluation, adjective ratings, usability metrics, product development cycle
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Empirical Evaluation of the System Usability Scale. https://app.cochat.ai/papers/10.1080/10447310802205776
- The paper
- Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An Empirical Evaluation of the System Usability Scale. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447310802205776
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.