Analysis in practical usability evaluation: a survey study
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Analysis is a key part of conducting usability evaluations, yet rarely systematically studied. Thus, we lack direction on how to do research on supporting practitioners' analysis and lose an opportunity for practitioners to learn from each other. We have surveyed 155 usability practitioners on the analysis in their latest usability evaluation. Analysis is typically flexible and light-weight. At the same time, practitioners see a need to strengthen reliability in evaluation. Redesign is closely integrated with analysis; more than half of the respondents provide visual redesign suggestions in their evaluation deliverables. Analysis support from academic research, including tools, forms and structured formats, does not seem to have direct impact on analysis practice. We provide six recommendations for future research to better support analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2006) · doi:10.1080/01449290500330331 · Cited by 3,026
Nielsen, J. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260531 · Cited by 2,759
A survey of user-centered design practice
Vredenburg, K., Mao, J.-Y., Smith, P. W., & Carey, T. (2002) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/503376.503460 · Cited by 691 · cited 3×
“In particular, survey studies provide information on evaluation method use and preferences among practitioners [3,35,36].”
BackgroundResult
Cited by
Human-AI Collaboration for UX Evaluation: Effects of Explanation and Synchronization
Fan, M., Yang, X., Yu, T., Liao, Q. V., & Zhao, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512943 · Cited by 81
How UX Practitioners Produce Findings in Usability Testing
Reeves, S. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3299096 · Cited by 34
“usability evaluation” [73], something Lallemand and Koenig concur with [59]. Følstad, Law and Hornbæk elaborate on this, noting that “Texts on usability testing […] lack detail on how to do analysis” [29]. Now, while some textbooks are light on the detail of what happens during a test (e.g., see [94, 58]), others actu…”
BackgroundFan, M., Zhao, Q., & Tibdewal, V. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445680 · Cited by 29
Metadata
- Authors
- Følstad, A., Law, E., & Hornbæk, K.
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 2127-2136
- OpenAlex
- W2042709572
- Cited by
- 135
- References
- 3
- FWCI
- 6.52
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.