A survey of software learnability: metrics, methodologies and guidelines
The DOI matches its Crossref record; 53 of the 55 references it cites were found in a registry or an index.
Abstract
It is well-accepted that learnability is an important aspect of usability, yet there is little agreement as to how learnability should be defined, measured, and evaluated. In this paper, we present a survey of the previous definitions, metrics, and evaluation methodologies which have been used for software learnability. Our survey of evaluation methodologies leads us to a new question-suggestion protocol, which, in a user study, was shown to expose a significantly higher number of learnability issues in comparison to a more traditional think-aloud protocol. Based on the issues identified in our study, we present a classification system of learnability issues, and demonstrate how these categories can lead to guidelines for addressing the associated challenges.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There is much research on how to improve the software learning experience for users [8,10,29,42,46,68], but this prior work has not typically considered older adults as target users.”
From Older Adults' Collaborative Learning Dynamics When Exploring Feature-Rich Software (2024) · cites this paper as [29]
In the citing paper
“Common challenges include a lack of awareness of functionality, locating and understanding functionality, and especially understanding task flow [29,42].”
From Older Adults' Collaborative Learning Dynamics When Exploring Feature-Rich Software (2024) · cites this paper as [29]
Citations
Cites
(1993) · Cited by 9,879
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,587
(2013) · Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship · doi:10.1007/978-1-4614-3858-8_100431 · Cited by 4,799
Protocol Analysis: Verbal Reports as Data
Venkatesan, M., Ericsson, K. A., & Simon, H. A. (1986) · Journal of Marketing Research · doi:10.2307/3151491 · Cited by 4,347
Designing the user interface strategies for effective human-computer interaction
Shneiderman, B. (1987) · ACM SIGBIO Newsletter · doi:10.1145/25065.950626 · Cited by 2,798
Direct Manipulation: A Step Beyond Programming Languages
Shneiderman (1983) · Computer · doi:10.1109/mc.1983.1654471 · Cited by 1,939 · cited 1×
“The graphical user interface is an excellent way of making functionality visible to the user, and thus improving their awareness through visual exposure [39].”
BackgroundA Practical Guide to Usability Testing
Dumas, J. F., & Redish, J. C. (1993) · Cited by 1,335
Protocol Analysis: Verbal Reports as Data
Danks, J. H., Ericsson, K. A., & Simon, H. A. (1985) · The American Journal of Psychology · doi:10.2307/1422516 · Cited by 1,218
NIELSEN, J. (1993) · Usability Engineering · doi:10.1016/b978-0-08-052029-2.50009-7 · Cited by 934
Usability engineering methods for software developers
Holzinger, A. (2005) · Communications of the ACM · doi:10.1145/1039539.1039541 · Cited by 855
Research works printed but not linked: 2
- Carroll , J. M. ( 1990 ). The Nurnberg Funnel . MIT Press . Carroll, J. M. (1990). The Nurnberg Funnel. MIT Press.
- Linja -aho, M. ( 2005 ). Evaluating and Improving the Learnability of a Building Modeling System . Helsinki University of Technology . Linja-aho, M. (2005). Evaluating and Improving the Learnability of a Building Modeling System. Helsinki University of Technology.
Cited by
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Chaves, A. P., & Gerosa, M. A. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1841438 · Cited by 572
“frustration and they often give up when they do not know how to proceed. The literature on interactive systems has highlighted the system learnability as the most fundamental component for usability [Grossman et al. 2009], and an easy-to-learn system should lead the user to perform well, even during their initial inte…”
BackgroundInfluentialAshtari, N., Bunt, A., McGrenere, J., Nebeling, M., & Chilana, P. K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376722 · Cited by 273
GamiCAD: a gamified tutorial system for first time autocad users
Li, W., Grossman, T., & Fitzmaurice, G. (2012) · Proceedings of the 25th annual ACM symposium on User interface software and technology · doi:10.1145/2380116.2380131 · Cited by 222
Self-citationThe impact of tutorials on games of varying complexity
Andersen, E., O'Rourke, E., Liu, Y.-E., Snider, R., Lowdermilk, J., Truong, D., Cooper, S., & Popovic, Z. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207687 · Cited by 171
Metadata
- Authors
- Grossman, T., Fitzmaurice, G., & Attar, R.
- Affiliations
- Autodesk Research, Toronto, ON, Canada
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2009-04-04
- Pages
- 649-658
- OpenAlex
- W2138350998
- Cited by
- 322
- References
- 55
- Citations by year
- 2026: 10 · 2025: 62 · 2024: 15 · 2023: 10 · 2022: 16 · 2021: 18 · 2020: 25 · 2019: 16 · 2018: 19 · 2017: 21 · 2016: 18 · 2015: 19 · 2014: 17 · 2013: 13 · 2012: 14
- FWCI
- 10.93
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- usability, think-aloud, user study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A survey of software learnability: metrics, methodologies and guidelines. https://app.cochat.ai/papers/10.1145/1518701.1518803
- The paper
- Grossman, T., Fitzmaurice, G., & Attar, R. (2009). A survey of software learnability: metrics, methodologies and guidelines. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1518701.1518803
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.