Design and evaluation of a command recommendation system for software applications
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
We examine the use of modern recommender system technology to aid command awareness in complex software applications. We first describe our adaptation of traditional recommender system algorithms to meet the unique requirements presented by the domain of software commands. A user study showed that our item-based collaborative filtering algorithm generates 2.1 times as many good suggestions as existing techniques. Motivated by these positive results, we propose a design space framework and its associated algorithms to support both global and contextual recommendations. To evaluate the algorithms, we developed the CommunityCommands plug-in for AutoCAD. This plug-in enabled us to perform a 6-week user study of real-time, within-application command recommendations in actual working environments. We report and visualize command usage behaviors during the study, and discuss how the recommendations affected users behaviors. In particular, we found that the plug-in successfully exposed users to new commands, as unique commands issued significantly increased.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evaluating collaborative filtering recommender systems
Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004) · ACM Transactions on Information Systems · doi:10.1145/963770.963772 · Cited by 5,798
Self-citationAmazon.com recommendations: item-to-item collaborative filtering
Linden, G., Smith, B., & York, J. (2003) · IEEE Internet Computing · doi:10.1109/mic.2003.1167344 · Cited by 5,403
GroupLens: an open architecture for collaborative filtering of netnews
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., & Riedl, J. (1994) · Proceedings of the 1994 ACM conference on Computer supported cooperative work - CSCW '94 · doi:10.1145/192844.192905 · Cited by 5,058
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Bailey, B. P., & Konstan, J. A. (2006) · Computers in Human Behavior · doi:10.1016/j.chb.2005.12.009 · Cited by 600 · cited 2×
“Previous research [Bailey and Konstan 2006; Cutrell et al. 2001; Cutrell, Czerwinski and Horvitz 2000; McGrenere et al. 2002] has demonstrated the harmful effects that notifications can have on a user‘s task performance.”
BackgroundMethodSelf-citationTraining wheels in a user interface
Carroll, J. M., & Carrithers, C. (1984) · Communications of the ACM · doi:10.1145/358198.358218 · Cited by 345 · cited 2×
“multi layered [Carroll and Carrithers 1984; Findlater and McGrenere 2010; Shneiderman 2003] user interfaces, our focus is instead on a command recommendation system which helps users find relevant tools that they were not previously aware of.”
BackgroundAutomatically generating personalized user interfaces with Supple
Gajos, K. Z., Weld, D. S., & Wobbrock, J. O. (2010) · Artificial Intelligence · doi:10.1016/j.artint.2010.05.005 · Cited by 317 · cited 3×
“Previous research has focused on adapting and optimizing the interface to the current user and context [Claypool et al. 2001; Cutrell et al. 2000; Gajos et al. 2010], and inferring possible operations based on the user‘s behavior [Horvitz et al.”
MethodPredicting human interruptibility with sensors: a Wizard of Oz feasibility study
Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J., & Yang, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642657 · Cited by 295 · cited 6×
“We avoid a system that pops up or forces the user to respond to the recommendation before continuing to work, since this type of system could be frustrating [Hudson et al. 2003; Xiao et al. 2004].”
BackgroundMethodInfluentialAn evaluation of a multiple interface design solution for bloated software
McGrenere, J., Baecker, R. M., & Booth, K. S. (2002) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/503376.503406 · Cited by 169 · cited 2×
“Previous research [Bailey and Konstan 2006; Cutrell et al. 2001; Cutrell, Czerwinski and Horvitz 2000; McGrenere et al. 2002] has demonstrated the harmful effects that notifications can have on a user‘s task performance.”
BackgroundMethodCommunityCommands: command recommendations for software applications
Matejka, J., Li, W., Grossman, T., & Fitzmaurice, G. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622214 · Cited by 125
Self-citation
Cited by
Supporting Novice to Expert Transitions in User Interfaces
Cockburn, A., Gutwin, C., Scarr, J., & Malacria, S. (2015) · ACM Computing Surveys · doi:10.1145/2659796 · Cited by 193
Improving software developers' fluency by recommending development environment commands
Murphy-Hill, E., Jiresal, R., & Murphy, G. C. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393645 · Cited by 67
Patina: dynamic heatmaps for visualizing application usage
Matejka, J., Grossman, T., & Fitzmaurice, G. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466442 · Cited by 67
Self-citationIP-QAT: in-product questions, answers, & tips
Matejka, J., Grossman, T., & Fitzmaurice, G. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047218 · Cited by 43
Self-citationLog2Intent: Towards Interpretable User Modeling via Recurrent Semantics Memory Unit
Tao, Z., Li, S., Wang, Z., Fang, C., Yang, L., Zhao, H., & Fu, Y. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330889 · Cited by 16
“Some previous works [1, 10, 21, 24] working with software actions are mainly designed to build a command recommendation system to assist users to complete complex task and learn software usage, yet neglect to learn a generic user representation.”
BackgroundMethod
Metadata
- Authors
- Li, W., Matejka, J., Grossman, T., Konstan, J. A., & Fitzmaurice, G.
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2011-06
- Volume
- 18
- Issue
- 2
- Pages
- 1-35
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W2089825482
- Cited by
- 57
- References
- 11
- FWCI
- 12.02
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Design and evaluation of a command recommendation system for software applications. https://app.cochat.ai/papers/10.1145/1970378.1970380
- The paper
- Li, W., Matejka, J., Grossman, T., Konstan, J. A., & Fitzmaurice, G. (2011). Design and evaluation of a command recommendation system for software applications. ACM Transactions on Computer-Human Interaction. https://doi.org/10.1145/1970378.1970380
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.