The Long Tail: Understanding the Discoverability of API Functionality
The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Abstract
Almost all software development revolves around the discovery and use of application programming interfaces (APIs). Once a suitable API is selected, programmers must begin the process of determining what functionality in the API is relevant to a programmer's task and how to use it. Our work aims to understand how API functionality is discovered by programmers and where tooling may be appropriate. We employed a mixed-methods approach to investigate Apache Beam, a distributed data processing API, by mining Beam client code and running a lab study to see how people discover Beam's available functionality. We found that programmers' prior experience with similar APIs significantly impacted their ability to find relevant features in an API and attempting to form a top-down mental model of an API resulted in less discovery of features.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
What Makes APIs Hard to Learn? Answers from Developers
Robillard, M. P. (2009) · IEEE Software · doi:10.1109/ms.2009.193 · Cited by 382
A field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352
Myers, B. A., & Stylos, J. (2016) · Communications of the ACM · doi:10.1145/2896587 · Cited by 269 · cited 1×
“Despite this ongoing effort, API usability is still seen as an evolving field tackling multiple important issues such as methods for evaluating and improving API usability [3]–[6] and documentation’s affect on API usability [7].”
MethodSelf-citationTinkering and gender in end-user programmers' debugging
Beckwith, L., Kissinger, C., Burnett, M., Wiedenbeck, S., Lawrance, J., Blackwell, A., & Cook, C. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124808 · Cited by 176
Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools
Myers, B. A., Ko, A. J., LaToza, T. D., & Yoon, Y. (2016) · Computer · doi:10.1109/mc.2016.200 · Cited by 144
Self-citationUsability Implications of Requiring Parameters in Objects' Constructors
Stylos, J., & Clarke, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.92 · Cited by 110
The implications of method placement on API learnability
Stylos, J., & Myers, B. A. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453117 · Cited by 102
Self-citationSpotWeb: Detecting Framework Hotspots and Coldspots via Mining Open Source Code on the Web
Thummalapenta, S., & Xie, T. (2008) · 2008 23rd IEEE/ACM International Conference on Automated Software Engineering · doi:10.1109/ase.2008.43 · Cited by 89
Opportunistic programming: how rapid ideation and prototyping occur in practice
Brandt, J., Guo, P. J., Lewenstein, J., & Klemmer, S. R. (2008) · Proceedings of the 4th international workshop on End-user software engineering · doi:10.1145/1370847.1370848 · Cited by 88
How do API documentation and static typing affect API usability?
Endrikat, S., Hanenberg, S., Robbes, R., & Stefik, A. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568299 · Cited by 85
Understanding the API usage in Java
Qiu, D., Li, B., & LEUNG, H. K. N. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.01.011 · Cited by 82
Improving API documentation using API usage information
Stylos, J., Faulring, A., Yang, Z., & Myers, B. A. (2009) · 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2009.5295283 · Cited by 81
Self-citationAsking and answering questions about unfamiliar APIs: An exploratory study
Duala-Ekoko, E., & Robillard, M. P. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227187 · Cited by 69
Design annotations to improve API discoverability
Santos, A. L., & Myers, B. A. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.12.036 · Cited by 64
Self-citationEnd-user debugging strategies: A sensemaking perspective
Grigoreanu, V., Burnett, M., Wiedenbeck, S., Cao, J., Rector, K., & Kwan, I. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2147783.2147788 · Cited by 60 · cited 1×
“Our findings are generally consistent with prior work [9], [16]–[19] that finds that programmers have different learning styles – some are systematic and prefer”
ResultMethods for Investigating Mental Models for Learners of APIs
Horvath, A., Nagy, M., Voichick, F., Kery, M. B., & Myers, B. A. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312897 · Cited by 13
Self-citationUsing concept maps to evaluate the usability of APIs
Gerken, J., Jetter, H.-C., & Reiterer, H. (2010) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1753846.1754082 · Cited by 6
Research works printed but not linked: 2
- foundation, 2018, Apache beam An advanced unified programming model
- clarke, 2004, Measuring API usability, Dr Dobbs Journal, 6s
Cited by
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
“Researchers have also studied challenges developers face [14, 23, 32, 58, 72, 74], including difficulties in effective search-query writing and information foraging, and built tools to support developers overcome such challenges [3, 42, 52].”
BackgroundSelf-citationHerskovitz, J., Xu, A., Alharbi, R., & Guo, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581249 · Cited by 52
Harrand, N., Benelallam, A., Soto-Valero, C., Bettega, F., Barais, O., & Baudry, B. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111134 · Cited by 33
Understanding How Programmers Can Use Annotations on Documentation
Horvath, A., Liu, M. X., Hendriksen, R., Shannon, C., Paterson, E., Jawad, K., Macvean, A., & Myers, B. A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502095 · Cited by 26
“We provided all the annotations, which were of two types — annotations authored by the first author that were designed to be helpful for this task given what developers were confused about in a previous study [26] and other annotations that were designed to be “distracting” to simulate the more realistic case where no…”
MethodSelf-citationBecks, E., Zdankin, P., Matkovic, V., & Weis, T. (2023) · Technologies · doi:10.3390/technologies11010009 · Cited by 22
Metadata
- Authors
- Horvath, A., Myers, B., Grover, S., Dong, S., Zhou, E., Voichick, F., Kery, M. B., Shinju, S., Nam, D., & Nagy, M.
- Affiliations
- Carnegie Mellon University (US)
- Washington University in St. Louis (US)
- Google (United States) (US)
- Venue
- 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
- Publisher
- IEEE
- Published
- 2019-10
- ISSN
- 1943-6092 (ISSN-L)
- OpenAlex
- W2973377235
- Cited by
- 9
- References
- 18
- Citations by year
- 2024: 2 · 2023: 3 · 2022: 2 · 2021: 1 · 2020: 1
- FWCI
- 0.80
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- application programming interfaces, programmer experience, mental models, Apache Beam, distributed data processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.