A Theory of Robust API Knowledge
The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Abstract
Creating modern software inevitably requires using application programming interfaces (APIs). While software developers can sometimes use APIs by simply copying and pasting code examples, a lack of robust knowledge of how an API works can lead to defects, complicate software maintenance, and limit what someone can express with an API. Prior work has uncovered the many ways that API documentation fails to be helpful, though rarely describes precisely why. We present a theory of robust API knowledge that attempts to explain why, arguing that effective understanding and use of APIs depends on three components of knowledge: (1) the domain concepts the API models along with terminology, (2) the usage patterns of APIs along with rationale, and (3) facts about an API’s execution to support reasoning about its runtime behavior. We derive five hypotheses from this theory and present a study to test them. Our study investigated the effect of having access to these components of knowledge, finding that while learners requested these three components of knowledge when they were not available, whether the knowledge helped the learner use or understand the API depended on the tasks and likely the relevance and quality of the specific information provided. The theory and our evidence in support of its claims have implications for what content API documentation, tutorials, and instruction should contain and the importance of giving the right information at the right time, as well as what information API tools should compute, and even how APIs should be designed. Future work is necessary to both further test and refine the theory, as well as exploit its ideas for better instructional design.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Empirical Studies of Programming Knowledge
Soloway, E., & Ehrlich, K. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010283 · Cited by 877
Program comprehension during software maintenance and evolution
Von Mayrhauser, A., & Vans, A. M. (1995) · Computer · doi:10.1109/2.402076 · Cited by 575
Six Learning Barriers in End-User Programming Systems
Ko, A. J., Myers, B. A., & Aung, H. H. (2004) · 2004 IEEE Symposium on Visual Languages - Human Centric Computing · doi:10.1109/vlhcc.2004.47 · Cited by 549
Self-citationInformation Needs in Collocated Software Development Teams
Ko, A. J., DeLine, R., & Venolia, G. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.45 · Cited by 499 · cited 7×
“Execution facts elucidate the details of the API structure [28], how to use API features [12, 21, 30], how to understand dependencies and relations between API features [12, 13, 30], and details of how code relates to output [21, 22].”
BackgroundInfluentialSelf-citation(2009) · doi:10.1145/1518701.1518944 · Cited by 447
What Makes APIs Hard to Learn? Answers from Developers
Robillard, M. P. (2009) · IEEE Software · doi:10.1109/ms.2009.193 · Cited by 382
0897915046/International Conference on Software Engineering
(1992) · doi:10.1145/143062 · Cited by 374
A field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352
Paradigm Shifts in Designed Instruction: From Behaviorism to Cognitivism to Constructivism.
Cooper, P. A. (1993) · Educational Technology archive · Cited by 307
Graph-based mining of multiple object usage patterns
Nguyen, T. T., Nguyen, H. A., Pham, N. H., Al-Kofahi, J. M., & Nguyen, T. N. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595767 · Cited by 292
Research works printed but not linked: 6
- Steven Clarke . 2004. Measuring API usability. Dr. Dobb’s Journal , Special Windows/.NET Supplement (May 2004 ). Retrieved from http://www.drdobbs.com/windows/measuring-api-usability/184405654. Steven Clarke. 2004. Measuring API usability. Dr. Dobb’s Journal, Special Windows/.NET Supplement (May 2004). Retrieved from http://www.drdobbs.com/windows/measuring-api-usability/184405654.
- Ekwa Duala-Ekoko and Martin P . Robillard . 2010 . The Information Gathering Strategies of API Learners. Technical ReportTR-2010.6, School of Computer Science, McGill University . Retrieved from https://pdfs.semanticscholar.org/a7ff/4cc954744f761e8697be4e73aa25166a76c4.pdf. Ekwa Duala-Ekoko and Martin P. Robillard. 2010. The Information Gathering Strategies of API Learners. Technical ReportTR-2010.6, School of Computer Science, McGill University. Retrieved from https://pdfs.semanticscholar.org/a7ff/4cc954744f761e8697be4e73aa25166a76c4.pdf.
- Patrick Suppes . 1967 . What is a scientific theory? In Philosophy of Science Today, Sidney Morgenbesser (Ed.) , Chapter 6. 55 -- 67 . Patrick Suppes. 1967. What is a scientific theory? In Philosophy of Science Today, Sidney Morgenbesser (Ed.), Chapter 6. 55--67.
- Kyle Matthew Thayer. 2020. Practical Knowledge Barriers in Professional Programming. Thesis. Retrieved from https://digital.lib.washington.edu:443/researchworks/handle/1773/45471. Kyle Matthew Thayer. 2020. Practical Knowledge Barriers in Professional Programming. Thesis. Retrieved from https://digital.lib.washington.edu:443/researchworks/handle/1773/45471.
- 10.5555/2337223.2337321 · doi:10.5555/2337223.2337321
- Leonid_2. 2015 . English: A simple torus fading out to a wireframe structure. Leonid_2. 2015. English: A simple torus fading out to a wireframe structure.
Cited by
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
Concept-Annotated Examples for Library Comparison
Yan, L., Kim, M., Hartmann, B., Zhang, T., & Glassman, E. L. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545647 · Cited by 20
Identifying Common Errors in Open-Ended Machine Learning Projects
Skripchuk, J., Shi, Y., & Price, T. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3478431.3499397 · Cited by 20
Usability-Oriented Design of Liquid Types for Java
Gamboa, C., Canelas, P., Timperley, C. S., & Fonseca, A. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00132 · Cited by 19
How statically-typed functional programmers write code
Lubin, J., & Chasins, S. E. (2021) · Proceedings of the ACM on Programming Languages · doi:10.1145/3485532 · Cited by 18
“For example, Ko and Myers (2003) develop and evaluate a theory of programming errors, and Thayer et al. (2021) do the same for a theory of API knowledge.”
BackgroundSelf-citation
Metadata
- Authors
- Thayer, K., Chasins, S. E., & Ko, A. J.
- Affiliations
- University of Washington (US)
- University of California, Berkeley (US)
- Venue
- ACM Transactions on Computing Education
- Published
- 2021-01-28
- Volume
- 21
- Issue
- 1
- Pages
- 1-32
- ISSN
- 1946-6226 (ISSN-L)
- OpenAlex
- W3128009513
- Cited by
- 25
- References
- 50
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 6 · 2023: 7 · 2022: 6 · 2021: 3
- FWCI
- 4.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Computing Education (publishedVersion)
- Funders
- National Science Foundation (1703304, 1735123, and 1836813.)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- application programming interfaces, API documentation, software learning, domain concepts, API usage patterns, API design, instructional design, software maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Theory of Robust API Knowledge. https://app.cochat.ai/papers/10.1145/3444945
- The paper
- Thayer, K., Chasins, S. E., & Ko, A. J. (2021). A Theory of Robust API Knowledge. ACM Transactions on Computing Education. https://doi.org/10.1145/3444945
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.