Augmenting API documentation with insights from stack overflow
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Software developers need access to different kinds of information which is often dispersed among different documentation sources, such as API documentation or Stack Overflow. We present an approach to automatically augment API documentation with "insight sentences" from Stack Overflow---sentences that are related to a particular API type and that provide insight not contained in the API documentation of that type. Based on a development set of 1,574 sentences, we compare the performance of two state-of-the-art summarization techniques as well as a pattern-based approach for insight sentence extraction. We then present SISE, a novel machine learning based approach that uses as features the sentences themselves, their formatting, their question, their answer, and their authors as well as part-of-speech tags and the similarity of a sentence to the corresponding API documentation. With SISE, we were able to achieve a precision of 0.64 and a coverage of 0.7 on the development set. In a comparative study with eight software developers, we found that SISE resulted in the highest number of sentences that were considered to add useful information not found in the API documentation. These results indicate that taking into account the meta data available on Stack Overflow as well as part-of-speech tags can significantly improve unsupervised extraction approaches when applied to Stack Overflow data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Mining StackOverflow to turn the IDE into a self-confident programming prompter
Ponzanelli, L., Bavota, G., Di Penta, M., Oliveto, R., & Lanza, M. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597077 · Cited by 240
Rahman, M. M., Yeasmin, S., & Roy, C. K. (2014) · 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE) · doi:10.1109/csmr-wcre.2014.6747170 · Cited by 75 · cited 1×
“[29] developed a contextaware IDE-based meta search engine for recommendations about programming errors and exceptions.”
Method(2012) · doi:10.5555/2666719.2666725 · Cited by 74
Cited by
A Survey of Machine Learning for Big Code and Naturalness
Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019) · ACM Computing Surveys · doi:10.1145/3212695 · Cited by 150
Razzaq, A., Buckley, J., Lai, Q., Yu, T., & Botterweck, G. (2025) · ACM Computing Surveys · doi:10.1145/3687299 · Cited by 36
Enabling Data-Driven API Design with Community Usage Data: A Need-Finding Study
Zhang, T., Hartmann, B., Kim, M., & Glassman, E. L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376382 · Cited by 25
“There has been a lot of interest in mining programming insights from online coding communities, e.g., API usage tips [29, 55], test cases [36, 34], analogical libraries [7, 8], bug fixing hints [9, 20, 61].”
BackgroundA Survey on Online Learning Preferences for Computer Science and Programming
Escobar-Avila, J., Venuti, D., Di Penta, M., & Haiduc, S. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) · doi:10.1109/icse-seet.2019.00026 · Cited by 24
A disruptive research playbook for studying disruptive innovations
Storey, M.-A., Russo, D., Novielli, N., Kobayashi, T., & Wang, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3678172 · Cited by 15
Metadata
- Authors
- Treude, C., & Robillard, M. P.
- Venue
- Proceedings of the 38th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2016-05-14
- Pages
- 392-403
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W2366532918
- Cited by
- 239
- References
- 5
- FWCI
- 25.93
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.