CoChat

Analyzing APIs Documentation and Code to Detect Directive Defects

Open access
Zhou, Yu, Gu, Ruihang, Chen, Taolue, Huang, Zhiqiu, Panichella, Sebastiano, Gall, Harald · 2017 · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE)
Cited by 115

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Application Programming Interface (API) documents represent one of the most important references for API users. However, it is frequently reported that the documentation is inconsistent with the source code and deviates from the API itself. Such inconsistencies in the documents inevitably confuse the API users hampering considerably their API comprehension and the quality of software built from such APIs. In this paper, we propose an automated approach to detect defects of API documents by leveraging techniques from program comprehension and natural language processing. Particularly, we focus on the directives of the API documents which are related to parameter constraints and exception throwing declarations. A first-order logic based constraint solver is employed to detect such defects based on the obtained analysis results. We evaluate our approach on parts of well documented JDK 1.8 APIs. Experiment results show that, out of around 2000 API usage constraints, our approach can detect 1158 defective document directives, with a precision rate of 81.6%, and a recall rate of 82.0%, which demonstrates its practical feasibility.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icse.2017.11

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 38

  1. 10.1109/SANER.2017.7884612 · DOI
  2. 10.1109/ICSE-C.2017.5 · DOI
  3. 10.1109/ICSE.2015.97 · DOI
  4. 10.1007/s10664-014-9323-y · DOI
  5. 10.1109/TSE.2014.2347969 · DOI
  6. 10.1109/ICSE.2013.6606629 · DOI · Paper page
  7. di, 2016, What would users change in my app? summarizing app reviews for recommending software changes, Foundations of Software Engineering (FSE) 2016 ACM SIGSOFT International Symposium on the
  8. 10.1109/ICPC.2013.6613842 · DOI
  9. 10.1145/2597008.2597799 · DOI
  10. 10.1109/ICPC.2012.6240510 · DOI
  11. 10.1109/ICSM.2015.7332474 · DOI · Paper page
  12. 10.1109/ASE.2015.12 · doi:10.1109/ase.2015.12
  13. sorbo, 2016, DECA: development emails content analyzer, Proceedings of the 38th International Conference on Software Engineering ICSE 2016, 641
  14. barrett, 2009, Satisfiability modulo theories, Handbook of Satisfiability, 185, 825
  15. feldt, 2010, Validity threats in empirical software engineering research-an initial survey, SEKE, 374
  16. 10.1145/1390630.1390664 · DOI · Paper page
  17. lam, 2011, The soot framework for java program analysis: a retrospective, Cetus Users and Compiler Infastructure Workshop (CETUS 2011)
  18. shi, 2011, An empirical study on evolution of api documentation, FASE, 6603, 416
  19. 10.1145/1882291.1882312 · DOI · Paper page
  20. 10.1109/ASE.2009.94 · DOI · Paper page
  21. 10.1145/2597008.2597155 · DOI · Paper page
  22. 10.1109/TSE.2012.63 · DOI · Paper page
  23. 10.1109/SANER.2015.7081813 · doi:10.1109/saner.2015.7081813
  24. 10.1109/ICSE.2012.6227137 · DOI
  25. 10.1109/MS.2014.80 · DOI · Paper page
  26. 10.1145/2568225.2568299 · DOI · Paper page
  27. bloch, 2006, How to design a good api and why it matters, Companion to the 21 st ACM SIGPLAN symposium on Object-oriented programming systems languages and applications, 506
  28. 10.1145/2775441.2775457 · DOI
  29. 10.1007/s10664-011-9186-4 · DOI
  30. 10.1109/ESEM.2013.14 · DOI · Paper page
  31. 10.1007/s10664-014-9350-8 · DOI
  32. 10.1145/1294261.1294276 · DOI · Paper page
  33. zhong, 2013, Detecting api documentation errors, ACM SIGPLAN Notices, 48, 803, 10.1145/2544173.2509523 · DOI
  34. 10.1109/ICST.2012.106 · DOI · Paper page
  35. 10.1145/2950290.2983938 · DOI
  36. 10.1145/1985793.1985796 · DOI · Paper page
  37. 10.1145/2642937.2642969 · DOI
  38. 10.1109/ICPC.2013.6613836 · DOI · Paper page

Cited by

See all 115

Metadata

Authors
Zhou, Y., Gu, R., Chen, T., Huang, Z., Panichella, S., & Gall, H.
Venue
2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE)
Publisher
IEEE
Published
2017-05
Pages
27-37
OpenAlex
W2617604339
Cited by
115
References
3
FWCI
9.74
Type
conference paper
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.