Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 13

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Detecting outdated code element references in software repository documentation

Tan, Wen Siang, Wagner, Markus, Treude, Christoph·2024·Empirical Software Engineering
DOI 10.1007/s10664-023-10397-6
Open this paper in CoChat — free account

Abstract

Outdated documentation is a pervasive problem in software development, preventing effective use of software, and misleading users and developers alike. We posit that one possible reason why documentation becomes out of sync so easily is that developers are unaware of when their source code modifications render the documentation obsolete. Ensuring that the documentation is always in sync with the source code takes considerable effort, especially for large codebases. To address this situation, we propose an approach that can automatically detect code element references that survive in the documentation after all source code instances have been deleted. In this work, we analysed over 3,000 GitHub projects and found that most projects contain at least one outdated code element reference at some point in their history. We submitted GitHub issues to real-world projects containing outdated references detected by our approach, some of which have already led to documentation fixes. As an initiative toward keeping documentation in software repositories up-to-date, we have made our implementation available for developers to scan their GitHub projects for outdated code element references.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10664-023-10397-6

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 29

  1. Aghajani E, Nagy C, Vega-Márquez OL, Linares-Vásquez M, Moreno L, Bavota G, Lanza M (2019) Software documentation issues unveiled. In: Proceedings of the International Conference on Software Engineering, pp 1199–1210 · DOI · Paper page
  2. Aldaeej A (2021) Towards effective technical debt decision making in software startups: A multiple case study of web and mobile app startups. PhD thesis, University of Maryland, Baltimore County
  3. Bacchelli A, Lanza M, Robbes R (2010) Linking e-mails and source code artifacts. In: Proceedings of the 32nd ACM/IEEE International conference on software engineering vol 1, pp 375–384 · DOI
  4. Dagenais B, Robillard MP (2012) Recovering traceability links between an api and its learning resources. In: 2012 34th international conference on software engineering (icse), IEEE, pp 47–57 · DOI
  5. Dagenais B, Robillard MP (2014) Using traceability links to recommend adaptive changes for documentation evolution. IEEE Trans Software Eng 40(11):1126–1146 · DOI
  6. Kajko-Mattsson M (2005) A survey of documentation practice within corrective maintenance. Empir Softw Eng 10(1):31–55 · DOI
  7. Kruchten P, Nord RL, Ozkaya I (2012) Technical debt: From metaphor to theory and practice. IEEE Softw 29(6):18–21 · DOI · Paper page
  8. Lee S, Wu R, Cheung SC, Kang S (2019) Automatic detection and update suggestion for outdated API names in documentation. IEEE Transactions on Software Engineering
  9. Lethbridge TC, Singer J, Forward A (2003) How software engineers use documentation: The state of the practice. IEEE Softw 20(6):35–39 · DOI · Paper page
  10. Liu J, Huang Q, Xia X, Shihab E, Lo D, Li S (2021) An exploratory study on the introduction and removal of different types of technical debt in deep learning frameworks. Empir Softw Eng 26(2):1–36 · DOI
  11. McHugh ML (2012) Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 22(3):276–282 · DOI · Paper page
  12. Mendes TS, de F Farias MA, Mendonça M, Soares HF, Kalinowski M, Spínola RO (2016) Impacts of agile requirements documentation debt on software projects: a retrospective study. In: Proceedings of the 31st annual ACM symposium on applied computing, pp 1290–1295 · DOI
  13. Panthaplackel S, Nie P, Gligoric M, Li JJ, Mooney RJ (2020) Learning to update natural language comments based on code changes. arXiv preprint arXiv:2004.12169 · DOI
  14. Parnas DL (1994) Software aging. In: Proceedings of international conference on software engineering, pp 279–287 · DOI · Paper page
  15. Petrosyan G, Robillard MP, De Mori R (2015) Discovering information explaining api types using text classification. In: 2015 IEEE/ACM 37th IEEE international conference on software engineering, IEEE, vol 1, pp 869–879 · DOI
  16. Prana GAA, Treude C, Thung F, Atapattu T, Lo D (2019) Categorizing the content of github readme files. Empir Softw Eng 24(3):1296–1327 · DOI
  17. Ratol IK, Robillard MP (2017) Detecting fragile comments. In: Proceedings of the international conference on automated software engineering, pp 112–122 · DOI
  18. Rigby PC, Robillard MP (2013) Discovering essential code elements in informal documentation. In: 2013 35th international conference on software engineering (ICSE), IEEE, pp 832–841 · DOI
  19. Rios N, Mendes L, Cerdeiral C, Magalhães APF, Perez B, Correal D, Astudillo H, Seaman C, Izurieta C, Santos G, et al. (2020) Hearing the voice of software practitioners on causes, effects, and practices to deal with documentation debt. In: International working conference on requirements engineering: foundation for software quality, Springer, pp 55–70 · DOI
  20. Robillard MP, Marcus A, Treude C, Bavota G, Chaparro O, Ernst N, Gerosa MA, Godfrey M, Lanza M, Linares-Vásquez M, et al. (2017) On-demand developer documentation. In: 2017 IEEE International conference on software maintenance and evolution (ICSME), IEEE, pp 479–483 · DOI
  21. Sholler D, Steinmacher I, Ford D, Averick M, Hoye M, Wilson G (2019) Ten simple rules for helping newcomers become contributors to open projects. PLoS Comput Biol 15(9):e1007296 · DOI · Paper page
  22. de Souza SCB, Anquetil N, de Oliveira KM (2005) A study of the documentation essential to software maintenance. In: Proceedings of the international conference on design of communication: documenting & designing for pervasive information, pp 68–75 · DOI · Paper page
  23. Steinmacher I, Treude C, Gerosa MA (2018) Let me in: Guidelines for the successful onboarding of newcomers to open source projects. IEEE Software 36(4):41–49 · DOI · Paper page
  24. Tan SH, Marinov D, Tan L, Leavens GT (2012) @tcomment: Testing Javadoc comments to detect comment-code inconsistencies. In: Proceedings of the international conference on software testing, verification and validation, pp 260–269 · DOI
  25. Treude C, Robillard MP, Dagenais B (2014) Extracting development tasks to navigate software documentation. IEEE Trans Software Eng 41(6):565–581 · DOI
  26. Uddin G, Robillard MP (2015) How API documentation fails. IEEE Software 32(4):68–75 · DOI
  27. Wen F, Nagy C, Bavota G, Lanza M (2019) A large-scale empirical study on code-comment inconsistencies. In: Proceedings of the international conference on program comprehension, pp 53–64 · DOI · Paper page
  28. Zhong H, Su Z (2013) Detecting API documentation errors. In: Proceedings of the international conference on object oriented programming systems languages & applications, pp 803–816 · DOI
  29. Zhou Y, Wang C, Yan X, Chen T, Panichella S, Gall HC (2020) Automatic detection and repair recommendation of directive defects in Java API documentation. IEEE Trans Software Eng 46(9):1004–1023 · DOI

Metadata

Authors
Tan, W. S., Wagner, M., & Treude, C.
Venue
Empirical Software Engineering
Publisher
Springer Science and Business Media LLC
Published
2023-11-21
Volume
29
Issue
1
ISSN
  • 1382-3256
  • 1573-7616
DOI
10.1007/s10664-023-10397-6
Cited by
13
References
1
Language
en
Licence
Funders
The University of Adelaide

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.