Categorizing the Content of GitHub README Files
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
README files play an essential role in shaping a developer’s first impression of a software repository and in documenting the software project that the repository hosts. Yet, we lack a systematic understanding of the content of a typical README file as well as tools that can process these files automatically. To close this gap, we conduct a qualitative study involving the manual annotation of 4,226 README file sections from 393 randomly sampled GitHub repositories and we design and evaluate a classifier and a set of features that can categorize these sections automatically. We find that information discussing the ‘What’ and ‘How’ of a repository is very common, while many README files lack information regarding the purpose and status of a repository. Our multi-label classifier which can predict eight different categories achieves an F1 score of 0.746. To evaluate the usefulness of the classification, we used the automatically determined classes to label sections in GitHub README files using badges and showed files with and without these badges to twenty software professionals. The majority of participants perceived the automated labeling of sections based on our classifier to ease information discovery. This work enables the owners of software repositories to improve the quality of their documentation and it has the potential to make it easier for the software development community to discover relevant information in GitHub README files.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907
Grounded theory research: Procedures, canons, and evaluative criteria
Corbin, J. M., & Strauss, A. (1990) · Qualitative Sociology · doi:10.1007/bf00988593 · Cited by 11,133
Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574 · cited 1×
“therwise it is a negative instance. We learn an SVM classier to get the weight of each feature. To capture signicantly important features, we perform the Scott-Knott ESD (Eect Size Dierence) test [69]. For the purpose of this test, we perform ten times ten-fold cross validation where each cross validation generates di…”
MethodQuestions programmers ask during software evolution tasks
Sillito, J., Murphy, G. C., & De Volder, K. (2006) · Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1181775.1181779 · Cited by 270 · cited 1×
“…developers have analyzed newsgroup questions [29], questions in collocated development teams [35, 73], questions during software evolution tasks [64, 65], questions that focus on issues that occur within a project [22], questions that are hard to answer [39], and information needs in software…”
Using information fragments to answer the questions developers ask
Fritz, T., & Murphy, G. C. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1 · doi:10.1145/1806799.1806828 · Cited by 163
Ensemble Methods for App Review Classification: An Approach for Software Evolution (N)
Guzman, E., El-Haliby, M., & Bruegge, B. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.88 · Cited by 143
Hard-to-answer questions about code
LaToza, T. D., & Myers, B. A. (2010) · Evaluation and Usability of Programming Languages and Tools · doi:10.1145/1937117.1937125 · Cited by 141
Knowledge-based approaches in software documentation: A systematic literature review
Ding, W., Liang, P., Tang, A., & van Vliet, H. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.01.008 · Cited by 107
Cited by
Prana, G. A. A., Ford, D., Rastogi, A., Lo, D., Purandare, R., & Nagappan, N. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3092813 · Cited by 66
Self-citationDo as I Do, Not as I Say: Do Contribution Guidelines Match the GitHub Contribution Process?
Elazhary, O., Storey, M.-A., Ernst, N., & Zaidman, A. (2019) · 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2019.00043 · Cited by 42
“[9] attempted to explore the contents of contribution guideline documentation, they only focused on the contents of README files.”
BackgroundMethodG-Repo: a Tool to Support MSR Studies on GitHub
Romano, S., Caulo, M., Buompastore, M., Guerra, L., Mounsif, A., Telesca, M., Baldassarre, M. T., & Scanniello, G. (2021) · 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00064 · Cited by 16
Detecting outdated code element references in software repository documentation
Tan, W. S., Wagner, M., & Treude, C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10397-6 · Cited by 13
Self-citationAutomatic analysis of available source code of top artificial intelligence conference papers
Lin, J., Wang, Y., Yu, Y., Zhou, Y., Chen, Y., & Shi, X. (2022) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194022500358 · Cited by 7
Metadata
- Authors
- Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D.
- Venue
- Empirical Software Engineering
- Published
- 2018-10-12
- Volume
- 24
- Issue
- 3
- Pages
- 1296-1327
- ISSN
- 1382-3256 (ISSN-L)
- 1573-7616
- OpenAlex
- W2788622925
- Cited by
- 143
- References
- 11
- FWCI
- 25.71
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Categorizing the Content of GitHub README Files. https://app.cochat.ai/papers/10.1007/s10664-018-9660-3
- The paper
- Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. (2019). Categorizing the Content of GitHub README Files. Empirical Software Engineering. https://doi.org/10.1007/s10664-018-9660-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.