The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
The promises and perils of mining GitHub
Abstract
With over 10 million git repositories, GitHub is becoming one of the most important source of software artifacts on the Internet. Researchers are starting to mine the information stored in GitHub's event logs, trying to understand how its users employ the site to collaborate on software. However, so far there have been no studies describing the quality and properties of the data available from GitHub. We document the results of an empirical study aimed at understanding the characteristics of the repositories in GitHub and how users take advantage of GitHub's main features---namely commits, pull requests, and issues. Our results indicate that, while GitHub is a rich source of data on software development, mining GitHub for research purposes should take various potential perils into consideration. We show, for example, that the majority of the projects are personal and inactive; that GitHub is also being used for free storage and as a Web hosting service; and that almost 40% of all pull requests do not appear as merged, even though they were. We provide a set of recommendations for software engineering researchers on how to approach the data in GitHub.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Social coding in GitHub: transparency and collaboration in an open software repository
Dabbish, L., Stuart, C., Tsay, J., & Herbsleb, J. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145396 · Cited by 999
Research works printed but not linked: 23
- 10.1109/ICSE.2009.5070530 · DOI · Paper page
- 10.5555/2486788.2486882 · doi:10.5555/2486788.2486882
- 10.1145/1882291.1882308 · DOI
- 10.1109/MS.2013.13 · DOI
- 10.1145/1595696.1595716 · DOI · Paper page
- 10.1109/MSR.2009.5069475 · DOI · Paper page
- G. Gousios . The GHTorrent dataset and tool suite . In Proceedings of the 10th Conference on Mining Software Repositories, MSR ’13 , pages 233– 236 , 2013 . G. Gousios. The GHTorrent dataset and tool suite. In Proceedings of the 10th Conference on Mining Software Repositories, MSR ’13, pages 233–236, 2013.
- 10.1145/2568225.2568260 · DOI · Paper page
- G. Gousios and D. Spinellis . GHTorrent: GitHub’s data from a firehose. In MSR ’12 : Proc. of the 9th Working Conf. on Mining Software Repositories , pages 12– 21 , jun 2012 . G. Gousios and D. Spinellis. GHTorrent: GitHub’s data from a firehose. In MSR ’12: Proc. of the 9th Working Conf. on Mining Software Repositories, pages 12–21, jun 2012.
- 10.1145/2597073.2597122 · DOI
- 10.1049/ic:20040467 · DOI
- 10.1145/2441776.2441792 · DOI · Paper page
- 10.1145/2468356.2468382 · DOI
- 10.1109/WCRE.2010.37 · DOI
- 10.5555/2486788.2486804 · doi:10.5555/2486788.2486804
- 10.1145/2491411.2491418 · DOI
- A. Rainer and S. Gale . Evaluating the quality and quantity of data on open source software projects . In Proceedings of the First International Conference on Open Source Systems (OSS 2005 ), pages 29– 36 , 2005 . A. Rainer and S. Gale. Evaluating the quality and quantity of data on open source software projects. In Proceedings of the First International Conference on Open Source Systems (OSS 2005), pages 29–36, 2005.
- 10.1145/2491411.2491444 · DOI · Paper page
- 10.1145/1368088.1368162 · DOI
- 10.1109/CSMR.2013.41 · DOI
- 10.1145/2141512.2141583 · DOI
- 10.5555/2487085.2487131 · doi:10.5555/2487085.2487131
- D. Weiss . Quantitative analysis of open source projects on sourceforge . In Proc. of the First Int. Conf. on Open Source Systems (OSS 2005 ), pages 140– 147 , 2005 . D. Weiss. Quantitative analysis of open source projects on sourceforge. In Proc. of the First Int. Conf. on Open Source Systems (OSS 2005), pages 140–147, 2005.
Cited by
Curating GitHub for engineered software projects
Munaiah, N., Kroh, S., Cabrey, C., & Nagappan, M. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9512-6 · Cited by 365
The ABC of Software Engineering Research
Stol, K.-J., & Fitzgerald, B. (2018) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3241743 · Cited by 245
DéjàVu: a map of code duplicates on GitHub
Lopes, C. V., Maj, P., Martins, P., Saini, V., Yang, D., Zitny, J., Sajnani, H., & Vitek, J. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133908 · Cited by 183
A Large-Scale Empirical Study on Code-Comment Inconsistencies
Wen, F., Nagy, C., Bavota, G., & Lanza, M. (2019) · 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2019.00019 · Cited by 105
The sky is not the limit: multitasking across GitHub projects
Vasilescu, B., Blincoe, K., Xuan, Q., Casalnuovo, C., Damian, D., Devanbu, P., & Filkov, V. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884875 · Cited by 77
Self-citationThe Signals that Potential Contributors Look for When Choosing Open-source Projects
Qiu, H. S., Li, Y. L., Padala, S., Sarma, A., & Vasilescu, B. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359224 · Cited by 75
Prana, G. A. A., Ford, D., Rastogi, A., Lo, D., Purandare, R., & Nagappan, N. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3092813 · Cited by 66
AI-Driven Development Is Here: Should You Worry?
Ernst, N. A., & Bavota, G. (2022) · IEEE Software · doi:10.1109/ms.2021.3133805 · Cited by 62
Ma, Y., Dey, T., Bogart, C., Amreen, S., Valiev, M., Tutko, A., Kennard, D., Zaretzki, R., & Mockus, A. (2021) · Empirical Software Engineering · doi:10.1007/s10664-020-09905-9 · Cited by 49
Do as I Do, Not as I Say: Do Contribution Guidelines Match the GitHub Contribution Process?
Elazhary, O., Storey, M.-A., Ernst, N., & Zaidman, A. (2019) · 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2019.00043 · Cited by 42
Zou, W., Xuan, J., Xie, X., Chen, Z., & Xu, B. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09720-x · Cited by 37
Song, F., Agarwal, A., & Wen, W. (2024) · arXiv · doi:10.48550/arxiv.2410.02091 · Cited by 30
Song, F., Agarwal, A., & Wen, W. (2024) · SSRN Electronic Journal (preprint) · doi:10.2139/ssrn.4856935 · Cited by 30
“To ensure the generalizability of our analysis, we follow established procedures in the literature (Kalliamvakou et al. 2014, AlMarzouq et al. 2020) to identify active repositories in the open-source community during our panel period.”
BackgroundMethodInfluentialHuman or LLM? A Comparative Study on Accessible Code Generation Capability
Suh, H., Tafreshipour, M., Malek, S., & Ahmed, I. (2025) · arXiv · doi:10.48550/arxiv.2503.15885 · Cited by 15
SourceFinder: Finding Malware Source-Code from Publicly Available Repositories
Rokon, M. O. F., Islam, R., Darki, A., Papalexakis, V. E., & Faloutsos, M. (2020) · arXiv · doi:10.48550/arxiv.2005.14311 · Cited by 10
Pitfalls and guidelines for using time-based Git data
Flint, S. W., Chauhan, J., & Dyer, R. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10200-y · Cited by 9
An empirical study of policy as code: adoption, purpose, and maintenance
Opdebeeck, R., Alfadel, M., Rahman, A., Kashiwa, Y., Ferreira, J. F., Kula, R. G., & De Roover, C. (2026) · Proceedings of the 23rd International Conference on Mining Software Repositories · doi:10.1145/3793302.3793355 · Cited by 5
Sharing Software-Evolution Datasets: Practices, Challenges, and Recommendations
Broneske, D., Kittan, S., & Krüger, J. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660798 · Cited by 5
Afroz, S., Miller, C., Menezes, T., Gilmour, E., Sarma, A., & Feng, Z. (2026) · arXiv · doi:10.48550/arxiv.2607.04003 · Cited by 1
Behind Developer Contributions on Conflicting Merge Scenarios
Vale, G., Fernandes, E., Figueiredo, E., & Apel, S. (2023) · 2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam59687.2023.00014 · Cited by 1
AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
Ogenrwot, D., & Businge, J. (2026) · Proceedings of the 3rd ACM International Conference on AI-Powered Software · doi:10.1145/3805760.3814923
“To further examine the nature of these conflicts, we analyze conflict severity, measured as the number of conflicting lines per PR. Figure 4 shows the distribution of conflict severity across agents on a logarithmic scale.”
EnvBench: A Benchmark for Automated Environment Setup
Eliseeva, A., Kovrigin, A., Kholkin, I., Bogomolov, E., & Zharov, Y. (2025) · arXiv · doi:10.48550/arxiv.2503.14443
Metadata
- Authors
- Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., & Damian, D.
- Venue
- Proceedings of the 11th Working Conference on Mining Software Repositories
- Publisher
- ACM
- Published
- 2014-05-31
- Pages
- 92-101
- OpenAlex
- W2140952846
- Cited by
- 770
- References
- 1
- FWCI
- 43.89
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
