The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index, and 1 are not checked yet.
Pitfalls and guidelines for using time-based Git data
Open accessAbstract
Many software engineering research papers rely on time-based data (e.g., commit timestamps, issue report creation/update/close dates, release dates). Like most real-world data however, time-based data is often dirty. To date, there are no studies that quantify how frequently such data is used by the software engineering research community, or investigate sources of and quantify how often such data is dirty. Depending on the research task and method used, including such dirty data could affect the research results. This paper presents an extended survey of papers that utilize time-based data, published in the Mining Software Repositories (MSR) conference series. Out of the 754 technical track and data papers published in MSR 2004–2021, we saw at least 290 (38%) papers utilized time-based data. We also observed that most time-based data used in research papers comes in the form of Git commits, often from GitHub. Based on those results, we then used the Boa and Software Heritage infrastructures to help identify and quantify several sources of dirty Git timestamp data. Finally we provide guidelines/best practices for researchers utilizing time-based data from Git repositories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The promises and perils of mining GitHub
Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., & Damian, D. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597074 · Cited by 770
An extensive comparison of bug prediction approaches
D'Ambros, M., Lanza, M., & Robbes, R. (2010) · 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010) · doi:10.1109/msr.2010.5463279 · Cited by 637
The promises and perils of mining git
Bird, C., Rigby, P. C., Barr, E. T., Hamilton, D. J., German, D. M., & Devanbu, P. (2009) · 2009 6th IEEE International Working Conference on Mining Software Repositories · doi:10.1109/msr.2009.5069475 · Cited by 311
An in-depth study of the promises and perils of mining GitHub
Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., & Damian, D. (2016) · Empirical Software Engineering · doi:10.1007/s10664-015-9393-5 · Cited by 282
A Large-Scale Study About Quality and Reproducibility of Jupyter Notebooks
Pimentel, J. F., Murta, L., Braganholo, V. P., & Freire, J. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00077 · Cited by 264
Boa: A Language and Infrastructure for Analyzing Ultra-Large-Scale Software Repositories
Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. (2013) · Cited by 246
Self-citationPreprocessing CVS data for fine-grained analysis
Zimmermann, T. (2004) · International Conference on Mining Software Repositories (MSR) · doi:10.1049/ic:20040466 · Cited by 215
An Empirical Analysis of the Docker Container Ecosystem on GitHub
Cito, J., Schermann, G., Wittern, E., Leitner, P., Zumberi, S., & Gall, H. C. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.67 · Cited by 137
Mining duplicate questions in stack overflow
Ahasanuzzaman, M., Asaduzzaman, M., Roy, C. K., & Schneider, K. A. (2016) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2901739.2901770 · Cited by 127
Karampatsis, R.-M., Sutton, C., Informatics, S. O., Institute for Machine Learning, & Intelligence, D. S. A. A. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5281/zenodo.3653444 · Cited by 117
Robles, G. (2010) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2010.5463348 · Cited by 94
Cosentino, V., Luis, J., & Cabot, J. (2016) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2901739.2901776 · Cited by 89
Using dynamic and contextual features to predict issue lifetime in GitHub projects
Kikas, R., Dumas, M., & Pfahl, D. (2016) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2901739.2901751 · Cited by 82
Mining sequences of changed-files from version histories
Kagdi, H., Yusuf, S., & Maletic, J. I. (2006) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1137983.1137996 · Cited by 72
The MSR Cookbook: Mining a decade of research
Hemmati, H., Nadi, S., Baysal, O., Kononenko, O., Wang, W., Holmes, R., & Godfrey, M. W. (2013) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624048 · Cited by 65
Robles, G., Gonzalez-Barahona, J. M., Cervigón, C., Capiluppi, A., & Izquierdo-Cortázar, D. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597107 · Cited by 57
Empirical Study of Restarted and Flaky Builds on Travis CI
Durieux, T., Goues, C. L., Hilton, M. L., & Abreu, R. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387460 · Cited by 37
20-MAD: 20 Years of Issues and Commits of Mozilla and Apache Development
Claes, M., & Mäntylä, M. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387487 · Cited by 32
Happy Birthday! A trend analysis on past MSR papers
Demeyer, S., Murgia, A., Wyckmans, K., & Lamkanfi, A. (2013) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624049 · Cited by 25
The MetricsGrimoire Database Collection
Gonzalez-Barahona, J. M., Robles, G., & Izquierdo-Cortázar, D. (2015) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2015.68 · Cited by 25
(2020) · doi:10.1145/3379597.3387491 · Cited by 24
Antoniol, G., Rollo, V. F., & Venturi, G. (2005) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1083142.1083156 · Cited by 22
A historical dataset for the Gnome ecosystem
Goeminne, M., Claes, M., & Mens, T. (2013) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624032 · Cited by 21
Mining usage data and development artifacts
Baysal, O., Holmes, R., & Godfrey, M. W. (2012) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224305 · Cited by 21
A lightweight approach to technical risk estimation via probabilistic impact analysis
Walker, R. J., Holmes, R., Hedgeland, I., Kapur, P., & Smith, A. C. (2006) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1137983.1138008 · Cited by 20
An Empirical Study on Regular Expression Bugs
Wang, P., Brown, C., Jennings, J. A., & Stolee, K. T. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387464 · Cited by 20
An Empirical Study of Multiple Names and Email Addresses in OSS Version Control Repositories
Zhu, J., & Wei, J. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00068 · Cited by 17
Co-evolution of logical couplings and commits for defect estimation
Steff, M., & Russo, B. (2012) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224283 · Cited by 17
Pietri, A., Rousseau, G., & Zacchiroli, S. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387450 · Cited by 14
Replicating mining studies with SOFAS
Ghezzi, G., & Gall, H. C. (2013) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624050 · Cited by 13
Escaping the Time Pit: Pitfalls and Guidelines for Using Time-Based Git Data
Flint, S. W., Chauhan, J., & Dyer, R. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00022 · Cited by 13
Self-citationStanding on Shoulders or Feet? The Usage of the MSR Data Papers
Kotti, Z., & Spinellis, D. D. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00085 · Cited by 13
An empirical analysis of the FixCache algorithm
Sadowski, C., Lewis, C., Lin, Z., Zhu, X., & Whitehead, E. J. (2011) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1985441.1985475 · Cited by 13
Traceability Support for Multi-Lingual Software Projects
Liu, Y., Lin, J., & Cleland‐Huang, J. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387440 · Cited by 12
Research infrastructure for empirical science of F/OSS
Gasser, L. (2004) · International Conference on Mining Software Repositories (MSR) · doi:10.1049/ic:20040468 · Cited by 11
A multi-level dataset of linux kernel patchwork
Xu, Y., & Zhou, M. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196475 · Cited by 10
Impacts of Daylight Saving Time on Software Development
Hayashi, J., Higo, Y., Matsumoto, S., & Kusumoto, S. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00076 · Cited by 3
Replication package for "Pitfalls and Guidelines for Using Time-Based VCS Data"
Flint, S. W., Chauhan, J., & Dyer, R. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5558290 · Cited by 1
Self-citation
Research works printed but not linked: 1
- Cosmo RD, Zacchiroli S (2017) Software Heritage: why and how to preserve software source code. In: iPRES 2017: 14th international conference on digital preservation. Kyoto, Japan
Cited by
On the suitability of hugging face hub for empirical studies
Ait, A., Izquierdo, J. L., & Cabot, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10608-8 · Cited by 21
Replicability Study: Corpora For Understanding Simulink Models & Projects
Shrestha, S. L., Chowdhury, S. A., & Csallner, C. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304867 · Cited by 10
EvoSL: A Large Open-Source Corpus of Changes in Simulink Models & Projects
Shrestha, S. L., Boll, A., Chowdhury, S. A., Kehrer, T., & Csallner, C. (2023) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1109/models58315.2023.00024 · Cited by 8
OpenLambdaVerse: A Dataset and Analysis of Open-Source Serverless Applications
Moreno, Á. C. C., & Abad, C. L. (2025) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e65552.2025.00032 · Cited by 5
Altered Histories in Version Control System Repositories: Evidence from the Trenches
Rapaport, S., Pautet, L., Tardieu, S., & Zacchiroli, S. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00181 · Cited by 3
Metadata
- Authors
- Flint, S. W., Chauhan, J., & Dyer, R.
- Affiliations
- University of Nebraska–Lincoln (US)
- Corresponding author
- Flint, Samuel W.
- Venue
- Empirical Software Engineering
- Published
- 2022-10-06
- Volume
- 27
- Issue
- 7
- ISSN
- 1382-3256 (ISSN-L)
- 1573-7616
- OpenAlex
- W4302293071
- Cited by
- 9
- References
- 37
- Citations by year
- 2026: 1 · 2025: 6 · 2023: 2
- FWCI
- 1.71
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Empirical Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Lincoln (University of Nebraska) (submittedVersion)
- Lincoln (University of Nebraska) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- dirty data, data provenance, GitHub, Software Heritage, Boas, software repositories
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
