The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
GrimoireLab: A toolset for software development analytics
Abstract
Background: After many years of research on software repositories, the knowledge for building mature, reusable tools that perform data retrieval, storage and basic analytics is readily available. However, there is still room to improvement in the area of reusable tools implementing this knowledge. Goal: To produce a reusable toolset supporting the most common tasks when retrieving, curating and visualizing data from software repositories, allowing for the easy reproduction of data sets ready for more complex analytics, and sparing the researcher or the analyst of most of the tasks that can be automated. Method: Use our experience in building tools in this domain to identify a collection of scenarios where a reusable toolset would be convenient, and the main components of such a toolset. Then build those components, and refine them incrementally using the feedback from their use in both commercial, community-based, and academic environments. Results: GrimoireLab, an efficient toolset composed of five main components, supporting about 30 different kinds of data sources related to software development. It has been tested in many environments, for performing different kinds of studies, and providing different kinds of services. It features a common API for accessing the retrieved data, facilities for relating items from different data sources, semi-structured storage for easing later analysis and reproduction, and basic facilities for visualization, preliminary analysis and drill-down in the data. It is also modular, making it easy to support new kinds of data sources and analysis. Conclusions: We present a mature toolset, widely tested in the field, that can help to improve the situation in the area of reusable tools for mining software repositories. We show some scenarios where it has already been used. We expect it will help to reduce the effort for doing studies or providing services in this area, leading to advances in reproducibility and comparison of results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
pandas: a Foundational Python Library for Data Analysis and Statistics
McKinney, W. (2011) · Cited by 1,264
Social coding in GitHub: transparency and collaboration in an open software repository
Dabbish, L., Stuart, C., Tsay, J., & Herbsleb, J. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145396 · Cited by 999
PyDriller: Python framework for mining software repositories
Spadini, D., Aniche, M. F., & Bacchelli, A. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264598 · Cited by 359
Boa: A language and infrastructure for analyzing ultra-large-scale software repositories
Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606588 · Cited by 267
GHTorrent: Github's data from a firehose
Gousios, G., & Spinellis, D. D. (2012) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224294 · Cited by 248
Boa: A Language and Infrastructure for Analyzing Ultra-Large-Scale Software Repositories
Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. (2013) · Cited by 246
The ABC of Software Engineering Research
Stol, K.-J., & Fitzgerald, B. (2018) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3241743 · Cited by 245
Collaboration Tools for Global Software Engineering
Lanubile, F., Ebert, C., Prikladnicki, R., & Vizcaíno, A. (2010) · IEEE Software · doi:10.1109/ms.2010.39 · Cited by 227
Bajracharya, S. K., Ngo, T. C., Linstead, E. J., Dou, Y., Rigor, P., Baldi, P., & Lopes, C. V. (2006) · Companion to the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA Companion) · doi:10.1145/1176617.1176671 · Cited by 220
The impact of social media on software engineering practices and tools
Storey, M., Treude, C., Arie Van Deursen, A., & Cheng, L.-T. (2010) · FSE/SDP Workshop on Future of Software Engineering Research (FoSER) · doi:10.1145/1882362.1882435 · Cited by 216
Zhao, Y., Serebrenik, A., Zhou, Y., Filkov, V., & Vasilescu, B. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115619 · Cited by 204
A Systematic Mapping Study of Software Development With GitHub
Cosentino, V., Izquierdo, J. L., & Cabot, J. (2017) · IEEE Access · doi:10.1109/access.2017.2682323 · Cited by 189
Self-citationFLOSSmole: A Collaborative Repository for FLOSS Research Data and Analyses
Howison, J., Conklin, M. S., & Crowston, K. (2009) · IGI Global eBooks · doi:10.4018/978-1-60566-418-7.ch002 · Cited by 148
Process Mining Software Repositories
Poncin, W. W., Serebrenik, A., & van den Brand, M. (2011) · Euromicro Conference on Software Maintenance and Reengineering (CSMR) · doi:10.1109/csmr.2011.5 · Cited by 148
Measuring developer contribution from software repository data
Gousios, G., Kalliamvakou, E., & Spinellis, D. D. (2008) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1370750.1370781 · Cited by 146
Menzies, T., & Zimmermann, T. (2013) · IEEE Software · doi:10.1109/ms.2013.86 · Cited by 145
Modeling and querying data in NoSQL databases
Kaur, K., & Rani, R. (2013) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2013.6691765 · Cited by 142
Software Heritage: Why and How to Preserve Software Source Code
Cosmo, R. D., & Zacchiroli, S. (2017) · Phaidra (Universität Wien) · Cited by 111
Remote analysis and measurement of libre software systems by means of the CVSAnalY tool
Robles, G. (2004) · doi:10.1049/ic:20040351 · Cited by 100
World of Code: An Infrastructure for Mining the Universe of Open Source VCS Data
Ma, Y., Bogart, C., Amreen, S., Zaretzki, R., & Mockus, A. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00031 · Cited by 95
Software Analytics in Practice
Zhang, D., Han, S., Dang, Y., Lou, J., Zhang, H., & Xie, T. (2013) · IEEE Software · doi:10.1109/ms.2013.94 · Cited by 92
Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. (2015) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2803171 · Cited by 89
CODEMINE: Building a Software Development Data Analytics Platform at Microsoft
Czerwonka, J., Nagappan, N., Schulte, W., et al. (2013) · IEEE Software · doi:10.1109/ms.2013.68 · Cited by 67
The MSR Cookbook: Mining a decade of research
Hemmati, H., Nadi, S., Baysal, O., Kononenko, O., Wang, W., Holmes, R., & Godfrey, M. W. (2013) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624048 · Cited by 65
An Extensive Dataset of UML Models in GitHub
Robles, G., Ho-Quang, T., Hebig, R., Chaudron, M. R. V., & Fernández, M. A. C. (2017) · IEEE/ACM International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.48 · Cited by 61
The Software Heritage Graph Dataset: Public Software Development Under One Roof
Pietri, A., Spinellis, D. D., & Zacchiroli, S. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00030 · Cited by 61
Analytics for software development
Buse, R. P. L., & Zimmermann, T. (2010) · FSE/SDP Workshop on Future of Software Engineering Research (FoSER) · doi:10.1145/1882362.1882379 · Cited by 60
FLOSSMetrics: Free/Libre/Open Source Software Metrics
Herraiz, I., Izquierdo-Cortázar, D., & Rivas-Hernández, F. (2009) · Proceedings of the ... European Conference on Software Maintenance and Reengineering/Proceedings of the European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2009.43 · Cited by 59
Dueñas, S., Cosentino, V., Robles, G., & Gonzalez-Barahona, J. M. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3183440.3183475 · Cited by 59
Self-citationTrautsch, F., Herbold, S., Makedonski, P., & Grabowski, J. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9537-x · Cited by 58
Alitheia Core: An extensible software quality monitoring platform
Gousios, G., & Spinellis, D. D. (2009) · doi:10.1109/icse.2009.5070560 · Cited by 56
OpenStack Gender Diversity Report
Izquierdo-Cortázar, D., Huesman, N., Serebrenik, A., & Robles, G. (2018) · IEEE Software · doi:10.1109/ms.2018.2874322 · Cited by 56
Quality of Open Source Software: The QualiPSo Trustworthiness Model
del Bianco, V., Lavazza, L., Morasca, S., & Taibi, D. (2009) · IFIP advances in information and communication technology · doi:10.1007/978-3-642-02032-2_18 · Cited by 52
A Linked Data platform for mining software repositories
Keivanloo, I., Forbes, C., Hmood, A., Erfani, M., Neal, C., Peristerakis, G., & Rilling, J. (2012) · doi:10.1109/msr.2012.6224296 · Cited by 45
Editorial Journal-First Publication for the Software Engineering Community
Dwyer, M. B., & Rosenblum, D. S. (2015) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2837717 · Cited by 43
MetricMiner: Supporting researchers in mining software repositories
Sokol, F. Z., Aniche, M. F., & Gerosa, M. A. (2013) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2013.6648195 · Cited by 41
A Quality Model for Actionable Analytics in Rapid Software DevelopmentPreprint
Martínez‐Fernández, S., Jedlitschka, A., Guzm\'an, L., et al. (2018) · arXiv (Cornell University) · Cited by 36
RSTrace+: Reviewer suggestion using software artifact traceability graphs
Sülün, E., Tüzün, E., & Doğrusöz, U. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106455 · Cited by 36
Comparing repositories visually with repograms
Rozenberg, D., Beschastnikh, I., Kosmale, F., Poser, V., Becker, H., Palyart, M., & Murphy, G. C. (2016) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2901739.2901768 · Cited by 34
Setting Up Government 3.0 Solutions Based on Open Source Software: The Case of X-Road
Robles, G., Gamalielsson, J., & Lundell, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-27325-5_6 · Cited by 32
20-MAD: 20 Years of Issues and Commits of Mozilla and Apache Development
Claes, M., & Mäntylä, M. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387487 · Cited by 32
Maintaining interoperability in open source software: A case study of the Apache PDFBox project
Butler, S., Gamalielsson, J., Lundell, B., Brax, C., Mattsson, A., Gustavsson, T., Feist, J., & Lönroth, E. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.110452 · Cited by 31
The Debsources Dataset: two decades of free and open source software
Caneill, M., Germán, D. M., & Zacchiroli, S. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9461-5 · Cited by 29
Using experience sampling to link software repositories with emotions and work well-being
Kuutila, M., Mäntylä, M. V., Claes, M., Elovainio, M., & Adams, B. (2018) · Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3239235.3239245 · Cited by 28
Telling stories about GNOME with Complicity
Neu, S., Lanza, M., Hattori, L., & D’Ambros, M. (2011) · doi:10.1109/vissof.2011.6069459 · Cited by 28
Robles, G. (2004) · doi:10.1049/ic:20040471 · Cited by 25
The MetricsGrimoire Database Collection
Gonzalez-Barahona, J. M., Robles, G., & Izquierdo-Cortázar, D. (2015) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2015.68 · Cited by 25
Self-citationSoftware Quality Assessment of Open Source Software
Gousios, G., Karakoidas, V., Stroggylos, K., Λουρίδας, Π., Vlachos, V., & Spinellis, D. D. (2007) · Cited by 24
Reviewer Recommendation using Software Artifact Traceability Graphs
Sülün, E., Tüzün, E., & Doğrusöz, U. (2019) · doi:10.1145/3345629.3345637 · Cited by 23
International Journal of Information Technology and Web Engineering
Alkhatib, G. I., Jordan, & Maamar, Z. (2026) · International Journal of Information Technology and Web Engineering · doi:10.4018/ijitwe · Cited by 20
BuCo Reporter: Mining Software and Bug Repositories.
Ligu, E., Chaikalis, T., & Chatzigeorgiou, A. (2013) · Cited by 19
Assessing the Refactoring of Brain Methods
Vidal, S. A., berra, I., Zulliani, S., Marcos, C. A., & Pace, J. A. D. (2018) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3191314 · Cited by 19
OpenHub: a scalable architecture for the analysis of software quality attributes
Farah, G., Tejada, J. S., & Correal, D. (2014) · Working Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597135 · Cited by 18
Towards automatically identifying paid open source developers
Claes, M., Mäntylä, M., Kuutila, M., & Farooq, U. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196447 · Cited by 16
Towards an interdisciplinary, socio-technical analysis of software ecosystem healthPreprint
Mens, T., Adams, B., & Marsan, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04532 · Cited by 15
Developer-Centric Knowledge Mining from Large Open-Source Software Repositories (CROSSMINER)
Bagnato, A., Barmpis, K., Bessis, N., Cabrera-Diego, L. A., Rocco, J. D., Ruscio, D. D., Gergely, T., Hansen, S., Kolovos, D. S., Krief, P., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-74730-9_33 · Cited by 15
On the use of emoticons in open source software development
Claes, M., Mäntylä, M., & Farooq, U. (2018) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3239235.3267434 · Cited by 13
The MetricsGrimoire database collection
Gonzalez-Barahona, J. M., Robles, G., & Izquierdo-Cortázar, D. (2015) · Mining Software Repositories · doi:10.5555/2820518.2820592 · Cited by 12
Self-citationGitana: A software project inspector
Cosentino, V., Izquierdo, J. L., & Cabot, J. (2017) · Science of Computer Programming · doi:10.1016/j.scico.2017.12.002 · Cited by 12
Self-citationOrviz, P., David, M., Dūma, M., Ronchieri, E., Gomes, J., & Salomoni, D. (2020) · Journal of Grid Computing · doi:10.1007/s10723-020-09509-z · Cited by 12
A Mixed Graph-Relational Dataset of Socio-technical Interactions in Open Source Systems
Ashraf, U., Mayr‐Dorn, C., Egyed, A. F., & Panichella, S. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387492 · Cited by 12
Software Development Analytics for Xen: Why and How
Izquierdo-Cortázar, D., Gonzalez-Barahona, J. M., Kurth, L., & Robles, G. (2018) · IEEE Software · doi:10.1109/ms.2018.290101357 · Cited by 11
Self-citationSortingHat: Wizardry on Software Project Members
Moreno, D., Dueñas, S., Cosentino, V., Fernández, M. A. C., Zerouali, A., Robles, G., & Gonzalez-Barahona, J. M. (2019) · IEEE/ACM International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00036 · Cited by 11
Self-citationDevelopment of Intelligent Virtual Assistant for Software Testing Team
Itkin, I., Novikov, A., & Yavorskiy, R. (2019) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c.2019.00036 · Cited by 11
Timezone and time-of-day variance in GitHub teams: an empirical method and study
Dévanbu, P., Kudigrama, P., Rubio-González, C., & Vasilescu, B. (2017) · doi:10.1145/3121257.3121261 · Cited by 9
A Quality Model for Actionable Analytics in Rapid Software Development
Martínez‐Fernández, S., Jedlitschka, A., Guzmán, L., et al. (2018) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2018.00067 · Cited by 8
Research works printed but not linked: 2
- European Parliament and Council of the European Union, 2016, Regulation on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (Data Protection Directive), Official Journal of the European Union, EUR-Lex Document 32016R0679, May 2016, 1
- Rubin, 2007, Process mining framework for software processes, 4470
Metadata
- Authors
- Dueñas, S., Cosentino, V., Gonzalez-Barahona, J. M., del Castillo San Felix, A., Izquierdo-Cortazar, D., Cañas-Díaz, L., & Pérez García-Plaza, A.
- Venue
- PeerJ Computer Science
- Publisher
- PeerJ
- Published
- 2021-07-09
- Volume
- 7
- Pages
- e601
- ISSN
- 2376-5992
- Cited by
- 33
- References
- 66
- Language
- en
- Funders
- Ministerio de Ciencia y Tecnología of Spain (RTI2018-101963-B-I00, RTC-2017-6554-7)
- Ministerio de Economia y Competitividad of Spain (PTQ-15-07709)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.