CoChat
Sign Up
Cited by 84Software Engineering Techniques and PracticesSoftware Engineering ResearchAdvanced Software Engineering Methodologies

Selecting third-party libraries: the practitioners’ perspective

Open access
Larios Vargas, Enrique, Aniche, Maurício, Treude, Christoph, Bruntink, Magiel, Gousios, Georgios·2020·Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
DOI 10.1145/3368089.3409711
Open this paper in CoChat — free accountOpen PDF

Abstract

The selection of third-party libraries is an essential element of virtually any software development project. However, deciding which libraries to choose is a challenging practical problem. Selecting the wrong library can severely impact a software project in terms of cost, time, and development effort, with the severity of the impact depending on the role of the library in the software architecture, among others. Despite the importance of following a careful library selection process, in practice, the selection of third-party libraries is still conducted in an ad-hoc manner, where dozens of factors play an influential role in the decision.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3368089.3409711

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The number of GitHub stars is widely used when stakeholders evaluate, select, and adopt open-source projects into their software supply chain.

    “Many studies provided evidence that the number of GitHub stars is widely used when stakeholders evaluate, select, and adopt opensource projects into their software supply chain [33, 61, 71, 76, 89, 95].”

    From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [89]

    In 3.2 Human Factors (p. 6), the paper reports that three interviewees used star counts when selecting libraries, while most considered popularity influential. This supports star counts as one selection metric, but not the broader claim that GitHub stars are widely used across stakeholders evaluating and adopting open-source projects. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Practitioners use star counts as an important (and often the most important) signal for decisions, including high-stakes decisions like open-source component adoption and repository reputation evaluation.

    “Yet, study after study shows how practitioners use star counts as an important (and often as the most important) signal for all kinds of decisions, including high-stakes decisions like open-source component adoption [69, 89, 95] and repository reputation evaluation [76, 88].”

    From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [89]

    In 3.2 Human Factors (p. 6), the paper reports that practitioners consider library popularity when selecting third-party libraries and that three interviewees use star counts as one measure. It does not establish that star counts themselves are often the most important signal; the reported influence ratings concern popularity more broadly. The repository-reputation portion of the citing sentence is attached to other citations, not this paper. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 27

  1. [1][1] Rabe Abdalkareem, Olivier Nourry, Sultan Wehaibi, Suhaib Mujahid, and Emad Shihab. 2017. Why Do Developers Use Trivial Packages? An Empirical Case Study on Npm. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017) . Association for Computing Machinery, New York, NY, USA, 385-395. https://doi.org/10.1145/ 3106237.3106267 -
  2. [2][2] David Arnott. 2005. Cognitive biases and decision support systems development: a design science approach. Information Systems Journal 16, 1 (2018/09/28 2005), 55-78. https://doi.org/10.1111/j.1365-2575.2006.00208.x - · DOI
  3. [4][4] Fernando López de la Mora and Sarah Nadi. 2018. An Empirical Study of Metricbased Comparisons of Software Libraries. In Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering (Oulu, Finland) (PROMISE'18) . ACM, New York, NY, USA, 22-31. https://doi.org/10. 1145/3273934.3273937 -
  4. [5][5] Fernando López de la Mora and Sarah Nadi. 2018. Which Library Should I Use?: A Metric-based Comparison of Software Libraries. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging 6. Results (Gothenburg, Sweden) (ICSE-NIER '18) . ACM, New York, NY, USA, 37-40. https://doi.org/10.1145/3183399.3183418 - · DOI
  5. [7][7] Egon G Guba. 1981. Criteria for assessing the trustworthiness of naturalistic inquiries. Educational Technology research and development 29, 2 (1981), 75-91. -
  6. [6][6] Andreas Gizas, Sotiris Christodoulou, and Theodore Papatheodorou. 2012. Comparative Evaluation of Javascript Frameworks. In Proceedings of the 21st International Conference on World Wide Web (Lyon, France) (WWW '12 Companion) . Association for Computing Machinery, New York, NY, USA, 513-514. https://doi.org/10.1145/2187980.2188103 - · DOI
  7. [8][8] Harry H. Hiller and Linda Diluzio. 2004. The Interviewee and the Research Interview: Analysing a Neglected Dimension in Research*. Canadian Review of Sociology/Revue canadienne de sociologie 41, 1 (2004), 1-26. https://doi.org/10. 1111/j.1755-618X.2004.tb02167.x -
  8. [10][10] Siw Elisabeth Hove and Bente Anda. 2005. Experiences from conducting semistructured interviews in empirical software engineering research. In Software metrics, 2005. 11th ieee international symposium . IEEE, 10-pp. -
  9. [9][9] A. Hora and M. T. Valente. 2015. Apiwave: Keeping track of API popularity and migration. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) , Vol. 00. 321-323. https://doi.org/10.1109/ICSM.2015.7332478 - · DOI
  10. [11][11] Irene Korstjens and Albine Moser. 2018. Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. European Journal of General Practice 24, 1 (2018), 120-124. -
  11. [12][12] R. G. Kula, D. M. German, T. Ishio, and K. Inoue. 2015. Trusting a library: A study of the latency to adopt the latest Maven release, In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER) , 520-524. https://doi.org/10.1109/SANER.2015.7081869 - · DOI
  12. [14][14] Enrique Larios Vargas, Joseph Hejderup, Maria Kechagia, Magiel Bruntink, and Georgios Gousios. 2018. Enabling Real-time Feedback in Software Engineering. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results (Gothenburg, Sweden) (ICSE-NIER '18) . ACM, New York, NY, USA, 21-24. https://doi.org/10.1145/3183399.3183416 - · DOI
  13. [16][16] Caroline Lima and Andre Hora. 2019. What are the characteristics of popular APIs? A large-scale study on Java, Android, and 165 libraries. Software Quality Journal (2019). https://doi.org/10.1007/s11219-019-09476-z - · DOI
  14. [15][15] M. Li, W. Wang, P. Wang, S. Wang, D. Wu, J. Liu, R. Xue, and W. Huo. 2017. LibD: Scalable and Precise Third-Party Library Detection in Android Markets. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) . 335-346. https://doi.org/10.1109/ICSE.2017.38 - · DOI
  15. [17][17] Yana Momchilova Mileva, Valentin Dallmeier, Martin Burger, and Andreas Zeller. 2009. Mining Trends of Library Usage. In Proceedings of the Joint International and Annual ERCIM Workshops on Principles of Software Evolution (IWPSE) and Software Evolution (Evol) Workshops (Amsterdam, The Netherlands) (IWPSE-Evol '09) . ACM, New York, NY, USA, 57-62. https://doi.org/10.1145/1595808.1595821 - · DOI
  16. [19][19] Katherine L. Milkman, Dolly Chugh, and Max H. Bazerman. 2009. How Can Decision Making Be Improved? Perspectives on psychological science : a journal of the Association for Psychological Science 4 4 (2009), 379-83. -
  17. [18][18] Yana Momchilova Mileva, Valentin Dallmeier, and Andreas Zeller. 2010. Mining API Popularity. In Testing -Practice and Research Techniques , Leonardo Bottaci and Gordon Fraser (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 173-180. -
  18. [20][20] I. J. Mojica, B. Adams, M. Nagappan, S. Dienst, T. Berger, and A. E. Hassan. 2014. A Large-Scale Empirical Study on Software Reuse in Mobile Apps. IEEE Software 31, 2 (Mar 2014), 78-86. https://doi.org/10.1109/MS.2013.142 - · DOI
  19. [22][22] Phuong T. Nguyen, Juri Di Rocco, Davide Di Ruscio, and Massimiliano Di Penta. 2020. CrossRec: Supporting software developers by recommending third-party libraries. Journal of Systems and Software 161 (2020), 110460. https://doi.org/10. 1016/j.jss.2019.110460 -
  20. [21][21] Brad A. Myers and Jeffrey Stylos. 2016. Improving API Usability. Commun. ACM 59, 6 (May 2016), 62-69. https://doi.org/10.1145/2896587 - · DOI · Paper page
  21. [23][23] Amantia Pano, Daniel Graziotin, and Pekka Abrahamsson. 2016. What leads developers towards the choice of a JavaScript framework? CoRR abs/1605.04303 (2016). arXiv:1605.04303 http://arxiv.org/abs/1605.04303 - · arXiv
  22. [25][25] Anselm Strauss and Juliet M Corbin. 1997. Grounded theory in practice . Sage Publications, Inc. -
  23. [24][24] M. Piccioni, C. A. Furia, and B. Meyer. 2013. An Empirical Study of API Usability. In 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement . 5-14. https://doi.org/10.1109/ESEM.2013.14 - · DOI
  24. [26][26] Viswanath Venkatesh, James Y. L. Thong, and Xin Xu. 2012. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly 36, 1 (2012), 157-178. http://www.jstor.org/stable/41410412 -
  25. [27][27] Bowen Xu, Le An, Ferdian Thung, Foutse Khomh, and David Lo. 2020. Why reinventing the wheels? An empirical study on library reuse and re-implementation. Empirical Software Engineering 25, 1 (2020), 755-789. https://doi.org/10.1007/ s10664-019-09771-0 -
  26. [28][28] Y. Yano, R. G. Kula, T. Ishio, and K. Inoue. 2015. VerXCombo: An Interactive Data Visualization of Popular Library Version Combinations, In 2015 IEEE 23rd International Conference on Program Comprehension. 2015 IEEE 23rd International Conference on Program Comprehension , 291-294. https://doi.org/10.1109/ICPC. 2015.43 - · DOI
  27. [29][29] Asimina Zaimi, Apostolos Ampatzoglou, Noni Triantafyllidou, Alexander Chatzigeorgiou, Androklis Mavridis, Theodore Chaikalis, Ignatios Deligiannis, Panagiotis Sfetsos, and Ioannis Stamelos. 2015. An Empirical Study on the Reuse of Third-Party Libraries in Open-Source Software Development. In Proceedings of the 7th Balkan Conference on Informatics Conference (Craiova, Romania) (BCI '15) . Association for Computing Machinery, New York, NY, USA, Article Article 4, 8 pages. https://doi.org/10.1145/2801081.2801087 · DOI

Cited by

Sorted by citations

Authors
Larios Vargas, E., Aniche, M., Treude, C., Bruntink, M., & Gousios, G.
Affiliations
  • Software Improvement Group, Netherlands
  • Delft University of Technology, Netherlands
  • The University of Adelaide (AU)
  • Delft University of Technology (NL)
ORCID
Venue
Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
Publisher
ACM
Published
2020-11-08
Pages
245-256
DOI
10.1145/3368089.3409711
OpenAlex
W3031471692
Cited by
84
References
0
Citations by year
2026: 10 · 2025: 17 · 2024: 16 · 2023: 17 · 2022: 11 · 2021: 9 · 2020: 4
FWCI
7.77
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Research Repository (Delft University of Technology) (submittedVersion, cc-by-nc-nd)
  • Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • H2020 (825328)
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek (628.008.001)
  • ARC (DE180100153)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
Keywords
third-party libraries, library selection, software development, software architecture, software reuse, practitioners' perspectives

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review