CoChat
Sign Up
Cited by 13Software Engineering ResearchOpen Source Software InnovationsSoftware Engineering Techniques and Practices

Modeling interconnected social and technical risks in open source software ecosystems

Open access
Schueller, William, Wachs, Johannes·2024·Collective Intelligence
DOI 10.1177/26339137241231912
Open this paper in CoChat — free accountOpen PDF

Abstract

Open source software ecosystems consist of thousands of interdependent libraries, which users can combine to great effect. Recent work has pointed out two kinds of risks in these systems: that technical problems like bugs and vulnerabilities can spread through dependency links, and that relatively few developers are responsible for maintaining even the most widely used libraries. However, a more holistic diagnosis of systemic risk in software ecosystem should consider how these social and technical sources of risk interact and amplify one another. Motivated by the observation that the same individuals maintain several libraries within dependency networks, we present a methodological framework to measure risk in software ecosystems as a function of both dependencies and developers. In our models, a library’s chance of failure increases as its developers leave and as its upstream dependencies fail. We apply our method to data from the Rust ecosystem, highlighting several systemically important libraries that are overlooked when only considering technical dependencies. We compare potential interventions, seeking better ways to deploy limited developer resources with a view to improving overall ecosystem health and software supply chain resilience.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/26339137241231912

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The number of GitHub stars is merely a popularity signal, with only moderate correlation to actual usage or importance.

    “it is merely a popularity signal (i.e., the attention received from GitHub users), with only moderate correlation to actual usage or importance as shown in prior studies [56, 61, 80].”

    From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [80]

    The Rankings section (page unavailable) defines GitHub stars as a measure of popularity or social visibility, distinct from downloads as a measure of use. The Introduction (page unavailable) reports a Spearman correlation of about 0.42 between stars and the paper’s measure of systemic importance among the top 1,000 Rust libraries. Our copy does not report a correlation between stars and actual usage, so that part of the claim is unsupported. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 65

  1. Abdalkareem R, Nourry O, Wehaibi S, et al. (2017) Why do developers use trivial packages? an empirical case study on npm. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineeringm, Paderborn, Germany, 4-8 Sep 2017, pp. 385-395.
  2. Alfadel M, Costa DE and Shihab E (2021) Empirical analysis of security vulnerabilities in python packages. In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Honolulu, HI, 12 March 2021, pp. 446-457. IEEE.
  3. Amreen S, Bichescu B, Bradley R, et al. (2019) A methodology for measuring floss ecosystems. In: Towards Engineering Free/Libre Open Source Software (FLOSS) Ecosystems for Impact and Sus- tainability. Salmon Tower Building, New York City: Springer, 1-29.
  4. Avelino G, Passos L, Hora A, et al. (2016) A novel approach for estimating truck factors. In: 2016 IEEE 24th International Conference on Program Comprehension (ICPC), Austin, TX, 17 May 2017, pp. 1-10. IEEE.
  5. Benkler Y, Shaw A and Hill BM (2015) Peer production: a form of collective intelligence. Handbook of Collective Intelligence 175. Cambridge, MA: MIT Press.
  6. Blincoe K, Harrison F and Damian D (2015) Ecosystems in Gi- tHub and a method for ecosystem identification using reference coupling. In: 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, Florence, Italy, 16-17 May 2015, pp. 202-211. IEEE.
  7. Blind K and Schubert T (2023) Estimating the gdp effect of open source software and its complementarities with r&d and pat- ents: evidence and policy implications. The Journal of Tech- nology Transfer 1: 1-26.
  8. Borges H and Tulio Valente M (2018) What's in a GitHub star? understanding repository starring practices in a social coding platform. Journal of Systems and Software 146: 112-129.
  9. Braesemann F, Stoehr N and Graham M (2019) Global networks in collaborative programming. Regional Studies, Regional Sci- ence 6(1): 371-373.
  10. Brown EHP (1957) The meaning of the fitted cobb-douglas function. Quarterly Journal of Economics 71(4): 546-560.
  11. Cataldo M and Herbsleb JD (2013) Coordination breakdowns and their impact on development productivity and software failures. IEEE Transactions on Software Engineering 39(3): 343-360.
  12. Cataldo M, Herbsleb JD and Carley KM (2008) Socio-technical congruence: a framework for assessing the impact of technical and work dependencies on software development productivity. In: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, Kai- serslautern, Germany, 9-10 October, 2008, pp. 2-11.
  13. Champion K and Hill BM (2021) Underproduction: an approach for measuring risk in open source software. In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Honolulu, HI, 9-12 March 2021, pp. 388-399. IEEE.
  14. Chowdhury MAR, Abdalkareem R, Shihab E, et al. (2022) On the untriviality of trivial packages: an empirical study of npm javascript packages. IEEE Transactions on Software Engi- neering 48: 2695-2708.
  15. Coelho J and Valente MT (2017) Why modern open source projects fail Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, Germany, 4- 8 September 2017, pp. 186-196.
  16. Cogo FR, Oliva GA and Hassan AE (2022) Deprecation of packages and releases in software ecosystems: a case study on npm. IEEE Transactions on Software Engineering 48: 2208-2223.
  17. Constantinou E and Mens T (2017) Socio-technical evolution of the ruby ecosystem in GitHub. In: 2017 IEEE 24th Interna- tional Conference on Software Analysis, Evolution and Re- engineering (SANER), Klagenfurt, Austria, 20-24 Feb 2017, pp. 34-44. IEEE.
  18. Cook RI (2020) Above the line, below the line. Communications of the ACM 63(3): 43-46.
  19. Decan A, Mens T and Constantinou E (2018) On the impact of security vulnerabilities in the npm package dependency net- work. In: Proceedings of the 15th International Conference on Mining Software Repositories, Gothenburg, Sweden, 28- 29 May 2018, pp. 181-191.
  20. Decan A, Mens T and Grosjean P (2019) An empirical comparison of dependency network evolution in seven software packaging ecosystems. Empirical Software Engineering 24(1): 381-416.
  21. Decan A, Mens T, Zerouali A, et al. (2022) Back to the past- analysing backporting practices in package dependency net- works. IEEE Transactions on Software Engineering 48: 4087-4099.
  22. Dey T and Mockus A (2020) Effect of technical and social factors on pull request quality for the npm ecosystem. In: Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Bari, Italy, 5 -9 October, 2020, pp. 1-11.
  23. Durumeric Z, Li F, Kasten J, et al. (2014) The matter of heartbleed. In: Proceedings of the 2014 Conference on Internet Mea- surement Conference, Vancouver, Canada, 5-7 November 2014, pp. 475-488.
  24. Eghbal N (2020) Working in Public: The Making and Maintenance of Open Source Software. San Francisco and Dublin: Stripe Press.
  25. Ferreira M, Mombach T, Valente MT, et al. (2019) Algorithms for estimating truck factors: a comparative study. Software Quality Journal 27(4): 1583-1617.
  26. Golzadeh M, Decan A, Legay D, et al. (2021) A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments. Journal of Systems and Software 175: 110911.
  27. Gote C, Scholtes I and Schweitzer F (2021) Analysing time- stamped co-editing networks in software development teams using git2net. Empirical Software Engineering 26(4): 1-41.
  28. Greenstein S and Nagle F (2014) Digital dark matter and the economic contribution of Apache. Research Policy 43(4): 623-631.
  29. Hejderup J, van Deursen A and Gousios G (2018) Software eco- system call graph for dependency management. In: 2018 IEEE/ ACM 40th International Conference on Software Engineering: New Ideas and Emerging Technologies Results (ICSE-NIER), Gothenburg, Sweden, 27 May 2018, pp. 101-104. IEEE.
  30. Hejderup J, Beller M, Triantafyllou K, et al. (2022) Präzi: from package-based to call-based dependency networks. Empirical Software Engineering 27(5): 102-142.
  31. Kinney R, Crucitti P, Albert R, et al. (2005) Modeling cascading failures in the north american power grid. The European Physical Journal B 46(1): 101-107.
  32. Lamb C and Zacchiroli S (2021) Reproducible Builds: Increasing the Integrity of Software Supply Chains. New York City, NY: IEEE Software.
  33. Lehman MM (1980) Programs, life cycles, and laws of software evolution. Proceedings of the IEEE 68(9): 1060-1076.
  34. Luszcz J (2018) Apache struts 2: how technical and development gaps caused the equifax breach. Network Security 2018(1): 5-8.
  35. Ma Y (2018) Constructing supply chains in open source software. In: 2018 IEEE/ACM 40th International Conference on Soft- ware Engineering: Companion (ICSE-Companion), Gothen- burg, Sweden, 27 May 2018, pp. 458-459. IEEE.
  36. Mens T and Grosjean P (2015) The ecology of software ecosys- tems. Computer 48(10): 85-87.
  37. Miller C, Widder DG, Kästner C, et al. (2019) Why do people give up flossing? a study of contributor disengagement in open source. In: IFIP International Conference on Open Source Systems, Mon- treal, QC, Canada, 26-27 May 2019, pp. 116-129. Springer.
  38. Mockus A (2009) Succession: measuring transfer of code and developer productivity. In: 2009 IEEE 31st International Conference on Software Engineering, Washington, DC, 16- 24 May 2009, pp. 67-77. IEEE.
  39. Ohm M, Plate H, Sykosch A, et al. (2020) Backstabber's knife collection: a review of open source software supply chain attacks. In: International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, Lisbon, Portugal, 24-26 June 2009, pp. 23-43. Springer.
  40. Overney C, Meinicke J, Kästner C, et al. (2020) How to not get rich: an empirical study of donations in open source. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul, Korea, 5-11 Oct 2020, pp. 1209-1221.
  41. O'Neil M, Muselli L, Raissi M, et al. (2021) 'Open source has won and lost the war': legitimising commercial-communal hybridisation in a FOSS project. New Media & Society 23(5): 1157-1180.
  42. Pashchenko I, Plate H, Ponta SE, et al. (2018) Vulnerable open source dependencies: counting those that matter. In: Pro- ceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, New York, NY, 11-12 Oct 2018, pp. 1-10.
  43. Peters K, Buzna L and Helbing D (2008) Modelling of cascading effects and efficient response to disaster spreading in complex networks. International Journal of Critical Infrastructures 4(1- 2): 46-62.
  44. Pfeiffer RH (2021) Identifying critical projects via pagerank and truck factor. In: 2021 IEEE/ACM 18th International Confer- ence on Mining Software Repositories (MSR), Madrid, Spain, 22-30 May 2021, pp. 41-45. IEEE.
  45. Poledna S, Molina-Borboa JL, Martínez-Jaramillo S, et al. (2015) The multi-layer network nature of systemic risk and its im- plications for the costs of financial crises. Journal of Financial Stability 20: 70-81.
  46. Raymond E (1999) The cathedral and the bazaar. Knowledge, Technology & Policy 12(3): 23-49.
  47. Lo Sardo DR, Thurner S, Sorger J, et al. (2019) Quantification of the resilience of primary care networks by stress testing the health care system. Proceedings of the National Academy of Sciences of the United States of America 116(48): 23930-23935.
  48. Schneider CM, Yazdani N, Araújo NA, et al. (2013) Towards de- signing robust coupled networks. Scientific Reports 3(1): 1-7.
  49. Schueller W, Wachs J, Servedio VD, et al. (2022) Evolving col- laboration, dependencies, and use in the rust open source software ecosystem. Scientific Data 9(1): 703.
  50. Spaeth S, von Krogh G and He F (2015) Research note-perceived firm attributes and intrinsic motivation in sponsored open source software projects. Information Systems Research 26(1): 224-237.
  51. Steinmacher I, Conte T, Gerosa MA, et al. (2015) Social barriers faced by newcomers placing their first contribution in open source software projects. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada, 14-18 March 2015, pp. 1379-1392.
  52. Steinmacher I, Balali S, Trinkenreich B, et al. (2021) Being a mentor in open source projects. Journal of Internet Services and Applications 12(1): 7-33.
  53. Takhteyev Y (2012) Coding Places: Software Practice in a South American City. Cambridge, Massachusetts: MIT Press.
  54. Thurner S and Poledna S (2013) Debtrank-transparency: con- trolling systemic risk in financial networks. Scientific Reports 3(1): 1-7.
  55. Torchiano M, Ricca F and Marchetto A (2011) Is my project's truck factor low? theoretical and empirical considerations about the truck factor threshold. In: Proceedings of the 2Nd Inter- national Workshop on Emerging Trends in Software Metrics, Honolulu, HI, May 24, 2011, pp. 12-18.
  56. Trinkenreich B, Guizani M, Wiese I, et al. (2020) Hidden figures: roles and pathways of successful OSS contributors. Proceed- ings of the ACM on Human-Computer Interaction 4(CSCW2): 1-22.
  57. Valiev M, Vasilescu B and Herbsleb J (2018) Ecosystem-level determinants of sustained activity in open-source projects: a case study of the pypi ecosystem. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engi- neering Conference and Symposium on the Foundations of Software Engineering, Lake Buena Vista FL, 4-9 Nov 2018, pp. 644-655.
  58. Vasilescu B, Capiluppi A and Serebrenik A (2014) Gender, rep- resentation and online participation: a quantitative study. In- teracting with Computers 26(5): 488-511.
  59. Vasilescu B, Blincoe K, Xuan Q, et al. (2016) The sky is not the limit: multitasking across GitHub projects. In: Proceedings of the 38th International Conference on Software Engineering, Austin Texas, 14 May 2016, pp. 994-1005.
  60. Wachs J (2023) Digital traces of brain drain: developers during the Russian invasion of Ukraine. EPJ Data Science 12(1): 14.
  61. Wang Y, Wen M, Liu Y, et al. (2020) Watchman: monitoring dependency conflicts for python library ecosystem. In: Pro- ceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul, South Korea, 27 June 2020, pp. 125-135.
  62. Williams L and Kessler RR (2003) Pair Programming Illuminated. Glenview, IL: Addison-Wesley Professional.
  63. Zerouali A, Mens T, Decan A, et al (2021) On the Impact of Security Vulnerabilities in the NPM and Rubygems Depen- dency Networks. arXiv preprint arXiv:2106.06747. · arXiv
  64. Zheng X, Wan Z, Zhang Y, et al. (2023) A closer look at the security risks in the rust ecosystem. ACM Transactions on Software Engineering and Methodology 33: 1-30.
  65. Zimmermann M, Staicu CA, Tenny C, et al. (2019) Small world with high risks: a study of security threats in the npm ecosystem. In: 28th USENIX Security Symposium (USE- NIX Security 19), Santa Clara, CA, 28 May 2019, pp. 995-1010.

Cited by

Sorted by citations

Authors
Schueller, W., & Wachs, J.
Affiliations
  • University of Veterinary Medicine Vienna (AT)
  • Corvinus University of Budapest (HU)
Corresponding author
Wachs, Johannes
ORCID
Venue
Collective Intelligence
Publisher
SAGE Publications
Published
2024-01
Volume
3
Issue
1
ISSN
2633-9137 (ISSN-L)
DOI
10.1177/26339137241231912
OpenAlex
W4391855073
Cited by
13
References
3
Citations by year
2026: 5 · 2025: 5 · 2024: 3
FWCI
4.90
Type
journal article
Language
en
Open access
Licence
Locations
  • Collective Intelligence (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • Repository of the Academy's Library (Library of the Hungarian Academy of Sciences) (submittedVersion)
  • Corvinus Research Archive (Corvinus University of Budapest) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
  • European Research Executive Agency (101086712-LearnData-HORIZON-WIDERA-2022-TALENTS-01)
  • Hungarian Scientific Research Fund (OTKA FK-145960)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
open source ecosystem, software dependency networks, systemic risk, vulnerability propagation, ecosystem health

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review