CoChat
Sign Up
PreprintCited by 1Information and Cyber SecuritySoftware Engineering Techniques and PracticesSoftware Engineering Research

The DOI matches its DataCite record.

Analyzing Challenges in Deployment of the SLSA Framework for Software Supply Chain Security

Open access
Tamanna, Mahzabin, Hamer, Sivana, Tran, Mindy, Fahl, Sascha, Acar, Yasemin, Williams, Laurie·2024·arXiv
DOI 10.48550/arxiv.2409.05014arXiv 2409.05014
Open this paper in CoChat — free accountOpen PDF

Abstract

In 2023, Sonatype reported a 200\% increase in software supply chain attacks, including major build infrastructure attacks. To secure the software supply chain, practitioners can follow security framework guidance like the Supply-chain Levels for Software Artifacts (SLSA). However, recent surveys and industry summits have shown that despite growing interest, the adoption of SLSA is not widespread. To understand adoption challenges, \textit{the goal of this study is to aid framework authors and practitioners in improving the adoption and development of Supply-Chain Levels for Software Artifacts (SLSA) through a qualitative study of SLSA-related issues on GitHub}. We analyzed 1,523 SLSA-related issues extracted from 233 GitHub repositories. We conducted a topic-guided thematic analysis, leveraging the Latent Dirichlet Allocation (LDA) unsupervised machine learning algorithm, to explore the challenges of adopting SLSA and the strategies for overcoming these challenges. We identified four significant challenges and five suggested adoption strategies. The two main challenges reported are complex implementation and unclear communication, highlighting the difficulties in implementing and understanding the SLSA process across diverse ecosystems. The suggested strategies include streamlining provenance generation processes, improving the SLSA verification process, and providing specific and detailed documentation. Our findings indicate that some strategies can help mitigate multiple challenges, and some challenges need future research and tool enhancement.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2409.05014

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Stable ordering, fixed metadata conventions, and repeatable artifact generation reduce ambiguity in post-hoc validation and enable independent re-execution of verification steps by third parties.

    “stable ordering, fixed metadata conventions, and repeatable artifact generation reduce ambiguity in post-hoc validation and enable independent re-execution of verification steps by third parties [5].”

    From Secure Image Processing Pipelines for Evidence-validated Media, with ROI-constrained Planning, Reproducible Execution, and Digitally Signed Evidence Packaging (2026) · cites this paper as [5]

    In “S3. Streamline Provenance Generation Processes” (page unavailable), the paper recommends standardizing provenance generation to reduce confusion and improve build consistency. “LF.1 Limited Attestation Verification” (page unavailable) also says unclear communication of attestation data hinders downstream verification. Neither passage supports the specific claims about stable ordering, fixed metadata conventions, repeatable artifact generation, or independent re-execution by third parties. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 50

  1. "Sonatype's 9th Annual State of the Software Supply Chain Report Reveals Ways to Improve Developer, DevSecOps Effi- ciency," https://www.sonatype.com/press-releases/sonatype-9th-annual- state-of-the-software-supply-chain-report.
  2. M. S. Melara and M. Bowman, "What is software supply chain secu- rity?" arXiv preprint arXiv:2209.04006, 2022. · arXiv
  3. N. K. Tran, S. Pallewatta, and M. A. Babar, "Toward a reference architecture for software supply chain metadata management," arXiv preprint arXiv:2310.06300, 2023. · arXiv
  4. P. Ladisa, S. E. Ponta, A. Sabetta, M. Martinez, and O. Barais, "Journey to the center of software supply chain attacks," IEEE Security & Privacy, 2023.
  5. M. Tran, Y. Acar, M. Cucker, W. Enck, A. Kapravelos, C. Kastner, and L. Williams, "S3c2 summit 2202-09: Industry secure suppy chain summit," arXiv preprint arXiv:2307.15642, 2023. · arXiv
  6. B. Hassanshahi, T. N. Mai, A. Michael, B. Selwyn-Smith, S. Bates, and P. Krishnan, "Macaron: A logic-based framework for software supply chain security assurance," in Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, 2023, pp. 29-37.
  7. S. Torres-Arias, H. Afzali, T. K. Kuppusamy, R. Curtmola, and J. Cap- pos, "in-toto: Providing farm-to-table guarantees for bits and bytes." in USENIX Security Symposium, 2019, pp. 1393-1410.
  8. V. Cosentino, J. L. C. Izquierdo, and J. Cabot, "A systematic mapping study of software development with github," Ieee access, vol. 5, pp. 7173-7192, 2017.
  9. N. Imtiaz, J. Middleton, J. Chakraborty, N. Robson, G. Bai, and E. Murphy-Hill, "Investigating the effects of gender bias on github," in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 700-711.
  10. G. A. A. Prana, C. Treude, F. Thung, T. Atapattu, and D. Lo, "Categoriz- ing the content of github readme files," Empirical Software Engineering, vol. 24, pp. 1296-1327, 2019.
  11. M. Wessel, B. M. De Souza, I. Steinmacher, I. S. Wiese, I. Polato, A. P. Chaves, and M. A. Gerosa, "The power of bots: Characterizing and understanding bots in oss projects," Proceedings of the ACM on Human-Computer Interaction, vol. 2, no. CSCW, pp. 1-19, 2018.
  12. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent dirichlet allocation," Journal of machine Learning research, vol. 3, no. Jan, pp. 993-1022, 2003.
  13. S. J. Uddin, M. Tamanna, A. Albert, and N. Pradhananga, "Leveraging natural language processing to identify health and safety challenges during post-disaster reconstruction," in Construction research congress 2022, 2022, pp. 284-293.
  14. S. J. Uddin, A. Albert, M. Tamanna, and A. Alsharef, "Youtube as a source of information: early coverage of the covid-19 pandemic in the context of the construction industry," Construction Management and Economics, vol. 41, no. 5, pp. 402-427, 2023.
  15. C. C. Silva, M. Galster, and F. Gilson, "Topic modeling in software engineering research," Empirical Software Engineering, vol. 26, no. 6, p. 120, 2021.
  16. R. P. Gauthier, M. J. Costello, and J. R. Wallace, ""i will not drink with you today": A topic-guided thematic analysis of addiction recovery on reddit," in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1-17.
  17. S. Geman and D. Geman, "Stochastic relaxation, gibbs distributions, and the bayesian restoration of images," IEEE Transactions on pattern analysis and machine intelligence, no. 6, pp. 721-741, 1984.
  18. F. Gurcan and N. E. Cagiltay, "Big data software engineering: Analysis of knowledge domains and skill sets using lda-based topic modeling," IEEE access, vol. 7, pp. 82 541-82 552, 2019.
  19. J. L. Campbell, C. Quincy, J. Osserman, and O. K. Pedersen, "Coding in- depth semistructured interviews: Problems of unitization and intercoder reliability and agreement," Sociological methods & research, vol. 42, no. 3, pp. 294-320, 2013.
  20. S. Campbell, M. Greenwood, S. Prior, T. Shearer, K. Walkem, S. Young, D. Bywaters, and K. Walker, "Purposive sampling: complex or simple? research case examples," Journal of research in Nursing, vol. 25, no. 8, pp. 652-661, 2020.
  21. Y. Chandra, L. Shang, Y. Chandra, and L. Shang, "Inductive coding," Qualitative research using R: A systematic approach, pp. 91-106, 2019.
  22. V. Clarke and V. Braun, "Thematic analysis," The journal of positive psychology, vol. 12, no. 3, pp. 297-298, 2017.
  23. D. Wermke, N. Wöhler, J. H. Klemmer, M. Fourné, Y. Acar, and S. Fahl, "Committed to trust: A qualitative study on security & trust in open source software projects," in 2022 IEEE symposium on Security and Privacy (SP). IEEE, 2022, pp. 1880-1896.
  24. S. Amft, S. Höltervennhoff, R. Panskus, K. Marky, and S. Fahl, "Everyone for themselves? a qualitative study about individual security setups of open source software contributors," in In 45th IEEE Symposium on Security and Privacy (S&P'24), 2024.
  25. N. Zahan, T. Zimmermann, P. Godefroid, B. Murphy, C. Maddila, and L. Williams, "What are weak links in the npm supply chain?" in Pro- ceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice, 2022, pp. 331-340.
  26. D. Wermke, J. H. Klemmer, N. Wöhler, J. Schmüser, H. S. Ramulu, Y. Acar, and S. Fahl, ""Always Contribute Back": A Qualitative Study on Security Challenges of the Open Source Supply Chain," in 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023, pp. 1545-1560.
  27. L. Williams, "Trusting trust: Humans in the software supply chain loop," IEEE Security & Privacy, vol. 20, no. 5, pp. 7-10, 2022.
  28. F. Calefato, M. A. Gerosa, G. Iaffaldano, F. Lanubile, and I. Steinmacher, "Will you come back to contribute? investigating the inactivity of oss core developers in github," Empirical Software Engineering, vol. 27, no. 3, p. 76, 2022.
  29. N. Zahan, P. Kanakiya, B. Hambleton, S. Shohan, and L. Williams, "Openssf scorecard: On the path toward ecosystem-wide automated security metrics," IEEE Security & Privacy, 2023.
  30. S. Balali, I. Steinmacher, U. Annamalai, A. Sarma, and M. A. Gerosa, "Newcomers' barriers... is that all? an analysis of mentors' and newcom- ers' barriers in oss projects," Computer Supported Cooperative Work (CSCW), vol. 27, pp. 679-714, 2018.
  31. C. Miller, C. Kästner, and B. Vasilescu, ""we feel like we're winging it:" a study on navigating open-source dependency abandonment," in Proceedings of the ACM SIGSOFT International Symposium on the Foundations of Software Engineering, 2023.
  32. M. Gerosa, I. Wiese, B. Trinkenreich, G. Link, G. Robles, C. Treude, I. Steinmacher, and A. Sarma, "The shifting sands of motivation: Revisiting what drives contributors in open source," in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021, pp. 1046-1058.
  33. M. AlMarzouq, A. AlZaidan, and J. AlDallal, "Mining github for research and education: challenges and opportunities," International Journal of Web Information Systems, vol. 16, no. 4, pp. 451-473, 2020.
  34. E. P. Baumer, D. Mimno, S. Guha, E. Quan, and G. K. Gay, "Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence?" Journal of the Association for Information Science and Technology, vol. 68, no. 6, pp. 1397-1410, 2017.
  35. D. Maier, A. Waldherr, P. Miltner, G. Wiedemann, A. Niekler, A. Kein- ert, B. Pfetsch, G. Heyer, U. Reber, T. Häussler et al., "Applying lda topic modeling in communication research: Toward a valid and reliable methodology," in Computational methods for communication science. Routledge, 2021, pp. 13-38.
  36. C. Liu, S. Chen, L. Fan, B. Chen, Y. Liu, and X. Peng, "Demystifying the vulnerability propagation and its evolution via dependency trees in the npm ecosystem," in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 672-684.
  37. E. Raymond, "The cathedral and the bazaar," Knowledge, Technology & Policy, vol. 12, no. 3, pp. 23-49, 1999.
  38. M. Ohm, H. Plate, A. Sykosch, and M. Meier, "Backstabber's knife collection: A review of open source software supply chain attacks," in Detection of Intrusions and Malware, and Vulnerability Assessment: 17th International Conference, DIMVA 2020, Lisbon, Portugal, June 24-26, 2020, Proceedings 17. Springer, 2020, pp. 23-43.
  39. S. Du, T. Lu, L. Zhao, B. Xu, X. Guo, and H. Yang, "Towards an analysis of software supply chain risk management," in Proceedings of the World Congress on Engineering and Computer Science, vol. 1, 2013.
  40. M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel, "Small world with high risks: A study of security threats in the npm ecosystem," in 28th USENIX Security Symposium (USENIX Security 19), 2019, pp. 995-1010.
  41. P. Ladisa, H. Plate, M. Martinez, and O. Barais, "Sok: Taxonomy of attacks on open-source software supply chains," in 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023, pp. 1509-1526.
  42. L. Williams, S. Migues, J. Boote, and B. Hutchison, "Proactive software supply chain risk management framework (p-sscrm) version 1," arXiv preprint arXiv:2404.12300, 2024. · arXiv
  43. J. Samuel, N. Mathewson, J. Cappos, and R. Dingledine, "Survivable key compromise in software update systems," in Proceedings of the 17th ACM conference on Computer and communications security, 2010, pp. 61-72.
  44. " Hardware-Enforced Integrity and Provenance for Distributed Code Deployments. In NIST Workshop on Enhancing Software Supply Chain Security," Jul 2021.
  45. E. Wyss, L. De Carli, and D. Davidson, "What the fork? finding hidden code clones in npm," in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 2415-2426.
  46. C. Okafor, T. R. Schorlemmer, S. Torres-Arias, and J. C. Davis, "Sok: Analysis of software supply chain security by establishing secure design properties," in Proceedings of the 2022 ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, 2022, pp. 15-24.
  47. W. Enck and L. Williams, "Top five challenges in software supply chain security: Observations from 30 industry and government organizations," IEEE Security & Privacy, vol. 20, no. 2, pp. 96-100, 2022.
  48. T. Dunlap, Y. Acar, M. Cucker, W. Enck, A. Kapravelos, C. Kastner, and L. Williams, "S3c2 summit 2023-02: Industry secure supply chain summit," arXiv preprint arXiv:2307 16557, 2023.
  49. K. G. Kalu, T. Singla, C. Okafor, S. Torres-Arias, and J. C. Davis, "An industry interview study of software signing for supply chain security," arXiv preprint arXiv:2406.08198, 2024. · arXiv
  50. C. E. M. Zottmann, T. M. S. do Amaral, R. R. Nunes, and J. J. C. Gondim, "Comparing software supply chain protection approaches," in 2023 Workshop on Communication Networks and Power Systems (WCNPS). IEEE, 2023, pp. 1-7.

Cited by

Sorted by citations

Metadata

Authors
Tamanna, M., Hamer, S., Tran, M., Fahl, S., Acar, Y., & Williams, L.
Affiliations
  • North Carolina State University (US)
  • Max Planck Institute for Security and Privacy (DE)
  • Helmholtz Center for Information Security (DE)
  • George Washington University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2409.05014
arXiv
2409.05014
OpenAlex
W4403617307
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (2207008)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
SLSA, software supply chain security, GitHub issues

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review