Gang of eight: a defect taxonomy for infrastructure as code scripts
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Defects in infrastructure as code (IaC) scripts can have serious consequences, for example, creating large-scale system outages. A taxonomy of IaC defects can be useful for understanding the nature of defects, and identifying activities needed to fix and prevent defects in IaC scripts. The goal of this paper is to help practitioners improve the quality of infrastructure as code (IaC) scripts by developing a defect taxonomy for IaC scripts through qualitative analysis. We develop a taxonomy of IaC defects by applying qualitative analysis on 1,448 defect-related commits collected from open source software (OSS) repositories of the Openstack organization. We conduct a survey with 66 practitioners to assess if they agree with the identified defect categories included in our taxonomy. We quantify the frequency of identified defect categories by analyzing 80,425 commits collected from 291 OSS repositories spanning across 2005 to 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Orthogonal defect classification-a concept for in-process measurements
Chillarege, R., Bhandari, I. S., Chaar, J. K., Halliday, M. J., Moebus, D. S., Ray, B. K., & Wong, M.-Y. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.177364 · Cited by 792 · cited 2×
“[9] proposed eight defect categories: algorithm, assignment, build, checking, documentation, function, interface, and timing.”
BackgroundCurating GitHub for engineered software projects
Munaiah, N., Kroh, S., Cabrey, C., & Nagappan, M. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9512-6 · Cited by 365 · cited 2×
“[43] advocated for curation of OSS repositories before conducting empirical analysis.”
BackgroundMethodBalanced Incomplete Block Designs for Inter-Rater Reliability Studies
Fleiss, J. L. (1981) · Applied Psychological Measurement · doi:10.1177/014662168100500115 · Cited by 313
Cited by
Security Smells in Ansible and Chef Scripts: A Replication Study
Rahman, A., Rahman, M. R., Parnin, C., & Williams, L. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3408897 · Cited by 80
Self-citationAutomated infrastructure as code program testing
Sokolowski, D., Spielmann, D., & Salvaneschi, G. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3393070 · Cited by 20
“In contrast to previous, more fine-grained bug taxonomies, e.g., for IaC defects by Rahman et al. [35], we focus purely on the required oracle to find a bug.”
ResultAn empirical study of policy as code: adoption, purpose, and maintenance
Opdebeeck, R., Alfadel, M., Rahman, A., Kashiwa, Y., Ferreira, J. F., Kula, R. G., & De Roover, C. (2026) · Proceedings of the 23rd International Conference on Mining Software Repositories · doi:10.1145/3793302.3793355 · Cited by 5
“Prior work has also empirically studied the occurrence of defects [16, 25, 53, 58, 85] and proposed approaches to detect them [23, 27, 70, 76].”
Self-citationSecuring Infrastructure as Code (IaC) through DevSecOps:A Comprehensive Risk Management Framework
Zeini, A., Lennon, R. G., & Lennon, P. (2023) · 2023 Cyber Research Conference - Ireland (Cyber-RCI) · doi:10.1109/cyber-rci59474.2023.10671452 · Cited by 4
Metadata
- Authors
- Rahman, A., Farhana, E., Parnin, C., & Williams, L.
- Venue
- Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2020-06-27
- Pages
- 752-764
- OpenAlex
- W3090625769
- Cited by
- 65
- References
- 3
- FWCI
- 7.54
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Gang of eight: a defect taxonomy for infrastructure as code scripts. https://app.cochat.ai/papers/10.1145/3377811.3380409
- The paper
- Rahman, A., Farhana, E., Parnin, C., & Williams, L. (2020). Gang of eight: a defect taxonomy for infrastructure as code scripts. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. https://doi.org/10.1145/3377811.3380409
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.