The do’s and don’ts of infrastructure code: A systematic gray literature review
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Infrastructure-as-code (IaC) is the DevOps tactic of managing and provisioning software infrastructures through machine-readable definition files, rather than manual hardware configuration or interactive configuration tools. From a maintenance and evolution perspective, the topic has picked the interest of practitioners and academics alike, given the relative scarcity of supporting patterns and practices in the academic literature. At the same time, a considerable amount of gray literature exists on IaC. Thus we aim to characterize IaC and compile a catalog of best and bad practices for widely used IaC languages, all using gray literature materials. In this paper, we systematically analyze the industrial gray literature on IaC, such as blog posts, tutorials, white papers using qualitative analysis techniques. We proposed a definition for IaC and distilled a broad catalog summarized in a taxonomy consisting of 10 and 4 primary categories for best practices and bad practices, respectively, both language-agnostic and language-specific ones, for three IaC languages, namely Ansible, Puppet, and Chef. The practices reflect implementation issues, design issues, and the violation of/adherence to the essential principles of IaC. Our findings reveal critical insights concerning the top languages as well as the best practices adopted by practitioners to address (some of) those challenges. We evidence that the field of development and maintenance IaC is in its infancy and deserves further attention.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Experimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012) · doi:10.1007/978-3-642-29044-2 · Cited by 4,460
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1×
“Items found using the search terms have been assessed thoroughly based on various dimensions of quality [56].”
Guarino, N., Oberle, D., & Staab, S. (2009) · Handbook on Ontologies · doi:10.1007/978-3-540-92673-3_0 · Cited by 1,244
Keeling, P. J., Burki, F., Wilcox, H. M., Allam, B., Allen, E. E., Amaral-Zettler, L. A., Armbrust, E. V., Archibald, J. M., Bharti, A. K., Bell, C. J., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001889 · Cited by 1,042
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831
Best Practices for Scientific Computing
Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723 · cited 1×
“For the candidate practice categories, we used the practice categories proposed in the existing literature [48, 49, 50, 51, 52, 53].”
A Survey of DevOps Concepts and Challenges
Leite, L., Rocha, C., Kon, F., Milojicic, D., & Meirelles, P. (2020) · ACM Computing Surveys · doi:10.1145/3359981 · Cited by 484
Qian, B., Su, J., Wen, Z., Jha, D. N., Li, Y., Guan, Y., Puthal, D., James, P., Yang, R., Zomaya, A. Y., et al. (2021) · ACM Computing Surveys · doi:10.1145/3398020 · Cited by 132
Good Enough Practices in Scientific Computing
Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., & Teal, T. K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005510 · Cited by 8 · cited 1×
“For the candidate practice categories, we used the practice categories proposed in the existing literature [48, 49, 50, 51, 52, 53].”
Cited by
Embracing IaC Through the DevSecOps Philosophy: Concepts, Challenges, and a Reference Framework
Alonso, J., Piliszek, R., & Cankar, M. (2023) · IEEE Software · doi:10.1109/ms.2022.3212194 · Cited by 30
Automated infrastructure as code program testing
Sokolowski, D., Spielmann, D., & Salvaneschi, G. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3393070 · Cited by 20
“Kumara et al. [4] and Guerriero et al. [3] explored IaC best practices and issues in the industry through a grey literature survey and practitioner interviews.”
BackgroundMethod
Metadata
- Authors
- Kumara, I., Garriga, M., Romeu, A. U., Di Nucci, D., Palomba, F., Tamburri, D. A., & van den Heuvel, W.-J.
- Venue
- Information and Software Technology
- Publisher
- Elsevier BV
- Published
- 2021-09
- Volume
- 137
- Pages
- 106593
- ISSN
- 0950-5849 (ISSN-L)
- OpenAlex
- W3157440142
- Cited by
- 70
- References
- 9
- FWCI
- 8.33
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The do’s and don’ts of infrastructure code: A systematic gray literature review. https://app.cochat.ai/papers/10.1016/j.infsof.2021.106593
- The paper
- Kumara, I., Garriga, M., Romeu, A. U., Di Nucci, D., Palomba, F., Tamburri, D. A., & van den Heuvel, W.-J. (2021). The do’s and don’ts of infrastructure code: A systematic gray literature review. Information and Software Technology. https://doi.org/10.1016/j.infsof.2021.106593
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.