Capabilities and Practices in DevOps: A Multivocal Literature Review
Abstract
Context.To meet the demands of customers and market, Information Technology (IT) organizations are seeking to implement DevOps. While many succeed in DevOps adoption, others lack the knowledge on how to incorporate DevOps culture, process, measurements, and techniques in their business. Thus, successful adoption is still inconsistent, highlighting the need to provide management with relevant information to support the development of DevOps Capabilities effectively. But what are these Capabilities? Unfortunately, there is still a lack of clarity about DevOps Capabilities and their relationships to DevOps Practices and Outcomes among researchers and practitioners.Objective.This research aims to gather community consensus on the relationship between Capabilities and Practices, so a better DevOps implementation can be mapped. Seeking to define DevOps Capabilities and Practices concepts and to identify, organize and summarize Capabilities as they relate to Practices.Method.A Multivocal Literature Review (MLR) is conducted, with 93 documents gathered and thoroughly examined from throughout the community, including books, scientific articles, white papers, and conferences, among others.Results.This survey contributes a list of 37 organized Capabilities, their mentions in literature, and their definitions. The concepts of Practices and Capabilities were mapped and categorized in an ordered taxonomy. It is concluded that industry research has much outweighed scientific research on this topic, with Capabilities evolving dynamically over time, reinforcingteam collaboration and communicationas the most crucial one. The study's Outcomes will assist researchers and practitioners understand how Capabilities and Practices are related at different levels and how to better implement them.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In current literature, the same practice, such as deployment automation, can be mentioned for both DevOps and DevSecOps which indicates overlapping between practices.
“In current literature, the same practice, such as deployment automation, can be mentioned for both DevOps [16] and DevSecOps [51] which indicates overlapping between practices.”
From DevSecOps practices and tools (2025) · cites this paper as [16]
In [16], Amaro et al., discuss and verify that practices and capabilities are interchangeable terms to describe the application of a method within the DevOps context.
“In [16], Amaro et al., discuss and verify that practices and capabilities are interchangeable terms to describe the application of a method within the DevOps context.”
From DevSecOps practices and tools (2025) · cites this paper as [16]
By leveraging the systematic mapping of DevOps practices found in [16] it is possible to compare which practices are mentioned in both approaches and how they are implemented in each approach.
“Additionally, by leveraging the systematic mapping of DevOps practices found in [16] it is possible to compare which practices are mentioned in both approaches and how they are implemented in each approach.”
From DevSecOps practices and tools (2025) · cites this paper as [16]
Although DevSecOps is still a recent topic, MLR has been previously applied to DevOps and DevSecOps.
“Although DevSecOps is still a recent topic, MLR has been previously applied to DevOps [16] and DevSecOps [10].”
From DevSecOps practices and tools (2025) · cites this paper as [16]
The perceived most common DevOps practices in literature are Continuous Integration, Continuous Delivery, Continuous Monitoring, and Continuous Testing.
“The perceived most common DevOps practices in literature are Continuous Integration, Continuous Delivery, Continuous Monitoring, and Continuous Testing [14] [16] [17] [18] [19] [20].”
From DevSecOps practices and tools (2025) · cites this paper as [16]
Cites
The NIST Definition of Cloud Computing
Mell, P., & Grance, T. (2011) · doi:10.6028/nist.sp.800-145 · Cited by 15,338 · see citing works
Performing systematic literature reviews in software engineering
Budgen, D., & Brereton, P. (2006) · International Conference on Software Engineering · doi:10.1145/1134285.1134500 · Cited by 8,824 · see citing works
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1× · see citing works
“In the next phase a snowballing [59] is conducted leading to extra 14 relevant publications found, which increased the total number of papers.”
BackgroundContinuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation
Humble, J., & Farley, D. R. (2010) · Cited by 911 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831 · see citing works
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture
Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2016) · IEEE Software · doi:10.1109/ms.2016.64 · Cited by 735 · see citing works
Shahin, M., Ali Babar, M., & Zhu, L. (2017) · IEEE Access · doi:10.1109/access.2017.2685629 · Cited by 680 · see citing works
Devops: A Software Architect's Perspective
Bass, L., Weber, I. M., & Zhu, L. (2015) · CERN Document Server (European Organization for Nuclear Research) · Cited by 561 · see citing works
Motivation in software engineering: A systematic literature review
Beecham, S., Baddoo, N., Hall, T., Robinson, H., & Sharp, H. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2007.09.004 · Cited by 506 · see citing works
Ebert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016) · IEEE Software · doi:10.1109/ms.2016.68 · Cited by 504 · see citing works
A Survey of DevOps Concepts and Challenges
Leite, L., Rocha, C., Kon, F., Milojicic, D., & Meirelles, P. (2020) · ACM Computing Surveys · doi:10.1145/3359981 · Cited by 484 · see citing works
Toyota Kata: Managing People for Improvement, Adaptiveness and Superior Results
Rother, M. (2009) · Cited by 421 · see citing works
Towards Rigor in Reviews of Multivocal Literatures: Applying the Exploratory Case Study Method
Ogawa, R. T., & Malen, B. (1991) · Review of Educational Research · doi:10.3102/00346543061003265 · Cited by 247 · cited 2× · see citing works
“…DevOps research, in the same area, using MLR already exist [22], [23], [24], thus corroborating the practical usefulness of this method for the proposed research, expanding the diversity of sources that are available in a variety of forms, reflecting different purposes and perspectives [53].”
BackgroundMethodA typology of organisational cultures
Westrum, R. (2004) · BMJ Quality & Safety · doi:10.1136/qshc.2003.009522 · Cited by 247 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2916008 · Cited by 210 · see citing works
A qualitative study of DevOps usage in practice
Erich, F., Amrit, C., & Daneva, M. (2017) · Journal of Software Evolution and Process · doi:10.1002/smr.1885 · Cited by 208 · see citing works
Kim, G. H., Debois, P., Willis, J., & Humble, J. (2016) · Cited by 205 · see citing works
DevSecOps: A Multivocal Literature Review
Myrbakken, H., & Colomo-Palacios, R. (2017) · Communications in Computer and Information Science · doi:10.1007/978-3-319-67383-7_2 · Cited by 205 · see citing works
Systematic review in software engineering
Kitchenham, B. A. (2012) · doi:10.1145/2372233.2372235 · Cited by 198 · see citing works
DevOps in practice: A multiple case study of five companies
Lwakatare, L. E., Kilamo, T., Karvonen, T., Sauvola, T., Heikkilä, V. T., Itkonen, J., Kuvaja, P., Mikkonen, T., Oivo, M., & Lassenius, C. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2019.06.010 · Cited by 197 · see citing works
Value of maturity models in performance measurement
Bititci, U., Garengo, P., Ates, A., & Nudurupati, S. (2014) · International Journal of Production Research · doi:10.1080/00207543.2014.970709 · Cited by 194 · see citing works
DevOps and software quality: A systematic mapping
Mishra, A., & Otaiwi, Z. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100308 · Cited by 177 · see citing works
DevOps Capabilities, Practices, and Challenges: Insights from a Case Study
Senapathi, M., Buchan, J., & Osman, H. (2018) · Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018 · doi:10.1145/3210459.3210465 · Cited by 172 · see citing works
Migrating to Cloud-Native Architectures Using Microservices: An Experience Report
Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2016) · Communications in computer and information science · doi:10.1007/978-3-319-33313-7_15 · Cited by 165 · see citing works
Adopting DevOps in the real world: A theory, a model, and a case study
Luz, W., Pinto, G., & Bonifácio, R. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.07.083 · Cited by 164 · see citing works
DevOps: A Definition and Perceived Adoption Impediments
Smeds, J., Nybom, K., & Porres, I. (2015) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-319-18612-2_14 · Cited by 158 · see citing works
DevOps Adoption Benefits and Challenges in Practice: A Case Study
Riungu-Kalliosaari, L., Mäkinen, S., Lwakatare, L. E., Tiihonen, J., & Männistö, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49094-6_44 · Cited by 154 · see citing works
Towards Definitions for Release Engineering and DevOps
Dyck, A., Penners, R., & Lichter, H. (2015) · doi:10.1109/releng.2015.10 · Cited by 154 · see citing works
Zhu, L., Bass, L., & Champlin-Scharff, G. (2016) · IEEE Software · doi:10.1109/ms.2016.81 · Cited by 150 · see citing works
Technological Capabilities and Learning in African Enterprises
Biggs, T. (1995) · Cited by 148 · see citing works
Applying DevOps Practices of Continuous Automation for Machine Learning
Karamitsos, I., Thabit, S., & Apostolopoulos, C. (2020) · Information · doi:10.3390/info11070363 · Cited by 140 · see citing works
Towards DevOps in the Embedded Systems Domain: Why is It So Hard?
Lwakatare, L. E., Karvonen, T., Sauvola, T., Kuvaja, P., Olsson, H. H., Bosch, J., & Oivo, M. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.671 · Cited by 113 · see citing works
Characterizing DevOps by Hearing Multiple Voices
de França, B. B. N., Jeronimo, H., & Travassos, G. H. (2016) · doi:10.1145/2973839.2973845 · Cited by 109 · see citing works
The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win
Kim, G. H., Behr, K., & Spafford, G. (2013) · Cited by 96 · see citing works
Continuous practices and devops: beyond the buzz, what does it all mean?
Ståhl, D., Mårtensson, T., & Bosch, J. (2017) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2017.8114695 · Cited by 87 · see citing works
Gupta, V., Kapur, P. K., & Kumar, D. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.07.010 · Cited by 81 · see citing works
An Empirical Taxonomy of DevOps in Practice
Macarthy, R. W., & Bass, J. M. (2020) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa51224.2020.00046 · Cited by 74 · see citing works
Advances in Using Agile and Lean Processes for Software Development
Rodríguez, P., Mäntylä, M., Oivo, M., Lwakatare, L. E., Seppänen, P., & Kuvaja, P. (2018) · Advances in computers · doi:10.1016/bs.adcom.2018.03.014 · Cited by 70 · see citing works
Factors that motivate software engineering teams: A four country empirical study
Verner, J. M., Babar, M. A., Cerpa, N., Hall, T., & Beecham, S. (2014) · Journal of Systems and Software · doi:10.1016/j.jss.2014.01.008 · Cited by 69 · see citing works
An Empirical Study on Culture, Automation, Measurement, and Sharing of DevSecOps
Tomas, N., Li, J., & Huang, H. (2019) · 2019 International Conference on Cyber Security and Protection of Digital Services (Cyber Security) · doi:10.1109/cybersecpods.2019.8884935 · Cited by 66 · see citing works
Characteristics of Open Data CSV Files
Mitlöhner, J., Neumaier, S., Jürgen Umbrich, A., & Polleres, A. (2016) · International Conference on Open and Big Data (OBD) · doi:10.1109/obd.2016.18 · Cited by 53 · see citing works
DevOps patterns to scale web applications using cloud services
Cukier, D. (2013) · doi:10.1145/2508075.2508432 · Cited by 52 · see citing works
Agile Infrastructure and Operations: How Infra-gile are You?
Debois, P. (2008) · Agile Conference · doi:10.1109/agile.2008.42 · Cited by 52 · see citing works
Preliminary Findings about DevSecOps from Grey Literature
Mao, R., Zhang, H., Dai, Q., Huang, H., Rong, G., Shen, H., Chen, L., & Lu, K. (2020) · 2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs51102.2020.00064 · Cited by 49 · see citing works
Pereira, R., & Serrano, J. P. (2020) · Journal of Information Technology · doi:10.1177/0268396219886874 · Cited by 46 · see citing works
Self-citationWhy are many businesses instilling a DevOps culture into their organization?
Díaz, J., López-Fernández, D., Pérez, J., & González-Prieto, Á. (2021) · Empirical Software Engineering · doi:10.1007/s10664-020-09919-3 · Cited by 45 · see citing works
Prates, L., Faustino, J., Silva, M., & Pereira, R. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-29608-7_7 · Cited by 45 · see citing works
Self-citationWiedemann, A., Forsgren, N., Wiesche, M., et al. (2019) · Communications of the ACM · doi:10.1145/3331138 · Cited by 43 · see citing works
Adoption Issues in DevOps from the Perspective of Continuous Delivery Pipeline
Toh, M. Z., Sahibuddin, S., & Mahrin, M. N. (2019) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3316615.3316619 · Cited by 41 · see citing works
Towards Process Improvement in DevOps
Badshah, S., Khan, A. A., & Khan, B. (2020) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3383219.3383280 · Cited by 36 · see citing works
Mäkinen, S., Leppänen, M., Kilamo, T., Mattila, A.-L., Laukkanen, E., Pagels, M., & Männistö, T. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.09.001 · Cited by 34 · see citing works
A DevOps approach to integration of software components in an EU research project
Stillwell, M., & Coutinho, J. G. F. (2015) · doi:10.1145/2804371.2804372 · Cited by 33 · see citing works
A Large Agile Organization on Its Journey Towards DevOps
Kuusinen, K., Balakumar, V., Jepsen, S. C., Larsen, S. H., Lemqvist, T. A., Muric, A., Nielsen, A. O., & Vestergaard, O. (2018) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2018.00019 · Cited by 31 · see citing works
Cooperation between developers and operations in software engineering projects
Tessem, B., & Iden, J. (2008) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/1370114.1370141 · Cited by 30 · see citing works
Exploiting devops practices for dependable and secure continuous delivery pipelines
Düllmann, T. F., Paule, C., & van Hoorn, A. (2018) · doi:10.1145/3194760.3194763 · Cited by 29 · see citing works
Teixeira, D., Pereira, R., Henriques, T. A., da Silva, M. M., Jo�ã, N. A., Faustino, J., & da Silva, M. M. (2020) · International Journal of Agile Systems and Management · doi:10.1504/ijasm.2020.112343 · Cited by 27 · see citing works
Self-citationDORA Platform: DevOps Assessment and Benchmarking
Forsgren, N., Tremblay, M. C., VanderMeer, D. E., & Humble, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59144-5_27 · Cited by 25 · see citing works
DevOps Competences and Maturity for Software Producing Organizations
de Feijter, R., Overbeek, S. J., van Vliet, R., Jagroep, E. A., & Brinkkemper, S. (2018) · Lecture notes in business information processing · doi:10.1007/978-3-319-91704-7_16 · Cited by 25 · see citing works
The Role of Continuous Delivery in IT and Organizational PerformancePreprint
Forsgren, N., & Humble, J. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2681909 · Cited by 24 · see citing works
38. Management Information Systems: Managing the Digital Firm : Global Edition.
Laudon, K. C., & Laudon, J. P. (2019) · Cited by 24 · see citing works
Investigating Agile Practices in Software Startups
de Souza, R. M. C. R., Rocha, L., Silva, F. M., & Machado, I. (2019) · Brazilian Symposium on Software Engineering · doi:10.1145/3350768.3350786 · Cited by 22 · see citing works
A Systematic Literature Review on DevOps Capabilities and Areas
Teixeira, D., Pereira, R., Henriques, T. A., da Silva, M. M., & Faustino, J. (2020) · International Journal of Human Capital and Information Technology Professionals · doi:10.4018/ijhcitp.2020040101 · Cited by 22 · see citing works
Self-citationTowards the adoption of DevOps in software product organizations: A maturity model approach
de Feijter, R., van Vliet, R., Jagroep, E. A., et al. (2017) · Utrecht University Repository (Utrecht University) · Cited by 21 · see citing works
Designing a next-generation continuous software delivery system
Steffens, A., Lichter, H., & Döring, J. S. (2018) · doi:10.1145/3194760.3194768 · Cited by 20 · see citing works
Impact Mapping: Making a Big Impact with Software Products and Projects
Adzic, G., & Bisset, M. (2012) · Cited by 19 · see citing works
Managing Quality Assurance Challenges of DevOps through Analytics
Ibrahim, M. M. A., Syed-Mohamad, S. M., & Husin, M. H. (2019) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3316615.3316670 · Cited by 19 · see citing works
A Multivocal Literature Review on the use of DevOps for e-Learning systems
Sánchez‐Gordón, M., & Colomo‐Palacios, R. (2018) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3284179.3284328 · Cited by 16 · see citing works
Pang, C., & Hindle, A. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.45 · Cited by 7 · see citing works
DevOps Capability Assessment in a Software Development Team
Kurkela, M. (2020) · Theseus (Ammattikorkeakoulujen) · Cited by 3 · see citing works
DevOps practices in incident management process
Faustino, J. P. C. (2018) · Repositório do ISCTE-IUL · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Abiona, O. O., Oladapo, O. J., Modupe, O. T., Oyeniran, O. C., Adewusi, A. O., & Komolafe, A. M. (2024) · World Journal of Advanced Engineering Technology and Sciences · doi:10.30574/wjaets.2024.11.2.0093 · Cited by 48 · see citing works
Prates, L., & Pereira, R. (2025) · International Journal of Information Security · doi:10.1007/s10207-024-00914-z · Cited by 49 · see citing works
“Although DevSecOps is still a recent topic, MLR has been previously applied to DevOps [16] and DevSecOps [10].”
InfluentialSelf-citationNavigating the DevOps landscape
Zhang, X., Zhao, P., & Jaskolka, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112331 · Cited by 21 · see citing works
- Authors
- Amaro, R., Pereira, R., & da Silva, M. M.
- Affiliations
- Iscte – Instituto Universitário de Lisboa (PT)
- Instituto Universitário de Lisboa (ISCTE-IUL), Lisboa, Portugal
- Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2023-02-01
- Volume
- 49
- Issue
- 2
- Pages
- 883-901
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4226494241
- Cited by
- 43
- References
- 72
- Citations by year
- 2026: 10 · 2025: 2 · 2024: 19 · 2023: 12
- FWCI
- 8.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Technology Governance and Strategy
- Keywords
- DevOps, DevOps practices, DevOps adoption, team collaboration, communication, IT organizations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
