What is DevOps?: A Systematic Mapping Study on Definitions and Practices
Abstract
Context: DevOps, the combination of Development and Operations, is a new way of thinking in the software engineering domain that recently received much attention. Given that DevOps is a new term and novel concept recently introduced, no common understanding of what it entails has been achieved yet. Consequently, definitions of DevOps often only represent a part that is relevant to the concept.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The perceived benefits of adopting DevOps found in the literature are Increased Release frequency, improved quality assurance (QA), and enhanced collaboration and communication.
“The perceived benefits of adopting DevOps found in the literature are Increased Release frequency, improved quality assurance (QA), and enhanced collaboration and communication.[19] [20] [21] [22] [23]”
From DevSecOps practices and tools (2025) · cites this paper as [20]
Cites
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669 · see citing works
Self-citationA systematic review of systematic review process research in software engineering
Kitchenham, B., & Brereton, P. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.07.010 · Cited by 1,094 · see citing works
Continuous software engineering: A roadmap and agenda
Fitzgerald, B. F., & Stol, K.-J. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.06.063 · Cited by 742 · see citing works
Guide to the Software Engineering Body of Knowledge : 2004 Version
Abran, A., Moore, J. W., Bourque, P., Dupuis, R., & Tripp, L. L. (2005) · Medical Entomology and Zoology · Cited by 375 · see citing works
What’s up with software metrics? – A preliminary mapping study
Kitchenham, B. A. (2009) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.041 · Cited by 253 · see citing works
Development and Deployment at Facebook
Feitelson, D. G., Frachtenberg, E., & Beck, K. L. (2013) · IEEE Internet Computing · doi:10.1109/mic.2013.25 · Cited by 243 · see citing works
Continuous software engineering and beyond: trends and challenges
Fitzgerald, B., & Stol, K.-J. (2014) · Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering · doi:10.1145/2593812.2593813 · Cited by 212 · see citing works
The Journal of Systems and Software
Binkley, D., Gold, N., Harman, M., et al. (1986) · IEEE Software · doi:10.1109/ms.1986.234074 · Cited by 176 · see citing works
DevOps: A Definition and Perceived Adoption Impediments
Smeds, J., Nybom, K., & Porres, I. (2015) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-319-18612-2_14 · Cited by 158 · see citing works
Towards Definitions for Release Engineering and DevOps
Dyck, A., Penners, R., & Lichter, H. (2015) · doi:10.1109/releng.2015.10 · Cited by 154 · see citing works
An empirically based terminology and taxonomy for global software engineering
Šmite, D., Wohlin, C., Galviņa, Z., et al. (2012) · Empirical Software Engineering · doi:10.1007/s10664-012-9217-9 · Cited by 140 · see citing works
Adopting DevOps practices in quality assurance
Roche, J. (2013) · Communications of the ACM · doi:10.1145/2524713.2524721 · Cited by 133 · see citing works
Understanding DevOps & bridging the gap from continuous integration to continuous delivery
Virmani, M. (2015) · International Conference on the Innovative Computing Technology (INTECH) · doi:10.1109/intech.2015.7173368 · Cited by 132 · see citing works
33 rd International Conference on
Koch, C. E., Garofalakis, M., Deshpande, A. B., Chan, C.-Y., Kanne, C.-C., & Neuhold, E. J. (2007) · Cited by 113 · see citing works
Parnas, D. L. (2007) · Communications of the ACM · doi:10.1145/1297797.1297815 · Cited by 85 · see citing works
Streamlining DevOps automation for Cloud applications using TOSCA as standardized metamodel
Wettinger, J., Breitenbücher, U., Kopp, O., et al. (2015) · Future Generation Computer Systems · doi:10.1016/j.future.2015.07.017 · Cited by 84 · see citing works
Fuggetta, A., & Nitto, E. D. (2014) · Future of Software Engineering (FOSE) · doi:10.1145/2593882.2593883 · Cited by 81 · see citing works
Modern DevOps: Optimizing software development through effective system interactions
Cois, C. A., Yankel, J., & Connell, A. (2014) · IEEE International Professional Communication Conference (ProComm) · doi:10.1109/ipcc.2014.7020388 · Cited by 74 · see citing works
A systematic literature review on the industrial use of software process simulation
bin Ali, N., Petersen, K., & Wohlin, C. (2014) · Journal of Systems and Software · doi:10.1016/j.jss.2014.06.059 · Cited by 72 · see citing works
Self-citationEvaluating strategies for study selection in systematic literature studies
bin Ali, N., & Petersen, K. (2014) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2652524.2652557 · Cited by 63 · see citing works
Self-citationA Mapping Study on Cooperation between Information System Development and Operations
Erich, F., Amrit, C., & Daneva, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13835-0_21 · Cited by 62 · see citing works
On fault resilience of OpenStack
Ju, X., Soares, L., Shin, K. G., et al. (2013) · Symposium on Cloud Computing · doi:10.1145/2523616.2523622 · Cited by 60 · see citing works
A grounded theory analysis of modern web applications
Bang, S. K., Chung, S., Choh, Y., & Dupuis, M. (2013) · ACM Annual Conference on Research in Information Technology (RIIT) · doi:10.1145/2512209.2512229 · Cited by 56 · see citing works
A Framework for Managing Mission Needs, Compliance, and Trust in the DevOps Environment
Farroha, B. S., & Farroha, D. L. (2014) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom.2014.54 · Cited by 56 · see citing works
Agile Software Development Practice Adoption Survey
Kurapati, N., Manyam, V. S. C., & Petersen, K. (2012) · Lecture notes in business information processing · doi:10.1007/978-3-642-30350-0_2 · Cited by 54 · see citing works
Self-citationAutomated Capturing and Systematic Usage of DevOps Knowledge for Cloud Applications
Wettinger, J., Andrikopoulos, V., & Leymann, F. (2015) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e.2015.23 · Cited by 49 · see citing works
DevOpSlang – Bridging the Gap between Development and Operations
Wettinger, J., Breitenbücher, U., & Leymann, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44879-3_8 · Cited by 47 · see citing works
ResearchOps: The case for DevOps in scientific applications
BAYSER, M. P. M. A. D., Azevedo, L. G., & Cerqueira, R. (2015) · IFIP/IEEE International Symposium on Integrated Network Management (IM) · doi:10.1109/inm.2015.7140503 · Cited by 45 · see citing works
Unifying Cloud and Carrier Network: EU FP7 Project UNIFY
Császár, A., John, W., Kind, M., et al. (2013) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc.2013.89 · Cited by 43 · see citing works
Integrating Configuration Management with Model-driven Cloud Management based on TOSCA
Wettinger, J., Behrendt, M., Binz, T., et al. (2013) · International Conference on Cloud Computing and Services Science · doi:10.5220/0004376204370446 · Cited by 43 · see citing works
CloudWave: Where adaptive cloud management meets DevOps
Bruneo, D., Fritz, T., Keidar-Barner, S., Leitner, P., Longo, F., Marquezan, C. C., Metzger, A., Pohl, K., Puliafito, A., Raz, D., et al. (2014) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc.2014.6912638 · Cited by 39 · see citing works
HPC-ABDS High Performance Computing Enhanced Apache Big Data Stack
Fox, G., Qiu, J., Kamburugamuve, S., et al. (2015) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2015.122 · Cited by 36 · see citing works
Cooperation between information system development and operations
Erich, F., Amrit, C., & Daneva, M. (2014) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2652524.2652598 · Cited by 35 · see citing works
Composable DevOps: Automated Ontology Based DevOps Maturity Analysis
McCarthy, M. A., Herger, L. M., Khan, S. M., et al. (2015) · IEEE International Conference on Services Computing · doi:10.1109/scc.2015.87 · Cited by 34 · see citing works
Wettinger, J., Binz, T., Breitenbücher, U., et al. (2014) · International Conference on Cloud Computing and Services Science · doi:10.5220/0004859005590568 · Cited by 32 · see citing works
Scheuner, J., Cito, J., Leitner, P., et al. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742833 · Cited by 28 · see citing works
Service provider DevOps for large scale modern network services
Kim, J., Meiroşu, C., Papafili, I., et al. (2015) · IFIP/IEEE International Symposium on Integrated Network Management (IM) · doi:10.1109/inm.2015.7140502 · Cited by 25 · see citing works
Spinellis, D. D. (2012) · IEEE Software · doi:10.1109/ms.2012.38 · Cited by 25 · see citing works
Hussaini, S. W. (2014) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2014.6957687 · Cited by 23 · see citing works
DevOps meets formal modelling in high-criticality complex systems
Olszewska, M., & Waldén, M. (2015) · doi:10.1145/2804371.2804373 · Cited by 23 · see citing works
Towards a DevOps Approach for Software Quality Engineering
Perez, J. F., Wang, W., & Casale, G. (2015) · doi:10.1145/2693561.2693564 · Cited by 23 · see citing works
Challenges, Benefits and Best Practices of Performance Focused DevOps
Gottesheim, W. (2015) · doi:10.1145/2693182.2693187 · Cited by 19 · see citing works
Product-Focused Software Process Improvement
Heidrich, J., Oivo, M., Jedlitschka, A., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39259-7 · Cited by 19 · see citing works
A DevOps framework to shorten delivery time for cloud applications
Hosono, S. (2012) · International Journal of Computational Science and Engineering · doi:10.1504/ijcse.2012.049753 · Cited by 18 · see citing works
Compensation-Based vs. Convergent Deployment Automation for Services Operated in the Cloud
Wettinger, J., Breitenbücher, U., & Leymann, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45391-9_23 · Cited by 17 · see citing works
Application Deployment and Management in the Cloud
Miglierina, M. (2014) · International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc.2014.63 · Cited by 15 · see citing works
Continuous Delivery of Composite Solutions
Austel, P., Han, C., Mikalsen, T., et al. (2015) · doi:10.1145/2756594.2756595 · Cited by 14 · see citing works
Wright, S. A., & Druta, D. (2014) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2014.7063506 · Cited by 13 · see citing works
Eliciting operations requirements for applications
Bass, L., Jeffery, R., Wada, H., et al. (2013) · doi:10.5555/2663360.2663362 · Cited by 13 · see citing works
Application Lifecycle Kit for Mass Customization on PaaS Platforms
Hosono, S., & Shimomura, Y. (2012) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services.2012.80 · Cited by 13 · see citing works
Mapping software testing practice with software testing research — SERP-test taxonomy
Engström, E., & Petersen, K. (2015) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2015.7107470 · Cited by 12 · see citing works
Self-citationDyn Tail - Dynamically Tailored Deployment Engines for Cloud Applications
Wettinger, J., Breitenbücher, U., & Leymann, F. (2015) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2015.63 · Cited by 11 · see citing works
Towards behavior driven operations (BDOps)
Gohil, K., Alapati, N., & Joglekar, S. (2011) · International Conference on Advances in Recent Technologies in Communication and Computing (ARTCom) · doi:10.1049/ic.2011.0095 · Cited by 10 · see citing works
Obstfeld, J., Knight, S., Kern, E., et al. (2014) · doi:10.1145/2619239.2631463 · Cited by 10 · see citing works
The social developer: now, then, and tomorrow
Kilamo, T., Leppänen, M., & Mikkonen, T. (2015) · doi:10.1145/2804381.2804388 · Cited by 9 · see citing works
Filling the gap: a tool to automate parameter estimation for software performance models
Wang, W., Perez, J. F., & Casale, G. (2015) · doi:10.1145/2804371.2804379 · Cited by 9 · see citing works
Preparing and Inter-Connecting Hyper-Converged SmartX Boxes for IoT-Cloud Testbed
Park, S., Cha, B., & Kim, J. (2015) · IEEE International Conference on Advanced Information Networking and Applications (AINA) · doi:10.1109/aina.2015.255 · Cited by 8 · see citing works
SPACE4Cloud: a DevOps environment for multi-cloud applications
Guerriero, M., Ciavotta, M., Gibilisco, G. P., et al. (2015) · doi:10.1145/2804371.2804378 · Cited by 8 · see citing works
U.S. Army Modeling and Simulation Executable Architecture Deployment Cloud Virtualization Strategy
Murphy, S. P., Gallant, S., Gaughan, C., et al. (2012) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2012.89 · Cited by 6 · see citing works
Compensation and Convergence — Comparing and Combining Deployment Automation Approaches
Wettinger, J., Breitenbücher, U., & Leymann, F. (2015) · International Journal of Cooperative Information Systems · doi:10.1142/s0218843015410014 · Cited by 6 · see citing works
Modelling multi-tier enterprise applications behaviour with design of experiments technique
Ustinova, T., & Jamshidi, P. (2015) · doi:10.1145/2804371.2804374 · Cited by 5 · see citing works
Towards explicitly elastic programming frameworks
Jayaram, K. R. (2015) · doi:10.5555/2819009.2819119 · Cited by 3 · see citing works
Factors in Development and Adoption of New Cloud Software and Standards
Sill, A. F. (2014) · IEEE Cloud Computing · doi:10.1109/mcc.2014.73 · Cited by 2 · see citing works
Assessment and Intervention with Wii Fit in the Elderly
Lemus, L., Pintos, N. F., Navarro‐Pardo, E., et al. (2015) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-19728-9_30 · Cited by 2 · see citing works
Preparing the end-to-end virtualized networking over software-defined infrastrure
Kim, J. (2014) · International Conference on Optical Internet · doi:10.1109/coin.2014.6950568 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
DevOps Capabilities, Practices, and Challenges: Insights from a Case Study
Senapathi, M., Buchan, J., & Osman, H. (2018) · Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018 · doi:10.1145/3210459.3210465 · Cited by 172 · see citing works
Kumar, R., & Goyal, R. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101967 · Cited by 97 · see citing works
Prates, L., & Pereira, R. (2025) · International Journal of Information Security · doi:10.1007/s10207-024-00914-z · Cited by 49 · see citing works
“The perceived most common DevOps practices in literature are Continuous Integration, Continuous Delivery, Continuous Monitoring, and Continuous Testing [14] [16] [17] [18] [19] [20].”
The evolution of software development orchestration: current state and an agenda for future research
Maruping, L. M., & Matook, S. (2020) · European Journal of Information Systems · doi:10.1080/0960085x.2020.1831834 · Cited by 41 · see citing works
Identifying the primary dimensions of DevSecOps: A multi-vocal literature review
Zhao, X., Clear, T., & Lal, R. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112063 · Cited by 40 · see citing works
Enterprise-Driven Open Source Software: A Case Study on Security Automation
Angermeir, F., Voggenreiter, M., Moyon, F., & Mendez, D. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00037 · Cited by 30 · see citing works
- Authors
- Jabbari, R., bin Ali, N., Petersen, K., & Tanveer, B.
- Affiliations
- Blekinge Institute of Technology (SE)
- Blekinge Institute of Technology, Karlskrona, Sweden
- Fraunhofer Institute for Experimental Software Engineering (DE)
- Venue
- Proceedings of the Scientific Workshop Proceedings of XP2016
- Publisher
- ACM
- Published
- 2016-05-24
- Pages
- 1-11
- OpenAlex
- W2527104146
- Cited by
- 230
- References
- 66
- Citations by year
- 2026: 19 · 2025: 16 · 2024: 32 · 2023: 30 · 2022: 31 · 2021: 32 · 2020: 36 · 2019: 15 · 2018: 15 · 2017: 4
- FWCI
- 9.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Scientific Workshops of the International Conference on Agile Software Development (XP) (publishedVersion)
- Lund University Publications (Lund University) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
- Keywords
- DevOps, software engineering concepts, software delivery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
