Navigating the DevOps landscape
Open accessAbstract
DevOps, with its increasing prevalence in both industry and academia, has evolved into various DevOps variants (namely XOps) to address emerging technological and operational challenges. However, this proliferation has created confusion and a lack of clarity about the systematic understanding of these XOps and their interrelationship in the DevOps landscape, leading to fragmented knowledge and application. This research seeks to construct a comprehensive picture of the existing DevOps landscape, clarifying the nature and nuances of various XOps, to guide effective future studies and implementations. Utilizing Multivocal Literature Review (MLR), 80 gathered documents are thoroughly examined from throughout the whole community, encompassing both white and grey literature, to map the DevOps landscape. Our review systematically discovered 38 XOps terms and 13 well-studied XOps including AIOps, BizDevOps, CloudOps, DataOps, DevSecOps, FinOps, GitOps, MLOps, ModelOps, NetDevOps, NoOps, SecDevOps and TwinOps. We provided dictionary-like resource that elucidates the core concepts and main ideas associated with each XOps. An in-depth understanding of intricate evolution from DevOps to XOps is delved into, supplemented by the research of relationships between XOps and various technological enablers as well as relationships between XOps and organizational teams, contributing to the ongoing dialogue surrounding their application and evolution. This paper provides a foundational understanding of the DevOps landscape including open issues and challenges, current and future trends, assisting both researchers and practitioners in navigating this complex field. It establishes a platform for further research and practical applications in the evolving field of DevOps and XOps.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study's breadth-over-depth approach may overlook the nuanced challenges and unique solutions associated with specific XOps variants.
“The study's breadth-over-depth approach, while valuable for providing a high-level overview, may overlook the nuanced challenges and unique solutions associated with specific XOps variants.”
From Evolution of DevSecOps and Its Influence on Application Security: A Systematic Literature Review (2025) · cites this paper as [46]
The synthesis of a wide range of sources can dilute the depth of analysis for individual practices, potentially leaving gaps in understanding the detailed implementation issues of each variant.
“The synthesis of a wide range of sources can dilute the depth of analysis for individual practices, potentially leaving gaps in understanding the detailed implementation issues of each variant.”
From Evolution of DevSecOps and Its Influence on Application Security: A Systematic Literature Review (2025) · cites this paper as [46]
Cites
Literature review as a research methodology: An overview and guidelines
Snyder, H. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.07.039 · Cited by 11,524 · see citing works
Performing systematic literature reviews in software engineering
Budgen, D., & Brereton, P. (2006) · International Conference on Software Engineering · doi:10.1145/1134285.1134500 · Cited by 8,824 · see citing works
(2004) · Cited by 5,198 · see citing works
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · see citing works
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · see citing works
Systematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · see citing works
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669 · see citing works
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · see citing works
An Overview on Edge Computing Research
Cao, K., Liu, Y., Meng, G., & Sun, Q. (2020) · IEEE Access · doi:10.1109/access.2020.2991734 · Cited by 1,254 · see citing works
Adams, R. J., Smart, P., & Huff, A. S. (2017) · International Journal of Management Reviews · doi:10.1111/ijmr.12102 · Cited by 995 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831 · see citing works
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766 · see citing works
Continuous software engineering: A roadmap and agenda
Fitzgerald, B. F., & Stol, K.-J. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.06.063 · Cited by 742 · see citing works
A Survey of DevOps Concepts and Challenges
Leite, L., Rocha, C., Kon, F., Milojicic, D., & Meirelles, P. (2020) · ACM Computing Surveys · doi:10.1145/3359981 · Cited by 484 · see citing works
State‐of‐the‐Evidence Reviews: Advantages and Challenges of Including Grey Literature
Benzies, K. M., Premji, S., Hayden, K. A., & Serrett, K. (2006) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2006.00051.x · Cited by 408 · see citing works
How to do a grounded theory study: a worked example of a study of dental practices
Sbaraini, A., Carter, S. M., Evans, R. W., & Blinkhorn, A. S. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-128 · Cited by 388 · see citing works
Towards Rigor in Reviews of Multivocal Literatures: Applying the Exploratory Case Study Method
Ogawa, R. T., & Malen, B. (1991) · Review of Educational Research · doi:10.3102/00346543061003265 · Cited by 247 · see citing works
Challenges and solutions when adopting DevSecOps: A systematic review
Rajapakse, R. N., Zahedi, M., Babar, M. A., & Shen, H. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2021.106700 · Cited by 235 · see citing works
DevOps in practice: A multiple case study of five companies
Lwakatare, L. E., Kilamo, T., Karvonen, T., Sauvola, T., Heikkilä, V. T., Itkonen, J., Kuvaja, P., Mikkonen, T., Oivo, M., & Lassenius, C. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2019.06.010 · Cited by 197 · see citing works
Adopting DevOps in the real world: A theory, a model, and a case study
Luz, W., Pinto, G., & Bonifácio, R. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.07.083 · Cited by 164 · see citing works
DevOps: A Definition and Perceived Adoption Impediments
Smeds, J., Nybom, K., & Porres, I. (2015) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-319-18612-2_14 · Cited by 158 · see citing works
DevOps Adoption Benefits and Challenges in Practice: A Case Study
Riungu-Kalliosaari, L., Mäkinen, S., Lwakatare, L. E., Tiihonen, J., & Männistö, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49094-6_44 · Cited by 154 · see citing works
A survey of AIOps methods for failure management
Notaro, P., Cardoso, J., & Gerndt, M. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3483424 · Cited by 142 · see citing works
MLOps: A Taxonomy and a Methodology
Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022) · IEEE Access · doi:10.1109/access.2022.3181730 · Cited by 137 · see citing works
Service, R. (2009) · Organizational Research Methods · doi:10.1177/1094428108324514 · Cited by 136 · see citing works
Towards MLOps: A Framework and Maturity Model
John, M. M., Olsson, H. H., & Bosch, J. (2021) · 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa53835.2021.00050 · Cited by 130 · see citing works
Toward successful DevSecOps in software development organizations: A decision-making framework
Akbar, M. A., Smolander, K., Mahmood, S., & Alsanad, A. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.106894 · Cited by 128 · see citing works
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2015) · Lecture notes in business information processing · doi:10.1007/978-3-319-18612-2_19 · Cited by 121 · see citing works
Benchmarking Kappa: Interrater Agreement in Software Process Assessments
Emam, K. E. (1999) · Empirical Software Engineering · doi:10.1023/a:1009820201126 · Cited by 113 · see citing works
SecDevOps: Is It a Marketing Buzzword? - Mapping Research on Security in DevOps
Mohan, V., & Othmane, L. B. (2016) · 2016 11th International Conference on Availability, Reliability and Security (ARES) · doi:10.1109/ares.2016.92 · Cited by 112 · see citing works
Butijn, B.-J., Tamburri, D. A., & van den Heuvel, W. (2020) · ACM Computing Surveys · doi:10.1145/3369052 · Cited by 107 · see citing works
A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic Survey
Díaz-de-Arcaya, J., Torre-Bastida, A. I., Zárate, G., Miñón, R., & Almeida, A. (2023) · ACM Computing Surveys · doi:10.1145/3625289 · Cited by 105 · see citing works
ModelOps: Cloud-Based Lifecycle Management for Reliable and Trusted AI
Hummer, W., Muthusamy, V., Rausch, T., Dube, P., El Maghraoui, K., Murthi, A., & Oum, P. (2019) · 2019 IEEE International Conference on Cloud Engineering (IC2E) · doi:10.1109/ic2e.2019.00025 · Cited by 105 · see citing works
International Conference on Electrical, Computer and Energy Technologies
(2023) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet58911.2023.10389303 · Cited by 78 · see citing works
From Ad-Hoc Data Analytics to DataOps
Munappy, A. R., Mattos, D. I., Bosch, J., Olsson, H. H., & Dakkak, A. (2020) · International Conference on Software and System Process · doi:10.1145/3379177.3388909 · Cited by 73 · see citing works
Relationship of DevOps to Agile, Lean and Continuous Deployment
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49094-6_27 · Cited by 71 · see citing works
Agile Processes in Software Engineering and Extreme Programming
Lassenius, C., Dingsøyr, T., & Paasivaara, M. (2015) · Lecture notes in business information processing · doi:10.1007/978-3-319-18612-2 · Cited by 54 · see citing works
Taking DevOps mainstream: a critical review and conceptual framework
Gall, M., & Pigni, F. (2021) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1997100 · Cited by 53 · see citing works
Characterizing DevOps Culture: A Systematic Literature Review
Sánchez‐Gordón, M., & Colomo‐Palacios, R. (2018) · Communications in computer and information science · doi:10.1007/978-3-030-00623-5_1 · Cited by 52 · see citing works
Agile Infrastructure and Operations: How Infra-gile are You?
Debois, P. (2008) · Agile Conference · doi:10.1109/agile.2008.42 · Cited by 52 · see citing works
Preliminary Findings about DevSecOps from Grey Literature
Mao, R., Zhang, H., Dai, Q., Huang, H., Rong, G., Shen, H., Chen, L., & Lu, K. (2020) · 2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs51102.2020.00064 · Cited by 49 · see citing works
GitOps: The Evolution of DevOps?
Beetz, F., & Harrer, S. (2021) · IEEE Software · doi:10.1109/ms.2021.3119106 · Cited by 48 · see citing works
A Multivocal Literature Review of MLOps Tools and Features
Recupito, G., Pecorelli, F., Catolino, G., Moreschini, S., Nucci, D. D., Palomba, F., & Tamburri, D. A. (2022) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa56994.2022.00021 · Cited by 45 · see citing works
Integrating Security with DevSecOps: Techniques and Challenges
Ahmed, Z., & Francis, S. C. (2019) · 2019 International Conference on Digitization (ICD) · doi:10.1109/icd47981.2019.9105789 · Cited by 44 · see citing works
Capabilities and Practices in DevOps: A Multivocal Literature Review
Amaro, R., Pereira, R., & da Silva, M. M. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3166626 · Cited by 43 · see citing works
Computational Science and Its Applications – ICCSA 2019
Petersburg, I. 2. S., Misra, S., Gervasi, O., Murgante, B., Stankova, E., Korkhov, V. V., Torre, C. M., Rocha, A. M. A. C., Taniar, D., Apduhan, B. O., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-24289-3 · Cited by 40 · see citing works
Lombardi, F., & Fanton, A. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09619-3 · Cited by 39 · see citing works
BizDevOps: Because DevOps is Not the End of the Story
Gruhn, V., & Schäfer, C. (2015) · Communications in computer and information science · doi:10.1007/978-3-319-22689-7_30 · Cited by 39 · see citing works
Cooperation between information system development and operations
Erich, F., Amrit, C., & Daneva, M. (2014) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2652524.2652598 · Cited by 35 · see citing works
A Systematic Review of Machine Learning DevOps
Mboweni, T., Masombuka, T., & Dongmo, C. (2022) · 2022 International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet55527.2022.9872968 · Cited by 34 · see citing works
Artificial Intelligence for Cloud and Edge Computing
(2022) · Internet of things · doi:10.1007/978-3-030-80821-1 · Cited by 33 · see citing works
Product-Focused Software Process Improvement
Winkler, D., Sarro, F., Kalinowski, M., Turhan, B., Fernández, D. M., & Felderer, M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69926-4 · Cited by 32 · see citing works
SliceOps: Explainable MLOps for Streamlined Automation-Native 6G Networks
Rezazadeh, F., Chergui, H., Alonso, L., & Verikoukis, C. V. (2024) · IEEE Wireless Communications · doi:10.1109/mwc.007.2300144 · Cited by 32 · see citing works
Towards a consistent interpretation of AIOps models
Lyu, Y., Rajbahadur, G. K., Lin, D., Chen, B.-Y., & Jiang, Z. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3488269 · Cited by 31 · see citing works
A Systematic Mapping Study in AIOps
Notaro, P., Cardoso, J., & Gerndt, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-76352-7_15 · Cited by 31 · see citing works
How Should Software Engineering Secondary Studies Include Grey Material?
Kitchenham, B. A., Madeyski, L., & Budgen, D. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3165938 · Cited by 30 · see citing works
Wurm, J. (2022) · Springer eBooks · doi:10.1007/978-3-031-13249-0 · Cited by 30 · see citing works
TwinOps - DevOps meets model-based engineering and digital twins for the engineering of CPS
Hugues, J., Hristosov, A., Hudak, J., & Joe, Y. (2020) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) · doi:10.1145/3417990.3421446 · Cited by 30 · see citing works
Babur, Ö., Chaudron, M. R. V., Cleophas, L., Iovino, L., & Kolovos, D. (2020) · doi:10.1145/3417990 · Cited by 29 · see citing works
2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion
(2023) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) · doi:10.1109/models-c59198.2023.00002 · Cited by 27 · see citing works
From DevOps to DevDataOps: Data Management in DevOps Processes
Capizzi, A., Distefano, S., & Mazzara, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39306-9_4 · Cited by 25 · see citing works
AIOps: A Multivocal Literature Review
Rijal, L., Colomo‐Palacios, R., & Sánchez‐Gordón, M. (2022) · Internet of things · doi:10.1007/978-3-030-80821-1_2 · Cited by 25 · see citing works
Bringing Automation to the Classroom: A ChatOps-Based Approach
Mulyana, E., Hakimi, R., & Hendrawan, H. (2018) · International Conference on Wireless and Telematics (ICWT) · doi:10.1109/icwt.2018.8527810 · Cited by 24 · see citing works
AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges
Cheng, Q., Sahoo, D., Saha, A., Yang, W., Liu, C., Woo, G., Singh, M., Saverese, S., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2304.04661 · Cited by 23 · see citing works
Revisit security in the era of DevOps: An evidence-based inquiry into DevSecOps industry
Zhou, X., Mao, R., Zhang, H., Dai, Q., Huang, H., Shen, H., Li, J., & Rong, G. (2023) · IET Software · doi:10.1049/sfw2.12132 · Cited by 23 · see citing works
Alonso, J., Orue-Echevarria, L., & Huarte, M. (2022) · Applied Sciences · doi:10.3390/app12094347 · Cited by 20 · see citing works
DevOps, DevSecOps, AIOPS- Paradigms to IT Operations
Sen, A. (2020) · Lecture notes in electrical engineering · doi:10.1007/978-981-15-7804-5_16 · Cited by 19 · see citing works
BizDevOps: A Multivocal Literature Review
Lohrasbinasab, I., Acharya, P. B., & Colomo‐Palacios, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58817-5_50 · Cited by 19 · see citing works
Evolving Technologies for Computing, Communication and Smart World
(2020) · Lecture notes in electrical engineering · doi:10.1007/978-981-15-7804-5 · Cited by 18 · see citing works
An empirical study on ML DevOps adoption trends, efforts, and benefits analysis
Rzig, D. E., Hassan, F., & Kessentini, M. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107037 · Cited by 18 · see citing works
Beginning DevOps for Developers
Hüttermann, M. (2012) · Apress eBooks · doi:10.1007/978-1-4302-4570-4_1 · Cited by 17 · see citing works
Conceptualizing the Secure Machine Learning Operations (SecMLOps) Paradigm
Zhang, X., & Jaskolka, J. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00023 · Cited by 16 · see citing works
Self-citationGitOps: A Path to More Self-service IT
Limoncelli, T. A. (2018) · Queue · doi:10.1145/3236386.3237207 · Cited by 13 · see citing works
ModelOps for enhanced decision-making and governance in emergency control rooms
Lefevre, K., Arora, C., Lee, K., Zaslavsky, A., Bouadjenek, M. R., Hassani, A., & Razzak, I. (2022) · Environment Systems & Decisions · doi:10.1007/s10669-022-09855-1 · Cited by 12 · see citing works
From DevOps to NoOps: Is It Worth It?
Jindal, A., & Gerndt, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-72369-9_8 · Cited by 11 · see citing works
Dealing with Security in a Real DevOps Environment
Larrucea, X., Berreteaga, A., & Santamaría, I. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-28005-5_35 · Cited by 11 · see citing works
Zasadziński, M., Solé, M., Brandón, Á., Muntés-Mulero, V., & Carrera, D. (2018) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster.2018.00039 · Cited by 10 · see citing works
Artificial Intelligence for IT Operations (AIOPS) Workshop White PaperPreprint
Bogatinovski, J., Nedelkoski, S., Acker, A., Schmidt, F., Wittkopp, T., Becker, S., Cardoso, J., & Kao, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06054 · Cited by 10 · see citing works
Discovering DataOps: A Comprehensive Review of Definitions, Use Cases, and Tools
Mainali, K., Ehrlinger, L., Himmelbauer, J., & Matskin, M. (2021) · Cited by 9 · see citing works
Conceptualising and defining DevOps: a review for understanding, not a framework for practitioners
Hemon, A., & Rowe, F. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2022.2100061 · Cited by 9 · see citing works
On-Premise AIOps Infrastructure for a Software Editor SME: An Experience Report
Bendimerad, A., Remil, Y., Mathonat, R., & Kaytoue, M. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3613876 · Cited by 9 · see citing works
Limoncelli, T. A. (2018) · Communications of the ACM · doi:10.1145/3233241 · Cited by 8 · see citing works
Beyond building predictive models: TwinOps in biomanufacturingPreprint
Canzani, E., & Timmer, S. W. (2021) · doi:10.36227/techrxiv.16478856.v1 · Cited by 8 · see citing works
Doležel, M. (2020) · Lecture notes in business information processing · doi:10.1007/978-3-030-58858-8_26 · Cited by 6 · see citing works
NoOps – A Multivocal literature review
Stefanac, T., & Colomo‐Palacios, R. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2021.12.002 · Cited by 6 · see citing works
Towards Continuous Consistency Checking of DevOps Artefacts
Colantoni, A., Horváth, B., Horváth, Á., Berardinelli, L., & Wimmer, M. (2021) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) · doi:10.1109/models-c53483.2021.00069 · Cited by 6 · see citing works
ChainOps for Smart Contract-Based Distributed Applications
van den Heuvel, W., Tamburri, D. A., D’Amici, D., Izzo, F., & Potten, S. (2021) · Lecture notes in business information processing · doi:10.1007/978-3-030-79976-2_25 · Cited by 5 · see citing works
Research on DataOps Capability - Practice and Development
Yin, Z., Zhou, S., Zhou, J., Tian, M., Lin, M., & Liu, S. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00303 · Cited by 5 · see citing works
BizDevOps Support for Business Process Microservices-Based Applications
Delgado, A., García, F. O., & Ruíz, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26507-5_22 · Cited by 5 · see citing works
Towards DesignOps Design Development, Delivery and Operations for the AECO Industry
Kosicki, M., Tsiliakos, M., El-Ashry, K., Borgström, O., Rod, A., Tarabishy, S., Nguyen, C., Davis, A., & Tsigkari, M. (2022) · Springer eBooks · doi:10.1007/978-3-031-13249-0_6 · Cited by 4 · see citing works
Centrality of AI Quality in MLOPs Lifecycle and Its Impact on the Adoption of AI/ML Solutions
Akkineni, A., Koohborfardhaghighi, S., & Singh, S. K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-35510-3_42 · Cited by 4 · see citing works
LAOps: Learning Analytics with Privacy-aware MLOps
Niemelä, P., Silverajan, B., Nurminen, M., Hukkanen, J., & Järvinen, H.-M. (2022) · International Conference on Computer Supported Education · doi:10.5220/0011113300003182 · Cited by 3 · see citing works
Incident Management and Chatops @ Netflix Feat Scorebot
Tobey, A. (2016) · Cited by 2 · see citing works
Incident Management and Chatops at Shopify
Niyonkuru, D. (2017) · Cited by 2 · see citing works
Using ChatOps to Achieve Continuous Certification of Cloud Services
Ohagen, P., Lins, S., Thiebes, S., & Sunyaev, A. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.912 · Cited by 1 · see citing works
Flagged first, then by citations
3 of the 102 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Zhao, P., Li, C., Yu, F. R., & Fu, Y. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3530861 · Cited by 17 · see citing works
Self-citationCai, X., Zhao, P., Liu, S., Fu, Y., Li, C., & Yu, F. R. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3546088 · Cited by 9 · see citing works
“Consequently, substantial resources, computational power, and energy are invested in data gathering, processing, and training, presenting a significant challenge in the realm of ML for CAVs [4].”
Self-citationEvolution of DevSecOps and Its Influence on Application Security: A Systematic Literature Review
Mohammed, K., Shanmugam, B., & El-Den, J. (2025) · Technologies · doi:10.3390/technologies13120548 · Cited by 8 · see citing works
Zhang, X., Zhao, P., Jaskolka, J., Li, H., & Lu, R. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10795-y · Cited by 8 · see citing works
“As a hybrid of agile and lean practices, DevOps integrates cultural philosophies, prac - tices, and tools that enhance an organization’s capacity to deliver applications and services rapidly and reliably (Zhang et al. 2025).”
Self-citationWearing a fur coat in the summertime: Should digital pathology redefine medical imaging?
Gershkovich, P. (2025) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2025.100450 · Cited by 6 · see citing works
“53 These frameworks underscore the enduring power of modularity, and the growing importance of domain-speci fi c, fl ex-ible architectures in an ever-changing technological landscape.”
Khadem, E. A., & Movaghar, A. (2025) · Array · doi:10.1016/j.array.2025.100547 · Cited by 10 · see citing works
Zhao, P., Fu, Y., Yu, F. R., & Li, C. (2025) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2025.3633255 · Cited by 1 · see citing works
“This oversimplification re-sults in sub-optimal decision-making, particularly in edge cases where multiple competing objectives must be balanced [25], [26].”
Self-citationEnabling Private Cooperative Sensing Sharing in Vehicular Networks via Encrypted Spatial Matching
Zhang, X., Zhao, P., Lu, R., Jaskolka, J., & Ray, S. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3650147 · Cited by 1 · see citing works
“This capability is particularly valuable in challenging scenarios such as urban intersections with limited visibility, highway merging, or navigation through complex tra ffi c patterns [7].”
Self-citationDependable Task Offloading for CAVs in Web3: An Intelligence-Based Active Inference Approach
Li, X., Jing, T., Yu, F. R., Wang, H., Ren, Y., Zhu, M., Yan, Y., Ma, S., & Li, Y. (2026) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2026.3663258 · Cited by 1 · see citing works
“Web3,thenextgenerationoftheInternet,isadecentralized, data-driven,andself-governedarchitecture[5],[6].”
Designing a cloud-native MLOps pipeline using Databricks: A case study in industrial practice
Moreschini, S., Raitaniemi, S., Mäkelä, E., Laukkanen, T., & Hästbacka, D. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108585 · Cited by 1 · see citing works
Incentivizing Cooperative Sensing Sharing Ecosystem for Connected and Autonomous Vehicles
Li, C., Zhao, P., Yu, F. R., & Fu, Y. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3556200 · Cited by 1 · see citing works
“For example, the high data rates and low latency of 6G allow for real-time sharing of high-resolution sensor data among vehicles, improving the accuracy and timeliness of cooperative perception [18].”
Self-citationHajder, P., Mojżeszko, M., Hallo, F., Hajder, L., Regulski, K., Banaś, K., Pernach, M., Szeliga, D., Bzowski, K., Michalik, K., et al. (2025) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops65533.2025.00057 · Cited by 1 · see citing works
“Additionally, it improves security, auditing, and resilience, thereby facilitating the management of applications across multiple environments with quick rollback capabilities in the event of failures [18].”
Joghal, M. M., & Shameli‐Sendi, A. (2025) · Iranian Conference on Advances in Enterprise Architecture (ICAEA) · doi:10.1109/icaea69058.2025.11301455
A DevOps-Based Framework for Automating Real-Time Cloud Data Collection Workflows
Saini, M., & Agarwal, P. (2025) · International Conference on Emerging Technologies and Innovation for Sustainability (EmergIN) · doi:10.1109/emergin67762.2025.11450898
“[ 12] Azure Repository: Azure Repos enables teams to collaborate on code, manage branches, and track changes to their source code.”
Reptile Meta Learning Based Beaconing and C2 Behaviour Detection in Network
Elijah, G. J., Saras, A., Daniel, R. M., & Jothish, D. (2026) · International Conference on Innovative Trends in Information Technology (ICITIIT) · doi:10.1109/icitiit68860.2026.11499562
Problematizing DevOps in the GenAI era: a literature review
McCarthy, T., McCarthy, S., & Sammon, D. (2026) · Journal of Decision System · doi:10.1080/12460125.2026.2688530
Pombo, N., & Faraji, A. (2026) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet69978.2026.11585124
“However, comprehensive investigations that explore the broader X-Ops spectrum remain sparse [13].”
AI-driven automation for CI/CD pipelines using attention-enabled reptile meta learning
Elijah, G. J., Mythily, M., Farid, F. A., R, N. A. G., Silas, S., Rajsingh, E. B., Uddin, J., & Karim, H. B. A. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1877426
Lämmel, R., Favré, J.-M., & Zaytsev, V. (2026) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3837062.3839062
“We use ‘XOps’ [76] here as an umbrella term for DevOps, DataOps, MLOps, and related practices.”
Nocera, S., Vegas, S., & Scanniello, G. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834408
“This is particularly evident in DevOps and DevSecOps frameworks [18, 54, 98].”
Reducing Security Testing Overhead in DevSecOps Through Risk-Aware Scan Orchestration
Lahcen, B., Mehdi, A. S., Abderrahim, M., Benmakhlouf, A., & Anas, R. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-18927-1_28
- Authors
- Zhang, X., Zhao, P., & Jaskolka, J.
- Affiliations
- Carleton University (CA)
- Corresponding author
- Zhao, Pincan
- Venue
- Journal of Systems and Software
- Publisher
- Elsevier BV
- Published
- 2025-05
- Volume
- 223
- Pages
- 112331
- ISSN
- 0164-1212 (ISSN-L)
- OpenAlex
- W4406348806
- Cited by
- 21
- References
- 102
- Citations by year
- 2026: 11 · 2025: 10
- FWCI
- 22.02
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Systems and Software (publishedVersion, cc-by-nc-nd)
- Zenodo (CERN European Organization for Nuclear Research)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada (RGPIN-2019-06306)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
- Keywords
- DevOps, AIOps, DataOps, DevSecOps, FinOps, GitOps, MLOps, organizational teams, technological enablers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
