DevOps in an ISO 13485 Regulated Environment: A Multivocal Literature Review
Open accessAbstract
Background: Medical device development projects must follow proper directives and regulations to be able to market and sell the end-product in their respective territories. The regulations describe requirements that seem to be opposite to efficient software development and short time-to-market. As agile approaches, like DevOps, are becoming more and more popular in software industry, a discrepancy between these modern methods and traditional regulated development has been reported. Although examples of successful adoption in this context exist, the research is sparse. Aims: The objective of this study is twofold: to review the current state of DevOps adoption in regulated medical device environment; and to propose a checklist based on that review for introducing DevOps in that context. Method: A multivocal literature review is performed and evidence is synthesized from sources published between 2015 to March of 2020 to capture the opinions of experts and community in this field. Results: Our findings reveal that adoption of DevOps in a regulated medical device environment such as ISO 13485 has its challenges, but potential benefits may outweigh those in areas such as regulatory, compliance, security, organizational and technical. Conclusion: DevOps for regulated medical device environments is a highly appealing approach as compared to traditional methods and could be particularly suited for regulated medical development. However, an organization must properly anchor a transition to DevOps in top-level management and be supportive in the initial phase utilizing professional coaching and space for iterative learning; as such an initiative is a complex organizational and technical task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Buffer systems, artificial saliva, and selected saliva/serum samples show that PDA-based biosensing platforms can achieve improved biomolecule immobilization, signal transduction, fluorescence/electrochemical/SERS detection, and salivary biomarker analysis.
“Buffer systems, artificial saliva, and selected saliva/serum samples show improved biomolecule immobilization, signal transduction, fluorescence/electrochemical/SERS detection, and salivary biomarker analysis.”
From Polydopamine-Based Biomaterials in Oral Medicine: Applications, Mechanisms, and Translational Challenges (2026) · cites this paper as [98]
The Study Protocol (p. 3) concerns a literature review of DevOps and regulated medical-device software, not biosensing experiments. Searches for artificial saliva, PDA, polydopamine, SERS, and salivary biomarkers found no passage in our copy addressing the attributed claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Manifesto for agile software development
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., & Thomas, D. A. (2001) · Cited by 4,284 · see citing works
Resolution Adopted By The General Assembly
East, E. Y. O. I. A. M. (1984) · International Legal Materials · doi:10.1163/9789004283688_019 · Cited by 1,670 · see citing works
Agile Software Development, Principles, Patterns, and Practices
Martin, R. (2002) · Cited by 1,466 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831 · see citing works
Blockchain technology innovations
Ahram, T., Sargolzaei, A., Sargolzaei, S., Daniels, J., & Amaba, B. (2017) · IEEE Technology & Engineering Management Conference (TEMSCON) · doi:10.1109/temscon.2017.7998367 · Cited by 612 · see citing works
Ebert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016) · IEEE Software · doi:10.1109/ms.2016.68 · Cited by 504 · see citing works
Standard IEC 62304 - Medical device software - Software lifecycle processes
Jordan, P. (2006) · IET Seminar on Software for Medical Devices · doi:10.1049/ic:20060141 · Cited by 244 · see citing works
DevSecOps: A Multivocal Literature Review
Myrbakken, H., & Colomo-Palacios, R. (2017) · Communications in Computer and Information Science · doi:10.1007/978-3-319-67383-7_2 · Cited by 205 · see citing works
Self-citationScaling agile methods to regulated environments: An industry case study
Fitzgerald, B. F., Stol, K.-J., O'Sullivan, R., & O’Brien, D. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606635 · Cited by 172 · see citing works
Agile Software Development: Principles, Patterns, and Practices
Winter, R. J. (2014) · Performance Improvement Journal · doi:10.1002/pfi.21408 · Cited by 118 · see citing works
Lean/Agile Software Development Methodologies in Regulated Environments – State of the Art
Cawley, O., Wang, X., & Richardson, I. (2010) · Lecture notes in business information processing · doi:10.1007/978-3-642-16416-3_4 · Cited by 78 · see citing works
Modern DevOps: Optimizing software development through effective system interactions
Cois, C. A., Yankel, J., & Connell, A. (2014) · IEEE International Professional Communication Conference (ProComm) · doi:10.1109/ipcc.2014.7020388 · Cited by 74 · see citing works
Relationship of DevOps to Agile, Lean and Continuous Deployment
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49094-6_27 · Cited by 71 · see citing works
DevOps: Concepts, Practices, Tools, Benefits and Challenges
Ghantous, G. B., & Gill, A. Q. (2017) · Journal of the Association for Information Systems · Cited by 60 · see citing works
DevOps in Regulated Software Development: Case Medical Devices
Laukkarinen, T., Kuusinen, K., & Mikkonen, T. (2017) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier.2017.20 · Cited by 57 · see citing works
Gupta, R. K., Venkatachalapathy, M., & Jeberla, F. K. (2019) · 2019 ACM/IEEE 14th International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2019.00020 · Cited by 50 · see citing works
Regulated software meets DevOps
Laukkarinen, T., Kuusinen, K., & Mikkonen, T. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.01.011 · Cited by 45 · see citing works
Rainer, A. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.01.011 · Cited by 33 · see citing works
Medical device software. Software life-cycle processes
(2006) · doi:10.3403/30089751 · Cited by 29 · see citing works
Global Software Engineering: An Industry Perspective
Ebert, C., Kuhrmann, M., & Prikladnicki, R. (2015) · IEEE Software · doi:10.1109/ms.2016.27 · Cited by 28 · see citing works
Working software over comprehensive documentation – Rationales of agile teams for artefacts usage
Wagenaar, G., Overbeek, S. J., Lucassen, G., Brinkkemper, S., & Schneider, K. A. (2018) · Journal of Software Engineering Research and Development · doi:10.1186/s40411-018-0051-7 · Cited by 25 · see citing works
A hybrid assessment approach for medical device software development companies
Özcan‐Top, Ö., & McCaffery, F. (2017) · Journal of Software Evolution and Process · doi:10.1002/smr.1929 · Cited by 20 · see citing works
Software Process Improvement and Capability Determination
Mas, A., Dorling, A., O’Connor, R. V., Rout, T., & Mesquida, A. (2012) · Communications in computer and information science · doi:10.1007/978-3-642-30439-2 · Cited by 20 · see citing works
A Multivocal Literature Review on the use of DevOps for e-Learning systems
Sánchez‐Gordón, M., & Colomo‐Palacios, R. (2018) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3284179.3284328 · Cited by 16 · see citing works
Self-citationAdopting Agile Practices when Developing Medical Device Software
McHugh, M., & McCaffery, F. (2015) · Journal of Computer Engineering and Information Technology · doi:10.4172/2324-9307.1000131 · Cited by 15 · see citing works
Towards Agile Yet Regulatory-Compliant Development of Medical Software
Stirbu, V., & Mikkonen, T. (2018) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2018.00027 · Cited by 13 · see citing works
Intelligent Computing Methodologies
Huang, D.-S., Huang, Z.-K., & Hussain, A. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26766-7 · Cited by 13 · see citing works
Alsaadi, M., Lisitsa, A., Khalaf, M., & Qasaimeh, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26766-7_53 · Cited by 11 · see citing works
Implementing Secure DevOps assessment for highly regulated environments
Yaşar, H. (2017) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1145/3098954.3105819 · Cited by 10 · see citing works
Duffau, C., Grabiec, B., & Blay–Fornarino, M. (2017) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2017.8 · Cited by 6 · see citing works
Interleaving Software Craftsmanship Practices in Medical Device Agile Development
Badanahatti, A., & Pillutla, S. (2020) · Innovations in Software Engineering Conference (ISEC) · doi:10.1145/3385032.3385047 · Cited by 5 · see citing works
Understanding the Benefits of Agile Software Development in Regulated Environments.
Karrenbauer, J., Wiesche, M., & Krcmar, H. (2019) · Journal of the Association for Information Systems · Cited by 2 · see citing works
Implementing DevOps in a Regulated Traditionally Waterfall Environment
Victor, J., & Lega, P. (2016) · Cited by 1 · see citing works
IMPLEMENTATION OF ISO 13485:2016 IN MEDICAL PRODUCTS HANDLING MULTINATIONAL CORPORATION
Jalnasow, D. (2019) · RePEc: Research Papers in Economics · Cited by 1 · see citing works
Online appendix: DevOps in an ISO 13485 Regulated Environment
Lie, M. F., Sánchez-Gordón, M., & Colomo‐Palacios, R. (2020) · Figshare · doi:10.6084/m9.figshare.c.5063381 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [5][5] Jim Bird. 2016. DevOpsSec . O'Reilly Media, Inc. -
Cited by
Sorted by citations
Granlund, T., Stirbu, V., & Mikkonen, T. (2021) · SN Computer Science · doi:10.1007/s42979-021-00726-1 · Cited by 53 · see citing works
Plant, O. H., van Hillegersberg, J., & Aldea, A. (2022) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2022.100560 · Cited by 38 · see citing works
Holding on to Compliance While Adopting DevSecOps: An SLR
Ramaj, X., Sánchez-Gordón, M., Gkioulos, V., Chockalingam, S., & Colomo-Palacios, R. (2022) · Electronics · doi:10.3390/electronics11223707 · Cited by 33 · see citing works
Self-citationBoisrond, K., Tardif, P.-M., & Jaafar, F. (2024) · IEEE Access · doi:10.1109/access.2024.3434618 · Cited by 27 · see citing works
Kioskli, K., Grigoriou, E., Islam, S., Yiorkas, A. M., Christofi, L., & Mouratidis, H. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01009-z · Cited by 24 · see citing works
Salisbury, J. P. (2021) · JMIR Biomedical Engineering · doi:10.2196/26942 · Cited by 17 · see citing works
Chen, F., Chen, H., Yu, T., Wang, R., Wang, Y., Zhang, X., Li, J., Liu, K., Hai, D., Bao, X., et al. (2026) · MedComm · doi:10.1002/mco2.70597 · Cited by 10 · see citing works
Investigating Software Engineering Artifacts in DevOps Through the Lens of Boundary Objects
Matthies, C., Heinrich, R., & Wohlrab, R. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3593441 · Cited by 9 · see citing works
Towards People Maturity for Secure Development and Operations: A vision
Akbar, M. A., Rafi, S., Hyrynsalmi, S., & Khan, A. A. (2024) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3661167.3661238 · Cited by 7 · see citing works
DevOps as a culture of interaction and deployment in an insurance company.
Melgar, E. A. A. S. (2021) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.17762/turcomat.v12i4.1429 · Cited by 5 · see citing works
Unveiling the Safety Aspects of DevSecOps: Evolution, Gaps and Trends
Ramaj, X., Sánchez‐Gordón, M., Chockalingam, S., & Colomo‐Palacios, R. (2022) · Recent Advances in Computer Science and Communications · doi:10.2174/2666255816666220804143918 · Cited by 4 · see citing works
Self-citationMartina, M. R., Park, C. M., Alastruey, J., Bruno, R. M., Climie, R. E., Doğan, S., Tuna, B. G., Jerončić, A., Manouchehri, M., Panayiotou, A. G., et al. (2024) · Expert Review of Medical Devices · doi:10.1080/17434440.2024.2334931 · Cited by 3 · see citing works
DevOps Patterns: A Rapid Review
Copei, S., & Kosiol, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66326-0_3 · Cited by 2 · see citing works
Keeping Patient Data Safe in the Cloud: A DevOps Approach
Boda, V. V. R. (2021) · International Journal of Emerging Trends in Computer Science and Information Technology · doi:10.63282/3050-9246.ijetcsit-v2i3p104 · Cited by 2 · see citing works
Gestión documental para el desarrollo de dispositivos médicos
Cázares-Sánchez, A., González, F. S. C., García-Herrera, J. E., Garduño, I. E., & Méndez‐Lozoya, J. (2024) · Transdigital · doi:10.56162/transdigital340 · Cited by 1 · see citing works
Yang, Z., Song, W., Lv, L., Zhai, S., Liu, Y., Cui, H., Li, X., Gao, F., & Chu, S. (2026) · International Journal of Nanomedicine · doi:10.2147/ijn.s611922
Peer Review #1 of "Microservice security: a systematic literature review (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.1/reviews/1
Uddin, M. M., & Ge, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-62753-8_9
Alfina, K. N., & Ratnayake, R. M. C. (2024) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-71622-5_6
Sutradhar, S., Bose, R., Khan, A. A., Roy, S., Golder, S. S., Majumder, S., Prashar, D., & Shah, M. A. (2026) · IET Blockchain · doi:10.1049/blc2.70035
Amaro, R., Pereira, R., & da Silva, M. M. (2026) · Software Quality Journal · doi:10.1007/s11219-026-09765-4
Interaction Layer to Document Medical Devices for Regulatory Review
Manea, V. C., & Sørensen, M. T. (2026) · ACM Interactive Health Conference · doi:10.1145/3786579.3804933
- Authors
- Lie, M. F., Sánchez-Gordón, M., & Colomo-Palacios, R.
- Affiliations
- Østfold University of Applied Sciences (NO)
- Østfold University College, Halden, Norway
- ORCID
- Lie, Martin Forsberg
- Venue
- Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)
- Publisher
- ACM
- Published
- 2020-10-05
- Pages
- 1-11
- ISSN
- 1938-6451 (ISSN-L)
- OpenAlex
- W4288079612
- Cited by
- 21
- References
- 36
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 7 · 2023: 1 · 2022: 4 · 2021: 4
- FWCI
- 1.71
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Health Sciences › Health Professions › Health Information Management › Healthcare Regulation
- Keywords
- DevOps, ISO 13485, medical device development, regulatory compliance, agile software development, regulatory challenges, change management, medical device software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review