The DOI matches its Crossref record; 57 of the 58 references it cites were found in a registry or an index.
Legal, regulatory, and ethical frameworks for development of standards in artificial intelligence (AI) and autonomous robotic surgery
Abstract
Background: This paper aims to move the debate forward regarding the potential for artificial intelligence (AI) and autonomous robotic surgery with a particular focus on ethics, regulation and legal aspects (such as civil law, international law, tort law, liability, medical malpractice, privacy and product/device legislation, among other aspects). Methods: We conducted an intensive literature search on current or emerging AI and autonomous technologies (eg, vehicles), military and medical technologies (eg, surgical robots), relevant frameworks and standards, cyber security/safety‐ and legal‐systems worldwide. We provide a discussion on unique challenges for robotic surgery faced by proposals made for AI more generally (eg, Explainable AI) and machine learning more specifically (eg, black box), as well as recommendations for developing and improving relevant frameworks or standards. Conclusion: We classify responsibility into the following: (1) Accountability; (2) Liability; and (3) Culpability. All three aspects were addressed when discussing responsibility for AI and autonomous surgical robots, be these civil or military patients (however, these aspects may require revision in cases where robots become citizens). The component which produces the least clarity is Culpability, since it is unthinkable in the current state of technology. We envision that in the near future a surgical robot can learn and perform routine operative tasks that can then be supervised by a human surgeon. This represents a surgical parallel to autonomously driven vehicles. Here a human remains in the ‘driving seat’ as a ‘doctor‐in‐the‐loop’ thereby safeguarding patients undergoing operations that are supported by surgical machines with autonomous capabilities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“many AIs are forms of intelligence expressed by some artifact (Bryson & Theodorou, 2019) that interacts with humans at various levels, such as robots and self-driving cars (Manoharan, 2019; O’Sullivan et al., 2019).”
From Ethical principles for artificial intelligence in education (2023) · cites this paper as O’Sullivan et al., 2019
In the citing paper
“The safety issues of medical AI are the risks and harms that occur in its practices, such as program errors, being affected by cybersecurity, the need for adequate testing, difficult software certification, etc., covering various legal and ethical issues”
From Ethics and governance of trustworthy medical artificial intelligence (2023) · cites this paper as [47]
In the citing paper
“The rapid evolution of AI technologies has outpaced the formulation of comprehensive ethical guidelines, raising concerns about the potential misuse and unintended consequences of these powerful tools [32], [33].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [32]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Medical error—the third leading cause of death in the US
Makary, M. A., & Daniel, M. (2016) · BMJ · doi:10.1136/bmj.i2139 · Cited by 3,264
Lanfranco, A. R., Castellanos, A. E., Desai, J. P., & Meyers, W. C. (2003) · Annals of Surgery · doi:10.1097/01.sla.0000103020.19595.7d · Cited by 1,309
Large Scale Multiple Kernel Learning
Sonnenburg, S., Rätsch, G., Schäfer, C., & Schölkopf, B. (2006) · MPG.PuRe (Max Planck Society) · doi:10.5555/1248547.1248604 · Cited by 1,201
The responsibility gap: Ascribing responsibility for the actions of learning automata
Matthias, A. (2004) · Ethics and Information Technology · doi:10.1007/s10676-004-3422-1 · Cited by 1,101 · cited 2×
“ForMatthias(2004), 42 the traditional rules of liability law would then be unsuitable for unpredictable machines because the human would no longer really have control of the actions of the robot.”
BackgroundInteractive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906
Self-citationYe, Q.-H., Qin, L., Forgues, M., He, P., Kim, J.-W., Peng, A. C., Simon, R., Li, Y., Robles, A. I., Chen, Y., et al. (2003) · Nature Medicine · doi:10.1038/nm843 · Cited by 849
Robot ethics: the ethical and social implications of robotics
(2012) · Choice Reviews Online · doi:10.5860/choice.50-0844 · Cited by 790
Wallach, W., & Allen, C. (2009) · Oxford University Press · doi:10.1093/acprof:oso/9780195374049.001.0001 · Cited by 711 · cited 1×
“Such decisions could be informed by what Wallach and Allen (2009) 55 call “ top ‐ down, ” ie, theory ‐ driven, approaches to ethical decision making, or they could be shaped by “ bottom ‐ up, ” or data ‐ driven processes that have been trained to emulate human decision makers.”
BackgroundSelf-citationSupervised autonomous robotic soft tissue surgery
Shademan, A., Decker, R., Opfermann, J. D., Léonard, S., Krieger, A., & Kim, P. C. W. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aad9398 · Cited by 626
Transparent, explainable, and accountable AI for robotics
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · Science Robotics · doi:10.1126/scirobotics.aan6080 · Cited by 413
The SARS Epidemic in Hong Kong: What Lessons have we Learned?
Hung, L. S. (2003) · Journal of the Royal Society of Medicine · doi:10.1177/014107680309600803 · Cited by 380
Prolegomena to any future artificial moral agent
Allen, C., Varner, G., & Zinser, J. (2000) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/09528130050111428 · Cited by 326
Self-citationEvolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature
Moustris, G. P., Hiridis, S., Deliparaschos, K. M., & Konstantinidis, M. K. (2011) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.408 · Cited by 323
The SARS epidemic in Hong Kong: what lessons have we learned?
Hung, L. S. (2003) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.96.8.374 · Cited by 264
Ethics of healthcare robotics: Towards responsible research and innovation
Stahl, B. C., & Coeckelbergh, M. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.08.018 · Cited by 235
The evolution of robotic surgery: surgical and anaesthetic aspects
Ashrafian, H., Clancy, O., Grover, V., & Darzi, A. W. (2017) · British Journal of Anaesthesia · doi:10.1093/bja/aex383 · Cited by 200
Self-citationSARS: experience at Prince of Wales Hospital, Hong Kong
Tomlinson, B., & Cockram, C. S. (2003) · The Lancet · doi:10.1016/s0140-6736(03)13218-7 · Cited by 162
Kim, S. Y., Diggans, J., Pankratz, D., Huang, J., Pagan, M., Sindy, N., Tom, E., Anderson, J. D., Choi, Y., Lynch, D. A., et al. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(15)00140-x · Cited by 137
Smart Tissue Anastomosis Robot (STAR): A Vision-Guided Robotics System for Laparoscopic Suturing
Léonard, S., Wu, K. L., Kim, Y., Krieger, A., & Kim, P. C. W. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2014.2302385 · Cited by 129
Surgical Robotics: Systems Applications and Visions
(2011) · doi:10.1007/978-1-4419-1126-1 · Cited by 126
Holzinger, A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40511-2_22 · Cited by 116
Self-citationVital, Sophia, and Co.—The Quest for the Legal Personhood of Robots
Pagallo, U. (2018) · Information · doi:10.3390/info9090230 · Cited by 113
Self-citationUnderreporting of Robotic Surgery Complications
Cooper, M. A., Ibrahim, A. M., Lyu, H. G., & Makary, M. A. (2013) · Journal for Healthcare Quality · doi:10.1111/jhq.12036 · Cited by 112
Holzinger, A., Plass, M., Holzinger, K., Crişan, G. C., Pintea, C.-M., & Palade, V. (2019) · Creative Mathematics and Informatics · doi:10.37193/cmi.2019.02.04 · Cited by 111
Self-citationNHS ransomware attack spreads worldwide
Collier, R. (2017) · Canadian Medical Association Journal · doi:10.1503/cmaj.1095434 · Cited by 91
International Governance of Autonomous Military Robots
Marchant, G. E., Allenby, B., Arkin, R. C., Borenstein, J., Gaudet, L. M., Kittrie, O. F., Lin, P. K., Lucas, G. R., Jr., O’Meara, R. M., & Silberman, J. (2014) · Springer eBooks · doi:10.1007/978-90-481-9707-1_102 · Cited by 87
Rosen, J., & Hannaford, B. (2006) · IEEE Spectrum · doi:10.1109/mspec.2006.1705774 · Cited by 83
Li, Z., & Chiu, P. W. Y. (2018) · Visceral Medicine · doi:10.1159/000486121 · Cited by 77
Introduction to MAchine Learning & Knowledge Extraction (MAKE)
Holzinger, A. (2017) · Machine Learning and Knowledge Extraction · doi:10.3390/make1010001 · Cited by 71
Self-citationCyber attacks: protecting national infrastructure
Amoroso, E. G. (2011) · Cited by 71
Paraskevopoulos, D., Unterberg, A. W., Metzner, R., Dreyhaupt, J., Eggers, G., & Wirtz, C. R. (2011) · Neurosurgical Review · doi:10.1007/s10143-010-0302-5 · Cited by 62
Tactical Driving Behavior With Different Levels of Automation
Kircher, K., Larsson, A. F. L., & Hultgren, J. A. (2013) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2277725 · Cited by 62
Datteri, E. (2011) · Science and Engineering Ethics · doi:10.1007/s11948-011-9301-3 · Cited by 50
Algo-Rhythms and the Beat of the Legal Drum
Pagallo, U. (2017) · Philosophy & Technology · doi:10.1007/s13347-017-0277-z · Cited by 44
Self-citationOpfermann, J. D., Léonard, S., Decker, R., Uebele, N. A., Bayne, C. E., Joshi, A. S., & Krieger, A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206210 · Cited by 44
A novel modification of the Turing test for artificial intelligence and robotics in healthcare
Ashrafian, H., Darzi, A. W., & Athanasiou, T. (2014) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.1570 · Cited by 43
Self-citationUllman, D. L., & Malle, B. F. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3029798.3038423 · Cited by 32
Robotic Surgery: On the Cutting Edge of Ethics
SHARKEY, N. E., & SHARKEY, A. J. C. (2012) · Computer · doi:10.1109/mc.2012.424 · Cited by 29
The Five Robots—A Taxonomy for Roboethics
Steinert, S. (2013) · International Journal of Social Robotics · doi:10.1007/s12369-013-0221-z · Cited by 28
Mori, K., Ota, S., Deguchi, D., Kitasaka, T., Suenaga, Y., Iwano, S., Hasegawa, Y., Takabatake, H., Mori, M., & Natori, H. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04271-3_86 · Cited by 24
Coeckelbergh, M. (2011) · Philosophy & Technology · doi:10.1007/s13347-011-0019-6 · Cited by 22
Semi-autonomous surgical tasks using a miniature in vivo surgical robot
Dumpert, J., Lehman, A. C., Wood, N. A., Oleynikov, D., & Farritor, S. M. (2009) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2009.5332800 · Cited by 22
Towards a Framework for Alignment Between Automotive Safety and Security Standards
Schmittner, C., & Ma, Z. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24249-1_12 · Cited by 20
Can Artificial Intelligences Suffer from Mental Illness? A Philosophical Matter to Consider
Ashrafian, H. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-016-9783-0 · Cited by 20
Self-citationTort Liability for Vendors of Insecure Software: Has the Time Finally Come?
Scott, M. D. (2007) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 19
Intelligent robots must uphold human rights
Ashrafian, H. (2015) · Nature · doi:10.1038/519391a · Cited by 19
Self-citation"Sophisticated Robots": Balancing Liability, Regulation, and Innovation
Hubbard, F. P. (2014) · Scholar Commons (University of South Carolina) · Cited by 18
Toh, J. W. T., & Kim, S. H. (2017) · Journal of Robotic Surgery · doi:10.1007/s11701-017-0760-7 · Cited by 18
Franke, U. (2017) · doi:10.1002/9781118868065.ch2 · Cited by 10
Ethical reflections on health care robotics
Datteri, E., & Tamburrini, G. (2009) · BOA (University of Milano-Bicocca) · Cited by 10
Sheriff, K. D. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2735945 · Cited by 7
Autonomous Surgical RobotsPreprint
Britton, D. (2016) · SSRN Electronic Journal · Cited by 3
Beard, T. R., Ford, G. S., Koutsky, T. M., & Spiwak, L. J. (2009) · UIC Law Open Access Repository (University of Illinois at Chicago) · Cited by 2
Pagallo, U. (2013) · Law, governance and technology series · doi:10.1007/978-94-007-6564-1_1 · Cited by 2
Self-citationPopa, O. (2017) · European Taxation · doi:10.59403/2847wk6 · Cited by 1
Research works printed but not linked: 13
- Garcia P, 2009, Trauma Pod: a semi‐automated telerobotics surgical system, Int J Med Robot, 5
- BonaciT YanJ HerronJ et al. "Experimental analysis of cyber security attacks on teleoperated surgical robotics".in preparation to the ACM Transaction on Cyber‐Physical Systems2017.
- Holzinger K, 2018, Can we trust machine learning results?, Artificial Intelligence in Safety Critical News, 112, 42
- Decker M, 2014, Rethinking Responsibility in Science and Technology, Pisa, 65
- World Commission on the Ethics of Scientific Knowledge and Technology, 2017, Report of COMEST on robotics ethics
- Palmerini E, 2014, D6.2 Guidelines on Regulating Robotics
- MichaelK BowmanD JonesML et al.Special issue on social implications of robotics and AI. IEEE Technology and Societyy Magazine.2018.
- Veruggio G, 2005, ICRA, 1
- Nevejans N, 2017, Traité de droit et d'éthique de la robotique civile (Treatise of Law and Ethics of Civil Robotics), LEH Edition (in French), 646
- Graaf MMA, 2017, How People Explain Action (and AIS should too)
- Sharif E, 2014, Medical liability in humanitarian missions, The Journal of Humanitarian Assistance
- International Federation of Red Cross and Red Crescent Societies, 2011, The Legal Framework for Volunteering in Emergencies
- Pugliano J, 2017, The Robots are Coming: A Human's Survival Guide to Profiting in the Age of Automation
Cited by
Sorted by citations
Ethical principles for artificial intelligence in education
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.-P. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11316-w · Cited by 1,228
“Whereas learning analytics are governed by data ethics, many AIs are forms of intelligence expressed by some artifact (Bryson & Theodorou, 2019) that interacts with humans at various levels, such as robots and self-driving cars (Manoharan, 2019; O’Sullivan et al., 2019).”
BackgroundPrivacy and artificial intelligence: challenges for protecting health information in a new era
Murdoch, B. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00687-3 · Cited by 991
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Serrano, D. R., Luciano, F. C., Anaya, B. J., Öngoren, B., Kara, A., Molina, G., Ramirez, B. I., Sánchez-Guirales, S. A., Simon, J. A., Tomietto, G., et al. (2024) · Pharmaceutics · doi:10.3390/pharmaceutics16101328 · Cited by 594
Ethics and governance of trustworthy medical artificial intelligence
Zhang, J., & Zhang, Z.-M. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02103-9 · Cited by 507
Metadata
- Authors
- O'Sullivan, S., Nevejans, N., Allen, C., Blyth, A., Leonard, S., Pagallo, U., Holzinger, K., Holzinger, A., Sajid, M. I., & Ashrafian, H.
- Affiliations
- Department of Pathology, Faculdade de Medicina Universidade de São Paulo São Paulo Brazil
- Université d'Artois (FR)
- University of Pittsburgh (US)
- University of South Wales (GB)
- Department of Computer Science Johns Hopkins University Baltimore Maryland
- University of Turin (IT)
- SBA Research (AT)
- Medical University of Graz (AT)
- Wirral University Teaching Hospital NHS Foundation Trust (GB)
- Imperial College London (GB)
- ORCID
- Venue
- The International Journal of Medical Robotics and Computer Assisted Surgery
- Publisher
- Wiley
- Published
- 2019-01-09
- Volume
- 15
- Issue
- 1
- Pages
- e1968-e1968
- ISSN
- 1478-5951 (ISSN-L)
- 1478-596X
- OpenAlex
- W2899881299
- Cited by
- 437
- References
- 58
- Citations by year
- 2026: 46 · 2025: 71 · 2024: 93 · 2023: 72 · 2022: 60 · 2021: 59 · 2020: 28 · 2019: 8
- FWCI
- 45.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Medical Robotics and Computer Assisted Surgery (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion)
- Funders
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Topics
- Social Sciences › Law › Digitalization, Law, and Regulation
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- artificial intelligence, autonomous robotic surgery, surgical robots, AI standards, legal framework, regulatory framework, AI ethics, liability, accountability, culpability, explainable AI, black box problem, medical malpractice, tort law, privacy, artificial intelligence in warfare, autonomous vehicles
- MeSH terms
- Algorithms, Artificial Intelligence, Ethics, Medical, Europe, Humans, United States, Computer Security, Medical Errors, Robotic Surgical Procedures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
