Ethics and governance of trustworthy medical artificial intelligence

Probably not a substantial claim: may not pass S3 (load-bearing).

Data-induced bias refers to the bias when the training data is not representative or insufficient that affects the algorithm model.

“Data-induced bias refers to the bias when the training data is not representative or insufficient that affects the algorithm model”

• cites [27] Mooney SJ, Pejaver V. Big data in public health: terminology, machine learning, and privacy. Annu Rev Public Health. 2018;39(1):95–112. https://doi.org/10.1146/annurev-publhealth-040617-014208. sourceNot checked yet

  Not checked in the free run.

• cites [28] Senders JT, Zaki MM, Karhade AV, et al. An introduction and overview of machine learning in neurosurgical care. Acta Neurochir (Wien). 2018;160(1):29–38. https://doi.org/10.1007/s00701-017-3385-8. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Algorithms inherit human biases, and such biases are likely to be reinforced and amplified with the accumulation of data and iterations of algorithms.

“Not only do algorithms inherit human biases, but such biases are also likely to be reinforced and amplified with the accumulation of data and iterations of algorithms”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Most of the texts in TCM are unstructured data and Chinese medicine has more non-standardized expressions, manual annotation is extremely costly and difficult, which greatly affects data quality.

“since most of the texts in TCM are unstructured data and Chinese medicine has more non-standardized expressions, manual annotation is extremely costly and difficult, which greatly affects data quality”

May combine two claims.

These biases may be continuously replicated and amplified in algorithm and lead to biased prediction results, which may cause discriminatory treatment of certain people in medical care and even lead to medical safety accidents.

“These biases may be continuously replicated and amplified in algorithm and lead to biased prediction results, which may cause discriminatory treatment of certain people in medical care and even lead to medical safety accidents”

Probably not a substantial claim: may not pass S4 (based).

The opacity of algorithms affects patients' and doctors' trust in medical AI.

“The opacity of algorithms affects patients’ and doctors’ trust in medical AI”

May combine two claims.

In healthcare, biased algorithms may underestimate or overestimate certain patients' risks, and unrepresentative data may consolidate or exacerbate health disparities.

“In healthcare, biased algorithms may underestimate or overestimate certain patients’ risks, and unrepresentative data may consolidate or exacerbate health disparities”

Probably not a substantial claim: may not pass S4 (based).

Data quality directly affects the quality of medical AI algorithm models.

“data quality will directly affect the quality of medical AI algorithm models”

May combine two claims.

There are three possible reasons for opacity: algorithms are trade secrets; inability of lay people to understand; complex nature of algorithms.

“There are three possible reasons for opacity: (1) algorithms are trade secrets that companies intentionally hide; (2) the inability of lay people to understand programming and algorithmic techniques; (3) the complex nature of the algorithms themselves, which are incomprehensible to humans”

• cites [32] Burrell J. How the machine “thinks”: understanding opacity in machine learning algorithms. Big Data Soc. 2016. https://doi.org/10.1177/2053951715622512. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Five subjects influence the trustworthiness of medical AI: data quality, algorithmic bias, opacity, safety and security, and responsibility attribution.

“We adopted a multidisciplinary approach and summarized five subjects that influence the trustworthiness of medical AI: data quality, algorithmic bias, opacity, safety and security, and responsibility attribution”

Probably not a substantial claim: may not pass S3 (load-bearing).

Algorithmic bias includes both human-induced bias and data-induced bias.

“Algorithmic bias includes both human-induced bias and data-induced bias”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

At present, there are problems in the data of medical AI, such as data errors and omissions, lack of unified metadata standards, difficulties in data fusion, lack of data management, non-standardized strategy for data cleaning, and much medical data is stored in unstructured forms.

“At present, there are some problems in the data of medical AI, such as data errors and omissions in the original data entry process, lack of unified metadata standards, difficulties in data fusion, lack of data management, non-standardized strategy for data cleaning, and much medical data is stored in unstructured forms such as text and images”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Algorithmic bias can affect AI clinical predictions and exacerbate health disparities.

“Algorithmic bias can affect AI clinical predictions and exacerbate health disparities”

Probably not a substantial claim: may not pass S4 (based).

Algorithmic errors or security vulnerabilities can pose significant risks and harm to patients.

“algorithmic errors or security vulnerabilities can pose significant risks and harm to patients”

Probably not a substantial claim: may not pass S4 (based).

The involvement of medical AI in clinical practices may threaten doctors' and patients' autonomy and dignity.

“The involvement of medical AI in clinical practices may threaten doctors ‘and patients’ autonomy and dignity”

May combine two claims.

Deep learning is a typical "black box", it is opaque and uninterpretable, which makes the biases difficult to be detected.

“deep learning is a typical “black box”, it is opaque and uninterpretable, which makes the biases difficult to be detected”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Medical data are primarily unstructured, lacking uniform and standardized annotation.

“Medical data are primarily unstructured, lacking uniform and standardized annotation”

May combine two claims.

Watson cannot independently and synthetically evaluate medical problems, nor can it independently conceptualize and search for new relevant information, because it lacks the higher level of cognition required for critical thinking and the diversity of solutions ranked by intrinsic value.

“Although Watson is good at answering questions, it cannot independently and synthetically evaluate medical problems, nor can it independently conceptualize and search for new relevant information because it lacks the higher level of cognition required for critical thinking and the diversity of solutions ranked by the intrinsic value [67].”

• cites [67] Chung J, Zink A. Hey Watson, can i sue you for malpractice? Examining the liability of artificial intelligence in medicine. Asia Pac J Health Law Eth. 2018;11(2):51–80.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Much medical institution data is stored in the cloud or third-party servers, which is more vulnerable to hacking.

“Much medical institution data is stored in the cloud or third-party servers, which is more vulnerable to hacking”

May combine two claims.

The hybrid approach combines the top-down and bottom-up approaches and is the current mainstream of AI algorithm design.

“The hybrid approach combines the first two and is the current mainstream of AI algorithm design.”

• cites [77] Wallach W, Allen C, Smit I. Machine morality: bottom-up and top-down approaches for modelling human moral faculties. AI Soc. 2008;22(4):565–82. https://doi.org/10.1007/s00146-007-0099-0. sourceNot checked yet

  Not checked in the free run.

• cites [78] Reddy S, Allan S, Coghlan S, et al. A governance model for the application of AI in health care. J Am Med Inform Assoc. 2019;27(3):491–7. https://doi.org/10.1093/jamia/ocz192. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Medical AI replaces certain tasks previously performed by physicians, changing the doctor-patient relationship and posing a dilemma in the division of ethical responsibilities.

“Medical AI replaces certain tasks previously performed by physicians, which will undoubtedly change the relationship between doctors and patients, and poses a dilemma in the division of ethical responsibilities.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Beauchamp and Childress first proposed four major principles of bioethics: respect for autonomy, nonmaleficence, beneficence, and justice.

“Beauchamp T.L. and Childress J.F. first proposed four major principles of bioethics in their book Principles of Biomedical Ethics: respect for autonomy, nonmaleficence, beneficence, and justice[76]”

• cites [76] Beauchamp T, Childress J. Principles of biomedical ethics. New York: Oxford University Press; 2013.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Humans have no consensus on ethical dilemmas like the trolley problem, and it is unlikely to generate an ethical standard that everyone agrees on.

“In fact, humans have no consensus on ethical dilemmas like the “trolley problem”. It is unlikely to generate an ethical standard that everyone agrees on to deal with the issues, which also poses an ethical challenge for moral algorithms.”

May combine two claims.

AI challenges the perceived authority of clinicians and may influence their independent judgment.

“In addition, AI challenges the perceived authority of clinicians and may influence their independent judgment.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

There is a huge gap between ethical orientation and application.

“Although people agree with ethical values, there is a huge gap between ethical orientation and application.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The limited IT power of medical institutions also makes it difficult to ensure data security.

“The limited IT power of medical institutions also makes it difficult to ensure data security”

Probably not a substantial claim: may not pass S2 (checkable).

How to find a balance between medical data sharing and patient privacy protection is also an ethical problem faced in medical AI applications.

“how to find a balance between medical data sharing and patient privacy protection is also an ethical problem faced in medical AI applications”

Classified as probably not a substantial claim. May combine two claims.

The value orientation of AI ethics and medical ethics is the same: to promote human health and well-being, and the bottom line is to do no harm to humans.

“The value orientation of AI ethics and medical ethics is the same, that is, to promote human health and well-being, and the bottom line is to do no harm to humans.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The corresponding legal system has not yet been improved, resulting in the lack of effective regulation and restraint in medical data collection, use and privacy protection.

“The corresponding legal system has not yet been improved, resulting in the lack of effective regulation and restraint in medical data collection, use and privacy protection”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The third type of opacity is inherent to machine learning and is also known as the algorithmic "black box".

“The third type of opacity is inherent to machine learning and is also known as the algorithmic “black box””

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Due to the many stakeholders, doctors cannot simply be blamed in the application of medical AI; more responsible stakeholders such as developers, algorithm designers, training data providers, and regulators need to be brought in.

“It can be seen that, due to the many stakeholders, we cannot simply blame the doctors in the application of medical AI, but need to bring in more responsible stakeholders, such as developers of medical AI, designers of algorithms, providers of training data, regulators of AI, etc.”

May combine two claims.

Current medical AI is only an auxiliary tool used by doctors to diagnose and treat diseases, and is not qualified to be a responsible subject.

“Current medical AI is only an auxiliary tool used by doctors to diagnose and treat diseases, not qualified to be a responsible subject.”

• AssumptionRests on an assumption: Most scholars believe that AI robots at this stage can only belong to the category of tools.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Medical AI, as a technology, contains the values of the developer or designer; therefore, those involved have an ethical responsibility for AI products.

“Medical AI, as a technology, contains the values of the developer or designer; therefore, those involved have an ethical responsibility for AI products.”

Classified as probably not a substantial claim. May combine two claims.

The law is a mandatory norm with lagging defects and a long legislative cycle, so it can only be 'hindsight' and cannot provide timely and effective protection measures.

“The law is a mandatory norm with certain lagging defects, and the legislative process is characterized by a harshness and long cycle, and more often than not, it can only be “hindsight” and cannot provide timely and effective protection measures.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Medical AI is still just a machine or program at this stage. It cannot adjust its actions to the actual situation and must rely on doctors to manipulate the machinery or make the final decision.

“medical AI is still just a machine or program at this stage. It cannot adjust its actions to the actual situation and must rely on doctors to manipulate the machinery or make the final decision”

May combine two claims.

Healthcare data has high research and business value, but data breach occurs frequently.

“Healthcare data also has high research and business value, but data breach occurs frequently”

May combine two claims.

It is impossible to develop a code that covers all possibilities. Therefore, safety flaws in AI may endanger more patients than the possibility of a single misdiagnosis by a doctor because automated systems will replicate more errors.

“It is impossible to develop a code that covers all possibilities. Therefore, safety flaws in AI may endanger more patients than the possibility of a single misdiagnosis by a doctor because automated systems will replicate more errors”

• cites [50] Macnish K. Unblinking eyes: the ethics of automating surveillance. Eth Inf Technol. 2012;14:151–67. https://doi.org/10.1007/s10676-012-9291-0. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The opacity of the algorithm creates "ignorance" among human agents and will affect the trust of patients and clinicians in AI tools.

“The opacity of the algorithm creates “ignorance” among human agents and will affect the trust of patients and clinicians in AI tools”

Classified as probably not a substantial claim.

This automated setup means that even if the AI gives up control, none of the humans has time to intervene.

“This automated setup means that even if the AI gives up control, none of the humans has time to intervene, so who is responsible for the accident?”

May combine two claims.

AI neither meets the traditional criteria of free will nor is aware of what it is doing.

“From Aristotle's “action must originate from the agent, and a person cannot be unaware of what he or she is doing”, AI neither meets the traditional criteria of free will nor is aware of what it is doing.”

• cites [65] Coeckelbergh M. Artificial intelligence, responsibility attribution, and a relational justification of explainability. Sci Eng Eth. 2020;26(4):2051–68. https://doi.org/10.1007/s11948-019-00146-8. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

A 2022 NHTSA report showed that in an investigation of 16 Tesla crashes, several showed Autopilot gave back control of the car to the human driver in less than a second on average before the crash, resulting in 15 injuries and one fatality.

“In 2022, a report by the National Highway Traffic Safety Administration (NHTSA) showed that in an investigation of 16 Tesla crashes, several showed that Autopilot (Automated Assisted Driving) gave back control of the car to the human driver in “less than a second” on average before the crash and resulted in a total of 15 injuries and one fatality [70].”

• cites [70] Gitlin JM. NHTSA data shows Teslas using Autopilot crashed 273 times in less than a year. 2022. https://arstechnica.com/cars/2022/06/teslas-using-autopilot-crashed-273-times-in-less-than-a-year/. Accessed 28 Jul 2022.Not checked yet

  Not checked in the free run.

May combine two claims.

Doctors seem to be more repulsed by medical AI than patients. Based on an inherent mindset emphasizing science and logic, doctors are less willing to trust and rely on things they cannot explain.

“doctors seem to be more repulsed by medical AI than patients. Based on an inherent mindset emphasizing science and logic, doctors are less willing to trust and rely on things they cannot explain”

• cites [40] Feldman R, Aldana E, Stein K. Artificial intelligence in the health care space: how we can trust what we cannot know. Stanf Law Policy Rev. 2019;30:399–419.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Ethics and morality become an effective complement to the legal system.

“Therefore, ethics and morality become an effective complement to the legal system, and AI technology innovation must be carried out in accordance with ethical requirements.”

Probably not a substantial claim: may not pass S1 (asserted). May combine two claims.

Some scholars argue that although AI may surpass humans in many aspects, AI does not possess free will essentially and does not have moral subjectivity.

“Some scholars argue that although AI may surpass humans in many aspects, they do not possess free will essentially and does not have moral subjectivity [63, 64].”

• cites [63] Gan SP. How can robots have rights. Eth Res. 2017;3:126–30. https://doi.org/10.3969/j.issn.1671-9115.2017.03.025. sourceNot checked yet

  Not checked in the free run.

• cites [64] Sparrow R. Killer robots. J Appl Philos. 2007;24(1):62–77. https://doi.org/10.1111/j.1468-5930.2007.00346.x. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Medical chatbots provide diagnostic and treatment recommendations in order to reduce unnecessary doctor visits. However, these medical chatbots may also harm patients if they are not continuously updated, checked, or regulated.

“medical chatbots provide diagnostic and treatment recommendations in order to reduce unnecessary doctor visits. However, these medical chatbots may also harm patients if they are not continuously updated, checked, or regulated”

• cites [7] Katznelson G, Gerke S. The need for health AI ethics in medical school education. Adv Health Sci Educ. 2021;26:1447–58. https://doi.org/10.1007/s10459-021-10040-3. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The main situations of medical data breaches include cyber attacks such as hacker intrusion and unauthorized access, and even theft or loss of data by insiders in possession of the data.

“The main situations of medical data breaches include cyber attacks such as hacker intrusion and unauthorized access, and even theft or loss of data by insiders in possession of the data”

Probably not a substantial claim: may not pass S3 (load-bearing).

The data breach is also a security concern.

“The data breach is also a security concern”

May combine two claims.

Most current medical AI algorithms are trained using historical data from retrospective studies. When encountering real-world data that differ from that in the training datasets, the performance of AI may be worse, leading to clinical risks.

“most current medical AI algorithms are trained using historical data from retrospective studies. When encountering real-world data that differ from that in the training datasets, the performance of AI may be worse, leading to clinical risks”

• cites [52] Kelly CJ, Karthikesalingam A, Suleyman M, et al. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. https://doi.org/10.1186/s12916-019-1426-2. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

It is controversial to grant personality and rights to robots.

“However, it is controversial to grant personality and rights to robots.”

May combine two claims.

Even if a medical AI could perform computer and mathematical-level explanations, it would be difficult to translate them into meaningful medical explanations. It is because AI is always a statistical correlation analysis rather than a medical causality analysis.

“even if a medical AI could perform computer and mathematical-level explanations, it would be difficult to translate them into meaningful medical explanations. It is because AI is always a statistical correlation analysis rather than a medical causality analysis”

• cites [41] Kamensky S. Artificial intelligence and technology in health care: overview and possible legal implications. DePaul J Health Care Law. 2020;21(3):1–13.Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The common value goal in published ethical principles is to put the interests of human beings at the forefront; the development and use of AI must promote the good of mankind.

“The common value goal mentioned in these ethical principles is to put the interests of human beings at the forefront. That is to say, the development and use of AI must promote the good of mankind.”

• cites [71] IEEE. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, First Edition. 2016. https://ethicsinaction.ieee.org/. Accessed 2 Dec 2022.Not checked yet

  Not checked in the free run.

• cites [72] Future of life institute. Asilomar AI Principles. 2017. https://futureoflife.org/open-letter/asilomar-ai-principles/. Accessed 2 Dec 2022.Not checked yet

  Not checked in the free run.

• cites [73] The High-Level Expert Group on AI. Ethics guidelines for trustworthy AI. 2019. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. Accessed 2 Dec 2022.Not checked yet

  Not checked in the free run.

May combine two claims.

At the beginning of its application, doctors may suffer from machine malfunctions due to inexperience and unskilled operation, some of which can have serious consequences. Patients would not necessarily have suffered this harm if not for the use of medical AI.

“At the beginning of its application, doctors may suffer from machine malfunctions due to inexperience and unskilled operation, some of which (e.g., operating errors in surgical robots) can have serious consequences. Patients would not necessarily have suffered this harm if not for the use of medical AI”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Turing test suggests that complex AI may have a certain level of consciousness.

“The Turing test suggests that complex AI may have a certain level of consciousness [61].”

• cites [61] Warwick K, Shah H. Passing the turing test does not mean the end of humanity. Cognit Comput. 2016;8(3):409–19. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The medical AI Watson system passed the U.S. physician licensing exam in 2012.

“The medical AI Watson system passed the U.S. physician licensing exam in 2012 [66].”

• cites [66] Jessica SA. From Jeopardy to Jaundice: the medical liability implications of Dr. Watson and other artificial intelligence systems. La Law Rev. 2013;73:1049.Not checked yet

  Not checked in the free run.

May combine two claims.

Explaining deep learning-based medical AI is almost impossible, which is related to the connection complexity and mathematical abstraction of deep neural networks. People cannot explain how the machine makes judgments.

“explaining deep learning-based medical AI is almost impossible, which is related to the connection complexity and mathematical abstraction of deep neural networks. People cannot explain how the machine makes judgments”

• cites [38] Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare. J Med Eth. 2020;46(3):205–11. https://doi.org/10.1136/medethics-2019-105586. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Patients are generally repulsed by unexplained AI interventions in diagnostic and treatment sessions. They usually only accept AI to handle administrative matters such as registration, bill payment, and guidance.

“patients are generally repulsed by unexplained AI interventions in diagnostic and treatment sessions. They usually only accept AI to handle administrative matters such as registration, bill payment, and guidance”

• cites [39] Liu LL, He YM, Liu XD. A survey on patients perceptions and trust in artificial intelligence healthcare. Chin Med Eth. 2019;32(8):986–90. https://doi.org/10.12026/j.issn.1001-8565.2019.08.07. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

AI involvement may diminish the patient's subject position in clinical practices, undermine shared decision-making between doctors and patients, and threaten patient autonomy and dignity.

“Thus, AI involvement may diminish the patient's subject position in clinical practices, undermine shared decision-making between doctors and patients, and threaten patient’s autonomy and dignity.”

• AssumptionRests on an assumption: What is optimal for one patient in the same clinical situation may not be optimal for other patients.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The safety issues of medical AI are the risks and harms that occur in its practices, such as program errors, being affected by cybersecurity, the need for adequate testing, difficult software certification, etc., covering various legal and ethical issues.

“The safety issues of medical AI are the risks and harms that occur in its practices, such as program errors, being affected by cybersecurity, the need for adequate testing, difficult software certification, etc., covering various legal and ethical issues”

• cites [47] O’Sullivan S, Nevejans N, Allen C, et al. Legal, regulatory, and ethical frameworks for develop-ment of standards in artificial intelligence(AI) and autonomous robotic surgery. Int J Med Robot. 2019;15(1):e1968. https://doi.org/10.1002/rcs.1968. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

There is a consensus among computer scientists that an inherent tension is between the performance and explainability of machine learning, with the best-performing algorithms being the least transparent and those that provide precise explanatory algorithms often being less accurate.

“there is a consensus among computer scientists that an inherent tension is between the performance and explainability of machine learning, with the best-performing algorithms being the least transparent and those that provide precise explanatory algorithms often being less accurate”

• cites [46] Holzinger A, Biemann C, Pattichis CS. What Do We Need to Build Explainable AI Systems for the Medical Domain?. arXiv, 2017. https://doi.org/10.48550/arXiv.1712.09923 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The algorithmic black box makes models lack explainability and are difficult to proofread. If the algorithm is flawed or incorrect, the output will lead to even greater errors, which will most likely cause diagnostic errors, harm human health, and even deprive human lives.

“the algorithmic black box makes models lack explainability and are difficult to proofread. If the algorithm is flawed or incorrect, the output will lead to even greater errors, which will most likely cause diagnostic errors, harm human health, and even deprive human lives”

Probably not a substantial claim: may not pass S2 (checkable).

A gap exists between the machine's rule-based conditions and probabilistic statistics, and the human judgment mechanism, which is based on emotion and epiphanic meditation.

“A gap still exists between the machine’s rule-based conditions and probabilistic statistics, and the human judgment mechanism, which is based on emotion and epiphanic meditation.”

May combine two claims.

The form of the function that relates input data to output is too complex for non-specialists to understand, and even the designer may not be able to understand the logic of the calculation.

“The form of the function that relates the input data to the output is too complex for non-specialists to understand, and even the designer may not be able to understand the logic of the calculation”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Only with the participation of all relevant sectors of society and multiple parties can an ethical and publicly acceptable medical AI be developed.

“Only with the participation of all relevant sectors of society and multiple parties can an ethical and publicly acceptable medical AI be developed.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Trust in AI will improve through a more transparent development process with a better understanding of algorithms and AI functions.

“Trust in AI will improve through a more transparent development process with a better understanding of algorithms and AI functions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Aspirin was used for about 70 years for its antipyretic, analgesic, and anti-inflammatory clinical effects, but its pharmacological mechanisms were not understood until later.

“aspirin was used for about 70 years for its antipyretic, analgesic, and anti-inflammatory clinical effects, but its pharmacological mechanisms were not understood until later [93, 94].”

• cites [93] London AJ. Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Cent Rep. 2019;49(1):15–21. https://doi.org/10.1002/hast.973. sourceNot checked yet

  Not checked in the free run.

• cites [94] Vane JR, Botting RM. The mechanism of action of aspirin. Thromb Res. 2003;110(5–6):255–8. https://doi.org/10.1016/S0049-3848(03)00379-7. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Anonymous data cannot be used to identify a person and is therefore not subject to GDPR rules; if a company collects anonymous data, it does not need to obtain the consent of users.

“Anonymous data means that it cannot be used to identify a person and is therefore not subject to the GDPR rules, which means that if a company collects anonymous data, it does not need to obtain the consent of the users.”

• cites [86] EU: General Data Protection Regulation. 2018. https://gdpr-info.eu. Accessed 28 Jul 2022.Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Doctors are not exempt from liability because they are the main actors in diagnosing patients.

“At the same time, doctors are not exempt from liability because they are the main actors in diagnosing patients.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Algorithm transparency, though not equal to explainability, will form a powerful deterrent and encourage more diverse subjects to participate in supervision, greatly compensating for the lack of supervision of regulatory authorities.

“Although algorithm transparency is not equal to algorithm explainability, it will form a powerful deterrent and encourage more diverse subjects, such as medical institutions, insurance companies, and social security institutions, to participate in supervision, which will greatly compensate for the lack of supervision of regulatory authorities.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

If software development organizations were to require patient consent for each use of aggregated data, it would inevitably increase the cost of data use.

“In practice, however, if software development organizations were to require patient consent for each use of aggregated data, it would inevitably increase the cost of data use.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Manson and O'Neill argue that more specific consent is not always ethically better and is difficult to achieve in practice; consent requires unique communicative transactions.

“Manson and O’Neill [87] argue that more specific consent is not always ethically better and is difficult to achieve in practice. Consent requires unique communicative transactions.”

• cites [87] Manson NC, O’Neill O. Rethinking informed consent in bioethics. Cambridge: Cambridge University Press; 2007. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

From the perspective of human benefits, it is unethical not to use existing clinical data to develop tools to benefit all humanity.

“From the perspective of human benefits, it is also unethical not to use existing clinical data to develop tools to benefit all humanity [91].”

• cites [91] Larson DB, Magnus DC, Lungren MP, et al. Ethics of using and sharing clinical imaging data for artificial Intelligence: a proposed framework. Radiology. 2020;295(3):675–82. https://doi.org/10.1148/radiol.2020192536. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

In the third scenario, the related people are scrupulous in their duties but still cannot prevent the medical AI from making an incorrect diagnosis that leads to the patients' misfortune; there is no clear evidence of who is responsible, or responsibility cannot be attributed to any individual.

“In the third scenario, the related people are scrupulous in their duties, but still cannot prevent the medical AI from making an incorrect diagnosis that leads to the patients’ misfortune. There is no clear evidence of who is responsible, or we cannot attribute responsibility to any individual.”

Classified as probably not a substantial claim.

Floridi proposed a principle of moral responsibility of faultless responsibility, meaning that no one is at fault but they are still responsible for it.

“Floridi proposed a principle of moral responsibility of faultless responsibility, which means that no one is at fault, but they are still responsible for it.”

• cites [79] Floridi L, Peng C. Faultless responsibility: on the nature and allocation of moral responsibility for distributed moral actions. Philos Trans R Soc A Math Phys Eng Sci. 2018;374(2083):0112. https://doi.org/10.2139/ssrn.3835211. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Existing medical AIs are not moral enablers and do not have the ability to think and make decisions independently and cannot be considered as duty bearers.

“In the previous section, we have elaborated that existing medical AIs are not moral enablers and do not have the ability to think and make decisions independently and cannot be considered as duty bearers.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

In either approach, AI's response to the causality dilemma remains inadequate, and the threat to human moral subjectivity by allowing AI to form universal ethical guidelines spontaneously is even more deadly.

“However, in either approach, AI’s response to the causality dilemma remains inadequate, and the threat to human moral subjectivity by allowing AI to form universal ethical guidelines spontaneously is even more deadly.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

It is ethical to dispense with re-informed consent for data use under conditions that ensure data security and do not compromise patient privacy, as long as a sound ethical review system is in place.

“We believe that it is ethical to dispense with re-informed consent for data use under conditions that ensure data security and do not compromise patient privacy, as long as a sound ethical review system is in place.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Medical AI does not have moral status at this stage, and humans remain the responsibility bearers.

“At the legal level, we clarified that medical AI does not have moral status at this stage, and humans remain the responsibility bearers.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Medicine itself is potentially risky and uncertain.

“Medicine itself is potentially risky and uncertain.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

At the current level of medical AI development, doctors are still in the position of supervising and should not let machines make final decisions without their permission.

“At the current level of medical AI development, doctors are still in the position of supervising and they should not let machines make final decisions without their permission.”

May combine two claims.

GDPR requires operators to allow users to express a desire for personal data to be 'forgotten', and fines for violating GDPR can be as high as 4% of the global revenue of the fined organization.

“For example, operators are required to allow users to express a desire for personal data to be “forgotten”, i.e., “I don’t want you to remember my past data and I want you not to use my data for modeling purposes from now on”. At the same time, the consequences of violating GDPR are severe, and fines can be as high as 4% of the global revenue of the fined organization.”

• cites [86] EU: General Data Protection Regulation. 2018. https://gdpr-info.eu. Accessed 28 Jul 2022.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Patients' clinical data have potential medical value and should be widely shared to promote the health and well-being of all humans.

“Patients’ clinical data have potential medical value and should be widely shared to promote the health and well-being of all humans.”

May combine two claims.

Interpretability is not a necessary or sufficient condition for accountability.

“Interpretability is not a necessary or sufficient condition for accountability.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In 2018, the EU introduced the General Data Protection Regulation (GDPR), the first bill on personal data privacy protection; unlike previous industry regulations, it is a truly enforceable law with specific and strict requirements.

“In 2018, the EU introduced the first bill on personal data privacy protection-General Data Protection Regulation (GDPR) [86]. Unlike previous industry regulations, this is a truly enforceable law with specific and strict requirements.”

• cites [86] EU: General Data Protection Regulation. 2018. https://gdpr-info.eu. Accessed 28 Jul 2022.Not checked yet

  Not checked in the free run.

May combine two claims.

China's Deep Learning Assisted Decision-Making Medical Device Software Approval Points requires quality control of training data, diversity of data sources, and subdivision of data sets into training, validation, and testing sets with different acquisition requirements.

“For example, China’s Deep Learning Assisted Decision-Making Medical Device Software Approval Points [81] requires quality control of training data, and should ensure diversity of data sources, with data collected from multiple medical institutions at different geographic and hierarchical levels whenever possible. The Approval points further subdivide the data sets into training sets (for algorithm”

• cites [81] Deep Learning Assisted Decision-Making Medical Device Software Approval Points. China National Medical Products Administration [EB/OL] (2019–7–3). https://www.cmde.org.cn/xwdt/shpgzgg/gztg/20190703141714991.htmlNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Macnish and Gauttier argue that it is not appropriate to talk about our relationship with data in terms of ownership; there are only weak philosophical grounds for giving citizens control of 'their' data; control should be based around custody of data and the potential for harm.

“Macnish and Gauttier [82] argue that it's not appropriate to talk about our relationship with data in terms of ownership. There are only weak philosophical grounds on giving citizens control of ‘their’ data. Control should be based around custody of data and the potential for harm.”

• cites [82] Macnish K, Gauttier S. A pre-occupation with possession: the (non-) ownership of personal data. Big data and democracy. Edinburgh: Edinburgh University Press; 2020. p. 42–56.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Respect for personal privacy is a crucial ethical principle in health care because privacy is linked to personal identity and autonomy.

“Respect for personal privacy is a crucial ethical principle in health care because privacy is linked to personal identity and autonomy [84].”

• cites [84] Cath C. Governing artificial intelligence: ethical, legal and technical opportunities and challenges. Philos Trans A Math Phys Eng Sci. 2018;376(2133):20180080. https://doi.org/10.1098/rsta.2018.0080. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Current AI technology essentially obtains data by measuring the real world, extracts algorithmic models from the data, and uses the models to make relevant predictions.

“Current AI technology essentially obtains data by measuring the real world, extracts algorithmic models from the data, and uses the models to make relevant predictions.”

May combine two claims.

A few companies are keen to propose ethical standards rather than binding rules because there is no substantial penalty if they change or disregard ethical standards.

“A few companies are keen to propose ethical standards rather than binding rules. The reasons for this are apparent because there is no substantial penalty if they change or disregard ethical standards under this circumstance [80].”

• cites [80] Calo R. Artificial intelligence policy: a primer and roadmap. UC Davis Law Rev. 2017;2:399–436. https://doi.org/10.2139/ssrn.3015350. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

As of 2020, the FDA has approved 222 AI medical device products, and Europe has approved 240 AI medical device products with European conformity certification.

“As of 2020, the U.S. Food and Drug Administration (FDA) has approved a total of 222 AI medical device products, and Europe has approved a total of 240 AI medical device products with European conformity certification [97].”

• cites [97] Ujm A, Pd B, Pknv B. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis. Lancet Digits Health. 2021;3:195–203. https://doi.org/10.1016/S2589-7500(20)30292-2. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The main obstacle to data sharing is the ownership of data.

“The main obstacle to data sharing is the ownership of data.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some scholars proposed more lenient forms of informed consent, such as broad consent and blanket consent, to facilitate practical implementation; however, the moral rationality for these informed consents remains controversial.

“Some scholars proposed more lenient forms of informed consent, such as broad consent and blanket consent, to facilitate practical implementation [88, 89]. However, the moral rationality for these informed consents remains controversial.”

• cites [88] Virani AH, Longstaff H. Ethical considerations in biobanks: how a public health ethics perspective sheds new light on old controversies. J Genet Couns. 2015;24(3):428–32. https://doi.org/10.1007/s10897-014-9781-9. sourceNot checked yet

  Not checked in the free run.

• cites [89] Kaufman D, Bollinger J, Dvoskin R, et al. Preferences for opt-in and opt-out enrollment and consent models in biobank research: a national survey of Veterans Administration patients. Genet Med. 2012;14(9):787–94. https://doi.org/10.1038/gim.2012.45. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The algorithms of medical AI are based on pre-existing human experience.

“In addition, the algorithms of medical AI are based on the pre-existing human experience.”

Classified as probably not a substantial claim. May combine two claims.

Healthcare data are sensitive information about a person and are related to personal privacy.

“Healthcare data are sensitive information about a person [23, 83] which is also related to personal privacy.”

• cites [23] Char DS, Shah NH, Magnus D. Implementing machine learning in health care-addressing ethical challenges. N Engl J Med. 2018;378(11):981–3. https://doi.org/10.1056/NEJMp1714229. sourceNot checked yet

  Not checked in the free run.

• cites [83] Vayena E, Blasimme A, Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. 2018;15(11):e1002689. https://doi.org/10.1371/journal.pmed.1002689. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Distributed responsibility may result in a lack of individual responsibility, leading to a tendency for everyone to be conservative and making the application and innovation of new technologies impossible.

“However, this distributed responsibility may result in a lack of individual responsibility, leading to a tendency for everyone to be conservative and making the application and innovation of new technologies impossible.”

May combine two claims.

Algorithms may be continually updated beyond their initially approved clinical function, which may require particular policies and supervision.

“algorithms may be continually updated beyond their initially approved clinical function, which may require particular policies and supervision.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

Some believe that patients have an obligation to contribute to improving the quality of the health care system.

“Regarding the sharing of health care data, some believe that patients have an obligation to contribute to improving the quality of the health care system [90].”

• cites [90] Faden RR, Kass NE, Goodman SN, et al. An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep. 2013;43:16–27. https://doi.org/10.1002/hast.134. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

If the doctor has errors in operation, the doctor and the medical institution are responsible.

“If the doctor has errors in operation, the doctor and the medical institution are responsible.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Educating users and patients about medical AI is a way to ensure that they understand the benefits, risks, and limitations of medical AI devices and increase product transparency and user trust.

“Educating users and patients about medical AI is also a way to ensure that they understand the benefits, risks, and limitations of medical AI devices and increase product transparency and user trust [98].”

• cites [98] FDA. Artificial Intelligence and Machine Learning in Software as a Medical Device. 2021. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device. Accessed 29 Jul 2022.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Current medical AI falls under the category of medical devices.

“Besides, current medical AI falls under the category of medical devices”

May combine two claims.

GDPR requires algorithms to have interpretability, and data subjects can take intervention and require interpretation of the relevant data when they are not satisfied with automated decisions.

“GDPR requires algorithms to have interpretability, and data subjects can take intervention and require interpretation of the relevant data when they are not satisfied with automated decisions.”

• cites GDPRNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

If a doctor causes damage, the medical institution can recover compensation from the doctor after taking responsibility.

“If a doctor causes damage, the medical institution can recover compensation from the doctor after taking responsibility.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Algorithmic transparency and traceability do not require algorithms to be explainable, but they provide the possibility of algorithmic explanation and form effective supervision.

“algorithmic transparency and traceability do not require algorithms to be explainable, but they provide the possibility of algorithmic explanation and form effective supervision.”

Probably not a substantial claim: may not pass S4 (based).

AI robot participation in diagnosis and treatment is predicated on the approval of the medical institution where it is located.

“AI robot’s participation in diagnosis and treatment is predicated on the approval of the medical institution where it is located.”

Classified as probably not a substantial claim.

Legal experts concerned with AI governance criticize ethical principles as flawed and inadequate in addressing AI's ethical and social issues.

“Legal experts concerned with AI governance issues criticize ethical principles as flawed and inadequate in addressing AI’s ethical and social issues.”

• cites [80] Calo R. Artificial intelligence policy: a primer and roadmap. UC Davis Law Rev. 2017;2:399–436. https://doi.org/10.2139/ssrn.3015350. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Data and algorithms are the basis of AI computing and decision-making.

“Therefore, data and algorithms are the basis of AI computing and decision-making.”