Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond

In the paper

“While these tools can streamline communication and provide quick responses, they lack the emotional depth and understanding of human healthcare providers.”

What the sources say

• ?Could not verify

  The Literature review (§2) describes chatbots as simulating human conversation, and §2.5 describes a virtual nurse designed to feel like talking to a person. Neither section says chatbots lack the emotional depth or understanding of human healthcare providers. Searches for empathy, emotions, and limitations relative to human doctors found no passage addressing that claim; page numbers were unavailable. (LLM, conf=0.76)

  cites [132] sourceExperimental — accuracy not yet measured

In the paper

“The Food and Drug Administration (FDA) regulatory approach to AI in healthcare, for example, seeks to ensure that AI applications meet the same safety and effectiveness standards as traditional medical devices.This approach ensures that AI systems, such as those used for diagnosing diseases or making treatment recommendations, are held accountable for their performance and outcomes [143].”

What the sources say

• Partly supported

  The Introduction states that FDA approval requires compliance with standards and evaluation of device safety and efficacy; the Discussion says the FDA assesses medical devices on the same basis regardless of ML use. But the paper does not say this approach ensures AI systems are held accountable for their performance and outcomes. Its Discussion instead notes that responsibility for clinical decisions often remains with clinicians and that questions about responsibility require further examination (no page numbers provided). (LLM, conf=0.86)

  cites [143] sourceExperimental — accuracy not yet measured

In the paper

“However, despite these advancements, certain limitations hinder the widespread adoption and integration of AI into clinical practice, particularly in terms of explainability and interpretability [3], [4].”

What the sources say

• Not checked yetcites [3] source
• Not checked yetcites [4] Karim, 2018, Machine learning interpretability: A science rather than a tool, arXiv:1807.06722

In the paper

“Insufficient interpretability hinders the widespread and responsible adoption of ML, especially in high-stakes domains where transparency and accountability are paramount.”

In the paper

“In the healthcare context, the lack of interpretability [10] and explainability [11] in ML and DL models can lead to ethical issues and a loss of trust [12], [13].”

What the sources say

• Not checked yetcites [12] source
• Not checked yetcites [13] source

In the paper

“By integrating transparency and interpretability into AI models, these systems can instill trust and confidence among healthcare professionals and patients.”

In the paper

“However, the inner workings of these models are often complex and challenging to explain in human terms [38], [39].”

What the sources say

• Not checked yetcites [38] Bathaee, 2017, The artificial intelligence black box and the failure of intent and causation, Harv. JL Tech.
• Not checked yetcites [39] source

In the paper

“The black-box nature of AI models restricts the transparency of their outputs, making it difficult to comprehend the reasoning behind their decisions [18] as illustrated in Figure 1.”

What the sources say

• Not checked yetcites [18] Lakkaraju, 2017, Interpretable & explorable approximations of black box models, arXiv:1707.01154

In the paper

“The absence of explainability in AI systems also raises ethical concerns.”

In the paper

“Moreover, it can also facilitate regulatory compliance by enabling audits and assessments of AI models to ensure fairness, accountability, and compliance with ethical standards [29], [30].”

What the sources say

• Not checked yetcites [29] source
• Not checked yetcites [30] source

In the paper

“Moreover, the lack of interpretability hampers the ability to identify and mitigate biases in the data or algorithms used by AI models.”

In the paper

“For instance, [55] highlighted the significance of this issue through their study, which revealed racial bias in a commonly used algorithm in the U.S. health-care system.”

What the sources say

• Not checked yetcites [55] source

In the paper

“The inability to explain how decisions are made hinders the acceptance and adoption of AI in critical healthcare scenarios [4].”

What the sources say

• Not checked yetcites [4] Karim, 2018, Machine learning interpretability: A science rather than a tool, arXiv:1807.06722

In the paper

“However, the presence of bias regarding AI presents substantial challenges to achieving fairness in healthcare.”

In the paper

“Many AI algorithms, such as DNNs, operate as black-box models, making it challenging to understand the reasoning behind their decisions.”

In the paper

“However, their black-box nature presents challenges in understanding their decision-making process.”

In the paper

“While this may be acceptable in certain fields such as business decisions or human behavioral studies, it becomes problematic in clinical management, where decisions often involve critical, life-or-death situations [40].”

What the sources say

• Not checked yetcites [40] Ghassemi, 2020, A review of challenges and opportunities in machine learning for health, AMIA Summits Transl. Sci. Proc.

In the paper

“This lack of interpretability can limit our ability to understand why certain decisions are made, leading to potential biases, discrimination, or errors that go unnoticed [25]”

What the sources say

• Not checked yetcites [25] source

In the paper

“This lack of interpretability can be a significant barrier to the adoption of AI in healthcare.”

In the paper

“The algorithm relied on healthcare spending to determine patients requiring extra medical attention.”

What the sources say

• Not checked yetcites [55] source

In the paper

“Prediction models for cardiovascular disease that claim to predict heart attacks several years in advance are often trained on datasets predominantly comprising male subjects [58].”

What the sources say

• Not checked yetcites [58] source

In the paper

“However, if the dataset used to train AI algorithms is not diverse and representative, it can introduce statistical bias and compromise the accuracy and generalizability of the algorithms.”

In the paper

“For example, in dermatology, CNNs used to classify skin lesions are often trained on datasets predominantly composed of samples from patients with fair skin tones.”

In the paper

“Similar racial and gender bias can be observed in an online AI image creator tool i.e.Playground AI [56] as exemplified in Figure 4.”

What the sources say

• Not checked yetcites [56] source

In the paper

“A notable example can be seen in cardiology [57], where heart attacks are frequently misdiagnosed in women.”

What the sources say

• Not checked yetcites [57] source

In the paper

“While spending might serve as a reasonable indicator of illness severity and the need for additional care, the authors showed that implementing the algorithm would result in a reduction of over 50% in the number of Black patients identified as needing extra care.”

What the sources say

• Not checked yetcites [55] source

In the paper

“For example, automated sleep scoring algorithms trained on data from young and healthy individuals struggle to accurately diagnose sleep disorders in older patients.”

In the paper

“This limited representation of patients with darker skin tones and its variations leads to lower diagnostic accuracy when these networks are tested with images of patients with darker skin tones.”

In the paper

“This disparity is especially problematic given the higher mortality rate for melanoma in patients with darker skin tones compared to Caucasian patients [62].”

What the sources say

• Not checked yetcites [62] source

In the paper

“Despite the independent process of analysis conducted by ML algorithms, the potential for bias remains present.”

In the paper

“While data bias is a critical concern, it is important to note that bias in clinical datasets is not the sole source of bias in healthcare AI.”

In the paper

“Researchers and clinicians can inadvertently introduce unconscious judgments and biases into their research, which may manifest in biased AI algorithms.”

In the paper

“In healthcare, algorithms used for medical diagnosis or treatment recommendations can exhibit algorithmic bias, leading to biased outcomes.”

In the paper

“For instance, in a study conducted by [55] and colleagues, racial bias was identified in the initial data, leading to the erroneous conclusion that Black patients are more medically intricate and costly to the healthcare system compared to White patients.”

What the sources say

• Not checked yetcites [55] source

In the paper

“Similarly, Amazon's attempt to create an AI-driven tool for recruitment faced problems as the algorithm exhibited a negative bias against women due to the predominantly maleoriented data used for training purposes [70].”

What the sources say

• Not checked yetcites [70] source

In the paper

“This means that the results obtained from these early studies could influence the datasets used to train AI algorithms [64], [65].”

What the sources say

• Not checked yetcites [64] source
• Not checked yetcites [65] source

In the paper

“This oversight occurred because the model failed to consider issues of access.”

What the sources say

• Not checked yetcites [55] source

In the paper

“These biases during the pre-clinical stages can impact how women respond to newly developed drugs.”

In the paper

“For example, a model trained on data from one hospital may not generalize effectively to data from a different hospital due to variations in patient demographics, equipment, and clinical practices [82].”

What the sources say

• Not checked yetcites [82] Fang, Rethinking importance weighting for deep learning under distribution shift

In the paper

“For instance, if a healthcare provider relies solely on patient ratings and reviews to select a specialist, they may unknowingly favor providers who have received more positive feedback, potentially overlooking highly competent specialists who have fewer online ratings [88], [89].”

What the sources say

• Not checked yetcites [88] source
• Not checked yetcites [89] source

In the paper

“For example, if an online health forum primarily attracts individuals with a particular condition who are actively seeking information and support, the data collected from that platform may not reflect the experiences and perspectives of individuals who do not actively engage in online discussions [87].”

What the sources say

• Not checked yetcites [87] source

In the paper

“For example, if a particular medication is primarily tested on a specific population (e.g., predominantly White males), there may be limited evidence on how it affects other populations, leading to biased healthcare decisions for those groups [71].”

What the sources say

• Not checked yetcites [71] source

In the paper

“AI models trained on specific datasets may struggle to generalize their knowledge to new, unseen data in healthcare settings [81].”

What the sources say

• Not checked yetcites [81] Yang, Rethinking bias-variance trade-off for generalization of neural networks

In the paper

“For instance, if a study on diabetes management predominantly includes data from urban areas, the findings may not accurately represent the experiences and needs of rural populations [86].”

What the sources say

• Not checked yetcites [86] source

In the paper

“In healthcare, historical bias can be seen in the underrepresentation of certain demographic groups in clinical trials or medical research.”

What the sources say

• Not checked yetcites [71] source

In the paper

“For example, if an algorithm is trained on biased or incomplete medical data, it may disproportionately impact certain demographic groups, resulting in disparities in diagnosis or treatment recommendations [76].”

What the sources say

• Not checked yetcites [76] source

In the paper

“In the domain of healthcare, disparities in accessing advanced AI diagnostics can impact patient outcomes [104]- [106].”

What the sources say

• Not checked yetcites [104]- [106]

In the paper

“It is still challenging to encode the intricacies of context in AI algorithms [96].”

What the sources say

• Not checked yetcites [96] source

In the paper

“Biases ingrained in historical data can lead to prejudice in algorithms, perpetuating the existing inequalities in areas like hiring and loan approvals [102].”

What the sources say

• Not checked yetcites [102] source

In the paper

“The digital divide further exacerbates the issue, creating discrepancies in accessing AI services and information, particularly for disadvantaged populations [103].”

What the sources say

• Not checked yetcites [103] source

In the paper

“AI models may not be able to adapt quickly enough to these dynamic environments, leading to sub-optimal performance and potential errors [98].”

What the sources say

• Not checked yetcites [98] source

In the paper

“While AI has shown remarkable advancements in various domains, achieving human-level contextual understanding remains elusive [95].”

What the sources say

• Not checked yetcites [95] source

In the paper

“For instance, if a study investigates the effectiveness of a particular treatment based on data collected from a specific time period, the findings may not reflect the current healthcare landscape or advancements in medical practices [91].”

What the sources say

• Not checked yetcites [91] source

In the paper

“For example, if a mobile health app is primarily used by individuals who are already health-conscious and actively managing their wellbeing, the data collected may not accurately represent the broader population's health behaviors and needs [86], [90].”

What the sources say

• Not checked yetcites [86] source
• Not checked yetcites [90] source

In the paper

“In 2018, an AI-powered facial recognition system was used by the Metropolitan Police in London to identify suspects, 104 previously unknown people who were suspected of committing crimes, out of which only 2 were accurately predicted [99].”

What the sources say

• Not checked yetcites [99] Santow, 2020, Can artificial intelligence be trusted with our human rights?, AQ, Austral. Quart.

In the paper

“For example, if electronic health records (EHR) predominantly use medical terminology and abbreviations, the language used in those records may not accurately capture patients' subjective experiences or provide a comprehensive understanding of their health conditions [92].”

What the sources say

• Not checked yetcites [92] source

In the paper

“It plays a crucial role in human interactions, decision-making, and overall health [127], [128].”

What the sources say

• Not checked yetcites [127] source
• Not checked yetcites [128] Beck, 2017, The rise of AI makes emotional intelligence more important, Harvard Bus. Rev.

In the paper

“In 2018, Amazon's facial recognition system, Rekognition, came under scrutiny for exhibiting racial and gender biases.It significantly had higher error rates for identifying darker-skinned individuals and females.”

What the sources say

• Not checked yetcites [113] source
• Not checked yetcites [114] West, 2019, Discriminating systems, AI Now

In the paper

“These robots can sense emotions in residents through facial expressions and gestures [130].”

What the sources say

• Not checked yetcites [130] source

In the paper

“AI-driven healthcare platforms can utilize facial recognition and voice analysis to detect emotions in patients during interactions [129].”

What the sources say

• Not checked yetcites [129] source

In the paper

“Consider radiology, where AI assists radiologists in analyzing vast volumes of medical images where algorithms can quickly identify anomalies in x-rays or MRIs, flagging potential issues and requiring closer inspection [116].”

What the sources say

• Not checked yetcites [116] source

In the paper

“While it aims to provide evidence-based recommendations, reports have highlighted instances where Watson for Oncology suggested treatments that contradicted medical guidelines [141].”

What the sources say

• Not checked yetcites [141] source

In the paper

“Hospitals like Mount Sinai in New York have implemented AI systems that analyze patient data to predict potential deterioration hours before it occurs, allowing medical teams to intervene promptly [142].”

What the sources say

• Not checked yetcites [142] Shukla, 2022, Pandemic analytics: how countries are leveraging big data analytics and artificial intelligence to fight COVID-19?, Social Netw. Comput. Sci.

In the paper

“Overreliance on AI for emotional support could have negative consequences, especially for individuals with severe mental health conditions.”

In the paper

“Thus, the lack of emotional intelligence in AI systems is another obstacle to achieving effective human-AI interaction.”

In the paper

“Google's Duplex, an AI system capable of making phone calls and scheduling appointments on behalf of users, faced criticism for not always disclosing its AI identity during interactions [135], [136], highlighting the importance of transparency and ethical considerations in AI development.”

What the sources say

• Not checked yetcites [135] source
• Not checked yetcites [136] source

In the paper

“Moreover, if, AI can mimic certain aspects of emotional intelligence [137], [138], the ethical and societal implications are noteworthy.”

What the sources say

• Not checked yetcites [137] source
• Not checked yetcites [138] Boden, 2016

In the paper

“Furthermore, HIPAA grants patients the right to access and amend their health informa- tion under §164.524.”

What the sources say

• Not checked yetcites HIPAA §164.524

In the paper

“The principle of data minimization, as outlined in GDPR Article 5-1c, comes into play when AI is applied to extensive data analytics.”

What the sources say

• Not checked yetcites GDPR Article 5-1c

In the paper

“It requires that consent forms be written in plain language, clearly explaining the purpose of data usage and disclosing the individuals or entities that may access the data.”

What the sources say

• Not checked yetcites HIPAA §164.508

In the paper

“One of the key ethical considerations within HIPAA is the emphasis on patients' consent for the usage of their protected health information (PHI).This emphasis is outlined in various sections of HIPAA, most notably in Article 164.508, which pertains to the conditions for disclosure of PHI.”

What the sources say

• Not checked yetcites HIPAA Article 164.508

In the paper

“3) Freedom of Consent Criterion: Consent cannot serve as a valid legal basis for personal data processing when a significant imbalance exists between the data subject and the controlling entity.”

What the sources say

• Not checked yetcites [152] 2016

In the paper

“It generally takes precedence over state laws regarding health information privacy.”

What the sources say

• Not checked yetcites §160.203

In the paper

“However, if a state law offers stricter privacy protections, then both the state law and HIPAA must be followed §160.203.”

What the sources say

• Not checked yetcites §160.203

In the paper

“Crucially, GDPR's accountability principle, detailed in Article 5-2, extends its reach to AI applications.”

What the sources say

• Not checked yetcites GDPR Article 5-2

In the paper

“Research use of PHI is allowed under certain conditions, with safeguards such as Institutional Review Board (IRB) approval.”

What the sources say

• Not checked yetcites [157] source

In the paper

“Similarly, Article 20's right to data portability, permitting the transfer of personal data, can affect algorithmic accuracy and confidentiality, potentially influencing the larger dataset.”

What the sources say

• Not checked yetcites [161] source

In the paper

“Article 15 grants individuals the right to access information about their data processing, including details about automated decision-making logic and its consequences.”

What the sources say

• Not checked yetcites Article 15

In the paper

“However, there are exceptions, for instance, the rule does not apply when the information is used for treatment, payment, or healthcare operations, among other scenarios.”

What the sources say

• Not checked yetcites §164.502[b-1]

In the paper

“Article 22, crucial in the context of AI, prohibits automated decisions if they produce significant effects.”

What the sources say

• Not checked yetcites [161] source

In the paper

“Article 17's right to erasure, allowing data removal, can impact an algorithmic model's credibility [160].”

What the sources say

• Not checked yetcites [160] source

In the paper

“The law also entails that patients must be given a Notice of Privacy Practices (NPP), which outlines how their information can be used, especially for treatment, payment, and health care operations §164.520[b-1][ii-A].”

What the sources say

• Not checked yetcites §164.520[b-1][ii-A]

In the paper

“Article 22-2 provides exceptions, allowing automated decisions if they are contractually necessary, legally authorized with safeguards, or based on explicit consent.”

What the sources say

• Not checked yetcites [161] source

In the paper

“For instance, sexual orientation can be inferred from seemingly unrelated data like activity or likes.”

What the sources say

• Not checked yetcites [161] source

In the paper

“HIPAA goes further to affirm patients' rights to control their health information through §164.524[b], which grants individuals the right to request amendments to their health records.”

What the sources say

• Not checked yetcites §164.524[b]

In the paper

“Under HIPAA §164.524[a],patients are granted the authority to access their own health records held by covered entities.”

What the sources say

• Not checked yetcites §164.524[a]

In the paper

“§164.524[c]outlines the right to receive an accounting of disclosures, which provides patients with information about how their health records have been shared.”

What the sources say

• Not checked yetcites §164.524[c]

In the paper

“Similarly, non-sensitive data can act as proxies for sensitive data, leading to possible unlawful discrimination [161].”

What the sources say

• Not checked yetcites [161] source