Identifying Ethical Considerations for Machine Learning Healthcare Applications

Probably not a substantial claim: may not pass S3 (load-bearing).

The paper's literature review identified bias from training data as a cited ethical consideration for ML-HCAs.

“our review of the literature (see appendix for review methods) identified a variety of ethical considerations and concerns that have been cited, such as bias arising from the training data set”

• cites Challen et al. 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The paper's literature review identified accountability for ML-HCA failings as a cited ethical consideration.

“accountability for ML-HCA’s failings”

• cites Ross and Swelitz 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The paper's literature review identified ownership of data used to train ML-HCAs as a cited ethical consideration.

“ownership of the data used to train ML-HCAs”

• cites Ornstein and Thomas 2018Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

A common attribute of problem structuring methods is creating and clarifying a shared conceptual mental map of the problem.

“A common attribute across these methods is creating and clarifying (ideally with a diverse group of stakeholders) a shared conceptual mental map of the problem”

• cites Rosenhead and Mingers 2008Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Wicked problems typically defy a singular formulation, are nested within systems with interrelated problems, and have social values woven into their fabric such that solutions are better or worse rather than true or false.

“typically defy a singular formulation of the problem, are nested within systems that have interrelated problems, and have social values woven into their fabric such that solutions are not simply true or false but rather better or worse”

• cites Rittel and Webber 1973Not checked yet

  Not checked in the free run.

May combine two claims.

Equipped with a shared conceptual map, problem solvers may identify more decisions and their interconnected consequences, which may advance value-focused thinking and improve ethical decision-making.

“Equipped with such a map, problem solvers may identify more decisions and their interconnected consequences, which in turn may advance value-focused thinking (Keeney 1992) and improve ethical decision-making (Stenmark et al. 2011).”

• cites Keeney 1992Not checked yet

  Not checked in the free run.

• cites Stenmark et al. 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Most likely, large health systems will have generated and compiled much of this 'big data', which in turn was paid for by insurance premiums and co-pays.

“Most likely, large health systems will have generated and compiled much of this “big data”, which in turn was paid for by insurance premiums and co-pays.”

May combine two claims.

The aggregation and curation of large datasets raises issues regarding specifying the standards that high-quality reference standard data must achieve, data privacy, and data ownership.

“The aggregation and curation of these large datasets raises not only issues regarding specifying the standards that high-quality reference standard data must achieve, but also issues regarding data privacy and data ownership”

• cites Aboueid et al. 2019Not checked yet

  Not checked in the free run.

• cites Amarasingham R. et al. 2016Not checked yet

  Not checked in the free run.

• cites Cohen et al. 2014Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: Machine learning requires large amounts of training data.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The pipeline framework is useful for systematic ethical appraisals of ML-HCA from development through implementation, and for interdisciplinary collaboration of diverse stakeholders.

“This pipeline model framework will be useful for systematic ethical appraisals of ML-HCA from development through implementation, and for interdisciplinary collaboration of diverse stakeholders”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

The pipeline framework is neither too narrow nor too broad, applies across a wide variety of ML-HCAs, and accommodates the perspectives and concerns of different groups of stakeholders.

“This pipeline framework is neither too narrow nor too broad, applies across a wide variety of ML-HCAs, and accommodates the perspectives and concerns of different groups of stakeholders.”

May combine two claims.

A non-inspectable, autonomous system poses a higher risk of patient harm, raises questions about the responsibility of the system in situations of harm, and could engender significant backlash against autonomous systems.

“A non-inspectable, autonomous system poses a higher risk of patient harm, raises questions about the responsibility of the system in situations of harm (and the need for the system to have malpractice insurance), and could engender significant backlash against autonomous systems.”

May combine two claims.

Evaluating the safety, efficiency, and equity for a continuous learning ML-HCA is more challenging than for a locked ML-HCA, and therefore understanding ethical considerations and addressing concerns is more difficult.

“evaluating the safety, efficiency, and equity for a continuous learning ML-HCA is more challenging, and therefore understanding ethical considerations and addressing concerns is more difficult.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Transparency needs to be balanced against protection of the intellectual property of ML-HCA design.

“Transparency, however, needs to be balanced against protection of the intellectual property of ML-HCA design.”

Classified as probably not a substantial claim.

The developer's choice of a ML-HCA's level of autonomy has clear implications for assumption of responsibility and liability.

“Since the developer’s choice of a ML-HCA’s level of autonomy has clear implications for assumption of responsibility and liability, this autonomy level needs to be apparent.”

May combine two claims.

Any perpetuated biases incorporated into a ML-HCA may subsequently impact clinical decisions and support self-fulfilling prophesies.

“Furthermore, any perpetuated biases incorporated into a ML-HCA may subsequently impact clinical decisions and support self-fulfilling prophesies.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The subsequent effort to curate the data and then develop the ML-HCA adds value to the raw data, but certainly not all of the value.

“The subsequent effort to curate the data and then develop the ML-HCA adds value to the raw data, but certainly not all of the value.”

May combine two claims.

In many cases a goal of ML-HCA is lowering cost, yet for certain conditions obtaining high quality long-term outcome data needed for validation and subsequent updating may require more not less financial resources.

“In addition, in many cases a goal of ML-HCA is lowering cost, yet for certain conditions (such as most chronic diseases, where costs are driven by long-term adverse outcomes), obtaining high quality long-term outcome data needed for validation and subsequent updating may require more not less financial resources.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Stakeholders do not need to understand in detail the inner working of an ML-HCA in order to achieve auditability.

“Stakeholders do not need to understand in detail the inner working of an ML-HCA in order to achieve “auditability.””

Classified as probably not a substantial claim.

Potential conflicts of interest include individual financial interests and operational interests of the organization that may not be aligned with the duty of clinicians and health care delivery organizations to advance the best interest of each patient under their care.

“These potential conflicts of interest include individual financial interests (such as payment for services or personal ownership of stocks) as well as any operational interests of the organization that may not be aligned with the duty of clinicians and health care delivery organizations to advance the best interest of each patient under their care”

• cites Kohli et al. 2017Not checked yet

  Not checked in the free run.

• cites Fischer et al. 2016Not checked yet

  Not checked in the free run.

• cites Jaremko et al. 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Many data sets, particularly those involving image or biopsy interpretations, may also reflect the significant intellectual contributions of interpreting clinicians.

“Many data sets, particularly those involving image or biopsy interpretations, may also reflect the significant intellectual contributions of interpreting clinicians.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Other ML-HCAs may be based on data from non-clinical sources such as personal devices, social media, financial, or legal sources, which may contain potentially controversial data elements or have been collected via novel means that we cannot foresee.

“Other ML-HCAs may be based on data from non-clinical sources (such as personal devices, social media, financial, or legal sources), which may contain potentially controversial data elements or have been collected via novel means that we cannot foresee.”

May combine two claims.

Examples already exist of predictive scores failing both because of poorly composed training data and because, when expanded to broader populations, racially discriminatory outcomes occurred.

“Examples already exist of predictive scores failing both because of poorly composed training data and because, when expanded to broader populations, racially discriminatory outcomes occurred”

• cites Char, Shah, and Magnus 2018Not checked yet

  Not checked in the free run.

• cites Obermeyer et al. 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Transparency on the part of ML-HCA developers allows clinicians, patients, and society as a whole to independently assess potential conflicts of interest and other harms that may have negative consequences outside of the AI developer's direct control.

“Transparency on the part of ML-HCA developers allows clinicians, patients, and society as a whole to independently assess potential conflicts of interest and other harms that may have negative consequences outside of the AI developer’s direct control.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There has been ongoing patient activism for inclusion in recognition for specimen contribution to scientific advances.

“There has also been ongoing patient activism for inclusion in recognition for specimen contribution to scientific advances”

• cites Bledsoe and Grizzle 2013Not checked yet

  Not checked in the free run.

May combine two claims.

Highly focused approaches such as this assume an a priori comprehensive understanding of where and why such biases are occurring; if this assumption is wrong, these approaches risk introducing a complex set of unintended biases in attempts to correct the initial bias.

“Highly focused approaches such as this assume an a priori comprehensive understanding of where and why such biases are occurring; if this assumption is wrong, these approaches risk introducing a complex set of unintended biases in attempts to correct the initial bias”

• cites Goodman et al 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

ML-HCA is likely to raise complex and only partially considered ethical considerations with implementation.

“Along with many potential benefits to healthcare delivery, ML-HCA is likely to raise complex and as yet only partially considered ethical considerations with implementation.”

May combine two claims.

Many medical decisions and diagnoses lack a perfectly correct or incorrect label, and their downstream outcomes cannot always be predicted.

“many medical decisions and diagnoses cannot be perfectly labeled as correct or incorrect and down-stream outcomes cannot always be anticipated (Fenton et al. 2007).”

• cites Fenton et al. 2007Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The pipeline framework is an approach for systematically identifying ethical considerations and facilitating interdisciplinary dialogue and collaboration to manage ethical implications of ML-HCAs.

“The pipeline framework, starting with a map of the conception, development, implementation, and the parallel evaluation and oversight tasks of ML-HCAs, and then layering over this map key questions, value-based issues, and ethical considerations, is an approach for systematically identifying these ethical considerations and for facilitating inter-disciplinary dialogue and collaboration to better”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

For the ethical appraisal of any given ML-HCA, detailed content and context-specific knowledge will always be needed to provide more thorough and precise ethical evaluation, and this will require cross-disciplinary collaborations.

“For the ethical appraisal of any given ML-HCA, detailed content and context-specific knowledge will always be needed to provide more thorough and precise ethical evaluation, and this will require cross-disciplinary collaborations.”

Probably not a substantial claim: may not pass S4 (based).

ML-HCAs are likely to have a broad range of stakeholders, from patients and health care practitioners, to computer scientists, engineers, and entrepreneurial developers, to healthcare organizations and payers, to oversight bodies charged with regulating medical practice.

“ML-HCAs are likely to have a broad range of stakeholders, from patients and health care practitioners, to computer scientists, engineers, and entrepreneurial developers, to healthcare organizations and payers, to oversight bodies charged with regulating medical practice.”

Probably not a substantial claim: may not pass S4 (based).

ML-HCA developers and users inherit ethical operating characteristics from previous decisions about how the ML-HCA was constructed.

“ML-HCA developers and users poised at a particular point in the pipeline inheret the ethical operating characteristics that arise from previous decisions about how the ML-HCA has been constructed.”

Classified as probably not a substantial claim.

The broadness of the framework helps combat narrow focus on one ethical consideration and neglect of others.

“The broadness of the framework will help combat any tendency to focus narrowly on one ethical consideration while potentially neglecting other relevant considerations and thus sidestepping grappling with tradeoffs.”

Classified as probably not a substantial claim. May combine two claims.

The pipeline framework can be used to anticipate future development and implementation and to aid ethical evaluation and decisions.

“Heading in the other direction, the framework can also be used to look ahead, anticipating future development and implementation (or implementations in other settings). Identification of potential future consequences can aid ethical evaluation and decisions regarding design, development, implementation, and evaluation.”

May combine two claims.

When machine-based and human-based errors are equivalent in nature, probability, and harm magnitude, judging the machine error more severely is not legitimate and reflects pro-human or anti-machine bias.

“If, comparing the machine-based and the human-based scenarios, the nature and probability of the error and the magnitude of the ensuing harm are equivalent, this tendency does not appear to legitimate, instead reflecting a pro-human or anti-machine bias.”

May combine two claims.

The common basic elements of the pipeline allow comparison of ML-HCA pipelines to those of other medical technologies and show that ML-HCAs raise both novel and common issues.

“the common basic elements of the pipeline – an application is conceived of, developed, calibrated, implemented, and evaluated, with various forms of oversight – allows for ready comparison of the ML-HCA pipeline to the pipelines of other medical technologies, and to see that while ML-HCAs do raise some novel issues, they also raise many issues common to existing diagnostic or therapeutic”

Classified as probably not a substantial claim.

The pipeline framework provides a guide for individuals and groups to identify and evaluate present, past, and future ethical issues.

“the pipeline framework provides a guide to help these individuals and groups with the task of identifying and evaluating present, past, and future ethical issues.”

Probably not a substantial claim: may not pass S2 (checkable).

This comparison can check unwarranted ML-HCA exceptionalism in ethics thinking.

“This can put a check on unwarranted ML-HCA exceptionalism in our thinking about the ethics of this emerging technology.”

Classified as probably not a substantial claim.

The pipeline framework offers diverse stakeholder groups a bigger picture that can help forge a shared mental model of relevant questions and ethical considerations.

“The pipeline framework also offers groups of diverse stakeholders a “bigger picture” of ML-HCAs that can, with dialogue, help to forge a shared mental model of the range of revelant questions and ethical considerations that should guide design and evaluation decisions.”

May combine two claims.

A highly restrictive focus and limited framework may be applicable for ultimately addressing a specific ethical consideration and set of concerns, but will not suffice to manage the uncertainty regarding other potential ethical considerations.

“More generally, a highly restrictive focus and limited framework may be applicable for ultimately addressing a specific ethical consideration and set of concerns, but will not suffice to manage the uncertainty regarding other potential ethical considerations.”

Classified as probably not a substantial claim.

Accuracy in an ML-HCA inherently involves trade-offs among test characteristics, and these trade-offs are guided by designer value judgments with ethical implications.

“The notion of accuracy in an ML-HCA, inherently involves trade-offs between test characteristics, guided by designer value judgments with consequent ethical implications.”

May combine two claims.

ML-HCAs will create new information flows and require resource allocation.

“ML-HCAs will create new information flows and consequently need resource allocation, including the important resource of clinical attention.”

• AssumptionRests on an assumption: Clinical attention is an important resource that must be allocated.

May combine two claims.

Many ML-HCA computer scientists have already turned away from ethical analysis as unworkable or not adequately responsive to ongoing ML-HCA development, have instead focused exclusively on the ethical consideration of fairness and emerging concerns regarding bias, and have begun to pursue an ideal of 'algorithmic fairness'.

“Many ML-HCA computer scientists have already turned away from ethical analysis as unworkable or not adequately responsive to ongoing ML-HCA development, have instead focused exclusively on the ethical consideration of fairness and emerging concerns regarding bias, and have begun to pursue an ideal of “algorithmic fairness,” or the ability to computationally demonstrate a lack of between-group bias”

• cites Rajkomar et al. 2018Not checked yet

  Not checked in the free run.