Foundational Considerations for Artificial Intelligence Using Ophthalmic Images

The 2018 Abramoff et al trial was developed under a strict ethical framework.

“developed under a strict ethical framework.16,17”

A retina specialist review of the AI images (a level 4 reference standard) demonstrated an estimated sensitivity of 100% and specificity of 78.9%.

“A retina specialist review of the AI images (a level 4 reference standard) demonstrated an estimated sensitivity of 100% and specificity of 78.9%33.”

The IDx-DR system diagnoses referable DED, defined as ETDRS level 35 or higher, clinically significant macular edema, and/or center-involved macular edema, which requires further management or treatment by an ophthalmologist or retina specialist.

“The system diagnoses specific levels of diabetic retinopathy and diabetic macular edema (Early Treatment of Diabetic Retinopathy Study level 35 and higher, clinically significant macular edema, and or center-involved macular edema)32,33, referred to as “referable DED”34, that requires further management or treatment by an ophthalmologist or retina specialist.”

LumineticsCore diagnoses according to the ETDRS and Diabetic Retinopathy Clinical Research network grading system to detect ETDRS level 35 or higher or central-involved diabetic macular edema or clinically significant diabetic macular edema.

“LumineticsCore diagnoses according to the ETDRS and Diabetic Retinopathy Clinical Research network grading system to detect ETDRS level 35 or higher (mild or moderate DR depending on the classification system) or central-involved diabetic macular edema or clinically significant diabetic macular edema.12”

Programmatic approaches to AI model surveillance in clinical environments is an expanding priority for healthcare providers.

“Accordingly, programmatic approaches to AI model surveillance in clinical environments is an expanding priority for healthcare providers81.”

The Foundational Considerations on Algorithmic Interpretation (FPOAI) workgroup published their "Foundational Considerations" on AI as a start to developing metrics for ethics, including metrics for "Equity".

“More recently, the Foundational Considerations on Algorithmic Interpretation (FPOAI) workgroup of the Collaborative Community on Ophthalmic Imaging published their “Foundational Considerations”32 on AI as a start to developing metrics for ethics, including metrics for “Equity”33, in order to be able to evaluate how specific AI systems adhere to various bioethical principles.”

Abramoff et al. subsequently linked this model to specific metrics for the conception, design, development, training, validation and implementation phases of AI technologies in healthcare.

“Abramoff et al. subsequently linked this model to specific metrics for the conception, design, development, training, validation and implementation phases of AI technologies in healthcare32.”

It does not consider what matters most to patients and other stakeholders: whether or not the addition of an AI system to the care process results in a favorable change in care - i.e., improved clinical outcome.

“In other words, it does not consider what matters most to patients and other stakeholders: whether or not the addition of an AI system to the care process results in a favorable change in care – i.e., improved clinical outcome43,32.”

This approach incorrectly assumes that the reference standard to compare the AI system to is perfectly correct, and focuses solely on the potential for bias in the AI/ML algorithm.

“However, this approach, which focuses on AI built exclusively from retrospectively collected data, incorrectly assumes that the reference standard (sometimes referred to as ground truth) to compare the AI system to is perfectly correct32, and focuses solely on the potential for bias in the AI/ML algorithm.”

The impact of differential access can be measured through population-achieved sensitivity compared to overall sensitivity.

“The impact of such differential access can be measured for example through population-achieved sensitivity32 compared to overall sensitivity, i.e., the fraction of correctly identified disease cases in a sample, without regard to representativeness of that sample (or lack thereof).”

Abramoff et al. asserts such bias may also be the logical result of 'vernacular medicine' which are regional biases in care that may not expand to broader communities.

“Abramoff et al. asserts such bias may also be the logical result of ‘vernacular medicine’ which are regional biases in care that may not expand to broader communities32.”

These TPLC phases map in a straightforward manner to the 'pipeline phases' as defined in Char et al. and as operationalized in Abramoff et al.

“These TPLC phases map in a straightforward manner to the ‘pipeline phases’ as defined in Char et al.28 and as operationalized in Abramoff et al.32, see also ref. 46.”

Population-achieved sensitivity and specificity, which measures the impact of both access and sensitivity, has been proposed as a way to understand the impact of bias on population health, when including so-called invisible populations.

“As an example, where there is concern about access to a diagnostic test, population-achieved sensitivity and specificity, which measures the impact of both access and sensitivity, has been proposed as a way to understand the impact of bias on population health, when including so-called invisible populations32.”

Depending on which TPLC phase is considered, standard equity metrics can be added, such as for the development phase, key performance indicators in software development, quality systems and risk of population harm analysis, or, for the validation phase of diagnostic medical devices, and absence of racial or ethnic effects on sensitivity and efficacy.

“Depending on which TPLC phase is considered, standard equity metrics can be added, such as for the development phase, key performance indicators in software development, quality systems and risk of population harm analysis, or, for the validation phase of diagnostic medical devices, and absence of racial or ethnic effects on sensitivity and efficacy32.”

Bias in the reference standard for the training dataset may be caused by using inadequate proxies for clinical outcomes as reference standard.

“Finally, bias in the ‘reference standard’, for the training dataset, may be caused by using inadequate proxies for clinical outcomes as reference standard32,52.”

Typically, a Pareto optimum is sought between multiple bioethical principles, such as beneficence, autonomy and equity ("Justics").

“Typically, a Pareto optimum is sought between multiple bioethical principles, such as beneficence, autonomy and equity (“Justics”)32.”

AI validity, explainability and transparency help assess the equity implications of the Software as a Medical Device's (SaMD's) algorithm design.

“AI validity, explainability and transparency all help assess the equity implications of the Software as a Medical Device’s (SaMD)’s algorithm design32.”

Such bias reduces the bioethical principle of "Justice".

“Thus, such bias reduces the bioethical principle of “Justice,” as Char et al.28 and Abramoff et al.32 described.”