CoChat

Considerations in the reliability and fairness audits of predictive models for advance care planning

Open access
Lu, Jonathan, Sattler, Amelia, Wang, Samantha, Khaki, Ali Raza, Callahan, Alison, Fleming, Scott, Fong, Rebecca, Ehlert, Benjamin, Li, Ron C., Shieh, Lisa, Ramchandran, Kavitha, Gensheimer, Michael F., Chobot, Sarah, Pfohl, Stephen, Li, Siyun, Shum, Kenny, Parikh, Nitin, Desai, Priya, Seevaratnam, Briththa, Hanson, Melanie, Smith, Margaret, Xu, Yizhe, Gokhale, Arjun, Lin, Steven, Pfeffer, Michael A., Teuteberg, Winifred, Shah, Nigam H. · 2022 · Frontiers in Digital Health
Cited by 40

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Abstract

Multiple reporting guidelines for artificial intelligence (AI) models in healthcare recommend that models be audited for reliability and fairness. However, there is a gap of operational guidance for performing reliability and fairness audits in practice. Following guideline recommendations, we conducted a reliability audit of two models based on model performance and calibration as well as a fairness audit based on summary statistics, subgroup performance and subgroup calibration. We assessed the Epic End-of-Life (EOL) Index model and an internally developed Stanford Hospital Medicine (HM) Advance Care Planning (ACP) model in 3 practice settings: Primary Care, Inpatient Oncology and Hospital Medicine, using clinicians' answers to the surprise question (“Would you be surprised if [patient X] passed away in [Y years]?”) as a surrogate outcome. For performance, the models had positive predictive value (PPV) at or above 0.76 in all settings. In Hospital Medicine and Inpatient Oncology, the Stanford HM ACP model had higher sensitivity (0.69, 0.89 respectively) than the EOL model (0.20, 0.27), and better calibration (O/E 1.5, 1.7) than the EOL model (O/E 2.5, 3.0). The Epic EOL model flagged fewer patients (11%, 21% respectively) than the Stanford HM ACP model (38%, 75%). There were no differences in performance and calibration by sex. Both models had lower sensitivity in Hispanic/Latino male patients with Race listed as “Other.” 10 clinicians were surveyed after a presentation summarizing the audit. 10/10 reported that summary statistics, overall performance, and subgroup performance would affect their decision to use the model to guide care; 9/10 said the same for overall and subgroup calibration. The most commonly identified barriers for routinely conducting such reliability and fairness audits were poor demographic data quality and lack of data access. This audit required 115 person-hours across 8–10 months. Our recommendations for performing reliability and fairness audits include verifying data validity, analyzing model performance on intersectional subgroups, and collecting clinician-patient linkages as necessary for label generation by clinicians. Those responsible for AI models should require such audits before model deployment and mediate between model auditors and impacted stakeholders.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fdgth.2022.943768

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 40

Metadata

Authors
Lu, J., Sattler, A., Wang, S., Khaki, A. R., Callahan, A., Fleming, S., Fong, R., Ehlert, B., Li, R. C., Shieh, L., Ramchandran, K., Gensheimer, M. F., Chobot, S., Pfohl, S., Li, S., Shum, K., Parikh, N., Desai, P., Seevaratnam, B., Hanson, M., Smith, M., Xu, Y., Gokhale, A., Lin, S., Pfeffer, M. A., Teuteberg, W., & Shah, N. H.
Venue
Frontiers in Digital Health
Published
2022-09-12
Volume
4
Pages
943768-943768
ISSN
2673-253X (ISSN-L)
OpenAlex
W4295329163
Cited by
40
References
26
FWCI
5.60
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.