CoChat
Sign Up
Cited by 25Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIExplainable Artificial Intelligence (XAI)

Foundation Models in Healthcare: Opportunities, Risks & Strategies Forward

Open access
Thieme, Anja, Nori, Aditya, Ghassemi, Marzyeh, Bommasani, Rishi, Andersen, Tariq Osman, Luger, Ewa·2023·Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3544549.3583177
Open this paper in CoChat — free accountOpen PDF

Abstract

Foundation models (FMs) are a new paradigm in AI. First pretrained on broad data at immense scale and subsequently adapted to more specific tasks, they achieve high performances and unlock powerful new capabilities to be leveraged in many domains, including healthcare. This SIG will bring together researchers and practitioners within the CHI community interested in such emerging technology and healthcare. Drawing attention to the rapid evolution of these models and proposals for their wide-spread adoption, we aim to demonstrate their strengths whilst simultaneously highlighting deficiencies and limitations that give raise to ethical and societal concerns. In particular, we will invite the community to actively debate how the field of HCI – with its research frameworks and methods – can help address some of these existing challenges and mitigate risks to ensure the safe and ethical use of the end-product; a requirement to realize many of the ambitious visions for how these models can positively transform healthcare delivery. This conversation will benefit from a diversity of voices, critical perspectives, and open debate, which are necessary to bring about the right norms and best practices, and to identify a path forward in devising responsible approaches to future FM design and use in healthcare.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544549.3583177

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Human-centered AI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users and other impacted stakeholders.

    “The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [110]

    Section 2.3 criticizes developing AI without understanding intended users’ needs and calls for attention to people who may benefit or be harmed. It does not specifically call for actively involving those people in AI design or say that such involvement reflects their values and preferences. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 18 · Cited by 25

Research works printed but not linked: 28

  1. Hammaad Adam, Aparna Balagopalan, Emily Alsentzer, Fotini Christia, and Marzyeh Ghassemi. 2022. Mitigating the impact of biased artificial intelligence in emergency decision-making. Communications Medicine 2, 1 (2022), 1-6.
  2. Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, et al. 2022. Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing. arXiv preprint arXiv:2204.09817 (2022). · arXiv
  3. Rishi Bommasani, Kathleen Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang. 2022. Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.). https://openreview.net/forum?id=-H6kKm4DVo
  4. Rishi Bommasani, Drew A Hudson, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021). · arXiv
  5. Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. 2021. Ethical machine learning in healthcare. Annual review of biomedical data science 4 (2021), 123-144.
  6. Richard J Chen, Ming Y Lu, Tiffany Y Chen, Drew FK Williamson, and Faisal Mahmood. 2021. Synthetic data in machine learning for medicine and healthcare. Nature Biomedical Engineering 5, 6 (2021), 493-497.
  7. Isabel Chien, Nina Deliu, Richard Turner, Adrian Weller, Sofia Villar, and Niki Kilbertus. 2022. Multi-disciplinary fairness considerations in machine learn- ing for clinical trials. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 906-924.
  8. Enrico Coiera. 2019. The last mile: where artificial intelligence meets reality. Journal of medical Internet research 21, 11 (2019), e16323.
  9. Kate Crawford. 2021. The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press, New Haven and London.
  10. Dina Demner-Fushman, Yassine Mrabet, and Asma Ben Abacha. 2020. Consumer health information and question answering: helping consumers find answers to their health-related information needs. Journal of the American Medical Infor- matics Association 27, 2 (2020), 194-201.
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). · arXiv
  12. Christine Fountain and Peter Bearman. 2011. Risk as social context: immigration policy and autism in California 1. In Sociological Forum, Vol. 26. Wiley Online Library, 215-240.
  13. Sneha D Goenka, John E Gorzynski, Kishwar Shafin, Dianna G Fisk, Trevor Pesout, Tanner D Jensen, Jean Monlong, Pi-Chuan Chang, Gunjan Baid, Jonathan A Bernstein, et al. 2022. Accelerated identification of disease-causing variants with ultra-rapid nanopore genome sequencing. Nature Biotechnology 40, 7 (2022), 1035-1041.
  14. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, et al. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH) 3, 1 (2021), 1-23.
  15. Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International conference on machine learning. PMLR, 1885-1894.
  16. Kundan Krishna, Sopan Khosla, Jeffrey P Bigham, and Zachary C Lipton. 2020. Generating SOAP notes from doctor-patient conversations using modular sum- marization techniques. arXiv preprint arXiv:2005.01795 (2020). · arXiv
  17. Tim Miller, Piers Howe, and Liz Sonenberg. 2017. Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547 (2017). · arXiv
  18. Emmanuel Mutabazi, Jianjun Ni, Guangyi Tang, and Weidong Cao. 2021. A review on medical textual question answering systems based on deep learning approaches. Applied Sciences 11, 12 (2021), 5456.
  19. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447-453.
  20. Tariq Osman Andersen, Francisco Nunes, Lauren Wilcox, Elizabeth Kaziunas, Stina Matthiesen, and Farah Magrabi. 2021. Realizing AI in healthcare: challenges appearing in the wild. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1-5.
  21. Sam Preston, Mu Wei, Rajesh Rao, Robert Tinn, Naoto Usuyama, Michael Lucas, Roshanthi Weerasinghe, Soohee Lee, Brian Piening, Paul Tittel, et al. 2022. To- wards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision. arXiv preprint arXiv:2203.10442 (2022). · arXiv
  22. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
  23. Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. 2021. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ digital medicine 4, 1 (2021), 1-13.
  24. Anja Thieme, Maryann Hanratty, Maria Lyons, Jorge E Palacios, Rita Marques, et al. 2022. Designing Human-Centered AI for Mental Health: Developing Clin- ically Relevant Applications for Online CBT Treatment. ACM Transactions on Computer-Human Interaction (2022).
  25. Walter F Wiggins and Ali S Tejani. 2022. On the Opportunities and Risks of Foundation Models for Natural Language Processing in Radiology. Radiology: Artificial Intelligence 4, 4 (2022), e220119.
  26. Malwina Anna Wójcik. 2022. Foundation Models in Healthcare: Opportunities, Bi- ases and Regulatory Prospects in Europe. In International Conference on Electronic Government and the Information Systems Perspective. Springer, 32-46.
  27. Kun-Hsing Yu, Andrew L Beam, and Isaac S Kohane. 2018. Artificial intelligence in healthcare. Nature biomedical engineering 2, 10 (2018), 719-731.
  28. Hubert D Zajac, Dana Li, Xiang Dai, Jonathan F Carlsen, Finn Kensing, and Tariq O Andersen. 2023. Clinician-facing AI in the Wild: Taking Stock of the Sociotechnical Challenges and Opportunities for HCI. ACM Transactions on Computer-Human Interaction (2023).

Authors
Thieme, A., Nori, A., Ghassemi, M., Bommasani, R., Andersen, T. O., & Luger, E.
Affiliations
  • Microsoft (United Kingdom) (GB)
  • Massachusetts Institute of Technology (US)
  • Stanford University (US)
  • University of Copenhagen (DK)
  • University of Edinburgh (GB)
ORCID
Venue
Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-4
DOI
10.1145/3544549.3583177
OpenAlex
W4366598511
Cited by
25
References
18
Citations by year
2026: 5 · 2025: 10 · 2024: 8 · 2023: 1 · 2021: 1
FWCI
4.34
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
foundation models, artificial intelligence in healthcare, large language models, AI ethics, societal risks of AI, responsible AI, human-computer interaction, AI adoption in healthcare, health care delivery

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review