Hurtful words: quantifying biases in clinical contextual word embeddings
The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that NLP-based techniques have shown biases against under-represented subpopulations in both medical 32 and non-medical 33 domains, the automatic labeler could potentially be a large source of bias.”
From Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations (2021) · cites this paper as [32]
In the citing paper
“Following best practice 16,32 for FPR, FNR and FDR estimation, we select a single threshold for all groups, which maximizes the F1 score.”
From Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations (2021) · cites this paper as [32]
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Research works printed but not linked: 29
- 2020, Artificial Intelligence in Medicine. https://www.ibm.com/watson-health/learn/artificial-intelligence-medicine
- 2020, Drug development gets big data analytics boost. https://www.novartis.com/stories/discovery/drug-development-gets-big-data-analytics-boost
- Alsentzer Emily, 2019, McDermott
- Arnold Emily A, 2014, Triply cursed: racism, homophobia and HIV-related stigma are barriers to regular HIV testing, treatment adherence and disclosure among young Black gay men. Culture, health & sexuality 16, 6
- Basta Christine, 2019, Evaluating the underlying gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.08783 · arXiv
- Beutel Alex, 2017, Chi
- Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems. 4349--4357. Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems . 4349--4357.
- Buolamwini Joy, 2018, Conference on fairness, accountability and transparency. 77--91
- Yiming Cui Wanxiang Che Ting Liu Bing Qin Ziqing Yang Shijin Wang and Guoping Hu. 2019. Pre-Training with Whole Word Masking for Chinese BERT. (2019). [arxiv]1906.08101 http://arxiv.org/abs/1906.08101 Yiming Cui Wanxiang Che Ting Liu Bing Qin Ziqing Yang Shijin Wang and Guoping Hu. 2019. Pre-Training with Whole Word Masking for Chinese BERT. (2019). [arxiv]1906.08101 http://arxiv.org/abs/1906.08101 · arXiv
- DuBard C Annette, 2006, Effect of language on heart attack and stroke awareness among US Hispanics. American journal of preventive medicine 30, 3
- Edwards Harrison, 2015, Censoring representations with an adversary. arXiv preprint arXiv:1511.05897 · arXiv
- Elazar Yanai, 2018, Adversarial removal of demographic attributes from text data. arXiv preprint arXiv:1808.06640 · arXiv
- Glance Laurent G, 2013, Trends in racial disparities for injured patients admitted to trauma centers. Health services research 48, 5
- Moritz Hardt Eric Price Nati Srebro etal 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323. Moritz Hardt Eric Price Nati Srebro et al. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems . 3315--3323.
- Hashimoto Tatsunori, 2018, Fairness Without Demographics in Repeated Loss Minimization. In International Conference on Machine Learning. 1934--1943
- Johnson Alistair EW, 2016, Leo Anthony Celi, and Roger G Mark
- Kurita Keita, 2019, Measuring Bias in Contextualized Word Representations. arXiv preprint arXiv:1906.07337 · arXiv
- LeCun Yann, 2015, Deep learning. nature 521, 7553
- Lee Jinhyuk, 2019, Chan Ho So, and Jaewoo Kang
- Liu Yinhan, 2019, RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692 · arXiv
- Milner Kerry A, 2004, Gender and age differences in chief complaints of acute myocardial infarction (Worcester Heart Attack Study). The American journal of cardiology 93, 5
- Nelson Gregory S, 2019, Bias in artificial intelligence. North Carolina medical journal 80, 4
- Radford Alec, 2019, Language models are unsupervised multitask learners. OpenAI Blog 1, 8
- Suresh Harini, 2017, Machine Learning for Healthcare Conference. 322--337
- Topol Eric J, 2019, High-performance medicine: the convergence of human and artificial intelligence. Nature medicine 25, 1
- Vayena Effy, 2018, Machine learning in medicine: Addressing ethical challenges. PLoS medicine 15, 11
- Wang Tianlu, 2018, Adversarial Removal of Gender from Deep Image Representations. arXiv preprint arXiv:1811.08489 · arXiv
- Zemel Rich, 2013, International Conference on Machine Learning. 325--333
- Zhao Jieyu, 2018, Learning gender-neutral word embeddings. arXiv preprint arXiv:1809.01496 · arXiv
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724
Self-citationLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928
“Given that NLP-based techniques have shown biases against under-represented subpopulations in both medical 32 and non-medical 33 domains, the automatic labeler could potentially be a large source of bias.”
BackgroundMethodSelf-citationLaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., A., M. S. A., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2023) · BOA (University of Milano-Bicocca) · Cited by 737
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645
Self-citationBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K., & Succi, M. D. (2023) · Journal of Medical Internet Research · doi:10.2196/48659 · Cited by 412
The role of machine learning in clinical research: transforming the future of evidence generation
Weissler, E. H., Naumann, T., Andersson, T., Ranganath, R., Elemento, O., Luo, Y., Freitag, D. F., Benoit, J., Hughes, M. C., Khan, F., et al. (2021) · Trials · doi:10.1186/s13063-021-05489-x · Cited by 280
Self-citationAMMU: A survey of transformer-based biomedical pretrained language models
Kalyan, K. S., Rajasekharan, A., & Sangeetha, S. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103982 · Cited by 278
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · UCL Discovery (University College London) · Cited by 241
Training data influence analysis and estimation: a survey
Hammoudeh, Z., & Lowd, D. (2024) · Machine Learning · doi:10.1007/s10994-023-06495-7 · Cited by 232
Algorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 230
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00097-9 · Cited by 199
Transformers and large language models in healthcare: A review
Nerella, S., Bandyopadhyay, S., Zhang, J., Contreras, M., Siegel, S., Bumin, A., Silva, B., Sena, J., Shickel, B., Bihorac, A., et al. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102900 · Cited by 197
Toxicity in chatgpt: Analyzing persona-assigned language models
Deshpande, A., Murahari, V., Rajpurohit, T., Kalyan, A., & Narasimhan, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.88 · Cited by 178
Defining and detecting toxicity on social media: context and knowledge are key
Sheth, A., Shalin, V. L., & Kursuncu, U. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.095 · Cited by 178
Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset
Meng, C., Trinh, L., Xu, N., Enouen, J., & Liu, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-11012-2 · Cited by 174
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2024) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 169
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
Marko, J. G. O., Neagu, D. C., & Anand, P. B. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02884-1 · Cited by 154
A survey of recent methods for addressing AI fairness and bias in biomedicine
Yang, Y., Lin, M., Zhao, H., Peng, Y., Huang, F., & Lu, Z. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104646 · Cited by 146
Assessing the utility of ChatGPT throughout the entire clinical workflow
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K. J., & Succi, M. D. (2023) · doi:10.1101/2023.02.21.23285886 · Cited by 134
Empirical data drift detection experiments on real-world medical imaging data
Kore, A., Abbasi Bavil, E., Subasri, V., Abdalla, M., Fine, B., Dolatabadi, E., & Abdalla, M. (2024) · Nature Communications · doi:10.1038/s41467-024-46142-w · Cited by 124
Self-citationAuto-Debias: Debiasing Masked Language Models with Automated Biased Prompts
Guo, Y., Yang, Y., & Abbasi, A. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.72 · Cited by 113
The Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2023) · doi:10.1101/2023.01.30.23285067 · Cited by 106
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions
Ho, J. Q. H., Hartanto, A., Koh, A., & Majeed, N. M. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100145 · Cited by 111
Peer review of GPT-4 technical report and systems card
Gallifant, J., Fiske, A., Levites Strekalova, Y. A., Osorio-Valencia, J. S., Parke, R., Mwavu, R., Martinez, N., Gichoya, J. W., Ghassemi, M., Demner-Fushman, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000417 · Cited by 111
Self-citationEquity in essence: a call for operationalising fairness in machine learning for healthcare
Wawira Gichoya, J., McCoy, L. G., Celi, L. A., & Ghassemi, M. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100289 · Cited by 104
Self-citationImproving the Fairness of Chest X-ray ClassifiersPreprint
Zhang, H., Dullerud, N., Roth, K., Oakden‐Rayner, L., Pfohl, S. R., & Ghassemi, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.12609 · Cited by 101
Self-citationMore than a biomarker: could language be a biosocial marker of psychosis?
Palaniyappan, L. K. (2021) · Schizophrenia · doi:10.1038/s41537-021-00172-1 · Cited by 96
Mitigating the impact of biased artificial intelligence in emergency decision-making
Adam, H., Balagopalan, A., Alsentzer, E., Christia, F., & Ghassemi, M. (2022) · Communications Medicine · doi:10.1038/s43856-022-00214-4 · Cited by 81
Self-citationIdentifying Implicit Social Biases in Vision-Language Models
Hamidieh, K., Zhang, H., Gerych, W., Hartvigsen, T., & Ghassemi, M. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31657 · Cited by 81
Self-citationFramework for bias evaluation in large language models in healthcare settings
Templin, T., Fort, S., Padmanabham, P. S., Seshadri, P., Rimal, R., Oliva, J. B., Lich, K. H., Sylvia, S., & Sinnott-Armstrong, N. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01786-w · Cited by 80
Shen, T., Li, J., Bouadjenek, M. R., Mai, Z., & Sanner, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103139 · Cited by 79
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark
Zhou, J., Deng, J., Mi, F., Li, Y., Wang, Y., Huang, M., Jiang, X., Liu, Q., & Meng, H. M. L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.262 · Cited by 78
A systematic review of ethical considerations of large language models in healthcare and medicine
Fareed, M., Fatima, M., Uddin, J., Ahmed, A., & Sattar, M. A. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1653631 · Cited by 73
A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare
Aljohani, M., Hou, J. L., Kommu, S., & Wang, X. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.356 · Cited by 69
Fairness and Bias in Multimodal AI: A SurveyPreprint
Adewumi, T., Alkhaled, L., Gurung, N., van Boven, G., & Pagliai, I. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.19097 · Cited by 68
Sociodemographic Bias in Language Models: A Survey and Forward Path
Gupta, V. K., Venkit, P. N., Wilson, S., & Passonneau, R. J. (2024) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2024.gebnlp-1.19 · Cited by 67
Artificial intelligence bias in medical system designs: a systematic review
Kumar, A., Aelgani, V., Vohra, R., Gupta, S. K., Bhagawati, M., Paul, S., Saba, L., Suri, N., Khanna, N. N., Laird, J. R., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16029-x · Cited by 65
McCoy, L. G., Ng, F. Y. C., Sauer, C. M., Legaspi, K. E. Y., Jain, B., Gallifant, J., McClurkin, M., Hammond, A., Goode, D., Gichoya, J. W., et al. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06048-z · Cited by 63
Sociolinguistically Driven Approaches for Just Natural Language Processing
Blodgett, S. L. (2021) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/20410631 · Cited by 62
Artificial intelligence in the neonatal intensive care unit: the time is now
Beam, K., Sharma, P., Levy, P. T., & Beam, A. L. (2023) · Journal of Perinatology · doi:10.1038/s41372-023-01719-z · Cited by 61
McInerney, D. J., Young, G. S., van de Meent, J.-W., & Wallace, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.568 · Cited by 60
Pipelines for Social Bias Testing of Large Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2022) · doi:10.18653/v1/2022.bigscience-1.6 · Cited by 60
UNQOVERing Stereotyping Biases via Underspecified Questions
Li, T., Khashabi, D., Khot, T., Sabharwal, A., & Srikumar, V. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.311 · Cited by 57
The MI-CLAIM-GEN checklist for generative artificial intelligence in health
Miao, B. Y., Chen, I. Y., Williams, C. Y. K., Davidson, J., García-Agúndez, A., Sun, S., Zack, T. I., Saria, S., Arnaout, R., Quer, G., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03470-0 · Cited by 54
Theories of ”gender” in NLP bias research
Devinney, H., Björklund, J., & Björklund, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534627 · Cited by 49
Large Language Models for Psychological Assessment: A Comprehensive Overview
Brickman, J., Gupta, M., & Oltmanns, J. R. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251343582 · Cited by 49
The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective
Gui, G., & Toubia, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4650172 · Cited by 46
Counterfactual and Factual Reasoning over Hypergraphs for Interpretable Clinical Predictions on EHR.
Xu, R., Yu, Y. S., Zhang, C., Ali, M. K., Ho, J. C., & Yang, C. (2022) · PubMed · Cited by 45
RuleBERT: Teaching Soft Rules to Pre-Trained Language Models
Saeed, M., Ahmadi, N., Nakov, P., & Papotti, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.110 · Cited by 44
Corti, L., Oltmans, R., Jung, J., Balayn, A., Wijsenbeek, M. S., & Yang, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642551 · Cited by 43
Arif, M. (2021) · Journal of Information Security and Cybercrimes Research · doi:10.26735/gbtv9013 · Cited by 43
Steed, R., Panda, S., Kobren, A., & Wick, M. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.247 · Cited by 42
Intelligent systems in healthcare: A systematic survey of explainable user interfaces
Cálem, J., Moreira, C., & Jorge, J. A. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108908 · Cited by 41
CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes
Mullenbach, J., Pruksachatkun, Y., Adler, S., Seale, J., Swartz, J. L., McKelvey, T. G., Dai, H., Yang, Y., & Sontag, D. A. (2021) · doi:10.18653/v1/2021.acl-long.109 · Cited by 40
Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain ManagementPreprint
Logé, C., Ross, E., Dadey, D. Y. A., Jain, S., Saporta, A., Ng, A. Y., & Rajpurkar, P. (2021) · arXiv (Cornell University) · doi:10.13026/2tdv-hj07 · Cited by 39
MLHOps: Machine Learning Health Operations
Khattak, F. K., Subasri, V., Krishnan, A., Pou-Prom, C., Koçak, S. A., Dolatabadi, E., Pandya, D., Seyyed-Kalantari, L., & Rudzicz, F. (2024) · IEEE Access · doi:10.1109/access.2024.3521279 · Cited by 39
Word embeddings for biomedical natural language processing: A survey
Chiu, B., & Baker, S. (2020) · Language and Linguistics Compass · doi:10.1111/lnc3.12402 · Cited by 38
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification
Xiao, Y., Lim, S., Pollard, T., & Ghassemi, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593982 · Cited by 38
Self-citationAuditing GPT's Content Moderation Guardrails: Can ChatGPT Write Your Favorite TV Show?
Mahomed, Y., Crawford, C. M., Gautam, S., Friedler, S. A., & Metaxa, D. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658932 · Cited by 38
Machine learning and health need better values
Ghassemi, M., & Mohamed, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00595-9 · Cited by 37
Self-citationAnalyzing the Impact of Personalization on Fairness in Federated Learning for Healthcare
Wang, T., Zhang, K., Cai, J., Gong, Y., Choo, K. R., & Guo, Y. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00164-7 · Cited by 36
On the Gap between Adoption and Understanding in NLP
Bianchi, F., & Hovy, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.340 · Cited by 35
Ensuring machine learning for healthcare works for all
McCoy, L. G., Banja, J. D., Ghassemi, M., & Celi, L. A. (2020) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100237 · Cited by 35
Self-citationA Systematic Review of ‘Fair’ AI Model Development for Image Classification and Prediction
Correa, R., Shaan, M., Trivedi, H., Patel, B. N., Celi, L. A., Gichoya, J. W., & Banerjee, I. (2022) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-022-00754-z · Cited by 34
Mccradden, M., Odusi, O., Joshi, S., Akrout, I., Ndlovu, K., Glocker, B., Maicas, G., Liu, X., Mazwi, M., Garnett, T., et al. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594096 · Cited by 33
Adam, H., Yang, M. Y., Cato, K., Baldini, I., Senteio, C., Celi, L. A., Zeng, J., Singh, M., & Ghassemi, M. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534203 · Cited by 33
Self-citationBiasTestGPT: Using ChatGPT for Social Bias Testing of Language ModelsPreprint
Kocielnik, R., Prabhumoye, S., Zhang, V., Jiang, R., Alvarez, R. M., & Anandkumar, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.07371 · Cited by 31
Ortloff, A.-M., Martius, F., Meier, M., Sans-Raimbault, T., Geierhaas, L., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713671 · Cited by 30
What Is Fair? Defining Fairness in Machine Learning for Health
Gao, J., Chou, B., McCaw, Z. R., Thurston, H., Varghese, P., Hong, C., & Gronsbell, J. L. (2025) · Statistics in Medicine · doi:10.1002/sim.70234 · Cited by 30
Exploration and adaptation of large language models for specialized domains
van Aken, B. (2023) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/15781 · Cited by 30
An Introduction to Artificial Intelligence in Developmental and Behavioral Pediatrics
Aylward, B. S., Abbas, H., Taraman, S., Salomon, C., Gal‐Szabo, D. E., Kraft, C. A., Ehwerhemuepha, L., Chang, A. C., & Wall, D. P. (2022) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000001149 · Cited by 29
Havens, L., Terras, M., Bach, B., & Alex, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05911 · Cited by 28
Algorithmic Fairness in Chest X-ray Diagnosis: A Case Study
Zhang, H., Hartvigsen, T., & Ghassemi, M. (2023) · doi:10.21428/2c646de5.816cae0e · Cited by 26
Self-citationUncovering Bias in Personal Informatics
Yfantidou, S., Sermpezis, P., Vakali, A. I., & Baeza-Yates, R. A. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610914 · Cited by 25
Richer Countries and Richer Representations
Zhou, K., Ethayarajh, K., & Jurafsky, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.164 · Cited by 25
Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP
Chen, J., Berlot-Attwell, I., Wang, X., Hossain, S., & Rudzicz, F. (2020) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2020.clinicalnlp-1.33 · Cited by 25
Natural Language Processing for Radiation Oncology: Personalizing Treatment Pathways
Lin, H., Ni, L., Phuong, C., & Hong, J. C. (2024) · Pharmacogenomics and Personalized Medicine · doi:10.2147/pgpm.s396971 · Cited by 25
Minot, J. R., Cheney, N., Maier, M., Elbers, D. C., Danforth, C. M., & Dodds, P. S. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3524887 · Cited by 23
On the Impact of Random Seeds on the Fairness of Clinical Classifiers
Amir, S., van de Meent, J.-W., & Wallace, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.299 · Cited by 23
Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and BenchmarksPreprint
Zhou, J., Deng, J., Mi, F., Li, Y., Wang, Y., Huang, M., Jiang, X., Liu, Q., & Meng, H. M. L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.08011 · Cited by 23
Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation
Soğancıoğlu, G., Mosteiro, P. J., Salah, A. A., Scheepers, F. E., & Kaya, H. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31732 · Cited by 23
MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset
Meng, C., Trinh, L., Xu, N., & Liu, Y. (2021) · doi:10.21203/rs.3.rs-402058/v1 · Cited by 21
Anoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 21
A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and BiasPreprint
Smith, B. D., Bouadjenek, M. R., Kheya, T. A., Dawson, P., & Aryal, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.09056 · Cited by 21
Temporal-Spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit
Nie, W., Yu, Y., Zhang, C., Song, D., Zhao, L., & Bai, Y. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3309956 · Cited by 20
Data Quality, Bias, and Strategic Challenges in Reinforcement Learning for Healthcare: A Survey
Rahman, A. U., Saqia, B., Alsenani, Y. S., & Ullah, I. (2024) · International Journal of Data Informatics and Intelligent Computing · doi:10.59461/ijdiic.v3i3.128 · Cited by 20
Text-to-Text Extraction and Verbalization of Biomedical Event Graphs
International Committee for Computational Linguistics 2022, Balzani, L., Frisoni, G., & Moro, G. (2022) · Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) · doi:10.48448/nhyv-sw15 · Cited by 20
Walker, A. L., Thorne, A., Das, S., Love, J. S., Cooper, H. L. F., Livingston, M. D., & Sarker, A. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae310 · Cited by 19
Measuring Gender Bias in Contextualized Embeddings
Katsarou, S., Rodríguez-Gálvez, B., & Shanahan, J. (2022) · doi:10.3390/cmsf2022003003 · Cited by 19
MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV DatasetPreprint
Meng, C., Trinh, L., Xu, N., & Liu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06761 · Cited by 18
Liu, C., Zheng, J., Liu, Y., Wang, X., Zhang, Y., Yu, W., Yu, W., Yu, T., Jiang, W., Wang, D., et al. (2025) · International Journal for Equity in Health · doi:10.1186/s12939-025-02581-5 · Cited by 18
Metadata
- Authors
- Zhang, H., Lu, A. X., Abdalla, M., McDermott, M., & Ghassemi, M.
- Affiliations
- University of Toronto (CA)
- Massachusetts Institute of Technology (US)
- Venue
- Proceedings of the ACM Conference on Health, Inference, and Learning
- Publisher
- ACM
- Published
- 2020-04-02
- Pages
- 110-120
- OpenAlex
- W3011762034
- Cited by
- 122
- References
- 38
- Citations by year
- 2026: 8 · 2025: 19 · 2024: 24 · 2023: 22 · 2022: 17 · 2021: 25 · 2020: 7
- FWCI
- 6.68
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Health, Inference, and Learning (CHIL) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, contextual word embeddings, BERT, clinical notes, MIMIC-III, marginalized communities, fairness, clinical prediction, performance gap, gender, language, ethnicity, insurance status, adversarial debiasing, clinical language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.