Artificial Intelligence in mental health and the biases of language based models
Open accessThe DOI matches its Crossref record; 70 of the 72 references it cites were found in a registry or an index.
Abstract
Background: The rapid integration of Artificial Intelligence (AI) into the healthcare field has occurred with little communication between computer scientists and doctors. The impact of AI on health outcomes and inequalities calls for health professionals and data scientists to make a collaborative effort to ensure historic health disparities are not encoded into the future. We present a study that evaluates bias in existing Natural Language Processing (NLP) models used in psychiatry and discuss how these biases may widen health inequalities. Our approach systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective. Design/Methods: A literature review of the uses of NLP in mental health was carried out across multiple disciplinary databases with defined Mesh terms and keywords. Our primary analysis evaluated biases within ‘GloVe’ and ‘Word2Vec’ word embeddings. Euclidean distances were measured to assess relationships between psychiatric terms and demographic labels, and vector similarity functions were used to solve analogy questions relating to mental health. Results: Our primary analysis of mental health terminology in GloVe and Word2Vec embeddings demonstrated significant biases with respect to religion, race, gender, nationality, sexuality and age. Our literature review returned 52 papers, of which none addressed all the areas of possible bias that we identify in model development. In addition, only one article existed on more than one research database, demonstrating the isolation of research within disciplinary silos and inhibiting cross-disciplinary collaboration or communication. Conclusion: Our findings are relevant to professionals who wish to minimize the health inequalities that may arise as a result of AI and data-driven algorithms. We offer primary research identifying biases within these technologies and provide recommendations for avoiding these harms in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs are prone to biased outputs due to the nature of their training data, which often reflects societal inequalities and fails to uniformly represent diverse demographic groups.21”
From Role of large language models in mental health research: an international survey of researchers' practices and perspectives (2025) · cites this paper as [21]
Experimental — accuracy not yet measuredIn the citing paper
“AI biases in medical research have generated considerable concern, since biases related to race, ethnicity, gender, sexuality, age, nationality, and socioeconomic status in health-related datasets can perpetuate health disparities by supporting biased hypotheses, models, theories, and policies [177, 198, 211].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [211]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Unequal treatment: confronting racial and ethnic disparities in health care
(2003) · Choice Reviews Online · doi:10.5860/choice.40-5843 · Cited by 7,043
Capps, L. (2003) · JAMA · doi:10.1001/jama.290.18.2487-b · Cited by 4,526
Lakoff, R. (1973) · Language in Society · doi:10.1017/s0047404500000051 · Cited by 3,138
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Psychological Aspects of Natural Language Use: Our Words, Our Selves
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145041 · Cited by 2,624 · cited 2×
“In the psychiatric literature there exists extensive research that describes how depression closely relates to increased use of first person singular pronouns, however we are yet to see a tool that can accurately differentiate between word particles that result from mental pathology, and those that originate from indi…”
Background(2016) · Cited by 2,334
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731
Research works printed but not linked: 2
- Bolukbasi T, Chang K-W, Zou J, Saligrama V, Kalai A. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. ArXiv:160706520 [Cs, Stat] 2016.
- Learning Word Vectors for Sentiment Analysis | Proceedings of the 49th Annual Meeting of the Associa- tion for Computational Linguistics: Human Language Technologies-Volume 1. https://dl.acm.org/doi/ abs/10.5555/2002472.2002491. Accessed 2 Mar. 2020. · doi:10.5555/2002472.2002491
Cited by
A study of generative large language model for medical research and healthcare
Peng, C., Yang, X., Chen, A., Smith, K. E., PourNejatian, N., Costa, A. B., Martin, C., Flores, M. G., Zhang, Y., Magoc, T., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00958-w · Cited by 408
“…facilitate documentation of patient reports (e.g., a progress report) 3,4 , improving diagnostic accuracy 5 , and assisting in various clinical care 6,7 , while at the same time concerning the hallucinations and fabrications 7,8 , bias and stereotype 9 , and risks of patient privacy and ethics 10 .”
Timmons, A. C., Duong, J. B., Simo Fiallo, N., Lee, T., Vo, H. P. Q., Ahle, M. W., Comer, J. S., Brewer, L. C., Frazier, S. L., & Chaspari, T. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221134490 · Cited by 263
Artificial intelligence in positive mental health: a narrative review
Thakkar, A., Gupta, A., & De Sousa, A. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1280235 · Cited by 251
The Opportunities and Risks of Large Language Models in Mental Health
Lawrence, H. R., Schneider, R. A., Rubin, S. B., Matarić, M. J., McDuff, D. J., & Jones Bell, M. (2024) · JMIR Mental Health · doi:10.2196/59479 · Cited by 235
“…supervision and evaluation can, therefore, lead to problematic generation of biased content and disparate model performance for different groups [45,55-57] (of note, however, there is some evidence that clinicians perceive less bias in LLM-generated responses [58] relative to clinician-generated…”
BackgroundExploring the Ethical Challenges of Conversational AI in Mental Health Care: Scoping Review
Rahsepar Meadi, M., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., & Batelaan, N. (2025) · JMIR Mental Health · doi:10.2196/60432 · Cited by 211
Resnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228
“…medical research have generated considerable concern, since biases related to race, ethnicity, gender, sexuality, age, nationality, and socioeconomic status in health-related data-sets can perpetuate health disparities by supporting biased hypotheses, models, theories, and policies [177, 198, 211].”
Evaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: Scoping Review
Huang, J., Galal, G., Etemadi, M., & Vaidyanathan, M. (2022) · JMIR Medical Informatics · doi:10.2196/36388 · Cited by 186
The promise of large language models in health care
Arora, A., & Arora, A. (2023) · The Lancet · doi:10.1016/s0140-6736(23)00216-7 · Cited by 153
AI chatbots cannot replace human interactions in the pursuit of more inclusive mental healthcare
Brown, J. E. H., & Halpern, J. (2021) · SSM - Mental Health · doi:10.1016/j.ssmmh.2021.100017 · Cited by 139
Human-Computer Interaction in Digital Mental Health
Balcombe, L., & Leo, D. D. (2022) · Informatics · doi:10.3390/informatics9010014 · Cited by 192
Large language models streamline automated machine learning for clinical studies
Tayebi Arasteh, S., Han, T., Lotfinia, M., Kuhl, C., Kather, J. N., Truhn, D., & Nebelung, S. (2024) · Nature Communications · doi:10.1038/s41467-024-45879-8 · Cited by 114
“…of unbiased scienti fi c or clinical deliberation, and (iii) algorithmic bias secondary to the model ’ s potentially skewed foundational data may perpetuate unbalanced outcomes, for example, by not representing those patients adequately that had been under-represented in the foundational data 44 .”
Straw, I., & Wu, H. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100457 · Cited by 104
Self-citationNi, Y., & Jia, F. (2025) · Healthcare · doi:10.3390/healthcare13101205 · Cited by 194
Algorithmic bias in public health AI: a silent threat to equity in low-resource settings
Joseph, J. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1643180 · Cited by 146
Li, F., Wang, S., Gao, Z., Qing, M., Pan, S. L., Liu, Y., & Hu, C. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1510792 · Cited by 120
“The “black-box” nature of many AI models, particularly deep learning algorithms, creates barriers to clinician trust and adoption (84, 85).”
AI-Driven Natural Language Processing in Healthcare: Transforming Patient-Provider Communication
Sarella, P. N. K., & Mangam, V. T. (2024) · Indian Journal of Pharmacy Practice · doi:10.5530/ijopp.17.1.4 · Cited by 157
Zafar, F., Alam, L. F., Vivas, R. R., Wang, J., Whei, S. J., Mehmood, S., Sadeghzadegan, A., Lakkimsetti, M., & Nazir, Z. (2024) · Cureus · doi:10.7759/cureus.56472 · Cited by 111
The Algorithmic Divide: A Systematic Review on AI-Driven Racial Disparities in Healthcare
Haider, S. A., Borna, S., Gomez-Cabello, C. A., Pressman, S. M., Haider, C. R., & Forte, A. J. (2024) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-024-02237-0 · Cited by 93
“This stipulates that biased input data will inevitably lead to biased output produced by AI [24, 25].”
Commercial Use of Emotion Artificial Intelligence (AI): Implications for Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01330-7 · Cited by 74
Addressing bias in artificial intelligence for public health surveillance
Flores, L., Kim, S., & Young, S. D. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108875 · Cited by 75
“42 This study revealed the existence of significant biases due to human judgement or behaviours with respect to religion, race, gender, nationality, sexuality and age in GloVe and Word2Vec embeddings within a public health setting.”
BackgroundPublic patient views of artificial intelligence in healthcare: A nominal group technique study
Musbahi, O., Syed, L., Feuvre, P. A. L., Cobb, J. P., & Jones, G. G. (2021) · Digital Health · doi:10.1177/20552076211063682 · Cited by 73
“This has been well described in several studies and a literature review of 52 papers using natural language processing (NLP) models in mental health found that no model addressed the possible biases in their development 34 .”
BackgroundLeón, C. E. Y., Lipuma, J. M., & Torres, X. O. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1619888 · Cited by 89
Screening for Depression Using Natural Language Processing: Literature Review
Teferra, B. G., Rueda, A., Pang, H. Y. M., Valenzano, R., Samavi, R., Krishnan, S., & Bhat, V. (2024) · Interactive Journal of Medical Research · doi:10.2196/55067 · Cited by 86
“Further ethical challenges arise from the potential biases that NLP models may inherit from the training data, giving rise to concerns regarding the robustness of depression detection [70-72].”
BackgroundBragazzi, N. L., Crapanzano, A., Converti, M., Zerbetto, R., & Khamisy-Farah, R. (2023) · Journal of Medical Internet Research · doi:10.2196/52091 · Cited by 54
Delanerolle, G., Yang, X., Shetty, S., Raymont, V., Shetty, A., Phiri, P., Hapangama, D. K., Tempest, N., Majumder, K., & Shi, J. (2021) · Women s Health · doi:10.1177/17455065211018111 · Cited by 80
“94 Straw and Callison-Burch 95 performed a literature review on use of NLP for MH and observed that most of the current solutions have significant bias with respect to religion, race, gender, nationality, sexuality and age.”
Applications of Large Language Models in the Field of Suicide Prevention: Scoping Review
Holmes, G., Tang, B., Gupta, S., Venkatesh, S., Christensen, H. M., & Whitton, A. E. (2025) · Journal of Medical Internet Research · doi:10.2196/63126 · Cited by 80
“In a recent review of AI algorithms applied to mental health, Straw Callison-Burch [114] found that significant biases exist with respect to religion, race, gender, nationality, sexuality, and age.”
InfluentialThe Role of Artificial Intelligence in Mental Health: A Review
Shimada, K. (2023) · Science Insights · doi:10.15354/si.23.re820 · Cited by 58
“Moreover, the biases that are inherent in the training data utilized for the development of NLP models can unintentionally sustain prevailing societal inequities and prejudice in the realm of mental health diagnosis and treatment (40).”
BackgroundDiagnostic accuracy of large language models in psychiatry
Gargari, O. K., Fatehi, F., Mohammadi, I., Firouzabadi, S. R., Shafiee, A., & Habibi, G. (2024) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2024.104168 · Cited by 48
Critical review of self‐diagnosis of mental health conditions using artificial intelligence
Wimbarti, S., Kairupan, B. H. R., & Tallei, T. E. (2024) · International Journal of Mental Health Nursing · doi:10.1111/inm.13303 · Cited by 42
Gendered Mental Health Stigma in Masked Language Models
Lin, I., Njoo, L., Field, A., Sharma, A., Reinecke, K., Althoff, T., & Tsvetkov, Y. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.139 · Cited by 33
Allareddy, V., Oubaidin, M., Rampa, S., Venugopalan, S. R., Elnagar, M. H., Yadav, S. K., & Lee, M. K. (2023) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12721 · Cited by 44
Applications of Speech Analysis in Psychiatry
Dikaios, K., Rempel, S., Dumpala, S. H., Oore, S., Kiefte, M., & Uher, R. (2023) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000356 · Cited by 35
“The wide distribution of fields contributing to research into speech analysis for psychiatry presents the problem of divided research efforts.(11) The advent of open-access psychiatric speech datasets provides the opportunity for a wider reach of inquiry but may increase interpretation of results that are detached fro…”
BackgroundMethodInfluentialA Review of Large Language Models in Healthcare: Taxonomy, Threats, Vulnerabilities, and Framework
Hamid, R., & Brohi, S. N. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8110161 · Cited by 32
Daniali, M., Galer, P. D., Lewis‐Smith, D., Parthasarathy, S., Kim, E., Salvucci, D. D., Miller, J. M., Haag, S., & Helbig, I. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102523 · Cited by 27
Large Language Model and Text Generation
Wu, Y. (2024) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-55865-8_10 · Cited by 34
Cotes, R. O., Boazak, M., Griner, E. L., Jiang, Z., Kim, B., Hogg-Bremer, W., Seyedi, S. S., Rad, A. B., & Clifford, G. D. (2022) · JMIR Research Protocols · doi:10.2196/36417 · Cited by 23
Evaluation of African American Language Bias in Natural Language Generation
Deas, N., Grieser, J. A., Kleiner, S., Patton, D. U., Turcan, E., & McKeown, K. R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.421 · Cited by 92
Wright-Berryman, J. L., Cohen, J., Haq, A., Black, D. P., & Pease, J. L. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1143175 · Cited by 26
A review on the efficacy of artificial intelligence for managing anxiety disorders
Das, K. P., & Gavade, P. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1435895 · Cited by 28
On the role of artificial intelligence in psychiatry
Rocheteau, E. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2022.132 · Cited by 25
Cohen, J., Wright-Berryman, J. L., Rohlfs, L., Trocinski, D. R., Daniel, L., & Klatt, T. W. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.818705 · Cited by 29
Noguero, D. S., Ramírez‐Cifuentes, D., Ríssola, E. A., & Freire, A. (2023) · Journal of Medical Internet Research · doi:10.2196/45184 · Cited by 29
“In this respect, Straw and Callison-Burch [27] conducted a literature review of 52 articles that addressed the use of natural language processing in mental health across multiple disciplinary databases and explored each stage of AI model development to analyze which and how biases arise.”
Advancing health equity and the role of digital health technologies: a scoping review
Bitomsky, L., Pfitzer, E., Nißen, M., & Kowatsch, T. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-099306 · Cited by 29
Can generative artificial intelligence pass the orthopaedic board examination?
Isleem, U. N., Zaidat, B., Ren, R., Geng, E. A., Burapachaisri, A., Tang, J. E., Kim, J. S., & Cho, S. K. (2024) · Journal of Orthopaedics · doi:10.1016/j.jor.2023.10.026 · Cited by 19
Nag, P. K., Bhagat, A., Priya, R. V., & Khare, D. K. (2023) · International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI) · doi:10.1109/icaiihi57871.2023.10489117 · Cited by 19
“[14]’s work on biases in NLP models highlights the importance of mitigating biases in digital mental healthcare.”
Artificial Intelligence in Science and Society: The Vision of USERN
Dorigo, T., Brown, G. D., Casonato, C., Cerdà, A., Ciarrochi, J., Lio, M. D., D’Souza, N., Gauger, N. R., Hayes, S. C., Hofmann, S. G., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3529357 · Cited by 27
Moore, R., Al-Tamimi, A.-K., & Freeman, E. (2024) · JMIR Formative Research · doi:10.2196/51571 · Cited by 19
“All of these features could potentially enhance CA quality and improve adherence in the future; however, concerns about bias, misinformation, privacy, and security need to be addressed before using them safely with adolescents [33,34].”
BackgroundInfluentialEquity by design principles for digital health interventions
Bitomsky, L., Nißen, M., & Kowatsch, T. (2025) · International Journal for Equity in Health · doi:10.1186/s12939-025-02645-6 · Cited by 26
“Biases can also arise during the design and development of AI algorithms, leading to unfair outcomes for priority populations [45–47].”
Scoping review on natural language processing applications in counselling and psychotherapy
Laricheva, M., Liu, Y., Shi, E., & Wu, A. D. (2024) · British Journal of Psychology · doi:10.1111/bjop.12721 · Cited by 26
“In mental health, Straw and Callison-Burch (2020) showed that NLP could introduce or aggravate biases related to age, religion, race and nationality.”
Jelassi, M., Matteli, K., Khalfallah, H. B., & Demongeot, J. (2024) · Information · doi:10.3390/info15120813 · Cited by 21
Artificial Intelligence (AI) Generated Health Counseling For Mental Illness Patients
Shankar Ganesh, M., & Venkateswaramurthy, N. (2024) · Current Psychiatry Research and Reviews · doi:10.2174/0126660822277500240109050359 · Cited by 26
Dimensions of artificial intelligence on family communication
Alfeir, N. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1398960 · Cited by 31
Adams, J. (2024) · Digital Society · doi:10.1007/s44206-024-00110-5 · Cited by 18
“Finally, bias in DMH systems has been identi - fied as a result of assumptions in human society broadly, as well as psychiatry more specifically, that tend to reveal themselves in algorithmic functioning (Pendse et al., 2022; Straw & Callison-Burch, 2020).”
Wang, T., Emami, E., Jafarpour, D., Tolentino, R., Gore, G. C., & Rahimi, S. A. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000941 · Cited by 17
“…Zidaru et al. [46], Daneshjou et al. [47], Jesso et al. [48], Zhao et al. [49], Velagapudi et al. [50], Yi et al. [51], and Straw & Callison-Burch [52] have identified issues such as uneven dataset distribution, under-reporting of race, ethnicity, sex, and socioeconomic factors, and insufficient…”
Navigating and Addressing Public Concerns in AI: Insights From Social Media Analytics and Delphi
Maghsoudi, M., Mohammadi, A., & Habibipour, S. (2024) · IEEE Access · doi:10.1109/access.2024.3440660 · Cited by 21
“The potential biases in AI systems, particularly in healthcare and societal contexts, have raised concerns regarding fairness and equity [36], [37].”
BackgroundNLP AI Models for Optimizing Medical Research: Demystifying the Concerns
Rao, K. N., Arora, R. D., Dange, P., & Nagarkar, N. M. (2023) · Indian Journal of Surgical Oncology · doi:10.1007/s13193-023-01791-z · Cited by 15
“The issue of bias in NLP AI models used for research is a significant concern because it can lead to inaccurate or discriminatory results [4].”
Albikawi, Z. F., Abuadas, M. H., & Rayani, A. M. (2025) · Healthcare · doi:10.3390/healthcare13091089 · Cited by 24
“In health care contexts, particularly digital mental health, perception also extends to include privacy concerns, fairness, and ethical risk awareness because these factors affect trust and engagement [19,20].”
Patient Attitudes Toward Artificial Intelligence in Cancer Care: Scoping Review
Hilbers, D., Nekain, N., Bates, A. T., & Nuñez, J.-J. (2025) · JMIR Cancer · doi:10.2196/74010 · Cited by 19
The Impact of LLM Hallucinations on Motor Skill Learning: A Case Study in Badminton
Qiu, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3444783 · Cited by 27
Woensel, W. V., & Motie, S. (2024) · Enterprise Information Systems · doi:10.1080/17517575.2024.2417404 · Cited by 28
“In process extraction, for instance, this may lead to processes that perpetuate discrimination on gender, race, religion or age [60].”
Köckritz, J., İlgen, B., Cohrdes, C., & Hattab, G. (2025) · Scientific Reports · doi:10.1038/s41598-025-18413-z · Cited by 14
Abubakari, M. S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2530-9.ch013 · Cited by 13
Linardon, J., Messer, M., Anderson, C., Liu, C., McClure, Z., Jarman, H. K., Goldberg, S. B., & Torous, J. (2025) · BMJ Mental Health · doi:10.1136/bmjment-2025-301787 · Cited by 10
“21 Consequently, LLMs may disproportionately reflect perspectives of more frequently occurring or dominant groups, posing risks to equity, representation and fairness in research outputs.”
Hornstein, S., Scharfenberger, J., Lueken, U., Wundrack, R., & Hilbert, K. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01121-9 · Cited by 17
Lauderdale, S. A., Schmitt, R., Wuckovich, B., Dalal, N., Desai, H., & Tomlinson, S. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1544951 · Cited by 14
Lauderdale, S. A., Griffin, S. A., Lahman, K. R., Mbaba, E., & Tomlinson, S. (2025) · Personality and Mental Health · doi:10.1002/pmh.70018 · Cited by 13
Yeasmin, S., Semi, M. M. A., Rony, M. K. K., Das, S., Sabeena, A. A., Rahman, R., Biswas, B., Ahmed, F., & Hossain, A. (2025) · Health Science Reports · doi:10.1002/hsr2.71703 · Cited by 24
“AI's ability to process vast data sets, recognize subtle patterns, and generate predictive insights positions it as a transformative tool in behavioral health [12–14].”
Prediction of Gender-Biased Perceptions of Learners and Teachers Using Machine Learning
Kausar, G., Saleem, S., Subhan, F., Su’ud, M. M., Alam, M., & Uddin, M. I. (2023) · Sustainability · doi:10.3390/su15076241 · Cited by 13
Rashid, L., Möckel, C., & Bohn, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0298040 · Cited by 11
Levis, M., Levy, J., Dufort, V., Russ, C. J., & Shiner, B. R. (2023) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2842 · Cited by 12
Rotstein, N. M., Cohen, Z. D., Welborn, A., Zbozinek, T. D., Akre, S., Jones, K. G., Null, K. E., Pontanares, J., Sanchez, K. L., Flanagan, D. C., et al. (2025) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2025.1478534 · Cited by 9
AI-Driven Tools for Detecting and Monitoring Mental Health Conditions Though Behaviour Patterns
Kasaudhan, K. (2025) · International Journal of Preventive Medicine and Health · doi:10.54105/ijpmh.b1054.05030325 · Cited by 13
Yeasmin, S., Saha, S., Rony, M. K. K., Semi, M. M. A., Das, S., Rahman, R., Khan, R., Tasnim, A. F., & Hosen, A. S. M. S. (2025) · Digital Health · doi:10.1177/20552076251386662 · Cited by 14
“12,13 16 Particularly for individuals with ASD, who often face dif fi - culties with verbal communication and sensory regulation, 17 these AI-supported platforms create structured, low-pressure spaces that are both engaging and easy to navigate.”
Wislocki, K. E., Sami, S., Liberzon, G., & Zalta, A. K. (2025) · JMIR Mental Health · doi:10.2196/80801 · Cited by 9
“GAI models are trained on biased data, including data that may reflect societal stereotypes or clinical misconceptions [24,25,27,29,44].”
InfluentialAn End-to-End Overview of Clinical Speech AI
Ng, S.-I., Xu, L., Siegert, I., Cummins, N., Benway, N. R., Liss, J., & Berisha, V. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3660470 · Cited by 12
“Similar biases have been reported in clinical speech AI of mental health issues, where models underperform for certain demographic groups [315].”
Adu, M. K., Banire, B., Dockrill, M., Ilie, A., Lappin, E., McGrath, P., Munro, S., Myers, K., Obuobi-Donkor, G., Orji, R., et al. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1449129 · Cited by 10
AI Integration in Behavioral Healthcare: A Practical Framework for Clinicians
Sadeh‐Sharvit, S., & Hollon, S. D. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00532-z · Cited by 9
Rubinstein, B., & Matos, S. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11725-1 · Cited by 9
“According to previous studies, NLP model bias may increase health inequality [17 ].”
Shoib, S., Siddiqui, M. F., Turan, S., Chandradasa, M., Armiya’u, A. Y., Saeed, F., Berardis, D. D., Islam, S. M. S., & Zaidi, I. (2025) · Preventive Medicine Research & Reviews · doi:10.4103/pmrr.pmrr_121_24 · Cited by 9
Straw, I., Brass, I., Mkwashi, A., Charles, I., Soares, A., & Steer, C. (2024) · Journal of Medical Internet Research · doi:10.2196/50505 · Cited by 6
Self-citationArtificial Intelligence in Revolutionizing Kidney Care and Beyond: Kid-AI Revolution
Khan, K., Zameer, F., Jain, P., Ravi, K. R., Niranjan, V., Manoj, S., Ravish, H., & Padyana, S. (2024) · Journal of Bio-X Research · doi:10.34133/jbioxresearch.0022 · Cited by 10
Albayrak, E., & Güler, K. G. (2026) · Issues in Mental Health Nursing · doi:10.1080/01612840.2025.2596210 · Cited by 5
“It is also emphasized that AI is important to minimize health inequalities (Straw & Callison-Burch, 2020).”
Artificial intelligence and mental health in the workplace: positive and negative impacts
El-Helaly, M., Awaad, A. E., & Elhelaly, A. M. (2026) · Archives of Environmental & Occupational Health · doi:10.1080/19338244.2026.2623267 · Cited by 5
“74 AI systems are trained on data that may contain existing biases, and if not properly addressed, these biases can be perpetuated and even amplified by AI algorithms.”
Explainable AI for Mental Disorder Detection on Social Media: A Survey and Outlook
Ibrahimov, Y., Anwar, T., & Yuan, T. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3688443 · Cited by 37
Unveiling the Transformative Potential of AI-Generated Imagery in Enriching Mental Health Research
Milasan, L. H. (2024) · Qualitative Health Research · doi:10.1177/10497323241274767 · Cited by 11
“Additionally, AI algorithms are trained on speci fi c datasets that risk perpetuating and amplifying societal biases and stereotypes in generating art with potential implications on how experiences of mental health and distress are portrayed (Straw & Callison-Burch, 2020).”
MyDAS Corpus: Malay Social Media Texts for Detecting Depression, Anxiety, and Stress on Facebook
Ahmad, Z., Mohamed, A., Conway, M., Zakaria, R., Teo, N. H. I., & Maskat, R. (2024) · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · doi:10.1109/aidas63860.2024.10730385 · Cited by 6
“These biases may arise from imbalanced datasets, algorithmic preferences, or pre-existing societal biases embedded within the data [4].”
BackgroundYear 2020 (with COVID): Observation of Scientific Literature on Clinical Natural Language Processing
Grabar, N., Grouin, C., & Processing, S. E. O. T. I. Y. S. O. C. N. L. (2021) · Yearbook of Medical Informatics · doi:10.1055/s-0041-1726528 · Cited by 7
“…patients [67], analysis 6 https://www.kaggle.com/allen-insti-tute-for-ai/CORD-19-research-challenge of COVID impact on people with epilepsy [36], prediction of care cost in mental health setting [68], and, more generally, the use of artificial intelligence in mental health and its biases [69].”
BackgroundDigital Mental Health Through an Intersectional Lens: A Narrative Review
Yesha, R., Orezzoli, M. C. E., Sims, K., & Landau, A. Y. (2026) · Healthcare · doi:10.3390/healthcare14020211 · Cited by 8
Estrada-Araoz, E. G. (2026) · Discover Public Health · doi:10.1186/s12982-026-02266-6 · Cited by 4
“This limitation may affect both the quality of responses generated by AI systems in health contexts and the appropriateness of their automated suggestions [33, 34].”
What We Owe Those Who Chat Woe: A Relational Lens for Mental Health Apps
Ho, A., & Perry, J. (2023) · The American Journal of Bioethics · doi:10.1080/15265161.2023.2250306 · Cited by 3
Straw, I., Rees, G. E., & Nachev, P. (2023) · BMC Digital Health · doi:10.1186/s44247-023-00044-x · Cited by 3
Self-citationEthical Considerations in Affective Computing
Saxena, C. (2024) · The Springer series in applied machine learning · doi:10.1007/978-3-031-63821-3_13 · Cited by 5
Samiun, M., Rony, M. K. K., Yeasmin, S., Manik, M. M. T. G., Debnath, A., Aziz, M. B., Tasnim, A. F., & Nilima, S. I. (2025) · Discover Public Health · doi:10.1186/s12982-025-00814-0 · Cited by 18
Risks and NLP Design: A Case Study on Procedural Document QA
Haduong, N., Gao, A., & Smith, N. A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.81 · Cited by 6
“…environment, concrete harms to specific entities that are more easily conceptualized than broad abstract harms to populations or society (as in Tan et al., 2022, Lee et al., 2020, and Straw and Callison-Burch, 2020), and EXPERTISE , skill required to successfully complete a procedure (Figure 1).”
BackgroundMu, X., Bibars, M., Seyedi, S. S., Zheng, I., Jiang, Z., Chen, L., Omofojoye, B., Hershenberg, R., Levey, A. I., Clifford, G. D., et al. (2026) · Machine Learning Health · doi:10.1088/3049-477x/ae250c · Cited by 6
“…such as sex and age, shown in table 1, and (2) biases inherited from pre-trained models for extracting multimodal features in our classification pipeline (DINOv2, LLaMA-65B, and WavLM), potentially encoding historic biases in facial and speech datasets available from the internet [82, 83].”
Lin, A., Chen, Z., Jiang, A., Tang, B., Qi, C., Zhu, L., Mou, W., Gan, W., Zeng, D., Xiao, M., et al. (2026) · International Journal of Surgery · doi:10.1097/js9.0000000000003839 · Cited by 2
Toward a Responsible and Trustworthy AI System: Navigating Current Challenges and Pathways
Chatterjee, S., Dindarian, A., & Rengaraju, U. (2025) · Apress eBooks · doi:10.1007/979-8-8688-1186-9_11 · Cited by 2
AI in Education: Early Detection of Mental Health Challenges for Inclusive and Supportive Learning
Efthymiou, E., Katsarou, D. V., Sofologi, M., Megari, K., Papadopoulou, S. L., Mantsos, E., Argyriadis, A., & Argyriadi, A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0573-8.ch009 · Cited by 4
Speech and Language Markers as Longitudinal Predictors of Youth Mental Health: A Systematic Review
Silva, M. S., Ahrens, J., Meister, F., & Palaniyappan, L. K. (2025) · Early Intervention in Psychiatry · doi:10.1111/eip.70102 · Cited by 2
“Speech variables, such as EE, may differ by cultural context (Bhugra and McKenzie 2003), and these concerns should be considered even when using speech-measurement methods based on artificial intelligence and language models (Straw and Callison-Burch 2020).”
Digital mental health needs a purpose-driven approach
Househ, M. S., Shah, H. A., Tariq, Z. U. A., Hassan, D. A., Khalifa, M., Schneider, J., Hamdi, M., Abd-Alrazaq, A. A., Ahmad, A., Solaiman, B., et al. (2026) · Nature Human Behaviour · doi:10.1038/s41562-025-02380-6 · Cited by 4
“A model trained predominantly on data from a particular demographic may not be effective or could even be harmful when applied to a different demographic 137 .”
Metadata
- Authors
- Straw, I., & Callison-Burch, C.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Straw, Isabel
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-12-17
- Volume
- 15
- Issue
- 12
- Pages
- e0240376
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7745984
- OpenAlex
- W3112631310
- Cited by
- 164
- References
- 72
- Citations by year
- 2026: 40 · 2025: 46 · 2024: 37 · 2023: 23 · 2022: 12 · 2021: 6
- FWCI
- 13.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (2418758)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- algorithmic bias, natural language processing, mental health, word embeddings, gloves, Word2Vec, health disparities, psychiatric terminology, demographic bias, interdisciplinary collaboration
- MeSH terms
- Intersectoral Collaboration, Data Science, Humans, Linguistics, Mental Health, Natural Language Processing, Psychiatry, Bias, Health Status Disparities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.