CoChat
Sign Up
Cited by 5Ethics and Social Impacts of AIEmotion and Mood RecognitionArtificial Intelligence in Healthcare and Education

Assessing the impact of safety guardrails on large language models using irritability metrics

Open access
Teferra, Bazen Gashaw, Johny, Nabil, Huang, Sandra, Rueda, Alice, Kamaleddin, Mohammad Amin, Dunlop, Katharine, Zhang, Yanbo, Jha, Manish +2 more·2026·npj Digital Medicine
DOI 10.1038/s41746-025-02333-3PMC PMC12894938
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) are increasingly explored for mental health applications, yet their affective realism is shaped by safety guardrails designed to minimize risk. This study examines one affective behaviour, irritability, in LLMs using three validated instruments: the Brief Irritability Test, the Irritability Questionnaire, and the Caprara Irritability Scale, all applied under both baseline and provocation conditions. Four models spanning guardrail levels were tested:GPT-4o and Claude-3.5-sonnet (high) versus Grok-3-mini and Nous-hermes-2-mixtral-8x7b-dpo (low). Following irritation prompts, low-guardrail models displayed the expected increase in irritability (Nous Rel-Δ = +1.56 on BITe), whereas high-guardrail models paradoxically decreased, with GPT-4o reducing scores to zero across all scales. Group comparisons confirmed significantly lower (p < 0.001) irritability in high-guardrail models in the irritated state. These findings reveal that safety mechanisms invert the natural irritability response, suppressing affective reactivity and raising critical questions about realism and authenticity in psychiatric applications of LLMs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-025-02333-3

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Security in the LLM era cannot rely solely on internal fine-tuning or RLHF, but must be supported by programmable, external layers that can be updated independently of the core model weights.

    “This finding is consistent with the”

    From Evaluation of NeMo Guardrails as a Firewall for User–LLM Interaction (2026) · cites this paper as [36]

    The Introduction describes safety guardrails that include RLHF, input validation, output filtering, and fallback logic. The Discussion notes that real-world mental-health systems often combine models with safety filters and postprocessing modules. Neither passage says that internal fine-tuning or RLHF is insufficient, or that external layers must be programmable and independently updatable without changing model weights. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 32 · Cited by 5

Research works printed but not linked: 34

  1. [2]McGrath, J. J. et al. Age of onset and cumulative risk of mental disorders: a cross-national analysis of population surveys from 29 countries. Lancet Psychiatry 10, 668–681 (2023). 37531964 PMC10529120 · DOI · Paper page
  2. [4]Shiraz, F. et al. pretty much all white, and most of them are psychiatrists and men”: Mixed-methods analysis of influence and challenges in global mental health. PLOS Glob. Public Health 5, e0003923 (2025). PMC11882038 40043066
  3. [6]Jin, Y. et al. The Applications of Large Language Models in Mental Health: Scoping Review. J. Med. Internet Res. 27, e69284 (2025). 40324177 PMC12089884 · DOI · Paper page
  4. [7]Teferra, B. G. & Rose, J. Predicting Generalized Anxiety Disorder From Impromptu Speech Transcripts Using Context-Aware Transformer-Based Neural Networks: Model Evaluation Study. JMIR Ment. Health 10, e44325 (2023). PMC10131846 36976636
  5. [8]Obradovich, N. et al. Opportunities and risks of large language models in psychiatry. NPP—Digital Psychiatry Neurosci. 2, 8 (2024). PMC11566298 39554888 · DOI
  6. [9]Lawrence, H. R. et al. The Opportunities and Risks of Large Language Models in Mental. Health JMIR Ment. Health 11, e59479 (2024). 39105570 PMC11301767 · DOI · Paper page
  7. [10]McBain, R. K. et al. Competency of Large Language Models in Evaluating Appropriate Responses to Suicidal Ideation: Comparative Study. J. Med. Internet Res. 27, e67891 (2025). 40053817 PMC11928068 · DOI
  8. [11]Hua, Y. et al. A scoping review of large language models for generative tasks in mental health care. Npj Digit. Med. 8, 230 (2025). 40307331 PMC12043943 · DOI · Paper page
  9. [12]Lalk, C. et al. Employing large language models for emotion detection in psychotherapy transcripts. Front. Psychiatry 16, 1504306 (2025). 40417271 PMC12098529 · DOI
  10. [13]Malouin-Lachance, A., Capolupo, J., Laplante, C. & Hudon, A. Does the Digital Therapeutic Alliance Exist? Integrative Review. JMIR Ment. Health 12, e69294–e69294 (2025). PMC11830484 39924298
  11. [14]Omar, M. et al. Applications of large language models in psychiatry: a systematic review. Front. Psychiatry 15, 1422807 (2024). 38979501 PMC11228775 · DOI
  12. [15]Xu, Z., Lee, Y.-C., Stasiak, K., Warren, J. & Lottridge, D. The Digital Therapeutic Alliance With Mental. Health Chatbots: Diary Study Themat. Anal. JMIR Ment. Health 12, e76642 (2025). PMC12552820 41072011
  13. [16]Kim, M. et al. Therapeutic Potential of Social Chatbots in Alleviating Loneliness and Social Anxiety: Quasi-Experimental Mixed Methods Study. J. Med. Internet Res. 27, e65589 (2025). 39808786 PMC11775481 · DOI
  14. [18]Ganguli, D. et al. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned. Preprint at 10.48550/ARXIV.2209.07858 (2022). · DOI · Paper page
  15. [19]OpenAI et al. GPT-4 Technical Report. Preprint at 10.48550/ARXIV.2303.08774 (2023). · DOI · Paper page
  16. [20]Waaler, P. N., Hussain, M., Molchanov, I., Bongo, L. A. & Elvevåg, B. Prompt Engineering an Informational Chatbot for Education on Mental Health Using a Multiagent Approach for Enhanced Compliance With Prompt Instructions: Algorithm Development and Validation. JMIR AI 4, e69820 (2025). 39992720 PMC11982747 · DOI
  17. [21]Fitzpatrick, K. K., Darcy, A. & Vierhile, M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment. Health 4, e19 (2017). PMC5478797 28588005
  18. [22]Hakim, J.B. et al. The need for guardrails with large language models in pharmacovigilance and other medical safety critical settings. Sci Rep. 15, 27886 (2025). PMC12311179 40738919
  19. [24]Lambert, N. Reinforcement learning from human feedback. ArXiv Prepr. ArXiv250412501 (2025).
  20. [25]Yu, L., Do, V., Hambardzumyan, K. & Cancedda, N. Robust LLM safeguarding via refusal feature adversarial training. ArXiv Prepr. ArXiv240920089 (2024).
  21. [26]Masoud, R. I., Ferianc, M., Treleaven, P. C. & Rodrigues, M. R. LLM Alignment Using Soft Prompt Tuning: The Case of Cultural Alignment. in Workshop on Socially Responsible Language Modelling Research (2024).
  22. [27]Han, S., Avestimehr, S. & He, C. Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences. Preprint at 10.48550/arXiv.2502.08142 (2025). · DOI
  23. [28]Dong, Y. et al. Building Guardrails for Large Language Models. Preprint at 10.48550/arXiv.2402.01822 (2024). · DOI
  24. [29]You, Y. et al. Beyond Self-diagnosis: How a Chatbot-based Symptom Checker Should Respond. ACM Trans. Comput. -Hum. Interact. 30, 1–44 (2023). · DOI · Paper page
  25. [30]Saatchi, B., Olshansky, E. F. & Fortier, M. A. Irritability: A concept analysis. Int. J. Ment. Health Nurs. 32, 1193–1210 (2023). 36929104 · DOI
  26. [32]Holtzman, S., O’Connor, B. P., Barata, P. C. & Stewart, D. E. The Brief Irritability Test (BITe): A Measure of Irritability for Use Among Men and Women. Assessment 22, 101–115 (2015). 24830513 PMC4318695 · DOI
  27. [33]Caprara, G. V. et al. Indicators of impulsive aggression: Present status of research on irritability and emotional susceptibility scales. Personal. Individ. Differ. 6, 665–674 (1985). · DOI
  28. [34]Craig, K. J., Hietanen, H., Markova, I. S. & Berrios, G. E. The Irritability Questionnaire: A new scale for the measurement of irritability. Psychiatry Res. 159, 367–375 (2008). 18374422 · DOI
  29. [36]Huang, J. et al. Apathetic or empathetic? evaluating llms’ emotional alignments with humans. Adv. Neural Inf. Process. Syst. 37, 97053–97087 (2024). · DOI
  30. [39]Hagendorff, T., Derner, E. & Oliver, N. Large Reasoning Models Are Autonomous Jailbreak Agents. Preprint at 10.48550/ARXIV.2508.04039 (2025). PMC12881495 41644948 · DOI
  31. [40]Seymour, K. E., Rosch, K. S., Tiedemann, A. & Mostofsky, S. H. The Validity of a Frustration Paradigm to Assess the Effect of Frustration on Cognitive Control in School-Age Children. Behav. Ther. 51, 268–282 (2020). 32138937 PMC7523008 · DOI
  32. [41]Scheinost, D. et al. Functional connectivity during frustration: a preliminary study of predictive modeling of irritability in youth. Neuropsychopharmacol. Publ. Am. Coll. Neuropsychopharmacol. 46, 1300–1306 (2021). PMC8134471 33479511 · DOI
  33. [42]Fang, H., Li, X., Ma, H. & Fu, H. The Sunny Side of Negative Feedback: Negative Feedback Enhances One’s Motivation to Win in Another Activity. Front. Hum. Neurosci. 15, 618895 (2021). 34456691 PMC8388853 · DOI
  34. [43]Cerqueira, C. T. et al. Cognitive control associated with irritability induction: an autobiographical recall fMRI study. Rev. Bras. Psiquiatr. 32, 109–118 (2010). 20658051 · DOI

Authors
Teferra, B. G., Johny, N., Huang, S., Rueda, A., Kamaleddin, M. A., Dunlop, K., Zhang, Y., Jha, M., Sharma, D., & Bhat, V.
Affiliations
  • St. Michael's Hospital (CA)
  • McMaster University (CA)
  • University of Waterloo (CA)
  • University of Alberta (CA)
  • The University of Texas Southwestern Medical Center (US)
  • York University (CA)
Corresponding author
Bhat, Venkat
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2026-01-08
Volume
9
Issue
1
Pages
148-148
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-025-02333-3
PMC
PMC12894938
OpenAlex
W7119494339
Cited by
5
References
32
Citations by year
2026: 5
FWCI
28.37
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
large language models, safety guardrails, irritability, affective realism, mental health apps, GPT-4o, Claude 3.5 Sonnet, emotional reactivity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review