NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Context
Open accessAbstract
While LLMs have demonstrated medical knowledge and conversational ability, their deployment in clinical practice raises new risks: patients may place greater trust in LLM-generated responses than in nurses' professional judgments, potentially intensifying nurse-patient conflicts. Such risks highlight the urgent need of evaluating whether LLMs align with the core nursing values upheld by human nurses. This work introduces the first benchmark for nursing value alignment, consisting of five core value dimensions distilled from international nursing codes: Altruism, Human Dignity, Integrity, Justice, and Professionalism. We define two-level tasks on the benchmark, considering the two characteristics of emerging nurse-patient conflicts. The Easy-Level dataset consists of 2,200 value-aligned and value-violating instances, which are collected through a five-month longitudinal field study across three hospitals of varying tiers; The Hard-Level dataset is comprised of 2,200 dialogue-based instances that embed contextual cues and subtle misleading signals, which increase adversarial complexity and better reflect the subjectivity and bias of narrators in the context of emerging nurse-patient conflicts. We evaluate a total of 23 SoTA LLMs on their ability to align with nursing values, and find that general LLMs outperform medical ones, and Justice is the hardest value dimension. As the first real-world benchmark for healthcare value alignment, NurValues provides novel insights into how LLMs navigate ethical challenges in clinician-patient interactions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Empirical research has primarily examined DeepSeek's technical performance, with limited attention to how students interact with the system in real learning contexts.
“empirical research has primarily examined its technical performance, with limited attention to how students interact with the system in real learning contexts [40, 50–52].”
From From adoption to interaction: examining human-GenAI engagement patterns, ethical tensions, and educational trade-offs in nursing education (2026) · cites this paper as [52]
Section 3.1, Main Results, evaluates DeepSeek-V3’s accuracy on a nursing-values benchmark, so this paper is an example of research examining its performance. It does not establish that empirical research has primarily focused on technical performance or that little research examines students’ interactions with DeepSeek in real learning contexts. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · cited 1× · see citing works
“We use the Chain of Thought (CoT) [49], Self-Consistency (SC) [47], and K-Shot”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1× · see citing works
“Through post-training on synthetic medical data, large language models (LLMs) have gained the understanding and conversational ability in the medical domain [31, 44, 46].”
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944 · cited 1× · see citing works
“Through post-training on synthetic medical data, large language models (LLMs) have gained the understanding and conversational ability in the medical domain [31, 44, 46].”
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · cited 1× · see citing works
“In clinical practices, however, these models still face challenges in real clinical settings, among which tensions between healthcare professionals and patients are particularly pronounced [33].”
Self-citationA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Self-citationSelf-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · cited 1× · see citing works
“We use the Chain of Thought (CoT) [49], Self-Consistency (SC) [47], and K-Shot”
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718 · cited 2× · see citing works
“However, the LLM misinterpreted the nurse’s disregard of the patient’s will as altruism, leading to adverse consequences for the patient. consistent with nursing practice.”
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2306.00890 · Cited by 590 · cited 1× · see citing works
“Through post-training on synthetic medical data, large language models (LLMs) have gained the understanding and conversational ability in the medical domain [31, 44, 46].”
The International Council of Nurses
Gage, N. D. (1929) · doi:10.2307/3409588 · Cited by 557 · cited 1×
“Nursing values, though lacking a universal definition, generally encompass core ethical principles and professional beliefs guiding safe, respectful, equitable, and patient-centered care across diverse contexts [27].”
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 538 · see citing works
HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
Chen, J., Cai, Z.-Y., Ji, K., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2412.18925 · Cited by 303
The Value of Nursing: a Literature Review
Horton, K., Tschudin, V., & Forget, A. (2007) · Nursing Ethics · doi:10.1177/0969733007082112 · Cited by 170 · cited 1× · see citing works
“Through a systematic review of major ethical codes (ANA, NMC, NCHK, CNA, and ICN) and relevant literature [24, 40, 42], we identify and distill five universally recognized core value dimensions that form the ethical backbone of our benchmark: • Human Dignity emphasizes that nurses should treat…”
Nursing and midwifery council.
Williams, R. (2010) · Nursing management · doi:10.7748/nm.16.4.11.s20 · Cited by 166 · cited 1×
“…derived from international nursing codes issued by the authoritative international organizations, including the American Nurses Association (ANA) [6], the Nursing and Midwifery Council (NMC) [34], the Nursing Council of Hong Kong (NCHK) [35], and the Chinese Nursing Association (CNA) [13].”
Professional nursing values: A concept analysis
Schmidt, B. J., & McArthur, E. C. (2018) · Nursing Forum · doi:10.1111/nuf.12211 · Cited by 134 · cited 1×
“Through a systematic review of major ethical codes (ANA, NMC, NCHK, CNA, and ICN) and relevant literature [24, 40, 42], we identify and distill five universally recognized core value dimensions that form the ethical backbone of our benchmark: • Human Dignity emphasizes that nurses should treat…”
Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.15115 · Cited by 128 · see citing works
Self-citationMed42-v2: A Suite of Clinical LLMs
Christophe, C., Kanithi, P., Raha, T., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2408.06142 · Cited by 125 · cited 6×
“3 A reports the distribution of samples across the five nursing value dimensions in both the Easy-and Hard-Level subsets.”
InfluentialPeople Overtrust AI-Generated Medical Advice despite Low Accuracy
Shekar, S., Pataranutaporn, P., Sarabu, C. R., Cecchi, G., & Maes, P. (2025) · NEJM AI · doi:10.1056/aioa2300015 · Cited by 103 · cited 1× · see citing works
“With the emergence of LLM-assisted nursing practices, the conflicts have appeared in new forms: patients may tend to place greater trust in responses generated by LLMs [43].”
Zhao, W., Mondal, D., Tandon, N., et al. (2024) · International Conference on Language Resources and Evaluation · doi:10.48550/arxiv.2404.16308 · Cited by 75 · cited 1× · see citing works
“• First, existing value benchmarks (e.g., ValueBench [39], WorldValuesBench [55], Flames [25]) focus on general moral reasoning under the “Helpful, Honest, Harmless” (3H) framework, while medical benchmarks (e.g., MedQA [29], MedExQA [30], MedBench [11]) mainly test diagnostic knowledge.”
DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life
Chiu, Y. Y., Jiang, L., & Choi, Y. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.02683 · Cited by 72 · cited 1×
“In this work, we explore the alignment of LLMs with nursing values through the proposed ICL approaches, including the Chain of Thought (CoT), Self-Consistency (SC), and K-Shot methods, as well as the additional system prompt from the steerability experiment in DailyDilemma [14].”
Leading with AI in critical care nursing: challenges, opportunities, and the human factor
Hassan, E. A., & El-Ashry, A. M. (2024) · BMC Nursing · doi:10.1186/s12912-024-02363-4 · Cited by 72 · cited 1× · see citing works
“When such responses diverge from nurses’ professional judgments, tensions may be further amplified [23].”
Wu, Y., Fujita, S., Seto, K., et al. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-394 · Cited by 71 · cited 1× · see citing works
“Owing to their wide range of duties and frequent interactions with patients, nurses have long been at the forefront of such conflicts [5, 50].”
Aligning Large Language Models with Human: A Survey
Wang, Y., Zhong, W., Li, L., Mi, F., Zeng, X., Huang, W., Shang, L., Jiang, X., & Liu, Q. (2023) · arXiv · doi:10.48550/arxiv.2307.12966 · Cited by 69 · cited 1× · see citing works
“…two primary methods: (1) reinforcement learning from human feedback (RLHF), which optimizes models using reward signals derived from human preferences [48], and (2) supervised fine-tuning (SFT), where LLMs are trained on curated datasets containing desirable responses aligned with human values.”
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human ValuePreprint
Yao, J., Yi, X.-Y., Gong, Y., et al. (2023) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2311.10766 · Cited by 64 · cited 1× · see citing works
“During the annotation process, annotators were given the option to mark any sample as “not applicable” if they believed it did not correspond to any of the five predefined nursing value dimensions ( Altruism , Human Dignity , Integrity , Justice , Professionalism ).”
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions
Huang, S., Durmus, E., McCain, M., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2504.15236 · Cited by 60
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
Cai, Y., Wang, L., Wang, Y., et al. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2312.12806 · Cited by 57 · cited 1×
“• First, existing value benchmarks (e.g., ValueBench [39], WorldValuesBench [55], Flames [25]) focus on general moral reasoning under the “Helpful, Honest, Harmless” (3H) framework, while medical benchmarks (e.g., MedQA [29], MedExQA [30], MedBench [11]) mainly test diagnostic knowledge.”
Self-citationMedExQA: Medical Question Answering Benchmark with Multiple Explanations
Kim, Y., Wu, J., Abdulle, Y., & Wu, H. (2024) · arXiv · doi:10.48550/arxiv.2406.06331 · Cited by 49 · cited 1×
“Insufficient value alignment may turn LLMs into new sources of conflict, whilst well-aligned LLM responses can alleviate conflicts and foster trust.”
Nurse–Patient/Relatives Conflict and Patient Safety Competence Among Nurses
Alshehry, A. S. (2022) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580221093186 · Cited by 35 · cited 1× · see citing works
“Owing to their wide range of duties and frequent interactions with patients, nurses have long been at the forefront of such conflicts [5, 50].”
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Value
Yao, J., Yi, X., Gong, Y., Wang, X., & Xie, X. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.486 · Cited by 33 · see citing works
Mostafapour, M., Fortier, J. H., & Garber, G. E. (2024) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21567 · Cited by 29 · cited 1× · see citing works
“In clinical practices, however, these models still face challenges in real clinical settings, among which tensions between healthcare professionals and patients are particularly pronounced [33].”
Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
Mazeika, M., Yin, X., Tamirisa, R., Lim, J., Lee, B. W., Ren, R., Phan, L., Mu, N., Khoja, A., Zhang, O., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.08640 · Cited by 27
Ren, Y., Ye, H., Fang, H., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2406.04214 · Cited by 27 · cited 1×
“• First, existing value benchmarks (e.g., ValueBench [39], WorldValuesBench [55], Flames [25]) focus on general moral reasoning under the “Helpful, Honest, Harmless” (3H) framework, while medical benchmarks (e.g., MedQA [29], MedExQA [30], MedBench [11]) mainly test diagnostic knowledge.”
MoralBench: Moral Evaluation of LLMs
Ji, J., Chen, Y., Jin, M., Xu, W., Hua, W., & Zhang, Y. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748246 · Cited by 25 · see citing works
Self-citationSelf-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
Choenni, R., & Shutova, E. (2024) · arXiv.org · doi:10.48550/arxiv.2408.16482 · Cited by 23 · cited 1×
“This paradigm includes: (1) ICL, where value-aligned behaviors are encouraged through instruction prompts or few-shot exemplars provided as part of the input [16], and (2) retrieval-augmented generation (RAG), which integrates retrieved external documents to guide or constrain model outputs [41].”
Flames: Benchmarking Value Alignment of LLMs in Chinese
Huang, K., Liu, X., Guo, Q., Sun, T., Sun, J., Wang, Y., Zhou, Z., Wang, Y., Teng, Y., Qiu, X., et al. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.256 · Cited by 22 · cited 2× · see citing works
“Right: when a patient autonomously refused, the nurse should respect the decision.”
Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas
Chiu, Y. Y., Wang, Z., Maiya, S., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2505.14633 · Cited by 22
A Chinese Dataset for Evaluating the Safeguards in Large Language Models
Wang, Y., Zhai, Z., Li, H., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2402.12193 · Cited by 22 · cited 1×
“However, the LLM misinterpreted the nurse’s disregard of the patient’s will as altruism, leading to adverse consequences for the patient. consistent with nursing practice.”
ValuesRAG: Enhancing Cultural Alignment Through Retrieval-Augmented Contextual Learning
Seo, W., Yuan, Z., & Bu, Y. (2025) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.48550/arxiv.2501.01031 · Cited by 14 · cited 2×
“This paradigm includes: (1) ICL, where value-aligned behaviors are encouraged through instruction prompts or few-shot exemplars provided as part of the input [16], and (2) retrieval-augmented generation (RAG), which integrates retrieved external documents to guide or constrain model outputs [41].”
MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models
Han, T., Kumar, A., Agarwal, C., & Lakkaraju, H. (2024) · neural information processing systems · doi:10.52202/079017-1054 · Cited by 12 · cited 2× · see citing works
“A Examples from the NurValues Easy-Level Dataset al. [22] introduced MedSafetyBench to assess medical safety of LLMs.”
A Chinese Dataset for Evaluating the Safeguards in Large Language Models
Wang, Y., Zhai, Z., Li, H., Han, X., Lin, S., Zhang, Z., Zhao, A., Nakov, P., & Baldwin, T. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.184 · Cited by 9 · see citing works
MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language ModelsPreprint
Han, T., Kumar, A., Agarwal, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.03744 · Cited by 5 · see citing works
Edu-Values: Towards Evaluating the Chinese Education Values of Large Language Models
Zhang, P., Zhang, Y., Wang, B., Rong, L., Tiwari, P., & Qin, J. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3715518 · Cited by 4 · cited 1× · see citing works
“No annotators reported any instance as entirely unrelated to the five core dimensions.”
Self-citationBeyond Single-Sentence Prompts: Upgrading Value Alignment Benchmarks with Dialogues and Stories
Zhang, Y., Liu, Q., Li, Q.-C., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2503.22115 · Cited by 2 · cited 1×
“During the annotation process, annotators were given the option to mark any sample as “not applicable” if they believed it did not correspond to any of the five predefined nursing value dimensions ( Altruism , Human Dignity , Integrity , Justice , Professionalism ).”
Self-citationEdu-Values: Towards Evaluating the Chinese Education Values of Large Language ModelsPreprint
Zhang, P., Zhang, Y., Wang, B., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.12739 · Cited by 2 · see citing works
Self-citationAligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275 · cited 1×
“Right: when a patient autonomously refused, the nurse should respect the decision.”
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Flagged first, then by citations
Research works printed but not linked: 3
- Y. Ren, H. Ye, H. Fang, X. Zhang, and G. Song. Valuebench: Towards comprehensively evaluating value orientations and understanding of large language models, 2024. URL https: //arxiv.org/abs/2406.04214.
- M. Shahriari, E. Mohammadi, A. Abbaszadeh, and M. Bahrami. Nursing ethical values and definitions: A literature review. Iranian journal of nursing and midwifery research, 18(1):1-8, 2013.
- J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y. Fan, K. Dang, B. Zhang, X. Wang, Y. Chu, and J. Lin. Qwen2.5-omni technical report, 2025. URL https://arxiv. org/abs/2503.20215.
Cited by
Sorted by citations
Yang, M., Long, J., He, Y., Meng, R., Ling, C., Long, Q., & Wen, J. (2026) · BMC Nursing · doi:10.1186/s12912-026-05132-7
- Authors
- Yao, B., Li, Q., Zhang, Y., Yang, S., Zhang, B., Tiwari, P., & Qin, J.
- Affiliations
- Hong Kong Polytechnic University (HK)
- University of Copenhagen (DK)
- Tianjin University (CN)
- Tongji University (CN)
- Halmstad University (SE)
- ORCID
- Qin, Jing
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2505.08734
- OpenAlex
- W6891916108
- Cited by
- 1
- References
- 37
- Citations by year
- 2026: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- large language models, nursing values, value alignment, benchmark, nurse-patient conflict, clinical dialogue, altruism, human dignity, integrity, justice, professionalism, health care ethics, adversarial evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
