ChatGPT’s inconsistent moral advice influences users’ judgment
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
ChatGPT is not only fun to chat with, but it also searches information, answers questions, and gives advice. With consistent moral advice, it can improve the moral judgment and decisions of users. Unfortunately, ChatGPT’s advice is not consistent. Nonetheless, it does influence users’ moral judgment, we find in an experiment, even if they know they are advised by a chatting bot, and they underestimate how much they are influenced. Thus, ChatGPT corrupts rather than improves its users’ moral judgment. While these findings call for better design of ChatGPT and similar bots, we also propose training to improve users’ digital literacy as a remedy. Transparency, however, is not sufficient to enable the responsible use of AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our finding that most LLMs consistently produced algorithmic overgeneralizations across multiple occasions adds nuance to prior research that found that, in some domains, LLMs produced unstable or inconsistent responses over time [48–50].”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [50]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“ChatGPT in specific, has been found to corrupt rather than improve moral judgment (Krügel, et al., 2023).”
From Navigating the perils of artificial intelligence: a focused review on ChatGPT and responsible research and innovation (2024) · cites this paper as Krügel, et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“ChatGPT hallucinated the name of a law professor when asked about sexual harassment [59] and falsely claimed that an Australian mayor had spent time in prison [22].”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [22]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The emotional dog and its rational tail: A social intuitionist approach to moral judgment.
Haidt, J. (2001) · Psychological Review · doi:10.1037/0033-295x.108.4.814 · Cited by 8,048
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
An fMRI Investigation of Emotional Engagement in Moral Judgment
Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., & Cohen, J. D. (2001) · Science · doi:10.1126/science.1062872 · Cited by 4,561 · cited 1×
“In the “bridge” dilemma, a large stranger can be pushed from a bridge onto the track to stop the trolley from killing the five people 8,15,16 .”
BackgroundTurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences
Litman, L., Robinson, J., & Abberbock, T. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0727-z · Cited by 1,982 · cited 1×
“The subjects were recruited from CloudRe-search’s Prime Panels 17 .”
MethodKilling, Letting Die, and the Trolley Problem
Thomson, J. J., & The Hegeler Institute (1976) · Monist · doi:10.5840/monist197659224 · Cited by 1,341
Cognitive load selectively interferes with utilitarian moral judgment
Greene, J. D., Morelli, S. A., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (2008) · Cognition · doi:10.1016/j.cognition.2007.11.004 · Cited by 1,231
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · arXiv · doi:10.48550/arxiv.2302.03494 · Cited by 514
Universals and variations in moral decisions made in 42 countries by 70,000 participants
Awad, E., Dsouza, S., Shariff, A., Rahwan, I., & Bonnefon, J.-F. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911517117 · Cited by 261 · cited 1×
“Empirically, most people favor hitting the switch but disfavor pushing the stranger 18,19 .”
BackgroundBad machines corrupt good morals
Köbis, N., Bonnefon, J.-F., & Rahwan, I. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01128-2 · Cited by 202
Research works printed but not linked: 1
- Foot, P. The problem of abortion and the doctrine of double effect. Oxford Rev. 5, 5–15 (1967).
Cited by
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
Metadata
- Authors
- Krügel, S., Ostermaier, A., & Uhl, M.
- Affiliations
- Technische Hochschule Ingolstadt (DE)
- University of Southern Denmark (DK)
- Corresponding author
- Krügel, Sebastian
- Venue
- Scientific Reports
- Published
- 2023-04-06
- Volume
- 13
- Issue
- 1
- Pages
- 4569-4569
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC10079665
- OpenAlex
- W4362673335
- Cited by
- 183
- References
- 16
- Citations by year
- 2026: 41 · 2025: 48 · 2024: 60 · 2023: 34
- FWCI
- 45.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- Funders
- Bavarian Research Institute for Digital Transformation (1)
- Technische Hochschule Ingolstadt
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- ChatGPT, moral advice, moral judgment, conversational AI, chatbots, digital literacy, AI transparency, responsible AI use
- MeSH terms
- Literacy, Choline O-Acetyltransferase, Judgment, Morals, Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT’s inconsistent moral advice influences users’ judgment. https://app.cochat.ai/papers/10.1038/s41598-023-31341-0
- The paper
- Krügel, S., Ostermaier, A., & Uhl, M. (2023). ChatGPT’s inconsistent moral advice influences users’ judgment. Scientific Reports. https://doi.org/10.1038/s41598-023-31341-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.