Truth machines: synthesizing veracity in AI language models
Open accessThe DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
As AI technologies are rolled out into healthcare, academia, human resources, law, and a multitude of other domains, they become de-facto arbiters of truth. But truth is highly contested, with many different definitions and approaches. This article discusses the struggle for truth in AI systems and the general responses to date. It then investigates the production of truth in InstructGPT, a large language model, highlighting how data harvesting, model architectures, and social feedback mechanisms weave together disparate understandings of veracity. It conceptualizes this performance as an operationalization of truth, where distinct, often-conflicting claims are smoothly synthesized and confidently presented into truth-statements. We argue that these same logics and inconsistencies play out in Instruct’s successor, ChatGPT, reiterating truth as a non-trivial problem. We suggest that enriching sociality and thickening “reality” are two promising vectors for enhancing the truth-evaluating capacities of future language models. We conclude, however, by stepping back to consider AI truth-telling as a social practice: what kind of “truth” do we as listeners desire?
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“critics argue that the reliability of LLMs is likely restricted to bodies of knowledge that are amply represented in their training data (Munn et al., 2023).”
From A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness (2024) · cites this paper as Munn et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
The Structure of Scientific Revolutions: 50th Anniversary Edition
Kuhn, T. S., & Hacking, I. (2012) · doi:10.7208/chicago/9780226458144.001.0001 · Cited by 21,170 · cited 1×
“It is not simply the case that truths change, as for instance when a new conceptual paradigm is ushered in to explain previous inexplicable scientific observations (Kuhn 2012).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Generative Adversarial Networks: An Overview
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765202 · Cited by 4,540 · cited 2×
“Adversarial models in machine learning, for instance, consist of “generators” and “discriminators,” a translation of the social roles of “forgers” and “critics” into technical architectures (Creswell et al. 2018).”
BackgroundTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 13×
“InstructGPT is a large language model derived from GPT-3 (Ouyang et al. 2022), and is similar to the more famous ChatGPT—both released in 2022.”
InfluentialHacking, I. (1990) · Cambridge University Press eBooks · doi:10.1017/cbo9780511819766 · Cited by 3,586
An Invitation to Social Construction
Gergen, K. J. (2015) · SAGE Publications Ltd eBooks · doi:10.4135/9781473921276 · Cited by 2,698 · cited 1×
“To these four “classic” theories we could add a fifth, the social construction theory of truth (Kvale 1995; Gergen 2015)—particularly relevant given the social circuits and networks embedded in these language models.”
From a Logical Point of View: Nine Logico-Philosophical Essays, Second Revised Edition
Van Orman Quine, W. (1980) · Harvard University Press eBooks · doi:10.2307/j.ctv1c5cx5c · Cited by 2,439
A History of the Modern Fact: Problems of Knowledge in the Sciences of Wealth and Society
Poovey, M. (1998) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226675183.001.0001 · Cited by 1,454 · cited 1×
“Rather than a radical break with “old-fashioned” facts and truths, machine learning and data science continue a long history of shaping these concepts in particular ways through scientific inquiry and economic activity (Poovey 1998; Deringer 2017).”
(2019) · Cited by 1,082
Research works printed but not linked: 11
- Chomsky N (2014) Aspects of the theory of syntax. MIT Press, Cambridge
- Cohen JE (2018) Turning privacy inside out. Theor Inq Law 20(1):1–32
- Foucault M (2019) “Discourse and truth” and “Parresia”, foucault, fruchaud, lorenzini. In: Fruchaud H-P, Lorenzini D (eds) The Chicago foucault project. University of Chicago Press, Chicago
- Fox NJ (2008) Post-positivism. SAGE Encycl Qual Res Methods 2:659–664
- Jones P (2021) Work without the worker: labour in the age of platform capitalism. Verso Books, London
- LeCun Y (2022) A path towards autonomous machine intelligence version 0.9.2. https://openreview.net/pdf?id=BZ5a1r-kVsf. Accessed 27 July 2022
- Meng K, Bau D, Andonian A, Belinkov Y (2022) Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35:17359–17372
- Passi S, Vorvoreanu M (2022) Overreliance on AI literature review. Microsoft, Seattle. https://www.microsoft.com/en-us/research/uploads/prod/2022/06/Aether-Overreliance-on-AI-Review-Final-6.21.22.pdf
- Ryan A (2006) Post-Positivist approaches to research. In: Antonesa M (ed) Researching and writing your thesis: a guide for postgraduate students. National University of Ireland, Maynooth, pp 12–26
- Singleton J (2020) Truth discovery: who to trust and what to believe. In: An B, Yorke-Smith N, Seghrouchni AEF, Sukthankar G (eds) International conference on autonomous agents and multi-agent systems 2020. International Foundation for Autonomous Agents and Multiagent Systems, pp 2211–13
- Stiennon N, Ouyang L, Jeffrey Wu, Ziegler D, Lowe R, Voss C, Radford A, Amodei D, Christiano PF (2020) Learning to summarize with human feedback. Adv Neural Inf Process Syst 33:3008–3021
Cited by
Large language models in medical and healthcare fields: applications, advances, and challenges
Wang, D., & Zhang, S. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10921-0 · Cited by 246
“Given the safety–critical nature of medical and health-care domains, erroneous advice regarding patients’ symptoms and medications can result in serious injury or even death (Munn et al. 2023), as exemplified by GPT-3 incorrectly recommending suicide for a patient (Atallah et al. 2023a).”
AI and the future of humanity: ChatGPT-4, philosophy and education – Critical responses
Peters, M. A., Jackson, L., Papastephanou, M., Jandrić, P., Lazaroiu, G., Evers, C. W., Cope, B., Kalantzis, M., Araya, D., Tesar, M., et al. (2024) · Educational Philosophy and Theory · doi:10.1080/00131857.2023.2213437 · Cited by 188
“They also invent non-existent facts when needed to complete a plausible text (Munn et al., 2023).”
Beware of botshit: How to manage the epistemic risks of generative chatbots
Hannigan, T. R., McCarthy, I. P., & Spicer, A. (2024) · Business Horizons · doi:10.1016/j.bushor.2024.03.001 · Cited by 219
Literacy in the Time of Artificial Intelligence
Kalantzis, M., & Cope, B. (2025) · Reading Research Quarterly · doi:10.1002/rrq.591 · Cited by 119
“When Generative AI makes up facts: It only knows how to write good sentences but has no way of checking whether the content is true (Klein, 2023; Munn et al., 2023b).”
A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness
Heersmink, R., de Rooij, B., Clavel Vázquez, M. J., & Colombo, M. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09777-3 · Cited by 91
“While this is an impressive track record, critics argue that the reliability of LLMs is likely restricted to bodies of knowledge that are amply represented in their training data (Munn et al., 2023).”
Metadata
- Authors
- Munn, L., Magee, L., & Arora, V.
- Affiliations
- The University of Queensland (AU)
- Western Sydney University (AU)
- University of Stirling (GB)
- Corresponding author
- Munn, Luke
- Venue
- AI & SOCIETY
- Published
- 2023-08-28
- Volume
- 39
- Issue
- 6
- Pages
- 2759-2773
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W4386225382
- Cited by
- 65
- References
- 50
- Citations by year
- 2026: 13 · 2025: 26 · 2024: 20 · 2023: 5
- FWCI
- 16.04
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- AI & Society (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- Stirling Online Research Repository (University of Stirling) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- The University of Queensland
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- truth, honesty, large language models, ChatGPT, data harvesting, model architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Truth machines: synthesizing veracity in AI language models. https://app.cochat.ai/papers/10.1007/s00146-023-01756-4
- The paper
- Munn, L., Magee, L., & Arora, V. (2024). Truth machines: synthesizing veracity in AI language models. AI & SOCIETY. https://doi.org/10.1007/s00146-023-01756-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


