CoChat
Sign Up
Cited by 44Misinformation and Its ImpactsEthics and Social Impacts of AIInnovative Human-Technology Interaction

From ChatGPT to FactGPT: A Participatory Design Study to Mitigate the Effects of Large Language Model Hallucinations on Users

Open access
Leiser, Florian, Eckhardt, Sven, Knaeble, Merlin, Maedche, Alexander, Schwabe, Gerhard, Sunyaev, Ali·2023·Mensch und Computer 2023
DOI 10.1145/3603555.3603565
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) like ChatGPT recently gained interest across all walks of life with their human-like quality in textual responses. Despite their success in research, healthcare, or education, LLMs frequently include incorrect information, called hallucinations, in their responses. These hallucinations could influence users to trust fake news or change their general beliefs. Therefore, we investigate mitigation strategies desired by users to enable identification of LLM hallucinations. To achieve this goal, we conduct a participatory design study where everyday users design interface features which are then assessed for their feasibility by machine learning (ML) experts. We find that many of the desired features are well-perceived by ML experts but are also considered as difficult to implement. Finally, we provide a list of desired features that should serve as a basis for mitigating the effect of LLM hallucinations on users.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3603555.3603565

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The negative impact of hallucinations on data quality is well-documented, and incorrect AI recommendations have been shown to harm users in decision-making contexts.

    “The negative impact of hallucinations on data quality is well-documented [33] —incorrect AI recommendations have been shown to harm users in decision-making contexts [13].”

    From Emerging Reliance Behaviors in Human-AI Content Grounded Data Generation: The Role of Cognitive Forcing Functions and Hallucinations (2025) · cites this paper as [33]

    The citation [13] supports the clause about incorrect AI recommendations, not the preceding data-quality claim cited to [33]. The paper’s Related Work section describes following incorrect AI information as overreliance, and its Principal Findings section says incorrect responses to influential questions might affect user behavior. It presents harm as a risk, however, rather than showing that incorrect recommendations have harmed users in decision-making contexts; page numbers are unavailable in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. Hussam Alkaissi and Samy I McFarlane. 2023. Artificial Hallucinations in Chat- GPT: Implications in Scientific Writing. Cureus (Feb. 2023). https://doi.org/10. 7759/cureus.35179 · DOI · Paper page
  2. Caroline Bassett. 2019. The computational therapeutic: exploring Weizenbaum's ELIZA as a history of the present. AI & SOCIETY 34 (2019), 803-812.
  3. Ivo Benke, Michael Thomas Knierim, and Alexander Maedche. 2020. Chatbot- based Emotion Management for Distributed Teams: A Participatory Design Study. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (Oct 2020), 1-30. https://doi.org/10.1145/3415189 · DOI
  4. Som Biswas. 2023. ChatGPT and the Future of Medical Writing. Radiology (Feb. 2023). https://doi.org/10.1148/radiol.223312 · DOI · Paper page
  5. Ali Borji. 2023. A categorical archive of chatgpt failures. arXiv preprint arXiv:2302.03494 (2023). · arXiv
  6. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting Training Data from Large Language Models.. In USENIX Security Symposium, Vol. 6.
  7. Robert Dilts. 1994. Strategies of genius. Vol. 1, Aristotle, Sherlock Holmes, Walt Disney, Wolfgang Amadeus Mozart. Dilts Strategy Group, Scotts Valley.
  8. Yogesh K Dwivedi, Nir Kshetri, Laurie Hughes, Emma Louise Slade, Anand Je- yaraj, Arpan Kumar Kar, Abdullah M Baabdullah, Alex Koohang, Vishnupriya Raghavan, Manju Ahuja, et al. 2023. "So what if ChatGPT wrote it?" Multidisci- plinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management 71 (2023), 102642.
  9. Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2019. A taxonomy of social cues for conversational agents. International Journal of Human-Computer Studies 132 (2019), 138-161.
  10. Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines 30 (2020), 681-694.
  11. Jia-Chen Gu, Tianda Li, Quan Liu, Zhen-Hua Ling, Zhiming Su, Si Wei, and Xiaodan Zhu. 2020. Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2041-2044.
  12. Andreas Holzinger. 2018. From machine learning to explainable AI. In 2018 world symposium on digital intelligence for systems and machines (DISA). IEEE, 55-66.
  13. Joseph Kahne and Benjamin Bowyer. 2018. The Political Significance of Social Media Activity and Social Networks. Political Communication 35, 3 (feb 2018), 470-493. https://doi.org/10.1080/10584609.2018.1426662 · DOI
  14. David Kaplan, Ramayya Krishnan, Rema Padman, and James Peters. 1998. As- sessing data quality in accounting information systems. Commun. ACM 41, 2 (Feb. 1998), 72-78. https://doi.org/10.1145/269012.269024 · DOI
  15. Merlin Knaeble, Mario Nadj, Luisa Germann, and Alexander Maedche. 2023. Tools of Trade of the Next Blue-Collar Job? Antecedents, Design Features, and Out- comes of Interactive Labeling Systems. In 31st European Conference on Information Systems (ECIS) Research Papers. Kristiansand, Norway.
  16. Merlin Knaeble, Gabriel Sailer, Zihan Chen, Thorsten Schwarz, Kailun Yang, Mario Nadj, Rainer Stiefelhagen, and Alexander Maedche. 2023. AutoChemplete -Making Chemical Structural Formulas Accessible. In Proceedings of the 20th International Web for All Conference (Austin, TX, USA) (W4A '23). Association for Computing Machinery, New York, NY, USA, 104-115. https://doi.org/10.1145/ 3587281.3587293 · DOI
  17. Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471 (2021). · arXiv
  18. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appro- priate reliance. Human factors 46, 1 (2004), 50-80.
  19. Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Al- lissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, and Ariel D. Procaccia. 2019. WeBuildAI: Participatory Framework for Algorithmic Governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (Nov 2019), 1-35. https://doi.org/10.1145/3359283 · DOI
  20. Sebastian Lins, Konstantin D. Pandl, Heiner Teigeler, Scott Thiebes, Calvin Bayer, and Ali Sunyaev. 2021. Artificial Intelligence as a Service. Business & Information Systems Engineering 63, 4 (jul 2021), 441-456. https://doi.org/10.1007/s12599- 021-00708-w · DOI
  21. Yvette P. Lopez, Paula L. Rechner, and Julie B. Olson-Buchanan. 2005. Shaping Ethical Perceptions: An Empirical Assessment of the Influence of Business Edu- cation, Culture, and Demographic Factors. Journal of Business Ethics 60, 4 (Sept. 2005), 341-358. https://doi.org/10.1007/s10551-005-1834-4 · DOI
  22. Jan Lorenz, Heiko Rauhut, Frank Schweitzer, and Dirk Helbing. 2011. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences 108, 22 (2011), 9020-9025. https://doi.org/10.1073/pnas. 1008636108 arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.1008636108 · DOI · Paper page
  23. Robert W McGee. 2023. Annie Chan: Three Short Stories Written with Chat GPT. Available at SSRN 4359403 (2023).
  24. Danette McGilvray. 2021. Executing data quality projects: Ten steps to quality data and trusted information (TM). Academic Press.
  25. Peter Norvig. 1992. Paradigms of artificial intelligence programming: case studies in Common LISP. Morgan Kaufmann.
  26. Siobhan O'Connor. 2023. Corrigendum to "Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?" [Nurse Educ. Pract. 66 (2023) 103537]. Nurse education in practice 67 (February 2023), 103572. https: //doi.org/10.1016/j.nepr.2023.103572 · DOI
  27. Siobhan O'Connor and ChatGPT. 2023. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse education in practice 66 (January 2023), 103537. https://doi.org/10.1016/j.nepr.2022.103537 · DOI · Paper page
  28. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training lan- guage models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 27730-27744.
  29. Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230-253.
  30. Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature review. (2022).
  31. Jaron Porciello, Maryia Ivanina, Maidul Islam, Stefan Einarson, and Haym Hirsh. 2020. Accelerating evidence-informed decision-making for the Sustainable De- velopment Goals using machine learning. Nature Machine Intelligence 2, 10 (Oct. 2020), 559-565. https://doi.org/10.1038/s42256-020-00235-5 · DOI
  32. Maximilian Renner, Sebastian Lins, Matthias Söllner, Scott Thiebes, and Ali Sunyaev. 2021. Achieving Trustworthy Artificial Intelligence: Multi-Source Trust Transfer in Artificial Intelligence-capable Technology. In Building Sustainability and Resilience with IS: A Call fpr Action : ICIS 2021 proceedings ; 42nd International Conference on Information Systems (ICIS). AIS eLibrary (AISeL).
  33. Aalok Sathe, Salar Ather, Tuan Manh Le, Nathan Perry, and Joonsuk Park. 2020. Automated fact-checking of claims from Wikipedia. In Proceedings of the 12th Language Resources and Evaluation Conference. 6874-6882.
  34. Ben Shneiderman. 2020. Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human-Computer Interaction 36, 6 (2020), 495-504.
  35. Sarah Tausch, Fabius Steinberger, and Heinrich Hußmann. 2015. Thinking Like Disney: Supporting the Disney Method Using Ambient Feedback Based on Group Performance. Vol. 9298. Springer International Publishing, Cham, 614-621. https: //doi.org/10.1007/978-3-319-22698-9_42 · DOI
  36. Scott Thiebes, Sebastian Lins, and Ali Sunyaev. 2020. Trustworthy artificial intelligence. Electronic Markets 31, 2 (Oct. 2020), 447-464. https://doi.org/10. 1007/s12525-020-00441-4 · DOI · Paper page
  37. Laura von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Julius Pfrommer, Annika Pick, Ra- jkumar Ramamurthy, Michal Walczak, Jochen Garcke, Christian Bauckhage, and Jannis Schuecker. 2023. Informed Machine Learning -A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems. IEEE Trans- actions on Knowledge and Data Engineering 35, 1 (2023), 614-633. https: //doi.org/10.1109/TKDE.2021.3079836 · DOI
  38. Yingxu Wang and Guenther Ruhe. 2007. The cognitive process of decision making. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI) 1, 2 (2007), 73-85.
  39. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022). · arXiv
  40. Ogan M. Yigitbasioglu and Oana Velcu. 2012. A review of dashboards in per- formance management: Implications for design and research. International Journal of Accounting Information Systems 13, 1 (March 2012), 41-59. https: //doi.org/10.1016/j.accinf.2011.08.002 · DOI · Paper page
  41. Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric P. S. Baumer, and Min Kyung Lee. 2022. Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, 1-4. https://doi.org/10.1145/3491101.3516506 Received 4 April 2023; Revised 28 June 2023 · DOI · Paper page

Cited by

Sorted by citations

Authors
Leiser, F., Eckhardt, S., Knaeble, M., Maedche, A., Schwabe, G., & Sunyaev, A.
Affiliations
  • Karlsruhe Institute of Technology (DE)
  • University of Zurich (CH)
ORCID
Venue
Mensch und Computer 2023
Publisher
ACM
Published
2023-09-03
Pages
81-90
DOI
10.1145/3603555.3603565
OpenAlex
W4386041250
Cited by
44
References
6
Citations by year
2026: 6 · 2025: 23 · 2024: 14 · 2023: 1
FWCI
40.88
Type
conference paper
Language
en
Open access
Licence
Locations
  • Mensch und Computer (MuC) (publishedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
large language models, hallucinations, hallucination mitigation, user interface design, fake news, trust in AI, belief revision, participatory design, ChatGPT

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review