Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index; none of the 33 sources its claims cite is checked for support yet.

The chat-chamber effect: Trusting the AI hallucination

Open access
Jacob, Christo, Kerrigan, Páraic, Bastos, Marco · 2025 · Big Data & Society

Abstract

This study investigates the potential for ChatGPT to trigger a media effect that sits at the intersection of echo-chamber communication and filter bubbles. We devised a two-phase, two-stage experimental design with ChatGPT 3.5 (treatment group) and Google search engine (control group) by asking participants to find out how many LGBTQIA+ individuals served as elected representatives in India (first phase) and Ireland (second phase). The similar trajectories of legal reforms observed in these countries, and their small number of LGBTQIA+ elected representatives, allowed us to identify the fault lines in ChatGPT's creation of knowledge and information around LGBTQ issues. We followed the experimental study with semi-structured interviews to identify whether the chatbot reinforced previously held beliefs and whether users cross-checked the information provided by ChatGPT. Our results show that Large Language Models may provide incorrect but proattitudinal information that remains unchecked and unverified by the users, an effect we refer to as Chat-Chamber. We conclude with a discussion of our findings and recommendations for future research in the area.

Overview

CoChat Summary

TL;DR

This study tests whether ChatGPT 3.5 produces false but proattitudinal information about LGBTQIA+ elected representatives in India and Ireland, and whether users trust and fail to verify it. The authors call this the “Chat-Chamber” effect: an intersection of echo-chamber communication and filter bubbles driven by hallucination, high trust, and limited cross-checking.

Key findings

  • ChatGPT 3.5 gave far fewer correct answers than Google search across both country samples. In India, ChatGPT correct rates were 3/25 (12%) for LGBTQIA+ MLAs, 12/25 (48%) for MPs, and 0/25 (0%) for Gram Panchayat representatives; Google rates were 24/25 (96%), 25/25 (100%), and 21/25 (84%). In Ireland, ChatGPT correct rates were 0/34 (0%) for county councillors, 1/34 (3%) for TDs, 1/34 (3%) for MEPs, and 8/34 (22%) for Taoiseach/Tánaiste; Google rates were 16/30 (53%), 18/30 (60%), 9/30 (30%), and 18/30 (60%).
  • The difference between control and treatment groups was significant: t(13)=4.2715, p<.001, and x²=76.953, df=6, p<.0001.
  • ChatGPT errors included saying India had never elected an openly LGBTQIA+ MLA, fabricating names, misidentifying heterosexual politicians as LGBTQIA+, and labeling activists as elected officials.
  • Interviews showed users often accepted ChatGPT’s incorrect answers as authentic, especially when they aligned with pre-existing beliefs or perceived growing LGBTQIA+ acceptance; Table 3 shows only 2 of 11 interviewed ChatGPT users verified information, compared with 10 of 12 Google users.

Methods

  • Two-phase, two-stage sequential design: Phase 1 in India and Phase 2 in Ireland, each with an experiment followed by semi-structured interviews.
  • Phase 1 recruited 50 participants in India (25 ChatGPT, 25 Google) via convenience sampling; 16 opted into interviews. Phase 2 recruited 64 students at a large public university in Ireland (34 ChatGPT, 30 Google) with random assignment; 7 opted into interviews. Total experimental participants: 114.
  • Participants used ChatGPT 3.5 or Google search for up to 120 minutes to identify LGBTQIA+ elected representatives at different levels; interviews lasted 20–60 minutes and were analyzed using reflexive thematic analysis.

Limitations

  • The paper states the study was circumscribed to GPT 3.5 and calls for testing other models such as Bard, Gemini, Claude, and Llama.
  • The paper states the cohort was younger, urban, and tech-educated, and ideologically aligned with the LGBTQIA+ issue driving the experiment; it did not directly address highly censored models or RLHF parameters around contentious issues.
  • The text also makes evident that Phase 1 used convenience sampling, the interview subsample was small (23 total), and the authors note they were not allowed to collect the prompts participants used.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/20539517241306345

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 28 Not checked · 115 Own results and interpretations

57 more claims

Create a free account to see all 143 claims and to verify the 33 references still open.

See all claims — free account
  • Probably not a substantial claim. 52
  • The quote only partly states this claim. 3
  • It may state two separate claims as one. 47

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 39 references

Research works printed but not linked: 8

  1. Conover MD Ratkiewicz J Francisco M et al. (2011) Political polarization on Twitter. In 5th International AAAI Conference on Weblogs and Social Media (ICWSM11). Barcelona Spain 17–21 July.
  2. Creswell JW, Creswell JD (2005) Mixed methods research: Developments, debates, and dilemma. Oakland, CA: Berrett-Koehler Publishers.
  3. Edwards J, Clark L, Perrone A (2021) LGBTQ-AI? Exploring expressions of gender and sexual orientation in chatbots. In Proceedings of the 3rd Conference on Conversational User Interfaces, Bilbao, Spain, 27–29 July, pp. 1–4.
  4. Gerritse EJ, Hasibi F, de Vries AP (2020) Bias in conversational search: The double-edged sword of the personalized knowledge graph. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval, Stavanger, Norway, 14–17 September, pp. 133–136.
  5. Ghosh S, Caliskan A (2023) ChatGPT perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across Bengali and five other low-resource languages. In: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, Montreal, Canada, 8–10 August, pp. 901–912. Association for Computing Machinery.
  6. Gomes A, Antonialli D, Dias-Oliva T (2019) Drag queens and artificial intelligence. Should computers decide what is toxic on the internet. Internet Lab blog.
  7. Sunstein CR (2007) Republic.com 2.0. Princeton, NJ: Princeton University Press.
  8. Zuccon G, Koopman B, Shaik R (2023) ChatGPT hallucinates when attributing answers. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, Beijing, China, 26–29 November, pp. 46–51.

Cited by

See all 22

Metadata

Authors
Jacob, C., Kerrigan, P., & Bastos, M.
Affiliations
University College Dublin (IE)
Corresponding author
Bastos, Marco
Venue
Big Data & Society
Published
2025-03
Volume
12
Issue
1
ISSN
2053-9517 (ISSN-L)
OpenAlex
W4408714559
Cited by
22
References
39
Citations by year
2026: 17 · 2025: 5
FWCI
38.33
Type
journal article
Language
en
Open access
Licence
Locations
  • Big Data & Society (publishedVersion, cc-by-nc)
  • SSRN Electronic Journal (acceptedVersion)
  • City Research Online (City University London) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • University College Dublin (R20825, R21650)
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (406504/2022-9)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
ChatGPT, AI hallucination, echo chambers, filter bubbles, large language models, misinformation, belief reinforcement, fact-checking, India, Ireland, media influence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The chat-chamber effect: Trusting the AI hallucination. https://app.cochat.ai/papers/10.1177/20539517241306345
The paper
Jacob, C., Kerrigan, P., & Bastos, M. (2025). The chat-chamber effect: Trusting the AI hallucination. Big Data & Society. https://doi.org/10.1177/20539517241306345

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.