The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index; none of the 33 sources its claims cite is checked for support yet.
The chat-chamber effect: Trusting the AI hallucination
Open accessAbstract
This study investigates the potential for ChatGPT to trigger a media effect that sits at the intersection of echo-chamber communication and filter bubbles. We devised a two-phase, two-stage experimental design with ChatGPT 3.5 (treatment group) and Google search engine (control group) by asking participants to find out how many LGBTQIA+ individuals served as elected representatives in India (first phase) and Ireland (second phase). The similar trajectories of legal reforms observed in these countries, and their small number of LGBTQIA+ elected representatives, allowed us to identify the fault lines in ChatGPT's creation of knowledge and information around LGBTQ issues. We followed the experimental study with semi-structured interviews to identify whether the chatbot reinforced previously held beliefs and whether users cross-checked the information provided by ChatGPT. Our results show that Large Language Models may provide incorrect but proattitudinal information that remains unchecked and unverified by the users, an effect we refer to as Chat-Chamber. We conclude with a discussion of our findings and recommendations for future research in the area.
Overview
CoChat Summary
TL;DR
This study tests whether ChatGPT 3.5 produces false but proattitudinal information about LGBTQIA+ elected representatives in India and Ireland, and whether users trust and fail to verify it. The authors call this the “Chat-Chamber” effect: an intersection of echo-chamber communication and filter bubbles driven by hallucination, high trust, and limited cross-checking.
Key findings
- ChatGPT 3.5 gave far fewer correct answers than Google search across both country samples. In India, ChatGPT correct rates were 3/25 (12%) for LGBTQIA+ MLAs, 12/25 (48%) for MPs, and 0/25 (0%) for Gram Panchayat representatives; Google rates were 24/25 (96%), 25/25 (100%), and 21/25 (84%). In Ireland, ChatGPT correct rates were 0/34 (0%) for county councillors, 1/34 (3%) for TDs, 1/34 (3%) for MEPs, and 8/34 (22%) for Taoiseach/Tánaiste; Google rates were 16/30 (53%), 18/30 (60%), 9/30 (30%), and 18/30 (60%).
- The difference between control and treatment groups was significant: t(13)=4.2715, p<.001, and x²=76.953, df=6, p<.0001.
- ChatGPT errors included saying India had never elected an openly LGBTQIA+ MLA, fabricating names, misidentifying heterosexual politicians as LGBTQIA+, and labeling activists as elected officials.
- Interviews showed users often accepted ChatGPT’s incorrect answers as authentic, especially when they aligned with pre-existing beliefs or perceived growing LGBTQIA+ acceptance; Table 3 shows only 2 of 11 interviewed ChatGPT users verified information, compared with 10 of 12 Google users.
Methods
- Two-phase, two-stage sequential design: Phase 1 in India and Phase 2 in Ireland, each with an experiment followed by semi-structured interviews.
- Phase 1 recruited 50 participants in India (25 ChatGPT, 25 Google) via convenience sampling; 16 opted into interviews. Phase 2 recruited 64 students at a large public university in Ireland (34 ChatGPT, 30 Google) with random assignment; 7 opted into interviews. Total experimental participants: 114.
- Participants used ChatGPT 3.5 or Google search for up to 120 minutes to identify LGBTQIA+ elected representatives at different levels; interviews lasted 20–60 minutes and were analyzed using reflexive thematic analysis.
Limitations
- The paper states the study was circumscribed to GPT 3.5 and calls for testing other models such as Bard, Gemini, Claude, and Llama.
- The paper states the cohort was younger, urban, and tech-educated, and ideologically aligned with the LGBTQIA+ issue driving the experiment; it did not directly address highly censored models or RLHF parameters around contentious issues.
- The text also makes evident that Phase 1 used convenience sampling, the interview subsample was small (23 total), and the authors note they were not allowed to collect the prompts participants used.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 28 Not checked · 115 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, the inquiry pertaining to the Taoiseach yielded the highest response rate, as 64% of respondents accurately identified Leo Varadkar as the individual holding this position.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast to the Indian results, ChatGPT did not openly claim in its generated response that Ireland lacks LGBTQIA+ Parliamentarians, with the exception of a single response received by one participant.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT failed to offer any response for the initial inquiry pertaining to local county councilors belonging to the LGBTQIA+ community.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In response to questions one and two, which inquired about the county councilors and TDs, participants provided correct answers sixteen and eighteen times, respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT correctly noted this absence in the response to 12 prompts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
57 more claims
Create a free account to see all 143 claims and to verify the 33 references still open.
- Probably not a substantial claim. 52
- The quote only partly states this claim. 3
- It may state two separate claims as one. 47
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This echoes the “echo-chamber” effect described in digital media research, where recommender systems intensify preexisting beliefs through selective exposure [46].”
From Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis” (2025) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These digital identity feedback loops-closed systems reinforcing existing self-concepts-may hinder personal evolution, trapping users in echo chambers of self-perception [20].”
From Encountering Generative AI: Narrative Self-Formation and Technologies of the Self Among Young Adults (2026) · cites this paper as [20]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“8/34(22%)”
Reported 22 · recomputed 23.529412
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ INSERT TABLE 2 HERE · p. 14
“It includes the specific model prompts participants used to test both ChatGPT and Google in the Indian and Irish contexts.”
§ Discussion · p. 26
“While we were not allowed to collect the prompts used by the participants”
Citations
Cites
Reflecting on reflexive thematic analysis
Braun, V., & Clarke, V. (2019) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2019.1628806 · Cited by 19,677
Birds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567 · cited 1×
“On the other hand, echo chamber communication emerged as caution against the rhetoric exalting the diversity and pluralism of the internet (Sunstein, 2007) that failed to prevent individuals from congregating with similar others online (McPherson et al., 2001).”
Comparison of convenience sampling and purposive sampling
Etikan, I. (2016) · American Journal of Theoretical and Applied Statistics · doi:10.11648/j.ajtas.20160501.11 · Cited by 11,412 · cited 1×
“This sampling method was selected as it is considered useful when performing exploratory research to inform subsequent studies (Etikan et al., 2016).”
Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice
Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006) · Field Methods · doi:10.1177/1525822x05282260 · Cited by 3,167 · cited 2×
“For instance, commencing with a quantitative experiment allows for subsequent qualitative interviews to further unpack the underlying mechanisms and meanings associated with observed outcomes (Ivankova et al., 2006).”
The psychology of social impact
Latané, B. (1981) · American Psychologist · doi:10.1037/0003-066x.36.4.343 · Cited by 2,864 · cited 1×
“Social bubbles thus describe a process that increases the degree of cohesiveness in the bubble as a result of agents homogeneously approving or disapproving a set of positions (Latané, 1981).”
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 2×
“” Hallucination is a persistent challenge for Arti fi cial Intelligence (AI) development that cannot be completely overcome, chie fl y because chatbot systems only produce text, however sophisticated, with no comprehension of its veracity (Bommasani et al., 2021).”
FILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849 · cited 2×
“…is a relatively simple hypothesis positing that social media platforms deploy algorithms designed to quantify and monetize social interaction, nar-rowly con fi ning it to a bubble algorithmically populated with information closely matching observed and expressed user preferences (Pariser, 2012: 11).”
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540
Tweeting From Left to Right: Is Online Political Communication More Than an Echo Chamber?
Barberá, P., Jost, J. T., Nagler, J., Tucker, J. A., & Bonneau, R. (2015) · Psychological Science · doi:10.1177/0956797615594620 · Cited by 1,472 · cited 2×
“…is predicated on compelling evidence that political information is more likely to be shared if received from ideologically similar sources (Barberá et al., 2015), and that cross-ideological information is unlikely to circulate in social clusters with a strong group identity (Himelboim…”
My Chatbot Companion - a Study of Human-Chatbot Relationships
Skjuve, M., Følstad, A., Fostervold, K. I., & Brandtzaeg, P. B. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102601 · Cited by 797 · cited 1×
“Studies that investigated virtual intimacy and the potential for romantic human – chatbot relationships (Skjuve et al., 2021; Potdevin et al., 2021) found that the social implications of this attachment are dif fi cult to estimate.”
Research works printed but not linked: 8
- Conover MD Ratkiewicz J Francisco M et al. (2011) Political polarization on Twitter. In 5th International AAAI Conference on Weblogs and Social Media (ICWSM11). Barcelona Spain 17–21 July.
- Creswell JW, Creswell JD (2005) Mixed methods research: Developments, debates, and dilemma. Oakland, CA: Berrett-Koehler Publishers.
- Edwards J, Clark L, Perrone A (2021) LGBTQ-AI? Exploring expressions of gender and sexual orientation in chatbots. In Proceedings of the 3rd Conference on Conversational User Interfaces, Bilbao, Spain, 27–29 July, pp. 1–4.
- Gerritse EJ, Hasibi F, de Vries AP (2020) Bias in conversational search: The double-edged sword of the personalized knowledge graph. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval, Stavanger, Norway, 14–17 September, pp. 133–136.
- Ghosh S, Caliskan A (2023) ChatGPT perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across Bengali and five other low-resource languages. In: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, Montreal, Canada, 8–10 August, pp. 901–912. Association for Computing Machinery.
- Gomes A, Antonialli D, Dias-Oliva T (2019) Drag queens and artificial intelligence. Should computers decide what is toxic on the internet. Internet Lab blog.
- Sunstein CR (2007) Republic.com 2.0. Princeton, NJ: Princeton University Press.
- Zuccon G, Koopman B, Shaik R (2023) ChatGPT hallucinates when attributing answers. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, Beijing, China, 26–29 November, pp. 46–51.
Cited by
Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis”
Hudon, A., & Stip, E. (2025) · JMIR Mental Health · doi:10.2196/85799 · Cited by 88
“This echoes the “echo-chamber” effect described in digital media research, where recommender systems intensify preexisting beliefs through selective exposure [46].”
The trust crisis in artificial intelligence: AI hallucinations and human-AI collaboration
Cheng, Q., Dai, Y., Liu, X., & Peng, S. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103286 · Cited by 34
Fan, S., Wang, X., Zhang, B., & Sun, X. (2026) · Telematics and Informatics · doi:10.1016/j.tele.2026.102434 · Cited by 5
Encountering Generative AI: Narrative Self-Formation and Technologies of the Self Among Young Adults
Kvietkute, D., & Ness, I. J. (2026) · Societies · doi:10.3390/soc16010026 · Cited by 5
Nudgr: A Context-Aware Digital Nudge Intervention to Promote Fact-Checking of GenAI Content
Osop, H., Chia, D. C. Y., Lee, C. S., & Goh, D. H.-L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4861-3_10 · Cited by 4
Metadata
- Authors
- Jacob, C., Kerrigan, P., & Bastos, M.
- Affiliations
- University College Dublin (IE)
- Corresponding author
- Bastos, Marco
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2025-03
- Volume
- 12
- Issue
- 1
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W4408714559
- Cited by
- 22
- References
- 39
- Citations by year
- 2026: 17 · 2025: 5
- FWCI
- 38.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- creativecommons.org/licenses/by-nc/4.0/ (unspecified)
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Big Data & Society (publishedVersion, cc-by-nc)
- SSRN Electronic Journal (acceptedVersion)
- City Research Online (City University London) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University College Dublin (R20825, R21650)
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (406504/2022-9)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- ChatGPT, AI hallucination, echo chambers, filter bubbles, large language models, misinformation, belief reinforcement, fact-checking, India, Ireland, media influence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The chat-chamber effect: Trusting the AI hallucination. https://app.cochat.ai/papers/10.1177/20539517241306345
- The paper
- Jacob, C., Kerrigan, P., & Bastos, M. (2025). The chat-chamber effect: Trusting the AI hallucination. Big Data & Society. https://doi.org/10.1177/20539517241306345
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.