In generative AI we trust: can chatbots effectively verify political information?
Open accessAbstract
This article presents a comparative analysis of the potential of two large language model (LLM)-based chatbots—ChatGPT and Bing Chat (recently rebranded to Microsoft Copilot)—to detect veracity of political information. We use AI auditing methodology to investigate how chatbots evaluate true, false, and borderline statements on five topics: COVID-19, Russian aggression against Ukraine, the Holocaust, climate change, and LGBTQ + -related debates. We compare how the chatbots respond in high- and low-resource languages by using prompts in English, Russian, and Ukrainian. Furthermore, we explore chatbots’ ability to evaluate statements according to political communication concepts of disinformation, misinformation, and conspiracy theory, using definition-oriented prompts. We also systematically test how such evaluations are influenced by source attribution. The results show high potential of ChatGPT for the baseline veracity evaluation task, with 72% of the cases evaluated in accordance with the baseline on average across languages without pre-training. Bing Chat evaluated 67% of the cases in accordance with the baseline. We observe significant disparities in how chatbots evaluate prompts in high- and low-resource languages and how they adapt their evaluations to political communication concepts with ChatGPT providing more nuanced outputs than Bing Chat. These findings highlight the potential of LLM-based chatbots in tackling different forms of false information in online environments, but also point to the substantial variation in terms of how such potential is realized due to specific factors (e.g. language of the prompt or the topic).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GPT models can effectively detect propaganda.
“Several experiments have revealed that GPT models can effectively detect propaganda (Makhortykh et al., 2023; Kuznetsova et al., 2025).”
From ChatGPT for President! Presupposed content in politicians versus GPT-generated texts (2025) · cites this paper as Kuznetsova et al., 2025
The ‘Accuracy of detecting conspiratorial statements’ section reports that ChatGPT correctly identified conspiratorial information in 81–86% of cases across three languages. The paper also examines misinformation and disinformation, but the passages do not report a test of propaganda detection specifically. (LLM, conf=0.80)
Experimental — accuracy not yet measuredLLMs can verify factual errors based on extensive pretrained data.
“can verify factual errors based on extensive pretrained data (Augenstein et al. 2024; Kuznetsova et al. 2025)”
From Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation (2026) · cites this paper as Kuznetsova et al. 2025
The Detection accuracy of false, true, and borderline statements section (page unavailable) reports that the tested chatbots classified 70% of prompts in accordance with the veracity baseline, supporting their ability to identify factual errors in some cases. The paper discusses training-data cutoffs and dependence on training data, but does not establish that verification works *because of extensive pretrained data*. (LLM, conf=0.85)
Experimental — accuracy not yet measuredStudies show LLMs can be effectively utilised for fact-checking, indicating reliable open-world detection of widely shared content.
“studies show LLM can be effectively utilised for fact-checking[26][27], indicating reliable open-world detection of widely shared content.”
From Illusions of reflection: open-ended task reveals systematic failures in Large Language Models' reflective reasoning (2025) · cites this paper as [27]
The paper’s ‘Detection accuracy of false, true, and borderline statements’ section reports that the chatbots classified 70% of prompts in accordance with the baseline, supporting their potential use in fact-checking. The Discussion and conclusion also reports substantial variation by chatbot and language. It does not establish reliable open-world detection of widely shared content; that extends beyond the claims and conditions tested. (LLM, conf=0.87)
Experimental — accuracy not yet measuredChatGPT, prompted in English, was found to be surprisingly adept at identifying and addressing conspiratorial narratives associated with Covid-19, the Russian aggression against Ukraine, the Holocaust, climate change, and debates related to LGBTQIA+ people, as compared with ChatGPT prompted in Ukrainian and with Bing Chat, which showed a decrease in responsiveness.
“ChatGPT, prompted in English, was found to be surprisingly adept at identifying and addressing conspiratorial narratives associated with Covid-19, the Russian aggression against Ukraine, the Holocaust, climate change, and debates related to LGBTQIA+ people, as compared with ChatGPT prompted in Ukrainian and with Bing Chat, which showed a decrease in responsiveness (Kuznetsova et al., 2025).”
From "Just Asking Questions": Doing Our Own Research on Conspiratorial Ideation by Generative AI Chatbots (2026) · cites this paper as Kuznetsova et al., 2025
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredPotential differences between outputs generated via API and the human-chatbot interface remain unexplored, as no systematic investigations on this issue have been conducted to date.
“Additionally, potential differences between outputs generated via API and the human-chatbot interface remain unexplored, as no systematic investigations on this issue have been conducted to date (Kuznetsova et al., 2025).”
From ChatGPT for President! Presupposed content in politicians versus GPT-generated texts (2025) · cites this paper as Kuznetsova et al., 2025
The Data collection and analysis section says differences between API and human-chatbot interface outputs are possible and, to the authors’ knowledge, had not been systematically investigated. The claim preserves the possible nature of the differences but presents the absence of investigations as an unqualified fact. (LLM, conf=0.85)
Experimental — accuracy not yet measured
ChatGPT achieved over 81% accuracy in classifying conspiracy theories in a comparative study of political information veracity in multiple languages.
“with ChatGPT achieving over 81% accuracy in classifying conspiracy theories.”
From Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation (2026) · cites this paper as Kuznetsova et al. 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1× · see citing works
“Similar to search engines and platforms mediated by non-generative forms of AI, generative AI technology is largely non-transparent for its users (Kasneci et al., 2023).”
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004 · see citing works
BAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 1,826 · see citing works
Chatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355 · see citing works
ChatGPT outperforms crowd workers for text-annotation tasks
Gilardi, F., Alizadeh, M., & Kubli, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2305016120 · Cited by 1,197 · see citing works
Fake news, disinformation and misinformation in social media: a review
Aïmeur, E., Amri, S., & Brassard, G. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-023-01028-5 · Cited by 772 · cited 1× · see citing works
“This particular task has been attracting a growing amount of scholarly interest (e.g. Hoes et al., 2023), but its realization remains rather challenging due to difficulties of automated evaluation of information veracity (e.g. Aïmeur et al., 2023).”
Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed
Rader, E., & Gray, R. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702174 · Cited by 386 · cited 2× · see citing works
“Among other things, these systems are applied for automated information curation, a process of selecting and presenting content from a pull of data (Rader & Gray, 2015).”
More human than human: measuring ChatGPT political bias
Motoki, F., Pinho Neto, V., & Rodrigues, V. (2024) · Public Choice · doi:10.1007/s11127-023-01097-2 · Cited by 329 · see citing works
Contesting epistemic authority: Conspiracy theories on the boundaries of science
Harambam, J., & Aupers, S. (2015) · Public Understanding of Science · doi:10.1177/0963662514559891 · Cited by 328 · cited 1× · see citing works
“…one-dimensional conceptualizations can suffice when misinformation pertains to factually incorrect information, it is hardly applicable to more nuanced cases, for instance, the ones dealing with ontologically contested subjects, including some conspiracy theories (e.g. Harambam & Aupers, 2015).”
Long-term effectiveness of inoculation against misinformation: Three longitudinal experiments.
Maertens, R., Roozenbeek, J., Basol, M., & van der Linden, S. (2021) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000315 · Cited by 321 · see citing works
AI model GPT-3 (dis)informs us better than humans
Spitale, G., Biller-Andorno, N., & Germani, F. (2023) · Science Advances · doi:10.1126/sciadv.adh1850 · Cited by 241 · see citing works
Scaling up fact-checking using the wisdom of crowds
Allen, J., Arechar, A. A., Pennycook, G., & Rand, D. G. (2021) · Science Advances · doi:10.1126/sciadv.abf4393 · Cited by 234 · see citing works
Deiana, G., Dettori, M., Arghittu, A., Azara, A., Gabutti, G., & Castiglia, P. (2023) · Vaccines · doi:10.3390/vaccines11071217 · Cited by 168 · see citing works
Quantifying and alleviating political bias in language models
Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103654 · Cited by 68 · see citing works
Flagged first, then by citations
Research works printed but not linked: 58
- Aguerri, J. C., & Santisteban, M. (2022). The algorithmic responses to disinformation: A suitable pathway? Justice, Power and Resistance, 5(3), 299-306. https:// doi. org/ 10. 1332/ CQNF2 293 · DOI
- Ahmed, W., Vidal-Alaball, J., Downing, J., & Seguí, F. L. (2020). COVID-19 and the 5G conspir- acy theory: Social network analysis of twitter data. Journal of Medical Internet Research, 22(5), e19458. https:// doi. org/ 10. 2196/ 19458 · DOI
- Alaphilippe, A., Gizikis, A., Hanot, C., & Bontcheva, K. (2019). Automated tackling of disinfor- mation: Major challenges ahead. European Parliament: Directorate General for Parliamentary Research Services. https:// doi. org/ 10. 2861/ 368879 · DOI
- Alieva, I., Ng, L. H. X., & Carley, K. M. (2022). Investigating the spread of Russian disinformation about Biolabs in Ukraine on twitter using social network analysis. IEEE International Conference on Big Data (Big Data), 2022, 1770-1775. https:// doi. org/ 10. 1109/ BigDa ta556 60. 2022. 10020 223 · DOI
- Arechar, A. A., Allen, J., Berinsky, A. J., Cole, R., Epstein, Z., Garimella, K., Gully, A., Lu, J. G., Ross, R. M., Stagnaro, M. N., Zhang, Y., Pennycook, G., & Rand, D. G. (2023). Understanding and combatting misinformation across 16 countries on six continents. Nature Human Behaviour, 7(9), 1502-1513. https:// doi. org/ 10. 1038/ s41562-023-01641-6 · DOI · Paper page
- Au, C. H., Ho, K. K. W., & Chiu, D. K. (2022). The role of online misinformation and fake news in ideological polarization: Barriers, catalysts, and implications. Information Systems Frontiers, 24, 1331-1354. https:// doi. org/ 10. 1007/ s10796-021-10133-9 · DOI
- Bailer, W., Thallinger, G., Backfried, G., & Thomas-Aniola, D. (2021). Challenges for automatic detection of fake news related to migration: Invited paper. IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), 2021, 133-138. https:// doi. org/ 10. 1109/ CogSI MA515 74. 2021. 94759 29 · DOI
- Bandy, J. (2021). Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits (arXiv: 2102. 04256). arXiv. http:// arxiv. org/ abs/ 2102. 04256
- Barchetti, A., Neybert, E., Mantel, S. P., & Kardes, F. R. (2022). The half-truth effect and its impli- cations for sustainability. Sustainability, 14(11), 6943. https:// doi. org/ 10. 3390/ su141 16943 · DOI
- Bastian, M., Makhortykh, M., & Dobber, T. (2019). News personalization for peace: How algorith- mic recommendations can impact conflict coverage. International Journal of Conflict Management, 30(3), 309-328. https:// doi. org/ 10. 1108/ IJCMA-02-2019-0032 · DOI
- Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D. (2024). AI auditing: The broken bus on the road to AI accountability. IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2024, 612-643.
- Blumler, J. G. (2016). The fourth age of political communication: Politiques de. Communication N°, 6(1), 19-30. https:// doi. org/ 10. 3917/ pdc. 006. 0019 · DOI
- Bonart, M., Samokhina, A., Heisenberg, G., & Schaer, P. (2019). An investigation of biases in web search engine query suggestions. Online Information Review, 44(2), 365-381. https:// doi. org/ 10. 1108/ OIR-11-2018-0341 · DOI
- Boulianne, S. (2019). US dominance of research on political communication: A meta-view. Political Communication, 36(4), 660-665. https:// doi. org/ 10. 1080/ 10584 609. 2019. 16708 99 · DOI
- Bountouridis, D., Makhortykh, M., Sullivan, E., Harambam, J., Tintarev, N., & Hauff, C. (2019, July). Annotating credibility: Identifying and mitigating bias in credibility datasets. ROME 2019: Workshop on Reducing Online Misinformation Exposure, Paris France. https:// rome2 019. github. io/ papers/ Bount ourid is_ etal_ ROME2 019. pdf
- Bozdag, E. (2013). Bias in algorithmic filtering and personalization. Ethics and Information Tech- nology, 15(3), 209-227. https:// doi. org/ 10. 1007/ s10676-013-9321-6 · DOI · Paper page
- Bradshaw, S. (2019). Disinformation optimised: Gaming search engine algorithms to amplify junk news. Internet Policy Review, 8(4), 1442. https:// doi. org/ 10. 14763/ 2019.4. 1442 · DOI
- Caramancion, K. M. (2023). News Verifiers Showdown: A Comparative Performance Evaluation of ChatGPT 3.5, ChatGPT 4.0, Bing AI, and Bard in News Fact-Checking (arXiv: 2306. 17176). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2306. 17176 · DOI
- Conspiracy Theory. (2023). In Cambridge Dictionary. Cambridge University Press & Assessment. https:// dicti onary. cambr idge. org/ dicti onary/ engli sh/ consp iracy-theory 23
- Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., & Wright, R. (2023). Opinion Paper: "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https:// doi. org/ 10. 1016/j. ijinf omgt. 2023. 102642 · DOI · Paper page
- Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., Jirotka, M., Johnson, H., LaPointe, C., Llorens, A. J., Mackworth, A. K., Maple, C., Pálsson, S. E., Pasquale, F., Winfield, A., & Yeong, Z. K. (2021). Governing AI safety through independent audits. Nature Machine Intelligence, 3(7), 566-571. https:// doi. org/ 10. 1038/ s42256-021-00370-7 · DOI · Paper page
- Farkas, J., & Schou, J. (2018). Fake news as a floating signifier: hegemony, antagonism and the politics of falsehood. Javnost -The Public, 25(3), 298-314. https:// doi. org/ 10. 1080/ 13183 222. 2018. 14630 47 · DOI
- Freelon, D., & Lokot, T. (2020). Russian disinformation campaigns on Twitter target political communities across the spectrum. Collaboration between opposed political groups might be the most effective way to counter it. Misinformation Review. https:// doi. org/ 10. 37016/ mr-2020-003 · DOI
- Ghosh, S., & Caliskan, A. (2023). ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages. arXiv preprint arXiv: 2305. 10510
- Gräfe, H.-C. (2018). Webtracking und Microtargeting als Gefahr für Demokratie und Medien. PinG Privacy in Germany, 1, 6. https:// doi. org/ 10. 37307/j. 2196-9817. 2019. 01. 06 · DOI
- In J. A. Tucker & N. Persily (Eds.), Social Media and Democracy: The State of the Field, Pros- pects for Reform (pp. 10-33). Cambridge University Press.
- Hameleers, M., van der Meer, T., & Vliegenhart, R. (2022). Civilized truths, hateful lies? Inci- vility and hate speech in false information -evidence from fact-checked statements in the US. Information, Communication & Society, 25(11), 1596-1613. https:// doi. org/ 10. 1080/ 13691 18X. 2021. 18740 38 · DOI
- Hinsley, A., & Holton, A. (2021). Fake news cues: examining the impact of content, source, and typology of news cues on People's confidence in identifying Mis-and disinformation. Interna- tional Journal of Communication, 15, 20.
- Hoes, E., Altay, S., & Bermeo, J. (2023). Leveraging ChatGPT for Efficient Fact-Checking. PsyArXiv. https:// doi. org/ 10. 31234/ osf. io/ qnjkf · DOI
- Jamison, A., Broniatowski, D. A., Smith, M. C., Parikh, K. S., Malik, A., Dredze, M., & Quinn, S. C. (2020). Adapting and extending a typology to identify vaccine misinformation on Twitter. American Journal of Public Health, 110(S3), S331-S339. https:// doi. org/ 10. 2105/ AJPH. 2020. 305940 · DOI
- Jia, F. (2020). Misinformation literature review: definitions, taxonomy, and models. International Journal of Social Science and Education Research, 3(12), 85-90. https:// doi. org/ 10. 6918/ IJOSS ER. 202012_ 3(12). 0011 · DOI
- Kapantai, E., Christopoulou, A., Berberidis, C., & Peristeras, V. (2021). A systematic literature review on disinformation: Toward a unified taxonomical framework. New Media & Society, 23(5), 1301-1326. https:// doi. org/ 10. 1177/ 14614 44820 959296 · DOI
- Kotek, H., Dockum, R., & Sun, D. Q. (2023). Gender bias and stereotypes in Large Language Mod- els. In Proceedings of The ACM Collective Intelligence Conference, 12-24. https:// doi. org/ 10. 1145/ 35822 69. 36155 99 · DOI · Paper page
- Kuznetsova, E., & Makhortykh, M. (2023). Blame it on the algorithm? Russian government-spon- sored media and algorithmic curation of political information on facebook. International Journal of Communication, 17, 971-992.
- Lewandowsky, S. (2021). Climate change disinformation and how to combat it. Annual Review of Public Health, 42(2021), 1-21. https:// doi. org/ 10. 1146/ annur ev-publh ealth-090419-102409 · DOI
- Litvinenko, A. (2023). Propaganda on demand: Russia's media environment during the war in Ukraine. Global Media Journal -German Edition, 12, no. 2. https:// doi. org/ 10. 22032/ DBT. 55518 · DOI
- Makhortykh, M., Zucker, E. M., Simon, D. J., Bultmann, D., & Ulloa, R. (2023). Shall androids dream of genocides? How generative AI can change the future of memorialization of mass atroci- ties. Discover Artificial Intelligence, 3(1), 28. https:// doi. org/ 10. 1007/ s44163-023-00072-6 · DOI · Paper page
- Meskó, B., & Topol, E. J. (2023). The imperative for regulatory oversight of large language mod- els (or generative AI) in healthcare. Npj Digital Medicine, 6(1), 120. https:// doi. org/ 10. 1038/ s41746-023-00873-0 · DOI · Paper page
- Mittelstadt, B. (2016). Automation, algorithms, and politics| auditing for transparency in content personalization systems. International Journal of Communication, 10(2016), 4991-5002.
- Moskvichev, A., Odouard, V. V., & Mitchell, M. (2023). The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain. Arxiv.org. https:// arxiv. org/ abs/ 2305. 07141
- Mumtarin, M., Chowdhury, M. S., & Wood, J. (2023). Large Language Models in Analyzing Crash Narratives-A Comparative Study of ChatGPT, BARD and GPT-4. arXiv: 2308. 13563
- Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., & Mian, A. (2023). A compre- hensive overview of large language models. arXiv preprint arXiv: 2307. 06435
- Roozenbeek, J., Schneider, C. R., Dryhurst, S., Kerr, J., Freeman, A. L. J., Recchia, G., van der Bles, A. M., & van der Linden, S. (2020). Susceptibility to misinformation about COVID-19 around the world. Royal Society Open Science, 7(10), 201199. https:// doi. org/ 10. 1098/ rsos. 201199 · DOI · Paper page
- Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H. R., Schütze, H., & Hovy, D. (2024). Polit- ical Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models (arXiv: 2402. 16786). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2402. 16786 · DOI
- Rozado, D. (2024). The Political Preferences of LLMs (arXiv: 2402. 01789). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2402. 01789 · DOI
- Ruokolainen, H., Widén, G., & Eskola, E.-L. (2023). How and why does official information become misinformation? A typology of official misinformation. Library & Information Science Research, 45(2), 101237. https:// doi. org/ 10. 1016/j. lisr. 2023. 101237 · DOI
- Rutinowski, J., Franke, S., Endendyk, J., Dormuth, I., Roidl, M., & Pauly, M. (2024). The self- perception and political biases of ChatGPT. Human Behavior and Emerging Technologies, 2024(1), 7115633.
- Samoilenko, S. A., & Cook, J. (2024). Developing an Ad Hominem typology for classifying climate misinformation. Climate Policy, 24(1), 138-151. https:// doi. org/ 10. 1080/ 14693 062. 2023. 22457 92 · DOI
- Saurwein, F., & Spencer-Smith, C. (2020). Combating disinformation on social media: multilevel governance and distributed accountability in Europe. Digital Journalism, 8(6), 820-841. https:// doi. org/ 10. 1080/ 21670 811. 2020. 17654 01 · DOI
- Søe, S. O. (2021). A unified account of information, misinformation, and disinformation. Synthese, 198(6), 5929-5949. https:// doi. org/ 10. 1007/ s11229-019-02444-x · DOI
- Strand, C., & Svensson, J. (2021). Disinformation campaigns about LGBTI+ people in the EU and foreign influence (pp. 1-28) [Briefing]. European Parlament, Policy Department for External Rela- tions. https:// dspace. ceid. org. tr/ xmlui/ bitst ream/ handle/ 1/ 1805/ QA092 1283E NN. en. pdf? seque nce= 1& isAll owed=y
- Tao, Y., Agrawal, A., Dombi, J., Sydorenko, T., & Lee, J. I. (2024). ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness. https:// arxiv. org/ abs/ 2403. 18121
- Törnberg, P., Valeeva, D., Uitermark, J., & Bail, C. (2023). Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms. arXiv preprint arXiv: 2310. 05984
- Unkel, J., & Haas, A. (2017). The effects of credibility cues on the selection of search engine results. Journal of the Association for Information Science and Technology, 68(8), 1850-1862. https:// doi. org/ 10. 1002/ asi. 23820 · DOI
- Urman, A., & Makhortykh, M. (2023). The Silence of the LLMs: Cross-Lingual Analysis of Political Bias and False Information Prevalence in ChatGPT, Google Bard, and Bing Chat. https:// doi. org/ 10. 31219/ osf. io/ q9v8f · DOI
- Vidgen, B., Scherrer, N., Kirk, H. R., Qian, R., Kannappan, A., Hale, S. A., & Röttger, P. (2023). SimpleSafetyTests: A Test Suite for Identifying Critical Safety Risks in Large Language Models. https:// doi. org/ 10. 48550/ ARXIV. 2311. 08370 · DOI
- Wan, Y., Pu, G., Sun, J., & Garimella, A. (2023). "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters. https:// doi. org/ 10. 48550/ arXiv. 2310. 09219 · DOI · Paper page
- Yang, E., & Roberts, M. E. (2023). The authoritarian data problem. Journal of Democracy, 34(4), 141-150. https:// doi. org/ 10. 1353/ jod. 2023. a9076 95 · DOI
Cited by
Sorted by citations
Park, S., & Nan, X. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02620-3 · Cited by 58 · see citing works
ChatGPT for President! Presupposed content in politicians versus GPT-generated texts
Garassino, D., Brocca, N., & Masia, V. (2025) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2025.100156 · Cited by 2 · see citing works
"Just Asking Questions": Doing Our Own Research on Conspiratorial Ideation by Generative AI Chatbots
FitzGerald, K. M., Riedlinger, M., Bruns, A., Harrington, S., Graham, T., & Angus, D. (2026) · Media and Communication · doi:10.17645/mac.11337 · Cited by 1 · see citing works
“…conspiratorial narratives associated with COVID-19, the Russian aggression against Ukraine, the Holocaust, climate change, and debates related to LGBTQ +, as compared with ChatGPT prompted in Ukrainian and with Bing Chat, which showed a decrease in responsiveness (Kuznetsova et al., 2025).”
Weatherhead, S., Salim, F., & Belbasis, A. (2025) · arXiv · doi:10.48550/arxiv.2510.18254
When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMs
Mehdizadeh, A., & Hilbert, M. (2025) · arXiv · doi:10.48550/arxiv.2510.19107
“Future studies should replicate our experiments in multiple languages to determine if the persuasion asymmetry and dual cognitive hierarchies are universal properties or are modulated by linguistic and cultural contexts embedded in the training data [159].”
- Authors
- Kuznetsova, E., Makhortykh, M., Vziatysheva, V., Stolze, M., Baghumyan, A., & Urman, A.
- Affiliations
- Weizenbaum Institute (DE)
- University of Bern (CH)
- University of Zurich (CH)
- Corresponding author
- Kuznetsova, Elizaveta
- ORCID
- Venue
- Journal of Computational Social Science
- Published
- 2024-12-17
- Volume
- 8
- Issue
- 1
- ISSN
- 2432-2717 (ISSN-L)
- 2432-2725
- OpenAlex
- W4405478483
- Cited by
- 32
- References
- 14
- Citations by year
- 2026: 17 · 2025: 15
- FWCI
- 22.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Journal of Computational Social Science (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Bundesministerium für Bildung und Forschung (grant no.: 16DII131 – “Weizenbaum-Institut”)
- Weizenbaum-Institut e.V.
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, chatbots, disinformation detection, misinformation, conspiracy theories, ChatGPT, Bing Chat, Microsoft Copilot, artificial intelligence in auditing, source attribution, low-resource languages, Russian language, Ukrainian language, COVID-19 pandemic, Russian aggression against Ukraine, Holocaust, climate change
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review