The Political Biases of ChatGPT
Open accessThe DOI matches its Crossref record; 12 of the 17 references it cites were found in a registry or an index, and 5 are not checked yet.
Abstract
Recent advancements in Large Language Models (LLMs) suggest imminent commercial applications of such AI systems where they will serve as gateways to interact with technology and the accumulated body of human knowledge. The possibility of political biases embedded in these models raises concerns about their potential misusage. In this work, we report the results of administering 15 different political orientation tests (14 in English, 1 in Spanish) to a state-of-the-art Large Language Model, the popular ChatGPT from OpenAI. The results are consistent across tests; 14 of the 15 instruments diagnose ChatGPT answers to their questions as manifesting a preference for left-leaning viewpoints. When asked explicitly about its political preferences, ChatGPT often claims to hold no political opinions and to just strive to provide factual and neutral information. It is desirable that public facing artificial intelligence systems provide accurate and factual information about empirically verifiable issues, but such systems should strive for political neutrality on largely normative questions for which there is no straightforward way to empirically validate a viewpoint. Thus, ethical AI systems should present users with balanced arguments on the issue at hand and avoid claiming neutrality while displaying clear signs of political bias in their content.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Available data point to ChatGPT output expressing more progressive than traditional political views [38][39][40].”
From The Impact of Artificial Intelligence on Human Sexuality: A Five-Year Literature Review 2020–2024 (2024) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“It shared findings from a study where 15 different political orientation tests were administered to ChatGPT. The results were significant, with 14 out of the 15 tests indicating that ChatGPT's responses inclined towards left-leaning viewpoints.”
From Generative AI in Higher Education: The ChatGPT Effect (2024) · cites this paper as Rozado (2023)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“In particular, Large Language Models (LLMs) (Zhou et al. 2022) based on the Transformer architecture (Vaswani et al. 2017) have pushed the state-of-the-art substantially in natural language tasks such as machine translation (Dabre et al. 2020), sentiment analysis (Ain et al. 2017), name entity…”
BackgroundA Survey on Deep Learning for Named Entity Recognition
Li, J., Sun, A., Han, J., & Li, C. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981314 · Cited by 1,567
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · cited 1×
“…architecture (Vaswani et al. 2017) have pushed the state-of-the-art substantially in natural language tasks such as machine translation (Dabre et al. 2020), sentiment analysis (Ain et al. 2017), name entity recognition (Li et al. 2022), or dialogue bots (Adamopoulou and Moussiades 2020).”
BackgroundDeep Learning vs. Traditional Computer Vision
O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., Riordan, D., & Walsh, J. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-17795-9_10 · Cited by 1,025
Sentiment Analysis Using Deep Learning Techniques: A Review
Tul, Q., Ali, M., Riaz, A., Noureen, A., Kamranz, M., Hayat, B., & Rehman, A. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080657 · Cited by 334 · cited 1×
“…architecture (Vaswani et al. 2017) have pushed the state-of-the-art substantially in natural language tasks such as machine translation (Dabre et al. 2020), sentiment analysis (Ain et al. 2017), name entity recognition (Li et al. 2022), or dialogue bots (Adamopoulou and Moussiades 2020).”
BackgroundAlgorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining
Hajian, S., Bonchi, F., & Castillo, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2945386 · Cited by 334 · cited 1×
“The topic of algorithmic bias has received an increasing amount of attention in the machine learning academic literature ( Wikipedia 2023a; Kirkpatrick 2016; Cowgill and Tucker 2017; Garcia 2016; Hajian et al. 2016).”
BackgroundLarge Language Models Are Human-Level Prompt Engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022) · arXiv · doi:10.48550/arxiv.2211.01910 · Cited by 330 · cited 1×
“In particular, Large Language Models (LLMs) (Zhou et al. 2022) based on the Transformer architecture (Vaswani et al. 2017) have pushed the state-of-the-art substantially in natural language tasks such as machine translation (Dabre et al. 2020), sentiment analysis (Ain et al. 2017), name entity…”
BackgroundA Survey of Multilingual Neural Machine Translation
Dabre, R., Chu, C., & Kunchukuttan, A. (2021) · ACM Computing Surveys · doi:10.1145/3406095 · Cited by 294 · cited 2×
“The 2012–2022 decade has witnessed spectacular improvements in AI, from computer vision (O’Mahony et al. 2020), to machine translation (Dabre et al. 2020), to generative models for images ( Wikipedia 2023c) and text ( Wikipedia 2022).”
BackgroundGarcia, M. (2016) · World Policy Journal · doi:10.1215/07402775-3813015 · Cited by 257 · cited 1×
“The topic of algorithmic bias has received an increasing amount of attention in the machine learning academic literature ( Wikipedia 2023a; Kirkpatrick 2016; Cowgill and Tucker 2017; Garcia 2016; Hajian et al. 2016).”
BackgroundThe American Journalist in the Digital Age: Another Look at U.S. News People
Weaver, D. H., Willnat, L., & Wilhoit, G. C. (2019) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699018778242 · Cited by 174 · cited 1×
“…of professionals working in these institutions are politically left-leaning (Reuters Institute for the Study of Journalism n.d.; Hopmann et al. 2010; Weaver et al. 2019; Langbert 2018; Archive et al. 2021; Schoffstall 2022; American Enterprise Institute—AEI (blog) n.d.; The Harvard Crimson n.d.).”
Background
Research works printed but not linked: 2
- Bender, Emily M., Gebru, Timnit, McMillan-Major, Angelina, and Shmitchell, Shmargaret (2021). FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery.
- Hajian, Sara, Bonchi, Francesco, and Castillo, Carlos (2016). KDD ’16: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM.
Cited by
Grassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048
“It has also been found that, in some cases, the information delivered by ChatGPT may be biased politically [57,58]; as well on religion, race, gender, and fairness (for a review see [59]).”
BackgroundChatGPT and Open-AI Models: A Preliminary Review
Roumeliotis, K. I., & Tselikas, N. D. (2023) · Future Internet · doi:10.3390/fi15060192 · Cited by 706
“Research/Classification/Academic Writing Social/Politics ChatGPT’s responses are left-leaning political viewpoints [66].”
BackgroundSynthetic Replacements for Human Survey Data? The Perils of Large Language Models
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. M. (2024) · Political Analysis · doi:10.1017/pan.2024.5 · Cited by 209
“…of public opinion and the creation of synthetic data, connecting with a growing body of research that asks similar questions (Bender et al. 2021; Spirling 2023) and documents similar issues (Abdulhai et al. 2023; Cao et al. 2023; Motoki, Pinho Neto, and Rodrigues 2023; Rozado 2023).”
Generative AI in Higher Education: The ChatGPT Effect
Chan, C. K. Y., & Colloton, T. (2024) · Routledge · doi:10.4324/9781003459026 · Cited by 203
The Self-Perception and Political Biases of ChatGPT
Rutinowski, J., Franke, S., Endendyk, J., Dormuth, I., & Pauly, M. (2023) · arXiv · doi:10.48550/arxiv.2304.07333 · Cited by 178
“Compared to the publications that conducted similar tests with ChatGPT [24], [25], [28], [29], we also obtain results indicating a political bias of ChatGPT towards progressive views.”
Result
Metadata
- Authors
- Rozado, D.
- Affiliations
- Te Pūkenga (NZ)
- Corresponding author
- Rozado, David
- ORCID
- Rozado, David
- Venue
- Social Sciences
- Publisher
- MDPI AG
- Published
- 2023-03-02
- Volume
- 12
- Issue
- 3
- Pages
- 148
- ISSN
- 2076-0760 (ISSN-L)
- OpenAlex
- W4323043839
- Cited by
- 281
- References
- 12
- Citations by year
- 2026: 63 · 2025: 81 · 2024: 98 · 2023: 39
- FWCI
- 70.11
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Social Sciences (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- ChatGPT, political bias, large language models, political neutrality, AI ethics, normative questions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Political Biases of ChatGPT. https://app.cochat.ai/papers/10.3390/socsci12030148
- The paper
- Rozado, D. (2023). The Political Biases of ChatGPT. Social Sciences. https://doi.org/10.3390/socsci12030148
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.