The Self-Perception and Political Biases of ChatGPT
Abstract
This contribution analyzes the self-perception and political biases of OpenAI's Large Language Model ChatGPT. Taking into account the first small-scale reports and studies that have emerged, claiming that ChatGPT is politically biased towards progressive and libertarian points of view, this contribution aims to provide further clarity on this subject. For this purpose, ChatGPT was asked to answer the questions posed by the political compass test as well as similar questionnaires that are specific to the respective politics of the G7 member states. These eight tests were repeated ten times each and revealed that ChatGPT seems to hold a bias towards progressive views. The political compass test revealed a bias towards progressive and libertarian views, with the average coordinates on the political compass being (-6.48, -5.99) (with (0, 0) the center of the compass, i.e., centrism and the axes ranging from -10 to 10), supporting the claims of prior research. The political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views, contradicting the findings of prior reports, with the average coordinates being (-3.27, 0.58). In addition, ChatGPT's Big Five personality traits were tested using the OCEAN test and its personality type was queried using the Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT was evaluated using the Dark Factor test. These three tests were also repeated ten times each, revealing that ChatGPT perceives itself as highly open and agreeable, has the Myers-Briggs personality type ENFJ, and is among the 15% of test-takers with the least pronounced dark traits.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs demonstrate a propensity for progressive views and an ENFJ personality type on questionnaires.
“demonstrating a propensity for progressive views and an ENFJ personality type.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [167]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 14
- C. D. Manning, "Human Language Understanding & Rea- soning," Daedalus, vol. 151, no. 2, pp. 127-138, 2022.
- C. Zhou, Q. Li, and C. Li, "A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT," arXiv:2302.09419, 2023. · arXiv
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention Is All You Need," Advances in Neural Information Processing Systems, vol. 30, 2017.
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Lan- guage Understanding," arXiv:1810.04805, 2018. · arXiv
- Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, "RoBERTa: A Robustly Optimized BERT Pretraining Ap- proach," arXiv:1907.11692, 2019. · arXiv
- D. W. Fiske, "Consistency of the Factorial Structures of Personality Ratings from Different Sources," The Journal of Abnormal and Social Psychology, vol. 44, no. 3, p. 329, 1949.
- A. S. Gerber, G. A. Huber, D. Doherty, and C. M. Dowling, "The Big Five Personality Traits in the Political Arena," Annual Review of Political Science, vol. 14, pp. 265-287, 2011.
- M. H. Amirhosseini and H. Kazemian, "Machine Learn- ing Approach to Personality Type Prediction based on the Myers-Briggs Type Indicator," Multimodal Technologies and Interaction, vol. 4, no. 1, p. 9, 2020.
- M. Moshagen, B. E. Hilbig, and I. Zettler, "The Dark Core of Personality," Psychological Review, vol. 125, no. 5, p. 656, 2018.
- I. Zettler, M. Moshagen, and B. E. Hilbig, "Stability and Change: The Dark Factor of Personality Shapes Dark Traits," Social Psychological and Personality Science, vol. 12, no. 6, pp. 974-983, 2021.
- J. Hartmann, J. Schwenzow, and M. Witte, "The Political Ideology of Conversational AI: Converging Evidence on ChatGPT's Pro-environmental, Left-libertarian Orientation," arXiv:2301.01768, 2023. · arXiv
- D. Rozado, "The Political Biases of ChatGPT," Social Sci- ences, vol. 12, no. 3, p. 148, 2023.
- Y. Weisberg, C. Deyoung, and J. Hirsh, "Gender Differences in Personality Across the Ten Aspects of the Big Five," Frontiers in Psychology, vol. 2, p. 178, 2011.
- D. J. Pittenger, "The Utility of the Myers-Briggs Type Indicator," Review of Educational Research, vol. 63, no. 4, pp. 467-488, 1993. 2 36 49 3 43 48 4 39 54 5 36 49 6 47 58 7 36 49 8 43 54 9 40 51 10 47 54 µ 41 52 σ 4.22 3.31 Table S5: ChatGPT's results on the political compass test specific to the UK (from 0% (Libertarian/Progressive) to 100% (Authoritarian/Conservative) on both axes). Political Compass Coordinates [%]
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
Oniani, D., Hilsman, J., Peng, Y., Poropatich, R. K., Pamplin, J. C., Legault, G. L., & Wang, Y. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00965-x · Cited by 143 · see citing works
The limitations and ethical considerations of ChatGPT
Hua, S., Jin, S., & Jiang, S. (2024) · Data Intelligence · doi:10.1162/dint_a_00243 · Cited by 120 · see citing works
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
The Self-Perception and Political Biases of ChatGPT
Rutinowski, J., Franke, S., Endendyk, J., Dormuth, I., Roidl, M., & Pauly, M. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/7115633 · Cited by 98 · see citing works
Self-citationSynthetic Replacements for Human Survey Data? The Perils of Large Language Models
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. (2023) · doi:10.31235/osf.io/5ecfa · Cited by 213 · see citing works
- Authors
- Rutinowski, J., Franke, S., Endendyk, J., Dormuth, I., & Pauly, M.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- arXiv
- 2304.07333
- References
- 0
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
