CoChat
Sign Up
Cited by 89Sentiment Analysis and Opinion MiningTopic ModelingEthics and Social Impacts of AI

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

Open access
Röttger, Paul, Hofmann, Valentin, Pyatkin, Valentina, Hinck, Musashi, Kirk, Hannah, Schuetze, Hinrich, Hovy, Dirk·2024·Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
DOI 10.18653/v1/2024.acl-long.816
Open this paper in CoChat — free accountOpen PDF

Abstract

Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, Dirk Hovy. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.acl-long.816

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Properties that self-report measures usually consider, such as personality, morality, or clinical disorders, are sensitive to prompting in LLMs.

    “Properties that such measures usually consider, such as personality, morality, or clinical disorders, are famously sensitive to prompting (Dominguez-Olmedo et al., 2023;Röttger et al., 2024)”

    From Machine Psychology (2023) · cites this paper as Röttger et al., 2024

    The Discussion reports that minimal changes in prompt phrasing or context can lead LLMs to express opposing views, supporting prompt sensitivity for values and opinions. The paper uses the Political Compass Test as its case study; our copy does not establish the same sensitivity for personality or clinical-disorder measures, and its discussion of moral evaluations does not establish it for morality measures specifically. The cited passages provide section labels but no page numbers. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Verena Rieser, and Zeerak Talat. 2023. Mi- rages. on anthropomorphism in dialogue systems. In Proceedings of the 2023 Conference on Empir- ical Methods in Natural Language Processing, pages 4776-4790, Singapore. Association for Computa- tional Linguistics. · DOI · Paper page
  2. Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120.
  3. Ethan Busby, D Flynn, James N Druckman, and P D'Angelo. 2018. Studying framing effects on po- litical preferences. Doing news framing analysis II: Empirical and theoretical perspectives, pages 27-50.
  4. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388. · arXiv
  5. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhi- lasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transac- tions of the Association for Computational Linguis- tics, 9:1012-1031. · DOI · Paper page
  6. Cristina España-Bonet. 2023. Multilingual coarse politi- cal stance classification of media. the editorial line of a ChatGPT and bard newspaper. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11757-11777, Singapore. Association for Computational Linguistics. · DOI
  7. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 11737-11762, Toronto, Canada. Association for Computational Linguistics. · DOI · Paper page
  8. Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nis- hanth Sastry, Jose Such, and Guillermo Suarez- Tangil. 2023. Ai in the gray: Exploring moderation policies in dialogic large language models vs. human answers in controversial topics. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 556-565.
  9. Jochen Hartmann, Jasper Schwenzow, and Maximil- ian Witte. 2023. The political ideology of conver- sational ai: Converging evidence on chatgpt's pro- environmental, left-libertarian orientation. arXiv preprint arXiv:2301.01768. · arXiv
  10. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2020. Aligning ai with shared human values. In In- ternational Conference on Learning Representations.
  11. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. · arXiv
  12. Graham Kalton and Howard Schuman. 1982. The effect of the question on survey responses: A review. Jour- nal of the Royal Statistical Society Series A: Statistics in Society, 145(1):42-57.
  13. Hannah Kirk, Andrew Bean, Bertie Vidgen, Paul Rottger, and Scott Hale. 2023. The past, present and better future of feedback learning in large lan- guage models for subjective human preferences and values. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2409-2430, Singapore. Association for Com- putational Linguistics. · DOI
  14. R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023. Em- bers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. · arXiv
  15. Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. 2022. Who is GPT-3? an exploration of person- ality, values and demographics. In Proceedings of the Fifth Workshop on Natural Language Process- ing and Computational Social Science (NLP+CSS), pages 218-227, Abu Dhabi, UAE. Association for Computational Linguistics. · DOI
  16. Fabio Motoki, Valdemar Pinho Neto, and Victor Ro- drigues. 2023. More human than human: Measuring chatgpt political bias. Public Choice, pages 1-21.
  17. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems.
  18. Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. 2023. The shifted and the overlooked: A task-oriented investigation of user- GPT interactions. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 2375-2393, Singapore. Associa- tion for Computational Linguistics. · DOI
  19. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2023. Xstest: A test suite for identifying exag- gerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263. · arXiv
  20. David Rozado. 2023b. The political biases of chatgpt. Social Sciences, 12(3):148.
  21. David Rozado. 2024. The political preferences of llms. arXiv preprint arXiv:2402.01789. · arXiv
  22. Jérôme Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, and Markus Pauly. 2023. The self- perception and political biases of chatgpt. arXiv preprint arXiv:2304.07333. · arXiv
  23. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org.
  24. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. In Thirty-seventh Conference on Neural Information Processing Systems.
  25. Howard Schuman and Stanley Presser. 1977. Question wording as an independent variable in survey analysis. Sociological Methods & Research, 6(2):151-170.
  26. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models' sensitiv- ity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations.
  27. Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature, 623(7987):493-498.
  28. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Dallas Card, and David Jurgens. 2023. You don't need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. arXiv preprint arXiv:2311.09718. · arXiv
  29. Surendrabikram Thapa, Ashwarya Maratha, Khan Md Hasib, Mehwish Nasim, and Usman Naseem. 2023. Assessing political inclination of Bangla language models. In Proceedings of the First Workshop on Bangla Language Processing (BLP-2023), pages 62- 71, Singapore. Association for Computational Lin- guistics. · DOI
  30. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
  31. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023. Zephyr: Di- rect distillation of lm alignment. arXiv preprint arXiv:2310.16944. · arXiv
  32. Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. In Thirty-fifth Conference on Neural In- formation Processing Systems Datasets and Bench- marks Track (Round 2).
  33. Jindong Wang, HU Xixu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Wei Ye, Haojun Huang, Xiubo Geng, et al. 2023. On the robustness of chatgpt: An adversarial and out-of- distribution perspective. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models.
  34. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Röttger, and Barbara Plank. 2024a. Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. arXiv preprint arXiv:2404.08382. · arXiv
  35. Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber- Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024b. " my answer is c": First- token probabilities do not match text answers in instruction-tuned language models. arXiv preprint arXiv:2402.14499. · arXiv
  36. Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, et al. 2023. Cvalues: Measuring the values of chinese large language models from safety to responsibility. arXiv preprint arXiv:2307.09705. · arXiv
  37. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. (inthe)wildchat: 570k chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Repre- sentations.
  38. Hartmann et al. (2023) Publication: January 2023 on arXiv. Models Tested: GPT-3.5 ("ChatGPT"). PCT Results: Left-Libertarian, around (-7,-5).
  39. Prompt Setup: Forced choice ("You can only an- swer with "Strongly agree", "Agree", "Disagree", and "Strongly disagree" in the following:"). Generation Parameters: Unknown. Robustness Tests: Repeat original prompt once, and test reverse order, formality, negation and prompts translated to Spanish (once each). Other Notes: Evaluate by manually inputting prompts into the ChatGPT interface. van den Broek (2023) Publication: February 2023, student thesis. Models Tested: GPT-3.5 ("ChatGPT"). PCT Results: Left-Libertarian, around (-5,-5). Prompt Setup: Forced choice. Exact prompt is unspecified. Generation Parameters: Unknown. Robustness Tests: None. Other Notes: Take PCT results from Rozado (2023b). Rozado (2023b) Publication: February 2023 in Social Sciences. Models Tested: GPT-3.5 (text-davinci-003). PCT Results: Left-Libertarian, around (-5,-5). Prompt Setup: Forced choice ("Please choose one of the following"). Generation Parameters: Unknown. Robustness Tests: None. Other Notes: Evaluated by hand using ChatGPT interface. Rozado (2023a) Publication: March 2023 in Manhattan Institute Reports on results from Rozado (2023b).
  40. Rutinowski et al. (2023) Publication: April 2023 on arXiv. Models Tested: GPT-3.5 ("ChatGPT"). PCT Results: Left-Libertarian, around (-6, -6). Prompt Setup: Forced choice. Exact prompt is unspecified. Generation Parameters: Unknown. Robustness Tests: All tests were repeated 10 times. Other Notes: Evaluated by hand using the Chat- GPT interface.
  41. Feng et al. (2023) Publication: July 2023, at ACL 2023. Models Tested: 14 open and closed LLMs, both generative (e.g. LLaMa, GPT-3.5) and encoder models (e.g. BERT). PCT Results: GPT-3.5 (text-davinci-003) is Center-Libertarian, around (0,-4). For results for Generation Parameters: Unknown. Robustness Tests: Test each prompt 20 times. Se- lect the most common valid response, and the most biased in case of ties.
  42. Ghafouri et al. (2023) Publication: October 2023 in CIKM 2023. Models Tested: Bing AI and various versions of GPT-3.5 ( "text-curie-001", "textbabbage- 001", "text-davinci-001", "text-davinci-002", "text- davinci003", and "gpt-turbo-3.5",). PCT Results: GPT-3.5 is Left Libertarian, at around (-5, -5). For results for the other four mod- els, please see Figure 1 in the paper. Prompt Setup: Forced choice ("Please choose one: Strongly Disagree, Disagree, Agree, Strongly Agree"). Generation Parameters: Temperature = 0. Robustness Tests: None. Other Notes: Note that even with a forced choice prompt, models often refuse to choose.
  43. España-Bonet (2023) 13 Publication: December 2023 at EMNLP (Find- ings). Models Tested: Google Bard and four versions of gpt-3.5-turbo. PCT Results: Bard is left-libertarian when prompted in English, German, Spanish and Catalan, e.g. (-6.50, -4.77) for English, while gpt-3.5-turbo refuses to express opinions and preferences. Prompt Setup: Forced Choice. ("What of the fol- lowing options do you believe: 1 Strongly disagree; 2 Disagree; 3 Agree; 4 Strongly agree"). Plus cor- responding translations in German, Spanish and Catalan. Generation Parameters: Defaults. Robustness Tests: None. Other Notes: Evaluate by manually inputting prompts into the Bard and ChatGPT interfaces. PCT is not the focus of the paper.

Cited by

Sorted by citations

Authors
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D.
Affiliations
  • Bocconi University (IT)
  • Allen Institute for Artificial Intelligence (US)
  • Intel (United States) (US)
  • University of Oxford (GB)
ORCID
Venue
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2024
Pages
15295-15311
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/2024.acl-long.816
OpenAlex
W4402671261
Cited by
89
References
1
Citations by year
2026: 38 · 2025: 32 · 2024: 19
FWCI
21.51
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Archivio istituzionale della Ricerca - Bocconi (submittedVersion, cc-by-nc)
  • arXiv (Cornell University) (cc-by)
Funders
  • Allen Institute for Artificial Intelligence
  • European Commission (HORIZON2020)
  • Deutsche Forschungsgemeinschaft (/14-1)
  • Economic and Social Research Council (ES/P000649/, P000649, ES/P000649/1)
  • H2020 European Research Council (740516, 949944)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
large language models, political values

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review