CoChat
Sign Up
PreprintCited by 50Ethics and Social Impacts of AITopic ModelingNatural Language Processing Techniques

The Ghost in the Machine has an American accent: value conflict in GPT-3

Open access
Johnson, Rebecca L, Pistilli, Giada, Menédez-González, Natalia, Duran, Leslye Denisse Dias, Panai, Enrico, Kalpokiene, Julija, Bertulfo, Donald Jay·2022·arXiv
DOI 10.48550/arxiv.2203.07785arXiv 2203.07785
Open this paper in CoChat — free accountOpen PDF

Abstract

The alignment problem in the context of large language models must consider the plurality of human values in our world. Whilst there are many resonant and overlapping values amongst the world's cultures, there are also many conflicting, yet equally valid, values. It is important to observe which cultural values a model exhibits, particularly when there is a value conflict between input prompts and generated outputs. We discuss how the co-creation of language and cultural value impacts large language models (LLMs). We explore the constitution of the training data for GPT-3 and compare that to the world's language and internet access demographics, as well as to reported statistical profiles of dominant values in some Nation-states. We stress tested GPT-3 with a range of value-rich texts representing several languages and nations; including some with values orthogonal to dominant US public opinion as reported by the World Values Survey. We observed when values embedded in the input text were mutated in the generated outputs and noted when these conflicting values were more aligned with reported dominant US values. Our discussion of these results uses a moral value pluralism (MVP) lens to better understand these value mutations. Finally, we provide recommendations for how our work may contribute to other current work in the field.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2203.07785

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The amount of training data for GPT in French and Italian is comparable but substantially smaller than in English.

    “Considering that the amount of training data for GPT in French and Italian is comparable -but substantially smaller than in English (Johnson et al. 2022; Lai et al., 2023)”

    From ChatGPT for President! Presupposed content in politicians versus GPT-generated texts (2025) · cites this paper as Johnson et al. 2022

    Section 1.3 (p. 4) says English is the dominant language in GPT-3’s training data, supporting the comparison with English. But the text available in our copy does not say that the amounts of French and Italian training data are comparable; Figure 1’s chart is not present as readable text in the copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Language models support certain cultural values more than others.

    “and support certain cultural values more than others [57]”

    From Co-Writing with Opinionated Language Models Affects Users’ Views (2023) · cites this paper as [57]

  • A first exploration of the opinions built into GPT-3 by Johnson et al. suggests that the model's preferred views align with dominant US public opinion.

    “A first exploration of the opinions built into GTP-3 by Johnson et al. [57] suggests that the model's preferred views align with dominant US public opinion.”

    From Co-Writing with Opinionated Language Models Affects Users’ Views (2023) · cites this paper as [57]

  • Small-n audits have suggested that the values embedded in models like GPT-3 were more aligned with reported dominant US values than those upheld in other cultures.

    “Small-n audits have also suggested that the values embedded in models like GPT-3 were more aligned with reported dominant US values than those upheld in other cultures [57].”

    From Co-Writing with Opinionated Language Models Affects Users’ Views (2023) · cites this paper as [57]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 57

  1. [1][1] Floridi, L. and Chiriatti, M. GPT-3: Its nature, scope, limits, and consequences . Minds and Machines, 30, 4 (2020), 681-694.
  2. [2][2] Abid, A., Farooqi, M. and Zou, J. Persistent anti-muslim bias in large language models . arXiv:2101.05783 (2021). · arXiv
  3. [3][3] Solaiman, I. and Dennison, C. Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets . arXiv:2106.10328 (2021). · arXiv
  4. [4][4] Tamkin, A., Brundage, M., Clark, J. and Ganguli, D. Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models . arXiv:2102.02503 (2021). · arXiv
  5. [7][7] Jonkers, P. How to Respond to Conflicts Over Value Pluralism? Journal of Nationalism, Memory, and LanguagePolitics (2019).
  6. [9][9] Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A. and Brunskill, E. On the opportunities and risks of foundation models . arXiv:2108.07258 (2021). · arXiv
  7. [10][10] Hume, D. A treatise of human nature . Clarendon Press, 1896.
  8. [11][11] Gabriel, I. Artificial intelligence, values, and alignment . Minds and Machines, 30, 3 (2020), 411-437.
  9. [12][12] Russell, S. Human-compatible artificial intelligence. Human-Like Machine Intelligence, Oxford University Press Oxford, 2021.
  10. [13][13] Wiener, N. Some Moral and Technical Consequences of Automation: As machines learn they may develop unforeseen strategies at rates that baffle their programmers . Science, 131, 3410 (1960), 1355-1358.
  11. [14][14] Habermas, J. Moral consciousness and communicative action . MIT press, 1990.
  12. [16][16] Peeters, B. Language and cultural values . International Journal of Language and Culture, 2, 2 (2015), 133-141.
  13. [17][17] Lakoff, G. Women, fire, and dangerous things : what categories reveal about the mind . University of Chicago Press, Chicago, 1987.
  14. [18][18] Stearn, R. L. Technology and world trade : proceedings of a symposium, November 16-17, 1966 . U.S. Department of Commerce, National Bureau of Standards, Washington, 1967.
  15. [19][19] Lakoff, G. and Johnson, M. Metaphors we live by . University of Chicago press, 2008.
  16. [20][20] Peeters, B. Tall poppies and egalitarianism in Australian discourse: From key word to cultural value . English worldwide, 25, 1 (2004), 1-25.
  17. [22][22] Walker, J., Tepper, S. J. and Gilovich, T. People are more tolerant of inequality when it is expressed in terms of individuals rather than groups at the top . Proceedings of the National Academy of Sciences, 118, 43 (2021).
  18. [23][23] Stephens, G. J., Silbert, L. J. and Hasson, U. Speakerlistener neural coupling underlies successful communication . Proceedings of the National Academy of Sciences, 107, 32 (2010), 14425-14430.
  19. [24][24] Clark, H. H. Using language . Cambridge university press, 1996.
  20. [25][25] Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. and Kalai, A. T. Man is to computer programmer as woman is to homemaker? debiasing word embeddings . Advances in neural information processing systems, 29 (2016).
  21. [27][27] Topal, M. O., Bas, A. and van Heerden, I. Exploring transformers in natural language generation: Gpt, bert, and xlnet . arXiv:2102.08036 (2021). · arXiv
  22. [28][28] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. and Polosukhin, I. Attention is all you need. In Proceedings of the Advances in neural information processing systems, 2017
  23. [29][29] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. and Galstyan, A. A survey on bias and fairness in machine learning . ACM Computing Surveys (CSUR), 54, 6 (2021), 135.
  24. [30][30] Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B. and Kasirzadeh, A. Ethical and social risks of harm from Language Models . arXiv:2112.04359 (2021). · arXiv
  25. [31][31] Hymes, D. Models of the interaction of language and social life: toward a descriptive theory . Intercultural discourse and communication: The essential readings, 1 (2005), 4-16.
  26. [33][33] Dewey, J. The logic of judgments of practise . The Journal of philosophy, psychology and scientific methods, 12, 19 (1915), 505-523.
  27. [35][35] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G. and Askell, A. Language models are few-shot learners . arXiv:2005.14165 (2020). · arXiv
  28. [38][38] Bender, E. M., Gebru, T., McMillan-Major, A. and Shmitchell, S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the The 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021
  29. [41][41] Nadeem, M., Bethke, A. and Reddy, S. Stereoset: Measuring stereotypical bias in pretrained language models . arXiv:2004.09456 (2020). - · arXiv
  30. [42][42] Lu, K., Mardziel, P., Wu, F., Amancharla, P. and Datta, A. Gender bias in neural natural language processing. Logic, Language, and Security, Springer, 2020. -
  31. [43][43] Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y. and Shieber, S. M. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In Proceedings of the NeurIPS, 2020 -
  32. [44][44] Russell, S. Human compatible: Artificial intelligence and the problem of control . Penguin, 2019. -
  33. [46][46] Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K.-W. and Gupta, R. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the The 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021 -
  34. [47][47] Guo, W. and Caliskan, A. Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases. In Proceedings of the Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 2021
  35. [48][48] Lucy, L. and Bamman, D. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the The Third Workshop on Narrative Understanding, 2021 -
  36. [49][49] Fazelpour, S. and De-Arteaga, M. Diversity in Sociotechnical Machine Learning Systems . arXiv:2107.09163 (2021). · arXiv
  37. [50][50] Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L. and Du, Y. LaMDA: Language Models for Dialog Applications . arXiv:2201.08239 (2022). - · arXiv
  38. [51][51] Galston, W. A. Liberal pluralism: The implications of value pluralism for political theory and practice . Cambridge University Press, 2002.
  39. [53][53] Tausch, A. Hofstede, Inglehart and Beyond. New Directions in Empirical Global Value Research . SSRN (2015).
  40. [54][54] McSweeney, B. The essentials of scholarship: A reply to Geert Hofstede . Human relations, 55, 11 (2002), 1363-1372.
  41. [55][55] Schwartz, S. H. and Bardi, A. Value hierarchies across cultures: Taking a similarities perspective . Journal of crosscultural Psychology, 32, 3 (2001), 268-290.
  42. [57][57] Weber, M. and Kalberg, S. The Protestant ethic and the spirit of capitalism . Routledge, 2013. -
  43. [59][59] Davani, A. M., Díaz, M. and Prabhakaran, V. Dealing with disagreements: Looking beyond the majority vote in subjective annotations . arXiv:2110.05719 (2021). - · arXiv
  44. [60][60] Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M. and Dean, J. Carbon emissions and large neural network training . arXiv:2104.10350 (2021). · arXiv
  45. [61][61] Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M. and Le, Q. V. Finetuned language models are zero-shot learners . arXiv:2109.01652 (2021). · arXiv
  46. [62][62] Nikolich, A. and Puchkova, A. Fine-tuning GPT-3 for Russian Text Summarization . arXiv:2108.03502 (2021). · arXiv
  47. [63][63] Moradi, M., Blagec, K., Haberl, F. and Samwald, M. GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain . arXiv:2109.02555 (2021). · arXiv
  48. [64][64] Kirstain, Y., Lewis, P., Riedel, S. and Levy, O. A Few More Examples May Be Worth Billions of Parameters . arXiv:2110.04374 (2021). · arXiv
  49. [65][65] Reynolds, L. and McDonell, K. Prompt programming for large language models: Beyond the few-shot paradigm. In Proceedings of the The 2021 CHI Conference on Human Factors in Computing Systems., 2021
  50. [66][66] Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R. and Young, S. Scaling language models: Methods, analysis & insights from training gopher . arXiv:2112.11446 (2021). · arXiv
  51. [68][68] Prabhakaran, V., Davani, A. M. and Díaz, M. On releasing Annotator-Level labels and information in datasets . arXiv:2110.05699 (2021). · arXiv
  52. [69][69] Newman, J. and Head, B. The national context of wicked problems: Comparing policies on gun violence in the US, Canada, and Australia . Journal of comparative policy analysis: research and practice, 19, 1 (2017), 40-53.
  53. [72][72] Ipsos. What do you think is more important?...To protect the right of Americans to own guns or control gun ownership? , Pew Research Center for the People & the Press, U.S.A, 2019.
  54. [78][78] Hauser, J. Education, secularism, and illiberalism: Marginalisation of Muslims by the French state . French Cultural Studies, 32, 2 (2021), 149-162.
  55. [79][79] Freedman, J. Secularism as a barrier to integration? The French dilemma . International Migration, 42, 3 (2004), 5-27.
  56. [84][84] Nagel, T. Mortal questions . Cambridge University Press, Cambridge [Eng.] ;, 1979.
  57. [85][85] Nagel, T. The Fragmentation of Value. Mortal Questions, Cambridge University Press, Cambridge, 2012. Below are some of the texts used in the tests.

Cited by

Sorted by citations

Authors
Johnson, R. L., Pistilli, G., Menédez-González, N., Duran, L. D. D., Panai, E., Kalpokiene, J., & Bertulfo, D. J.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2203.07785
arXiv
2203.07785
OpenAlex
W4221164682
Cited by
50
References
0
Citations by year
2026: 7 · 2025: 13 · 2024: 16 · 2023: 14
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Vytautas Magnus University (submittedVersion, cc-by)
  • HAL (submittedVersion)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
GPT-3, large language models, value conflicts, alignment problem, cultural values, training data, World Values Survey, American public opinion, cross-cultural comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review