GPT-3 and InstructGPT: technological dystopianism, utopianism, and “Contextual” perspectives in AI ethics and industry
Open accessAbstract
This paper examines the ethical solutions raised in response to OpenAI’s language model Generative Pre-trained Transformer-3 (GPT-3) a year and a half from its release. I argue that hype and fear about GPT-3, even within the Natural Language Processing (NLP) industry and AI ethics, have often been underpinned by technologically deterministic perspectives. These perspectives emphasise the autonomy of the language model rather than the autonomy of human actors in AI systems. I highlight the existence of deterministic perspectives in the current AI discourse (which range from technological utopianism to dystopianism), with a specific focus on the two issues of: (1) GPT-3’s potential intentional misuse for manipulation and (2) unintentional harm caused by bias. In response, I find that a contextual approach to GPT-3, which is centred upon wider ecologies of societal harm and benefit, human autonomy, and human values, illuminates practical solutions to concerns about manipulation and bias. Additionally, although OpenAI’s newest 2022 language model InstructGPT represents a small step in reducing toxic language and aligning GPT-3 with user intent, it does not provide any compelling solutions to manipulation or bias. Therefore, I argue that solutions to address these issues must focus on organisational settings as a precondition for ethical decision-making in AI, and high-quality curated datasets as a precondition for less harmful language model outputs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ChatGPT has been shown to generate accurate results using given data points and parameters.
“ChatGPT has been shown to generate accurate results using given data points and parameters, which caused fears of its potential to generate predictions that are accurate and comprehensive, often well beyond what can be achieved with existing technologies (Chan, 2022).”
From The Future of Artificial Intelligence in Special Education Technology (2023) · cites this paper as Chan, 2022
Section 2, “A background to GPT-3 and InstructGPT” (page unavailable), reports gains in the quality, accuracy, and breadth of GPT-3’s generated content and says those gains prompted interest and concern. It does not establish the claim about ChatGPT producing accurate results from given data points and parameters, or the cited sentence’s further claim about accurate, comprehensive predictions beyond existing technologies. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 3× · see citing works
“InstructGPT reveals that self-regulation by AI industry can often be an ineffective solution, as economic values (reflected in the emphasis on usability, efficiency, and effectiveness) override ethical values (such as justice, beneficence, or non-maleficence (see Floridi et al. [22]).”
BackgroundInfluentialDwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611 · cited 1× · see citing works
“(see Dwivedi et al. [19] and Russel and Norvig [51]).”
BackgroundHaenlein, M., & Kaplan, A. (2019) · California Management Review · doi:10.1177/0008125619864925 · Cited by 2,655 · see citing works
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 7× · see citing works
“…is also beneficial when considering that the key responsible agent is not a machine, but a human faced with abstract ethical guidelines, economic incentives, job dissatisfaction, or an egoistic work environment, all of which can work to the detriment of ethical practices (Hagendorff [28], p. 109).”
BackgroundInfluentialThe Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence
CRAWFORD, K. (2021) · doi:10.2307/j.ctv1ghv45t · Cited by 742 · see citing works
Plagiarism in the age of massive generative pre-trained transformers (GPT-3)
Dehouche, N. (2021) · Ethics in Science and Environmental Politics · doi:10.3354/esep00195 · Cited by 296 · see citing works
Gender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287 · see citing works
Flagged first, then by citations
Research works printed but not linked: 42
- Abid, A., Farooqi, M., Zou, J.: Persistent anti-Muslim bias in large language models. arXiv preprint arXiv:2101.05783, 1-17. https:// arxiv. org/ abs/ 2101. 05783 (2021) · arXiv
- Aggarwal, A., Chauhan, A., Kumar, D., Mittal, M., Verma, S.: Classification of fake news by fine-tuning deep bidirectional trans- formers based language model. EAI Endorsed Trans. Scalable Inf. Syst. 7(27), 1-12 (2020). https:// doi. org/ 10. 4108/ eai. 13-7-2018. 163973 · DOI
- Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dha- riwal, p. , Neelakantan, A., Shyam, p. , Sastry, G., Askell, A., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165, pp. 1-75. https:// arxiv. org/ abs/ 2005. 14165 (2020) · arXiv
- Calvo, R.A., Peters, D., Vold, K., Ryan, R.M.: Supporting human autonomy in ai systems: a framework for ethical enquiry. In: Burr, C., Floridi, L. (eds.) Ethics of Digital Well-Being: A Multidisciplinary Approach. Philosophical Studies Series, vol. 140, pp. 31-54. Springer, Cham (2020). https:// doi. org/ 10. 1007/ 978-3-030-50585-1_2 · DOI
- Cheong, M., Leins, K., & Coghlan, S.: Computer Science Com- munities: Who is Speaking, and Who is Listening to the Women? Using an Ethics of Care to Promote Diverse Voices. In ACM Con- ference on Fairness, Accountability, and Transparency (FAcct'21), March 3-10, 2021. Virtual Event, Canada.ACM, New York, NY, USA, pp. 1-10. s. https:// doi. org/ 10. 1145/ 34421 88. 34458 74 (2021) · DOI
- Chiu, L., Alexander, R.: Detecting Hate Speech with GPT-3. arXiv preprint arXiv:2103.12407, pp. 1-16. https:// arxiv. org/ abs/ 2103. 12407. (2021) · arXiv
- Colman, A.: Un/becoming digital: the ontology of technological determinism and its implications for art education. J. Soc. Theory Art Educ. 25(1), 278-305 (2005)
- Crawford, K., Whittaker, M., Elish, M.C., Barocas, S., Plasek, A., Ferryman, K.: The AI now report: The social and economic implications of artificial intelligence technologies in the near- term, pp. 1-25. https:// ainow insti tute. org/ AI_ Now_ 2016_ Report. pdf (2016). Accessed 10 Jan 2022
- Cummings, M.L.: Integrating ethics in design through value- sensitive design approach. Sci. Eng. Ethics 12, 701-715 (2006). https:// doi. org/ 10. 1007/ s11948-006-0065-0 · DOI
- Dafoe, A.: On technology determinism: a typology, scope con- ditions, and a mechanism. Sci. Technol. Human Values 40(6), 1047-1076 (2015)
- Dignum, V.: The role and challenges of education for responsible AI. Lond. Rev. Educ. 19(1), 1 (2021). https:// doi. org/ 10. 14324/ LRE. 19.1. 01. (1-11) · DOI
- Dignum, V.: Responsible autonomy. arXiv preprint arXiv:1706.02513. In: Proceedings of the Twenty-Sixth Inter- national Joint Conference on Artificial Intelligence (IJCAI-17), 4698-4704. https:// arxiv. org/ abs/ 1706. 02513 (2017) · arXiv
- European Commission. Europe fit for the Digital Age: Commis- sion proposes new rules and actions for excellence and trust in Artificial Intelligence. https:// ec. europa. eu/ commi ssion/ press corner/ detail/ en/ ip_ 21_ 1682 (2021)
- Floridi, L., Chiriatti, M.: GPT-3: its nature, scope, limits, and consequences. Mind. Mach. 30, 681-694 (2020). https:// doi. org/ 10. 1007/ s11023-020-09548-1 · DOI · Paper page
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wal- lach, H. Daumé III, H., Crawford, K.: Datasheets for Datasets. arXiv preprint arXiv:1803.09010v, pp. 1-24. https:// arxiv. org/ abs/ 1803. 09010 (2020)
- Ghallab, M.: Responsible AI: requirements and challenges. AI Perspectives 1(3), 1-7 (2019)
- Hagendorff, T.: AI virtues: The missing link putting AI ethics into practice. arXiv preprint arXiv:2011.12750, pp. 1-22. https:// arxiv. org/ abs/ 2011. 12750 (2020) · arXiv
- Hao, K.: We read the paper that forced Timnit Gebru out of Google. Here's what it says. MIT Technology Review. https:// www. techn ology review. com/ 2020/ 12/ 04/ 10132 94/ google-ai-eth- ics-resea rch-paper-forced-out-timnit-gebru/ (2020). Accessed 10 Feb 2022
- Heaven, W.D.: The new version of GPT-3 is much better behaved (and should be less toxic). MIT Technology Review. https:// www. techn ology review. com/ 2022/ 01/ 27/ 10443 98/ new-gpt3- openai-chatb ot-langu age-model-ai-toxic-misin forma tion/ (2022). Accessed 9 Feb 2022
- Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., Steinhardt. J.: Aligning AI with Shared Human Values. arXiv pre- print arXiv:2008.02275. In: Proceedings of the International Con- ference on Learning Representations (ICLR), pp. 1-29. https:// arxiv. org/ abs/ 2008. 02275 (2021) · arXiv
- Johnson, D.G., Verdicchio, M.: Reframing AI discourse. Mind. Mach. 27, 575-590 (2017). https:// doi. org/ 10. 1007/ s11023-017-9417-6 · DOI
- Kreps, S., McCain, R.M., Brundage, M.: All the news that's fit to fabricate: AI-generated text as a tool of media misinformation. J. Exp. Polit. Sci. 9(1), 104-117 (2020). https:// doi. org/ 10. 1017/ XPS. 2020. 37 · DOI · Paper page
- LaGrandeur, K.: How safe is our reliance on AI, and should we regulate it? AI Ethics (2021). https:// doi. org/ 10. 1007/ s43681-020-00010-7 · DOI
- McCormack, J., Gifford, T., Hutchings, P.: Autonomy, authentic- ity, authorship and intention in computer generated art. In: Ekárt, A., Liapis, A., Pena, M.L.C. (eds.) Computational Intelligence in Music, Sound, Art and Design, pp. 35-50. Springer, Cham (2019). https:// doi. org/ 10. 1007/ 978-3-030-16667-0_3 · DOI · Paper page
- McGuffie, K., Newhouse, A.: The radicalization risks of GPT-3 and advanced neural language models. arXiv preprint arXiv:2009.06807, pp. 1-12. https:// arxiv. org/ abs/ 2009. 06807 (2020) · arXiv
- McNamara, A., Smith, J., Murphy-Hill, E.: Does ACM's code of ethics change ethical decision making in software development?" In: Proceedings of the 2018 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018), November 4-9, 2018, Lake Buena Vista, FL, USA. ACM, New York, NY, USA: 1-7. https:// doi. org/ 10. 1145/ 32360 24. 32648 33 (2018) · DOI · Paper page
- Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Gal- styan, A.: A Survey on Bias and Fairness in Machine Learning. arXiv preprint arXiv:1908.09635, pp. 1-31. https://arxiv.org/ abs/1908.09635 (2019) · arXiv
- Mezentsev, S.: Technological determinism: breakthrough into the future. european proceedings of social and behavioural sci- ences. In: Proceedings of the 11th International and Theoretical Conference "Communicative Strategies of Information Society", pp. 240-248. https:// doi. org/ 10. 15405/ epsbs. 2020. 03. 02. 29. (2019) · DOI
- Mikalef, P., Conboy, K., Lundström, J.E., Popovič, A.: Thinking responsibly about responsible AI and 'the dark side' of AI. Eur. J. Inf. Syst. Edit. (2022). https:// doi. org/ 10. 1080/ 09600 85X. 2022. 20266 21 · DOI · Paper page
- Mikalef, P., Gupta, M.: Artificial intelligence capability: Con- ceptualization, measurement calibration, and empirical study on its impact on organizational creativity and firm performance. Inf. Manag. 58(103434), 1-20 (2021). https:// doi. org/ 10. 1016/j. im. 2021. 103434 · DOI · Paper page
- Morley, J., Floridi, L., Kinsey, L., Elhalal, A.: From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Sci. Eng. Ethics 26, 2141-2168 (2020). https:// doi. org/ 10. 1126/ scien ce. 132. 3429. 741 · DOI
- Müller, V.C.: Ethics of artificial intelligence and robotics. In: Zalta E.N. (ed.) The Stanford Encyclopedia of Philosophy (Winter 2020 Edition). https:// plato. stanf ord. edu/ archi ves/ win20 20/ entri es/ eth- ics-ai/ (2020). Accessed 20 May 2021
- Nallur, V., Lloyd, M., Pearson, S.: Automation: an essential com- ponent of ethical AI?. arXiv preprint arXiv:2103.15739. In: Pro- ceedings of the 15th Multi Conference on Computer Science and Information Systems, 20-23 July 2021, pp. 1-4. https:// arxiv. org/ abs/ 2103. 15739 (2021) · arXiv
- Noble, S.U.: Algorithms of Oppression: How Search Engines Reinforce Racism. New York University Press, New York (2018). https:// doi. org/ 10. 18574/ 97814 79833 641 · DOI
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, G., Mishkin, p. , Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feed- back. pp 1-68, arXiv preprint arXiv: 2202.02155, https:// arxiv. org/ abs/ 2203. 02155. (2022) · arXiv
- Russel, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 4th edn. Pearson, Hoboken (2021)
- Susser, D., Roessler, B., Nissenbaum, H.: Technology, autonomy, and manipulation. Internet Policy Rev. 8(2), 1-22 (2019). https:// doi. org/ 10. 14763/ 2019.2. 1410 · DOI
- Van de Poel, I.: Embedding Values in Artificial Intelligence (AI) Systems. Mind. Mach. 30(3), 385-409 (2020)
- Van den Hoven, J., Vermaas, P.E., van de Poel, I. (eds.): Handbook of Ethics, Values, and Technological design: Sources, Theory, Values and Application Domains. Springer, Cham (2015)
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. arXiv preprint arXiv:1706.03762. In: Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, https:// arxiv. org/ abs/ 1706. 03762 (2017) · arXiv
- Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., West, S. M., Richardson, R., Schultz, J., Schwartz, J.: AI Now Report 2018, pp. 1-62 (2018)
- Zhu, L., Xu, X., Lu, Q., Governatori, G., Whittle, J.: AI and eth- ics-operationalizing responsible. In: Chen, F., Zhou, J. (eds.) Humanity Driven AI. Springer, Cham (2022)
Cited by
Sorted by citations
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463 · see citing works
Wach, K., Duong, C. D., Ejdys, J., Kazlauskaitė, R., Korzynski, P., Mazurek, G., Paliszkiewicz, J., & Ziemba, E. (2023) · Entrepreneurial Business and Economics Review · doi:10.15678/eber.2023.110201 · Cited by 584 · see citing works
“…of the GAI, such as ChatGPT, to produce credible and human-like texts, its usage may facilitate the development of misleading or false information (Chan, 2023), which then may be used to influence and manipulate people’s emotions, perceptions, or behaviour, often without people even realising it…”
BackgroundYu, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1181712 · Cited by 409 · see citing works
“The introduction of these models has injected powerful impetus into the development of natural language processing Frontiers in Psychology 02 frontiersin.org and brought new opportunities and challenges for future natural language processing technology development (Chan, 2022).”
BackgroundGenerative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333 · see citing works
Yu, H. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e24289 · Cited by 225 · see citing works
RetractedThe Future of Artificial Intelligence in Special Education Technology
Marino, M. T., Vasquez, E., Dieker, L., Basham, J., & Blackorby, J. (2023) · Journal of Special Education Technology · doi:10.1177/01626434231165977 · Cited by 142 · see citing works
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137 · see citing works
“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”
Development and validation of ChatGPT literacy scale
Lee, S., & Park, G. (2024) · Current Psychology · doi:10.1007/s12144-024-05723-0 · Cited by 100 · see citing works
Performance and biases of large language models in public opinion simulation.
Qu, Y., & Wang, J. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03609-x · Cited by 79 · see citing works
A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning
Mcintosh, T. R., Liu, T., Susnjak, T., Watters, P., & Halgamuge, M. N. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3670691 · Cited by 42 · see citing works
AI chatbots: A disguised enemy for academic integrity?
Niloy, A. C., Hafiz, R., Hossain, B. M., Gulmeher, F., Sultana, N., Islam, K. F., Bushra, F., Islam, S., Hoque, S. I., Rahman, M., et al. (2024) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100396 · Cited by 26 · see citing works
Ni, W., & Li, J. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1856854 · Cited by 6 · see citing works
- Authors
- Chan, A.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Chan, Anastasia
- ORCID
- Chan, Anastasia
- Venue
- AI and Ethics
- Published
- 2022-04-04
- Volume
- 3
- Issue
- 1
- Pages
- 53-64
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W4225353277
- Cited by
- 155
- References
- 8
- Citations by year
- 2026: 20 · 2025: 39 · 2024: 55 · 2023: 38 · 2022: 3
- FWCI
- 39.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- AI and Ethics (publishedVersion, cc-by)
- Funders
- Macquarie University
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- GPT-3, AI ethics, technological determinism, technological utopia, manipulation, algorithmic bias, human autonomy, contextual approach, OpenAI, organisational settings, curated datasets, toxic language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
