The DOI matches its DataCite record; 86 of the 88 references it cites were found in a registry or an index, and 1 are not checked yet.
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Open accessAbstract
As the capabilities of generative language models continue to advance, the implications of biases ingrained within these models have garnered increasing attention from researchers, practitioners, and the broader public. This article investigates the challenges and risks associated with biases in large-scale language models like ChatGPT. We discuss the origins of biases, stemming from, among others, the nature of training data, model specifications, algorithmic constraints, product design, and policy decisions. We explore the ethical concerns arising from the unintended consequences of biased model outputs. We further analyze the potential opportunities to mitigate biases, the inevitability of some biases, and the implications of deploying these models in various applications, such as virtual assistants, content generation, and chatbots. Finally, we review the current approaches to identify, quantify, and mitigate biases in language models, emphasizing the need for a multi-disciplinary, collaborative effort to develop more equitable, transparent, and responsible AI systems. This article aims to stimulate a thoughtful dialogue within the artificial intelligence community, encouraging researchers and developers to reflect on the role of biases in generative language models and the ongoing pursuit of ethical AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, generative conversational AI may also expose users to undesirable consequences, such as biases, misinformation, and manipulation (Brandtzaeg, et al. , 2022; Ferrara, 2023).”
From Why do people use ChatGPT? Exploring user motivations for generative conversational AI (2024) · cites this paper as Ferrara, 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This may be particularly true if generative conversational AI lacks domain-specific knowledge and usable and transparent verification mechanisms, leading to unintended consequences of biased or misinformed model outputs (Ferrara, 2023).”
From Why do people use ChatGPT? Exploring user motivations for generative conversational AI (2024) · cites this paper as Ferrara, 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-understood that models trained on human data - such as being pre-trained on human-written text or fine-tuned on human feedback - can exhibit human biases.”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Ferrara, 2023
The cited source says what the claim attributes to it.
In the citing paper
“These range from parroting harmful stereotypes and misconceptions about certain social groups (Bianchi et al. 2023;Ferrara 2023)”
From Epistemic Injustice in Generative AI (2024) · cites this paper as Ferrara 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Part of this paper’s reference list could not be split into single entries.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Corsaro, W. A., & Bourdieu, P. (1992) · Social Forces · doi:10.2307/2579985 · Cited by 9,968
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
(2016) · Cited by 2,334
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
No Language Left Behind: Scaling Human-Centered Machine TranslationPreprint
Costa-jussà, M., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., et al. (2022) · arXiv.org · doi:10.48550/arxiv.2207.04672 · Cited by 2,176
Individualism and Collectivism
Triandis, H. C. (2018) · Routledge eBooks · doi:10.4324/9780429499845 · Cited by 1,844
(2020) · doi:10.7551/mitpress/12255.001.0001 · Cited by 1,706
Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020) · Communications of the ACM · doi:10.1145/3381831 · Cited by 1,647
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. C. (2019) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,510
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
Word Translation Without Parallel DataPreprint
Conneau, A., Lample, G., Ranzato, M., Denoyer, L., & Jeǵou, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04087 · Cited by 1,105
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Jain, A. (2017) · Business Economics · doi:10.1057/s11369-017-0027-3 · Cited by 1,094
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
A Network-based End-to-End Trainable Task-oriented Dialogue System
Wen, T.-H., Vandyke, D., Mrkšić, N., Gašić, M., Rojas-Barahona, L. M., Su, P.-H., Ultes, S., & Young, S. J. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1042 · Cited by 1,017
Beyond accuracy: behavioral testing of NLP models with CheckList
Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.442 · Cited by 890
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845
Recommendation of the Council on Artificial Intelligence (OECD)
Yeung, K. (2020) · International Legal Materials · doi:10.1017/ilm.2020.5 · Cited by 845
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
A Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11640 · Cited by 703
Fairness in Machine Learning: Lessons from Political Philosophy
Binns, R. (2017) · arXiv · doi:10.48550/arxiv.1712.03586 · Cited by 676
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674
Learning Deep Transformer Models for Machine Translation
Wang, Q., Li, B., Xiao, T., Zhu, J., Li, C., Wong, D. F., & Chao, L. S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1176 · Cited by 673
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632
Race after technology Abolitionist tools for the new Jim code
Benjamin, R. 1. (2019) · Cited by 578
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 2×
“When users and regulators can see that developers are taking steps to ensure the fairness and equity of their models, they may be more likely to trust and adopt these technologies [92, 93].”
BackgroundA Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.30462 · Cited by 512
Actionable Recourse in Linear Classification
Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
Situating methods in the magic of Big Data and AI
Elish, M. C., & boyd, D. (2017) · Communication Monographs · doi:10.1080/03637751.2017.1375130 · Cited by 488
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
Should Robots Replace Teachers?: AI and the Future of Education
Selwyn, N. (2019) · Cited by 434
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J. Y., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 421
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414
Fine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413
Transparent, explainable, and accountable AI for robotics
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · Science Robotics · doi:10.1126/scirobotics.aan6080 · Cited by 413
Assessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410
One-step and Two-step Classification for Abusive Language Detection on Twitter
Park, J. H., & Fung, P. (2017) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w17-3006 · Cited by 409
Neural Machine Translation for Low-resource Languages: A Survey
Ranathunga, S., Lee, E.-S. A., Skënduli, M. P., Shekhar, R., Alam, M., & Kaur, R. (2022) · ACM Computing Surveys · doi:10.1145/3567592 · Cited by 396
The EU Approach to Ethics Guidelines for Trustworthy Artificial Intelligence
Smuha, N. A. (2019) · Computer Law Review International · doi:10.9785/cri-2019-200402 · Cited by 356
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Dettmers, T., Lewis, M., Belkada, Y., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2208.07339 · Cited by 305
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292
Paulus, J. K., & Kent, D. M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0304-9 · Cited by 274
Clinical AI: opacity, accountability, responsibility and liability
Smith, H. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01019-6 · Cited by 245
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219
Regulate artificial intelligence to avert cyber arms race
Taddeo, M., & Floridi, L. (2018) · Nature · doi:10.1038/d41586-018-04602-6 · Cited by 209
Shahriari, K., & Shahriari, M. (2017) · IEEE Canada International Humanitarian Technology Conference · doi:10.1109/ihtc.2017.8058187 · Cited by 209
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2303.04671 · Cited by 197
Factuality challenges in the era of large language models and opportunities for fact-checking
Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00881-z · Cited by 187
Self-citationBehind the Screen: Content Moderation in the Shadows of Social Media
Gerrard, Y. (2019) · New Media & Society · doi:10.1177/1461444819878844 · Cited by 166
Botometer 101: social bot practicum for computational social scientists
Yang, K.-C., Ferrara, E., & Menczer, F. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00177-5 · Cited by 159
Self-citationMachine morality: bottom-up and top-down approaches for modelling human moral faculties
Wallach, W., Allen, C., & Smit, I. (2008) · AI & SOCIETY · doi:10.1007/s00146-007-0099-0 · Cited by 152
Help wanted: an examination of hiring algorithms, equity, and bias
Bogen, M., & Rieke, A. (2018) · Cited by 138
Behind the Screen: Content Moderation in the Shadows of Social Media
Roberts, S. (2019) · Cited by 132
Fair Class Balancing: Enhancing Model Fairness without Observing Sensitive Attributes
Yan, S., Kao, H.-T., & Ferrara, E. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411980 · Cited by 112
Self-citationTackling misinformation: What researchers could do with social media data
Pasquetto, I. V., Swire‐Thompson, B., & Amazeen, M. A. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-49 · Cited by 99
Stoyanovich, J., Howe, B., & Jagadish, H. V. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3415478.3415570 · Cited by 89
Is bigger better? The emergence of big data as a tool for international development policy
Taylor, L., & Schroeder, R. (2014) · GeoJournal · doi:10.1007/s10708-014-9603-5 · Cited by 88
Fairness and machine learning limitations and opportunities
Barocas, S., Hardt, M., & Narayanan, A. (n.d.) · Cited by 80
Contextual word representations
Smith, N. A. (2020) · Communications of the ACM · doi:10.1145/3347145 · Cited by 46
Factuality Challenges in the Era of Large Language Models
Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.05189 · Cited by 32
Self-citationOverview of the DARPA LORELEI Program
Christianson, C., Duncan, J. D., & Onyshkevych, B. A. (2018) · Machine Translation · doi:10.1007/s10590-017-9212-4 · Cited by 27
Neighbors helping the poor: improving low-resource machine translation using related languages
Pourdamghani, N., & Knight, K. K. (2019) · Machine Translation · doi:10.1007/s10590-019-09236-7 · Cited by 25
Biderman, S., Bicheno, K., & Gao, L. (2022) · arXiv · doi:10.48550/arxiv.2201.07311 · Cited by 12
(1991)
Research works printed but not linked: 5
- D.K. Citron and F. Pasquale, 2014. "The scored society: Due process for automated predictions," Washington Law Review, volume 89, number 1, pp. 1-33, and at https://digitalcommons.law.uw.edu/wlr/vol89/iss1/2/, accessed 27 October 2023.
- S. Dev and J.M. Phillips, 2019. "Attenuating bias in word vectors," Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, pp. 879-887, and at https://proceedings.mlr.press/v89/dev19a.html, accessed 27 October 2023.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin, 2017. "Attention is all you need," Advances in Neural Information Processing Systems, volume 30, at https://papers.nips.cc/paper_files/paper/2017, accessed 27 October 2023.
- B.L. Whorf, 1964. Language, thought, and reality: Selected Writings of Benjamin Lee Whorf. Edited by John B. Carroll. Cambridge, Mass.: MIT Press.
- R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi, 2019. "Defending against neural fake news," ANIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, article number 812, pp. 9,054-9,065.
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Ferrara, E. (2023) · Sci · doi:10.3390/sci6010003 · Cited by 875
Self-citationChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
Metadata
- Authors
- Ferrara, E.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Ferrara, Emilio
- ORCID
- Ferrara, Emilio
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 1396-0458 (ISSN-L)
- arXiv
- 2304.03738
- OpenAlex
- W4388488349
- Cited by
- 282
- References
- 49
- Citations by year
- 2026: 55 · 2025: 77 · 2024: 95 · 2023: 55
- FWCI
- 70.56
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- First Monday (publishedVersion, cc-by-nc-sa)
- SSRN Electronic Journal (acceptedVersion)
- First Monday (acceptedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Viterbi School of Engineering, University of Southern California
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- ChatGPT, large language models, algorithmic bias, bias mitigation, training data bias, ethical AI, generative AI risks, chatbots, virtual assistants, content creation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
