Gender and Representation Bias in GPT-3 Generated Stories
Open accessAbstract
Using topic modeling and lexicon-based word similarity, we find that stories generated by GPT-3 exhibit many known gender stereotypes.Generated stories depict different topics and descriptions depending on GPT-3's perceived gender of the character in a prompt, with feminine characters 1 more likely to be associated with family and appearance, and described as less powerful than masculine characters, even when associated with high power verbs in a prompt.Our study raises questions on how one can avoid unintended social biases when using large language models for storytelling.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GPT-3 has been used to generate articles, stories, and other written content, with some users reporting that the generated text is difficult to distinguish from text written by humans.
“GPT-3 has been used to generate articles (Transformer et al., 2022), stories (Lucy & Bamman, 2021), and other types of written content, with some users reporting that the generated text is difficult to distinguish from text written by humans (Elkins & Chun, 2020).”
From Chatting and cheating: Ensuring academic integrity in the era of ChatGPT (2024) · cites this paper as Lucy & Bamman, 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
(2001) · Cited by 13,930 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 2× · see citing works
“Language models mimic patterns in their training data, parroting or even amplifying social biases (Bender et al., 2021).”
BackgroundScience faculty’s subtle gender biases favor male students
Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1211286109 · Cited by 3,150 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Measuring praise and criticism
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,515 · see citing works
Harvesting implicit group attitudes and beliefs from a demonstration web site.
Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002) · Group Dynamics: Theory, Research, and Practice · doi:10.1037/1089-2699.6.1.101 · Cited by 1,321 · see citing works
The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings
Kozlowski, A. C., Taddy, M., & Evans, J. A. (2019) · American Sociological Review · doi:10.1177/0003122419877135 · Cited by 682 · see citing works
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · see citing works
Persistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · see citing works
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · see citing works
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 431 · cited 1× · see citing works
“Language models generate different occupations and levels of respect for different genders, races, and sexual orientations (Sheng et al., 2019; Kirk et al., 2021).”
BackgroundEmpath: Understanding Topic Signals in Large-Scale Text
Fast, E., Chen, B., & Bernstein, M. S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858535 · Cited by 413 · see citing works
Counter-fitting Word Vectors to Linguistic Constraints
Mrkšić, N., Séaghdha, D. Ó., Thomson, B., Gašić, M., Rojas-Barahona, L. M., Su, P.-H., Vandyke, D., Wen, T.-H., & Young, S. J. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1018 · Cited by 356 · see citing works
Boyd-Graber, J. L., Hu, Y., & Mimno, D. (2017) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000030 · Cited by 259 · see citing works
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · see citing works
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202 · see citing works
A Bayesian Mixed Effects Model of Literary Character
Bamman, D., Underwood, T., & Smith, N. A. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1035 · Cited by 191 · see citing works
Self-citationLearning Latent Personas of Film Characters
Bamman, D., O’Connor, B., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473477 · Cited by 181 · see citing works
Self-citationFlicker, E. (2003) · Public Understanding of Science · doi:10.1177/0963662503123009 · Cited by 167 · see citing works
The Transformation of Gender in English-Language Fiction
Underwood, T., Bamman, D., & Lee, S. (2018) · Journal of Cultural Analytics · doi:10.22148/16.019 · Cited by 121 · see citing works
Self-citationLong, M., Steinke, J., Applegate, B. E., Lapinski, M. K., Johnson, M. J., & Ghosh, S. (2010) · Science Communication · doi:10.1177/1075547009357779 · Cited by 119 · see citing works
Lucy, L., Demszky, D., Bromley, P., & Jurafsky, D. (2020) · AERA Open · doi:10.1177/2332858420940312 · Cited by 119 · see citing works
Self-citationThe Quiet Transformations of Literary Studies: What Thirteen Thousand Scholars Could Tell Us
Goldstone, A., & Underwood, T. (2014) · New Literary History · doi:10.1353/nlh.2014.0025 · Cited by 116 · see citing works
Gender as a Variable in Natural-Language Processing: Ethical Considerations
Larson, B. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1601 · Cited by 111 · see citing works
Jane, John … Leslie? A Historical Method for Algorithmic Gender Prediction
Blevins, C., & Mullen, L. A. (2015) · Digital humanities quarterly · doi:10.63744/2h8m9cv8t9y4 · Cited by 109 · see citing works
Towards Controllable Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.291 · Cited by 95 · see citing works
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation
Akoury, N., Wang, S., Whiting, J., Hood, S., Peng, N., & Iyyer, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.525 · Cited by 85 · see citing works
Singular they and the syntactic representation of gender in English
Bjorkman, B. M. (2017) · Glossa a journal of general linguistics · doi:10.5334/gjgl.374 · Cited by 84 · see citing works
Connotation Frames of Power and Agency in Modern Films
Sap, M., Prasettio, M. C., Holtzman, A., Rashkin, H., & Choi, Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1247 · Cited by 84 · see citing works
Syntactic and cognitive issues in investigating gendered coreference
Ackerman, L. M. (2019) · Glossa a journal of general linguistics · doi:10.5334/gjgl.721 · Cited by 66 · see citing works
An Annotated Dataset of Coreference in English LiteraturePreprint
Bamman, D., Lewke, O., & Mansoor, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.01140 · Cited by 59 · see citing works
Self-citationSemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment
An, J., Kwak, H., & Ahn, Y. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1228 · Cited by 56 · see citing works
Why Are We Like This?: The AI Architecture of a Co-Creative Storytelling Game
Kreminski, M., Dickinson, M., Mateas, M., & Wardrip-Fruin, N. (2020) · International Conference on the Foundations of Digital Games · doi:10.1145/3402942.3402953 · Cited by 50 · cited 1× · see citing works
“Advances in large language models have allowed new possibilities for their use in storytelling, such as machine-in-the-loop creative writing (Clark et al., 2018; Kreminski et al., 2020; Akoury et al., 2020) and narrative generation for games (Raley and Hua, 2020).”
BackgroundFast, E., Vachovsky, T., & Bernstein, M. S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v10i1.14744 · Cited by 43 · see citing works
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
Shwartz, V., Rudinger, R., & Tafjord, O. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.556 · Cited by 41 · see citing works
Social Characters: The Hierarchy of Gender in Contemporary English-Language Fiction
Kraicer, E., & Piper, A. (2019) · Journal of Cultural Analytics · doi:10.22148/16.032 · Cited by 32 · see citing works
Toward Gender-Inclusive Coreference Resolution
Cao, Y. T., & Daumé, H., III (2020) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2020.acl-main.418 · Cited by 22 · see citing works
Playing With Unicorns: AI Dungeon and Citizen NLP
Hua, M. T., & Raley, R. (2020) · Digital humanities quarterly · doi:10.63744/8c9hrrx4banb · Cited by 19 · see citing works
Geek Chic: Smart Women in Popular Culture
Inness, S. A. (2007) · Palgrave Macmillan US eBooks · doi:10.1007/978-1-137-08421-7 · Cited by 19 · see citing works
Analyzing Gender Bias within Narrative Tropes
Gala, D., Khursheed, M. O., Lerner, H., O’Connor, B., & Iyyer, M. (2020) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2020.nlpcss-1.23 · Cited by 16 · see citing works
Gender bias at scale: Evidence from the usage of personal names
Johns, B. T., & Dye, M. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01234-0 · Cited by 15 · see citing works
Unsupervised Discovery of Gendered Language through Latent-Variable Modeling
Hoyle, A., Wolf-Sonkin, L., Wallach, H., Augenstein, I., & Cotterell, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1167 · Cited by 15 · see citing works
Geek chic: smart women in popular culture
(2008) · Choice Reviews Online · doi:10.5860/choice.45-2402 · Cited by 6 · see citing works
How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases.Preprint
Kirk, H. R., Jun, Y., Haider, I., Benussi, E., Volpin, F., Dreyer, F. A., Shtedritski, A., & Asano, Y. M. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 48 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Stacy L Smith, Marc Choueiti, Ashley Prescott, and Katherine Pieper. 2012. Gender roles & occupa- tions: A look at character attributes and job-related aspirations in film and television. Geena Davis In- stitute on Gender in Media, pages 1-46.
Cited by
Sorted by citations
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
BAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 1,826 · see citing works
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004 · see citing works
“The ChatGPT has unprecedented ability to carry out remarkably complex tasks like writing an article (GTP-3 et al., 2021), story, poem, an essay (Lucy & Bamman 2021), ability to provide a summary or expansion of a text, adjusting texts to reflect a different perspective, and even write and debug…”
BackgroundChatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355 · see citing works
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919 · see citing works
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
“GPT-3 [30] exhibited bias based on religion, analogising ‘“Muslim” to “terrorist” in 23% of test cases [2]; and gender bias, presenting fictional female characters as more domestic than male counterparts [120].”
BackgroundCan Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585 · see citing works
Using large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369 · see citing works
Biases in Large Language Models: Origins, Inventory, and Discussion
Navigli, R., Conia, S., & Ross, B. (2023) · Journal of Data and Information Quality · doi:10.1145/3597307 · Cited by 364 · see citing works
Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., & Shimizu, T. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043378 · Cited by 344 · see citing works
Ali, O., Murray, P. A., Momin, M., Dwivedi, Y. K., & Malik, T. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.123076 · Cited by 317 · see citing works
Kar, A. K., Varsha, P. S., & Rajan, S. (2023) · Global Journal of Flexible Systems Management · doi:10.1007/s40171-023-00356-x · Cited by 209 · see citing works
Can Foundation Models Wrangle Your Data?
Narayan, A., Chami, I., Orr, L., & Ré, C. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3574245.3574258 · Cited by 229 · see citing works
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186 · see citing works
Integrating generative AI in knowledge building
Chen, B., Zhu, X., & Díaz del Castillo H., F. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100184 · Cited by 159 · see citing works
Chan, A. (2023) · AI and Ethics · doi:10.1007/s43681-022-00148-6 · Cited by 155 · see citing works
Liang, J. L., Wang, L., Luo, J., Yan, Y., & Fan, C. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1285392 · Cited by 195 · see citing works
Fields, J., Chovanec, K., & Madiraju, P. (2024) · IEEE Access · doi:10.1109/access.2024.3349952 · Cited by 165 · see citing works
Large language models show human-like content biases in transmission chain experiments
Acerbi, A., & Stubbersfield, J. M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2313790120 · Cited by 137 · see citing works
Knopp, M. I., Warm, E. J., Weber, D. E., Kelleher, M., Kinnear, B., Schumacher, D. J., Santen, S. A., Mendonça, E. A., & Turner, L. (2023) · JMIR Medical Education · doi:10.2196/50373 · Cited by 136 · see citing works
Representation Bias in Data: A Survey on Identification and Resolution Techniques
Shahbazi, N., Yin, L., Asudeh, A., & Jagadish, H. V. (2023) · ACM Computing Surveys · doi:10.1145/3588433 · Cited by 155 · see citing works
Societal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131 · see citing works
Sezgın, E., Sirrianni, J., & Linwood, S. L. (2022) · JMIR Medical Informatics · doi:10.2196/32875 · Cited by 129 · see citing works
Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing
Liu, Z., Chen, C., Wang, J., Che, X., Huang, Y.-K., Hu, J., & Wang, Q. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00119 · Cited by 130 · see citing works
The limitations and ethical considerations of ChatGPT
Hua, S., Jin, S., & Jiang, S. (2024) · Data Intelligence · doi:10.1162/dint_a_00243 · Cited by 120 · see citing works
Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Cheng, M., Durmus, E., & Jurafsky, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.84 · Cited by 152 · see citing works
Törnberg, P. (2024) · Social Science Computer Review · doi:10.1177/08944393241286471 · Cited by 129 · see citing works
Generative AI meets Responsible AI: Practical Challenges and Opportunities
Kenthapadi, K., Lakkaraju, H., & Rajani, N. F. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599557 · Cited by 102 · see citing works
The Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q., Li, L., Chen, X., Wen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 101 · see citing works
Contrasting Linguistic Patterns in Human and LLM-Generated News Text
Muñoz-Ortiz, A., Gómez-Rodríguez, C., & Vilares, D. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10903-2 · Cited by 94 · see citing works
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4
Chang, K. K., Cramer, M. H., Soni, S., & Bamman, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.453 · Cited by 110 · see citing works
Self-citationAbramski, K., Citraro, S., Lombardi, L., Rossetti, G., & Stella, M. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7030124 · Cited by 81 · see citing works
“Topic modeling and sentiment analysis techniques have been used to find gender stereotypes in narratives generated by GPT-3 [36].”
MethodLozić, E., & Štular, B. (2023) · Future Internet · doi:10.3390/fi15100336 · Cited by 95 · see citing works
Delphi: Towards Machine Ethics and NormsPreprint
Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. J. L., Forbes, M., Borchardt, J., Liang, J., Etzioni, O., Sap, M., & Choi, Y. (2021) · arXiv (Cornell University) · Cited by 74 · see citing works
Gupta, P., Mahajan, R., Badhera, U., & Kushwaha, P. S. (2024) · The International Journal of Management Education · doi:10.1016/j.ijme.2024.101017 · Cited by 77 · see citing works
Initial policy considerations for generative artificial intelligence
Lorenz, P., Perset, K., & Berryhill, J. (2023) · OECD artificial intelligence papers · doi:10.1787/fae2d1e6-en · Cited by 105 · see citing works
A Medical Ethics Framework for Conversational Artificial Intelligence
Fournier-Tombs, É., & McHardy, J. (2023) · Journal of Medical Internet Research · doi:10.2196/43068 · Cited by 65 · see citing works
Diagnostic and Management Applications of ChatGPT in Structured Otolaryngology Clinical Scenarios
Qu, R. W., Qureshi, U., Petersen, G., & Lee, S. C. (2023) · OTO Open · doi:10.1002/oto2.67 · Cited by 62 · see citing works
Al-Qaysi, N., Al‐Emran, M., Al-Sharafi, M. A., Iranmanesh, M., Ahmad, A., & Mahmoud, M. A. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2361210 · Cited by 67 · see citing works
Kim, Y., Lee, J. K. W., Kim, S., Park, J., & Kim, J. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645148 · Cited by 67 · see citing works
Experimental narratives: A comparison of human crowdsourced storytelling and AI storytelling
Beguš, N. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03868-8 · Cited by 60 · see citing works
Leveraging Large Language Models for the Generation of Novel Metaheuristic Optimization Algorithms
Pluháček, M., Kazíková, A., Kadavý, T., Viktorin, A., & Šenkeřík, R. (2023) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3583133.3596401 · Cited by 58 · see citing works
Foundation models are platform models: Prompting and the political economy of AI
Burkhardt, S., & Rieder, B. (2024) · Big Data & Society · doi:10.1177/20539517241247839 · Cited by 65 · see citing works
Mayer, C., Ludwig, S., & Brandt, S. (2022) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2142872 · Cited by 64 · see citing works
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Liu, Y., Fabbri, A. R., Liu, P., Zhao, Y., Nan, L., Han, R., Han, S., Joty, S., Wu, C.-S., Xiong, C., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.228 · Cited by 75 · see citing works
Navigating Generative AI (ChatGPT) in Higher Education: Opportunities and Challenges
Wang, T. (2023) · Lecture Notes in Educational Technology · doi:10.1007/978-981-99-5961-7_28 · Cited by 56 · see citing works
Xu, X., Su, Y., Zhang, Y., Wu, Y., & Xin-yu, X. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26239 · Cited by 80 · see citing works
ChatGPT in Hydrology and Earth Sciences: Opportunities, Prospects, and Concerns
Foroumandi, E., Moradkhani, H., Sanchez-Vila, X., Singha, K., Castelletti, A., & Destouni, G. (2023) · Water Resources Research · doi:10.1029/2023wr036288 · Cited by 36 · see citing works
Liu, Z., Chen, C., Wang, J., Chen, M., Wu, B., Tian, Z., Huang, Y.-K., Hu, J. C., & Wang, Q. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639118 · Cited by 59 · see citing works
The role of generative AI in academic and scientific authorship: an autopoietic perspective
Watson, S., Brezovec, E., & Romic, J. (2025) · AI & Society · doi:10.1007/s00146-024-02174-w · Cited by 69 · see citing works
Prompted Opinion Summarization with GPT-3.5
Bhaskar, A., Fabbri, A. R., & Durrett, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.591 · Cited by 59 · see citing works
SCOTT: Self-Consistent Chain-of-Thought Distillation
Wang, P., Wang, Z., Li, Z., Gao, Y., Yin, B., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.304 · Cited by 68 · see citing works
Teaming Up with an AI: Exploring Human–AI Collaboration in a Writing Scenario with ChatGPT
Luther, T., Kimmerle, J., & Creß, U. (2024) · AI · doi:10.3390/ai5030065 · Cited by 55 · see citing works
Generative AI and Its Educational Implications
Łodzikowski, K., Foltz, P. W., & Behrens, J. T. (2024) · Postdigital Science and Education · doi:10.1007/978-3-031-64487-0_2 · Cited by 45 · see citing works
EvoPrompting: Language Models for Code-Level Neural Architecture Search
Chen, A., Dohan, D. M., & So, D. R. (2023) · arXiv · doi:10.48550/arxiv.2302.14838 · Cited by 45 · see citing works
FLUTE: Figurative Language Understanding through Textual Explanations
Chakrabarty, T., Saakyan, A., Ghosh, D., & Muresan, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.481 · Cited by 45 · see citing works
Generative AI in the context of assistive technologies: Trends, limitations and future directions
Fu, B., Hadid, A., & Damer, N. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105347 · Cited by 52 · see citing works
Melhárt, D., Togelius, J., Mikkelsen, B., Holmgård, C., & Yannakakis, G. N. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3276425 · Cited by 54 · see citing works
Ovalle, A., Goyal, P., Dhamala, J., Jaggers, Z., Chang, K.-W., Galstyan, A., Zemel, R., & Gupta, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594078 · Cited by 43 · see citing works
Future of AI & Generation Alpha: ChatGPT beyond Boundaries
Rathore, B. (2023) · Eduzone International peer reviewed/refereed academic multidisciplinary journal · doi:10.56614/eiprmj.v12i1y23.254 · Cited by 42 · see citing works
Salinas, A., Shah, P., Huang, Y., McCormack, R., & Morstatter, F. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623257 · Cited by 45 · see citing works
Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT
Bhavya, B., Xiong, J., & Zhai, C. (2022) · Proceedings of the 15th International Conference on Natural Language Generation · doi:10.18653/v1/2022.inlg-main.25 · Cited by 40 · see citing works
Efficacy of ChatGPT in Cantonese Sentiment Analysis: Comparative Study
Ziru, F. U., Hsu, Y., Chan, C. S., Lau, C., Liu, J., & YIP, P. S. F. (2023) · Journal of Medical Internet Research · doi:10.2196/51069 · Cited by 38 · see citing works
Correlation between artificial intelligence in education and teacher self-efficacy beliefs: a review
Bedirhan Berk, O. R. A. N. (2023) · RumeliDE Dil ve Edebiyat Araştırmaları Dergisi :/RumeliDe Dil ve Edebiyat Araştırmaları Dergisi · doi:10.29000/rumelide.1316378 · Cited by 34 · see citing works
Chat GPT in Supporting Education Instruction Sector: An Empirical Literature Review
Tan, A. A., Huda, M., Rohim, M. A., Hassan, T. R. R., Ismail, A., & Siregar, M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-5035-1_2 · Cited by 34 · see citing works
Fonseca, L. M., de Oliveira, E. A., Pereira, T., & Sá, J. C. (2024) · Management & Marketing · doi:10.2478/mmcks-2024-0021 · Cited by 33 · see citing works
Mitigating Propensity Bias of Large Language Models for Recommender Systems
Zhang, G., Yuan, G., Cheng, D., Liu, L., Li, J., & Zhang, S. (2025) · ACM Transactions on Information Systems · doi:10.1145/3736404 · Cited by 31 · see citing works
Nielbo, K. L., Karsdorp, F., Wevers, M., Lassche, A., Baglini, R., Kestemont, M., & Tahmasebi, N. (2024) · Nature Reviews Methods Primers · doi:10.1038/s43586-024-00302-w · Cited by 38 · see citing works
Embedding Democratic Values into Social Media AIs via Societal Objective Functions
Jia, C., Lam, M. S., Mai, M. C., Hancock, J. T., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641002 · Cited by 31 · see citing works
Artificial intelligence and ChatGPT are fostering knowledge sharing, ethics, academia and libraries
Zeb, A., Rehman, F. U., Othayman, M. B., & Rabnawaz, M. (2024) · International Journal of Information and Learning Technology · doi:10.1108/ijilt-03-2024-0046 · Cited by 40 · see citing works
An argumentation experience regarding concepts of calculus with ChatGPT
Urhan, S., Gençaslan, O., & Dost, Ş. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2024.2308093 · Cited by 30 · see citing works
Yeo, S., Lim, G., Gao, J., Zhang, W., & Perrault, S. T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642530 · Cited by 35 · see citing works
GPT-3 Powered System for Content Generation and Transformation
Saravanan, S., & Sudha, K. (2022) · International Conference on Computational Intelligence and Communication Technologies (CCICT) · doi:10.1109/ccict56684.2022.00096 · Cited by 29 · see citing works
The Next Chapter: A Study of Large Language Models in Storytelling
Xie, Z., Cohn, T., & Lau, J. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.23 · Cited by 98 · see citing works
Future of Textile: Sustainable Manufacturing & Prediction via ChatGPT
Rathore, B. (2023) · Eduzone International peer reviewed/refereed academic multidisciplinary journal · doi:10.56614/eiprmj.v12i1y23.253 · Cited by 29 · see citing works
Computational Understanding of Narratives: A Survey
Ranade, P., Dey, S., Joshi, A., & Finin, T. (2022) · IEEE Access · doi:10.1109/access.2022.3205314 · Cited by 29 · see citing works
Sebastian, G., & Sebastian, S. S. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4461801 · Cited by 28 · see citing works
Mun, I. B., & Hwang, K.-H. (2024) · Information Development · doi:10.1177/02666669241307595 · Cited by 32 · see citing works
Revisiting Automated Topic Model Evaluation with Large Language Models
Stammbach, D., Zouhar, V., Hoyle, A., Sachan, M., & Ash, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.581 · Cited by 37 · see citing works
van Duijn, M., van Dijk, B., Kouwenhoven, T., de Valk, W., Spruit, M., & vanderPutten, P. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.25 · Cited by 24 · see citing works
CausalMapper: Challenging designers to think in systems with Causal Maps and Large Language Model
Huang, Z., Quan, K., Chan, J., & MacNeil, S. (2023) · Creativity and Cognition · doi:10.1145/3591196.3596818 · Cited by 24 · see citing works
Controllable Dialogue Simulation with In-context Learning
Li, Z., Chen, W., Li, S., Wang, H., Jing, Q., & Yan, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.318 · Cited by 24 · see citing works
Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis
Raza, S., Bamgbose, O., Chatrath, V., Ghuge, S., Sidyakin, Y., & Muaad, A. Y. M. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3392469 · Cited by 23 · see citing works
ChatGPT como herramienta de apoyo al aprendizaje en la educación superior: una experiencia docente
Segarra‐Ciprés, M., Grangel, R., & Belmonte, Ó. (2024) · Revista Tecnología Ciencia y Educación · doi:10.51302/tce.2024.19083 · Cited by 23 · see citing works
Benchmark Data Contamination of Large Language Models: A Survey
Xu, C., Guan, S., Greene, D., & Kechadi, M.-T. (2024) · arXiv · doi:10.48550/arxiv.2406.04244 · Cited by 22 · see citing works
Bedeutung von Künstlicher Intelligenz in der Sozialen Arbeit
Linnemann, G. A., Löhe, J., & Rottkemper, B. (2023) · Soziale Passagen · doi:10.1007/s12592-023-00455-7 · Cited by 22 · see citing works
Stammbach, D., Antoniak, M., & Ash, E. (2022) · doi:10.18653/v1/2022.wnu-1.6 · Cited by 29 · see citing works
Regulating under Uncertainty: Governance Options for Generative AIPreprint
G’Sell, F. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4918704 · Cited by 21 · see citing works
Lee, A. V. Y. (2024) · Asia Pacific Journal of Education · doi:10.1080/02188791.2024.2305171 · Cited by 20 · see citing works
Leong, K., & Sung, A. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03660-8 · Cited by 27 · see citing works
Nguyen, H., Nguyen, V., Ludovise, S., & Santagata, R. (2024) · Learning Media and Technology · doi:10.1080/17439884.2024.2435834 · Cited by 19 · see citing works
Solving Math Word Problems via Cooperative Reasoning induced Language Models
Zhu, X., Wang, J., Zhang, L., Zhang, Y., Huang, Y., Gan, R., Zhang, J., & Yang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.245 · Cited by 32 · see citing works
van Dijk, B., Kouwenhoven, T., Spruit, M. R., & van Duijn, M. J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.779 · Cited by 22 · see citing works
“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation
Showkat, D., & Baumer, E. P. S. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533483 · Cited by 18 · see citing works
Learning emotional prompt features with multiple views for visual emotion analysis
Xu, Q., Wei, Y., Yuan, S., Wu, J., Wang, L., & Wu, C. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102366 · Cited by 17 · see citing works
Uncovering Implicit Gender Bias in Narratives through Commonsense Inference
Huang, T., Brahman, F., Shwartz, V., & Chaturvedi, S. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.326 · Cited by 20 · see citing works
High school students’ use and beliefs about generative artificial intelligence and writing in school
Bueie, A. A., Skar, G. B. U., & Graham, S. (2025) · Reading and Writing · doi:10.1007/s11145-025-10702-3 · Cited by 16 · see citing works
Passion and attractiveness on display: an examination of gender bias in crowdfunding
Letwin, C. R., Ciuchta, M. P., Johnson, M. A., Stevenson, R. M., & Ford, C. (2023) · Small Business Economics · doi:10.1007/s11187-023-00808-9 · Cited by 16 · see citing works
A scoping review of the landscape of health-related open datasets in Latin America
Restrepo, D. S., Quion, J. M., Vásquez-Venegas, C., Villanueva, C., Celi, L. A., & Nakayama, L. F. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000368 · Cited by 18 · see citing works
- Authors
- Lucy, L., & Bamman, D.
- Affiliations
- University of California, Berkeley (US)
- Venue
- Proceedings of the Third Workshop on Narrative Understanding
- Published
- 2021
- Pages
- 48-55
- OpenAlex
- W3168771811
- Cited by
- 287
- References
- 48
- Citations by year
- 2026: 40 · 2025: 63 · 2024: 65 · 2023: 97 · 2022: 18 · 2021: 4
- FWCI
- 41.66
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Funders
- National Science Foundation (1942591, 1752814)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Gender Studies › Media, Gender, and Advertising
- Keywords
- gender stereotypes, representation bias, GPT-3, large language models, story generation, character gender
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
