Playgrounds and Prejudices: Exploring Biases in Generative AI For Children.
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
The influence of generative Artificial Intelligence (AI) on the propagation and amplification of societal biases, particularly in the context of children’s content creation, is a growing concern. By developing and testing a prototype tool designed to assist children in Digital Storytelling (DST), our research aimed to explore and mitigate the propagation of stereotypes through the use of a character-generating AI tool utilising Stable Diffusion. Despite initial aspirations, the tool demonstrated significant biases inherent in the underlying AI model, leading to the decision against its use by children. The findings we discovered contribute to a broader discourse on the development of ethical AI and its use, advocating for a more responsible and inclusive approach to technological innovation in the context of children’s digital media consumption and creation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“when the system produces biased or incorrect information, they may accept and reinforce it [15].”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [15]
In the citing paper
“For example, [6] explicitly explored how image generation models reflect stereotypical representations of gender and race, encouraging children to reflect critically on the AI's limitations.”
From Child-AI Co-Creation: A Review of the Current Research Landscape and a Proposal for Six Design Considerations (2025) · cites this paper as [6]
In the citing paper
“In our review, a few papers [2,6,23,30] explicitly raised concerns about the opaque nature of AI tools, with children and adults questioning where the AI-generated information comes from, why certain outputs were created, or how the system made decisions.”
From Child-AI Co-Creation: A Review of the Current Research Landscape and a Proposal for Six Design Considerations (2025) · cites this paper as [6]
Citations
Cites
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“The issue of bias in such AI-generated content is particularly concerning due to the frequently opaque nature of the training data in which these technologies are developed [2].”
BackgroundMethodAdding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.00982 · Cited by 3,653
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 1,738
LAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022
Kshetri, N., Dwivedi, Y. K., Davenport, T. H., & Panteli, N. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102716 · Cited by 474
Gender bias and stereotypes in Large Language Models
Kotek, H., Dockum, R., & Sun, D. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615599 · Cited by 381
Roselli, D., Matthews, J., & Talagala, N. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317590 · Cited by 237 · cited 1×
“Despite the utility of these tools, the adoption of generative AI is not without its challenges, particularly in terms of the inherent biases that have not often been adequately addressed [27].”
BackgroundSafe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
Schramowski, P., Brack, M., Deiseroth, B., & Kersting, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02157 · Cited by 219
Sun, L., Wei, M., Sun, Y., Suh, Y. J., Shen, L., & Yang, S. (2023) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmad045 · Cited by 146 · cited 2×
“Typically, these AI systems learn from vast, generic data-sets which, whilst facilitating learning, can inadvertently perpetuate inherent biases intrinsic to the training data [32].”
BackgroundA survey on bias in visual datasets
Fabbrizzi, S., Papadopoulos, S., Ntoutsi, E., & Kompatsiaris, I. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103552 · Cited by 137
Banse, R., Gawronski, B., Rebetez, C., Gutt, H., & Bruce Morton, J. (2010) · Developmental Science · doi:10.1111/j.1467-7687.2009.00880.x · Cited by 120 · cited 1×
“In addition to the complexity inherent within AI systems is the already ingrained stereotypes that children have been shown to develop as early as 5-months old, with research highlighting that these perceptions are nearly fully developed within the first 7-12 years of a child’s life [5].”
BackgroundAligning Artificial Neural Networks and Ontologies towards Explainable AI
Ribeiro, M. D. S., & Leite, J. A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i6.16626 · Cited by 71 · cited 1×
“It is often difficult to tell how these systems work and as such, there is potential for unsafe and biased images to be presented to children unwittingly [9].”
BackgroundRubegni, E., Landoni, M., Malinverni, L., & Jaccheri, L. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102727 · Cited by 54 · cited 2×
“By leveraging the literature on the power of DST in mitigating gender stereotypical thinking [28] and the issues of using generative AI with and for children, the project aims to help to mitigate biases within children.”
BackgroundSelf-citationStable Bias: Analyzing Societal Representations in Diffusion Models
Luccioni, A. S., Akiki, C., Mitchell, M., & Jernite, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.11408 · Cited by 52
Uncurated Image-Text Datasets: Shedding Light on Demographic Bias
Garcia, N., Hirota, Y., Wu, Y., & Nakashima, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00672 · Cited by 43 · cited 2×
“The datasets used are, often, large and lack quality control [15, 33], which serves to create further conversation regarding what children are subjected to by allowing some of those tools to be easily accessible by young, vulnerable audiences.”
BackgroundWolfe, R., Yang, Y., Howe, B., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594072 · Cited by 42 · cited 3×
“The datasets used are, often, large and lack quality control [15, 33], which serves to create further conversation regarding what children are subjected to by allowing some of those tools to be easily accessible by young, vulnerable audiences.”
BackgroundMethodBaumgarten, M. (2003) · Computers in entertainment · doi:10.1145/950566.950584 · Cited by 33
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion ModelsPreprint
Schramowski, P., Brack, M., Deiseroth, B., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.05105 · Cited by 8
ChatGPT begins: A reflection on the involvement of AI in the creation of film and television scripts
Fang Luchen1, 2. (2023) · Frontiers in Art Research · doi:10.25236/far.2023.051701 · Cited by 8 · cited 1×
“Alongside the technical issues related to individual usage of AI systems, there are also issues associated with the usage of AI on larger scales such as with the production of popular media [23].”
Background
Research works printed but not linked: 14
- Olaiya E Aina and Petronella A Cameron. 2011. Why does gender matter? Counteracting stereotypes with young children. Dimensions of early childhood 39, 3 (2011), 11–19.
- Veronica Barassi. 2020. Child Data Citizen. MIT Press.
- Edward Dieterle, Chris Dede, and Michael Walker. 2022. The cyclical ethical effects of using artificial intelligence in education. AI & society (2022), 1–11.
- Joyce J Endendijk, Marleen G Groeneveld, Sheila R van Berkel, Elizabeth T Hallers-Haalboom, Judi Mesman, and Marian J Bakermans-Kranenburg. 2013. Gender stereotypes in the family context: Mothers, fathers, and siblings. Sex roles 68 (2013), 577–590.
- Douglas Guilbeault et. Al.2024. Online images amplify gender bias | Nature. (14 February 2024).
- Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit Bermano Gal Chechik and Daniel Cohen-Or. 2022. An image is worth one word: personalizing text-to-Image generation using textual inversion. Technical Report. Cornell University Library arXiv.org.
- Angela M Gooden and Mark A Gooden. 2001. Gender representation in notable children’s picture books: 1995–1999. Sex roles 45 (2001), 89–101.
- James Hannon, Sonya Soohoo, Justine Reel, and Thomas Ratliffe. 2009. Gender stereotyping and the influence of race in sport among adolescents. Research quarterly for exercise and sport 80, 3 (2009), 676–684.
- Weng Marc Lim, Asanka Gunasekara, Jessica Leigh Pallant, Jason Ian Pallant, and Ekaterina Pechenkina. 2023. Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The international journal of management education 21, 2 (2023), 100790.
- Diane M Mackie, David L Hamilton, Joshua Susskind, and Francine Rosselli. 1996. Social psychological foundations of stereotype formation. Stereotypes and stereotyping (1996), 41–78.
- Allison H Master and Andrew N Meltzoff. 2020. Cultural stereotypes and sense of belonging contribute to gender gaps in STEM.Grantee Submission 12, 1 (2020), 152–198.
- Siva Sai, Aanchal Gaur, Revant Sai, Vinay Chamola, Mohsen Guizani, and Joel JPC Rodrigues. 2024. Generative AI for Transformative Healthcare: A Comprehensive Study of Emerging Models, Applications, Case Studies and Limitations. IEEE Access (2024).
- Julia T Wood. 1994. Gendered media: The influence of media on views of gender. Gendered lives: Communication, gender, and culture 9 (1994), 231–244.
- Ming Zhong Yelong Shen Shuohang Wang Yadong Lu Yizhu Jiao Siru Ouyang Donghan Yu Jiawei Han and Weizhu Chen. 2024. Multi-LoRA Composition for Image Generation. Technical Report. Cornell University Library arXiv.org.
Cited by
Lin, X., & Tan, H. (2025) · Systems · doi:10.3390/systems13100840 · Cited by 22
Elsharif, W., Alzubaidi, M. S., & Agus, M. (2025) · IEEE Access · doi:10.1109/access.2025.3585745 · Cited by 36
“More over, [95] revealed that Stable Diffusion 2.0 reinforced gender stereotypes in children’s storytelling, depicting men in leadership roles while women were placed in caregiving or secondary roles.”
InfluentialCai, Z., Han, A., Zhou, X., Gazulla, E. D., & Peppler, K. (2025) · Proceedings of the 24th Interaction Design and Children · doi:10.1145/3713043.3731506 · Cited by 17
Parents’ Perceptions About the Use of Generative AI Systems by Adolescents
Eira, M., Rasouli, A., & Charisi, V. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731508 · Cited by 27
“However, an increasing body of literature raises questions about the emerging challenges of GenAI use by teenagers, such as the impact on human development [7], the concerns about privacy [12], and the amplification of societal biases [1].”
“Won’t somebody please (actually) think of the children?” AI Ethics for Children: A Scoping Review
Collyer-Hoar, G., & Rubegni, E. (2025) · ACM Journal on Responsible Computing · doi:10.1145/3745031 · Cited by 22
“Both researchers discussed and agreed upon a set of predefined data items to be extracted from each included source, based on the overarching research questions listed in Section 1.”
Self-citation
Metadata
- Authors
- Baines, A., Gruia, L., Collyer-Hoar, G., & Rubegni, E.
- Affiliations
- Lancaster University (GB)
- Venue
- Proceedings of the 23rd Annual ACM Interaction Design and Children Conference
- Publisher
- ACM
- Published
- 2024-06-17
- Pages
- 839-843
- OpenAlex
- W4399667892
- Cited by
- 21
- References
- 18
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 1
- FWCI
- 9.58
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Interaction Design and Children (IDC) (publishedVersion)
- Lancaster EPrints (Lancaster University) (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Communication › Technology's Impact on Media
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- generative AI, stereotypes, digital storytelling, character generation, Stable Diffusion, ethical AI, children's digital media, playgrounds
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.