Quantifying co-creative writing experiences
Abstract
Collaboration with artificial intelligence (AI) is a growing trend even in the field of creativity. This paper examines which quantitative metrics can be used to comparatively analyse human-computer co-creativity with children. To study this question, 24 schoolchildren of age 10–11 wrote a poem with three co-creative poetry writing processes: a human-computer, a human-human, and a human-human-computer process. The computational participant in the processes was an AI-based application called the Poetry Machine. The children were asked to evaluate their user experience with a 5-point Likert-type questionnaire after each writing process and a comparative questionnaire after finishing all processes. The metrics used in the evaluation were immediate fun, long-term enjoyment, creativity, self-expression, outcome satisfaction, ease of starting and finishing writing, quality of ideas and support from others, and ownership.Significant differences were found in fun, long-term enjoyment, quality of ideas, support, and ownership. The high number of statistically relevant results was enabled by exposing all participants to all writing processes, and the comparative questionnaire. The human-human-computer process was evaluated the best in long-term enjoyment and the human-computer process the weakest in support and idea quality. Creativity and ease of finishing writing turned out to be outlining metrics for the co-creative processes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One reviewed study used a rule-based system that only provided syntactic templates and was perceived by children as 'dumb.'
“One unique study used a rule-based system [29] that only provided syntactic templates and was perceived by children as “dumb.””
From Child-AI Co-Creation: A Review of the Current Research Landscape and a Proposal for Six Design Considerations (2025) · cites this paper as [29]
In Kantosalo and Riihiaho [29], children who co-wrote poems with AI reflected that something felt 'missing' compared to working with peers, suggesting a lack of social cues and shared meaning-making typical in peer collaboration.
“In the study by Kantosalo and Riihiaho [29], children who co-wrote poems with AI reflected that something felt “missing” compared to working with peers-suggesting the lack of social cues, and shared meaning-making typical in peer collaboration.”
From Child-AI Co-Creation: A Review of the Current Research Landscape and a Proposal for Six Design Considerations (2025) · cites this paper as [29]
Cites
Likert scales, levels of measurement, and the “laws” of statistics
Norman, G. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9222-y · Cited by 4,850 · see citing works
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,400 · see citing works
Creativity support tools: accelerating discovery and innovation
Shneiderman, B. (2007) · Communications of the ACM · doi:10.1145/1323688.1323689 · Cited by 765 · see citing works
Quantifying the Creativity Support of Digital Tools through the Creativity Support Index
Cherry, E., & Latulipe, C. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2617588 · Cited by 498 · cited 4× · see citing works
“Clark et al. (2018) used enjoyment as a metric in their study, and it is included in the Creativity Support Index (Cherry and Latulipe 2014).”
BackgroundMethodInfluentialInteraction Design and Children
Hourcade, J. P. (2008) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000006 · Cited by 285 · see citing works
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · see citing works
Jordanous, A. (2012) · Cognitive Computation · doi:10.1007/s12559-012-9156-1 · Cited by 207 · see citing works
All work and no play: Measuring fun, usability, and learning in software for children
Sim, G., MacFarlane, S., & Read, J. C. (2005) · Computers & Education · doi:10.1016/j.compedu.2005.11.021 · Cited by 146 · see citing works
Davis, N., Hsiao, C.-P., Yashraj Singh, K., Li, L., & Magerko, B. (2016) · Proceedings of the 21st International Conference on Intelligent User Interfaces · doi:10.1145/2856767.2856795 · Cited by 138 · see citing works
Höysniemi, J., Hämäläinen, P., & Turkki, L. (2003) · Interacting with Computers · doi:10.1016/s0953-5438(03)00008-0 · Cited by 127 · see citing works
Wizard of Oz prototyping of computer vision based action games for children
Höysniemi, J., Hämäläinen, P., & Turkki, L. (2004) · doi:10.1145/1017833.1017837 · Cited by 119 · see citing works
(2002) · Nature Cell Biology · doi:10.1038/ncb0102-e1 · Cited by 43 · see citing works
Lachner, F., Naegelein, P., Kowalski, R., Spann, M., & Butz, A. M. (2016) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2971485.2971501 · Cited by 40 · see citing works
Empirically Grounding the Evaluation of Creative Systems: Incorporating Interaction Design.
Bown, O. (2014) · ICCC · Cited by 33 · see citing works
Kantosalo, A., Toivanen, J. M., Xiao, P., & Toivonen, H. T. T. (2014) · Cited by 32 · see citing works
Self-citationOMG UR Funny! Computer-Aided Humor with an Application to Chat
Wen, M., Baym, N. K., Tamuz, O., Teevan, J. B., Dumais, S., & Kalai, A. T. (2015) · ICCC · Cited by 27 · see citing works
Interaction-based Authoring for Scalable Co-creative Agents
Jacob, M., & Magerko, B. S. (2015) · ICCC · Cited by 17 · see citing works
Kantosalo, A., & Riihiaho, S. (2018) · Connection Science · doi:10.1080/09540091.2018.1432566 · Cited by 17 · see citing works
Self-citationBown, O. (2015) · ICCC · Cited by 12 · see citing works
Experience driven design of creative systems
Yee-King, M. J., & d’Inverno, M. (2016) · Goldsmiths (University of London) · Cited by 11 · see citing works
Turn-Taking with Improvisational Co-Creative Agents
Winston, L., & Magerko, B. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v13i1.12931 · Cited by 11 · see citing works
InspireMe: Learning Sequence Models for Stories
Fortuin, V., Weber, R. M., Schriber, S., Wotruba, D., & Groß, M. H. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11407 · Cited by 8 · see citing works
Interaction Evaluation for Human-Computer Co-Creativity
Kantosalo, A., Toivanen, J. M., & Toivonen, H. T. T. (2015) · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Joksimovic, S., Ifenthaler, D., Marrone, R., De Laat, M., & Siemens, G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100138 · Cited by 152 · see citing works
The effect of AI-based inspiration on human design ideation
Kim, J., & Maher, M. L. (2023) · International Journal of Design Creativity and Innovation · doi:10.1080/21650349.2023.2167124 · Cited by 114 · see citing works
Kim, T., Kim, H., Lee, H. Y., Goh, H., Abdigapporov, S., Jeong, M., Cho, H., Han, K., Noh, Y., Lee, S.-J., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517701 · Cited by 46 · see citing works
Fiedler, A., & Döpke, J. (2025) · International Review of Economics Education · doi:10.1016/j.iree.2025.100321 · Cited by 40 · see citing works
Cai, Z., Han, A., Zhou, X., Gazulla, E. D., & Peppler, K. (2025) · Proceedings of the 24th Interaction Design and Children · doi:10.1145/3713043.3731506 · Cited by 17 · see citing works
A Systematic Review of Tech-supported Collaborative Creativity Practices in the Field of Education
Gündoğdu, B., & Merç, A. (2021) · Journal of Learning and Teaching in Digital Age · doi:10.53850/joltida.953760 · Cited by 14 · see citing works
Augmented Creativity: Leveraging Natural Language Processing for Creative Writing
Plate, D., & Hutson, J. (2022) · Art and Design Review · doi:10.4236/adr.2022.103029 · Cited by 8 · see citing works
The Language of AI and Human Poetry: A Comparative Lexicometric Study
Hamat, A. (2024) · 3L The Southeast Asian Journal of English Language Studies · doi:10.17576/3l-2024-3002-01 · Cited by 8 · see citing works
Development and Validation of Four Social Scales for the UX Evaluation of Interactive Products
Mortazavi, E., Doyon-Poulin, P., Imbeau, D., & Robert, J.-M. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2258026 · Cited by 6 · see citing works
Research Duos: Unveiling the Collaborative Essence of Research
Glebova, E. (2024) · Societies · doi:10.3390/soc14090175 · Cited by 3 · see citing works
Cai, Z., Wei, S., Han, A., & Peppler, K. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3728859 · Cited by 2 · see citing works
Yahaya, M. F., & Ariffin, A. A. M. (2025) · Creativity Studies · doi:10.3846/cs.2025.20440 · Cited by 2 · see citing works
User experience no contexto da inteligência artificial: uma revisão sistemática da literatura
Piccolo, D. M., Pereira, C. M., Katahira, I., Vidotti, S. A. B. G., & Rodas, C. M. (2021) · Informação & Informação · doi:10.5433/1981-8920.2021v26n3p302 · Cited by 3 · see citing works
Yahaya, M. F., & Ariffin, A. A. M. (2024) · Tourism and hospitality management · doi:10.20867/thm.31.1.5 · Cited by 2 · see citing works
Noonen, S., & Baek, J. (2024) · The Journal of Korean Association of Computer Education · doi:10.32431/kace.2024.27.4.015
Yin, Y., Wu, M., Huang, R., Tong, X., Zhou, J., Yu, C., & Shi, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790848
Hu, Y., Zhang, J., Wang, Z., & Yu, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790926
Raman, G., & Brady, E. L. (2026) · Creativity and Cognition · doi:10.1145/3803784.3807525
Liu, X., Wen, Y., Yang, X., & Gao, Y. (2026) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70347 · Cited by 1 · see citing works
- Authors
- Kantosalo, A., & Riihiaho, S.
- Affiliations
- University of Helsinki (FI)
- Corresponding author
- Kantosalo, Anna
- Venue
- Digital Creativity
- Publisher
- Informa UK Limited
- Published
- 2019-01-02
- Volume
- 30
- Issue
- 1
- Pages
- 23-38
- ISSN
- 1462-6268 (ISSN-L)
- 1744-3806
- OpenAlex
- W2912335615
- Cited by
- 19
- References
- 23
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 4 · 2023: 3 · 2022: 2 · 2021: 2
- FWCI
- 0.95
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Digital Creativity (publishedVersion)
- Funders
- Academy of Finland (276897 (CLiC))
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
- Keywords
- human-AI co-creation, children, poetry writing, user experience, creativity, enjoyment, ownership, idea quality, self-expression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
