AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably
Open accessAbstract
As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ 2 (1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ 2 (2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The findings offer partial support for the algorithm aversion hypothesis: participants reacted negatively to texts identified as AI-generated, but Version D received the highest mean scores despite containing AI-generated content, suggesting human editing obscured its AI origins and neutralized aversion; algorithm aversion is not a stable disposition but a contextually contingent response.
“Our findings offer partial support for the algorithm aversion hypothesis.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Porter and Machery (2024)
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe study's findings contrast with prior research on short-form poetry, where AI-generated content has been found particularly difficult to distinguish from human-authored work; this difference is likely attributable to the greater length and narrative complexity of prose fiction.
“This contrasts with prior research on short-form poetry, where AI-generated content has been found to be particularly difficult to distinguish from human-authored work (Hitsuwari et al., 2023; Porter and Machery, 2024)”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Porter and Machery, 2024
Porter and Machery (2024) found that non-expert readers performed below chance in identifying AI-generated poems by well-known human poets, and that AI-generated poems were rated more favorably in terms of rhythm and aesthetic quality.
“Porter and Machery (2024) found that non-expert readers performed below chance in identifying AI-generated poems by well-known human poets, and that AI-generated poems were rated more favorably in terms of rhythm and aesthetic quality and that this contributed to their mistaken identification as human-authored.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Porter and Machery (2024)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 20 · Cited by 133
Research works printed but not linked: 18
- [1]Sun, Y., Yang, C.-H., Lyu, Y. & Lin, R. From pigments to pixels: A comparison of human and AI painting. Appl. Sci. 12, 3724. 10.3390/app12083724 (2022). · DOI
- [2]Ragot, M., Martin, N. & Cojean, S. AI-generated vs. human artworks. A perception bias towards artificial intelligence? In CHI Conference on Human Factors in Computing Systems, 1–10 (2020). 10.1145/3334480.3382892. · DOI
- [4]Tucciarelli, R., Vehar, S. & Tsakiris, M. On the realness of people who do not exist: The social processing of artificial faces. iScience 25, 105441 (2022). 36590465 PMC9801245 · DOI
- [5]Miller, E. J. et al. AI hyperrealism: Why AI faces are perceived as more real than human ones. Psychol. Sci. 34 (12), 1390–1403. 10.1177/09567976231207095 (2023). 37955384 · DOI
- [6]Gorenz, D. & Schwarz, N. How funny is ChatGPT? A comparison of human- and A.I.-produced jokes (2024). 10.31234/osf.io/5yz8n. PMC11221738 38959273 · DOI
- [8]Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).
- [9]Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. arXiv (2023). 10.48550/arXiv.2307.09288. · DOI · Paper page
- [10]Jakesch, M., Hancock, J. T. & Naaman, M. Human heuristics for AI-generated language are flawed. Proc. Natl. Acad. Sci. 120, e2208839120 (2023). 36881628 PMC10089155 · DOI · Paper page
- [13]Linardaki, C. Poetry at the first steps of Artificial Intelligence. Humanist Stud. Digit. Age 10.5399/uo/hsda/7.1.6 (2022). · DOI
- [14]Gunser, V., Gottschling, S., Brucker, B., Richter, S., Çakir, D. & Gerjets, P. The pure poet: How good is the subjective credibility and stylistic quality of literary short texts written with an artificial intelligence tool as compared to texts written by human authors?. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 44) (2022). https://escholarship.org/uc/item/1wx3983m.
- [15]Rahmeh, H. Digital verses versus inked poetry: Exploring readers’ response to AI-generated and human-authored Sonnets. Sch. Int. J. Linguist. Lit. 6 (9), 372–382 (2023). · DOI
- [16]Köbis, N. & Mossink, L. D. Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Comput. Hum. Behav. 114, 106533. 10.1016/j.chb.2020.106553 (2021). · DOI · Paper page
- [17]Hitsuwari, J., Ueda, Y., Yun, W. & Nomura, M. Does human–AI collaboration lead to more creative art? Aesthetic evaluation of human-made and AI-generated haiku poetry. Comput. Hum. Behav. 139, 107–502. 10.1016/j.chb.2022.107502 (2023). · DOI · Paper page
- [18]Benjamin, D. J. et al. Redefine statistical significance. Nat. Hum. Behav. 2, 6–10. 10.1038/s41562-017-0189-z (2018). 30980045 · DOI · Paper page
- [19]Bates, D., Kliegl, R., Vasishth, S. & Baayen, H. Parsimonious mixed models. arXiv (2015). 10.48550/arXiv.1506.04967. · DOI
- [20]Kennedy, X. J. & Gioia, D. An Introduction to Poetry 13th edn. (Pearson, 2009).
- [21]ten Berge, J. M. F., Krijnen, W. P., Wansbeek, T. & Shapiro, A. Some new results on correlation-preserving factor scores prediction methods. Linear Algebra Appl. 289, 311–318 (1999). · DOI
- [24]Acquisti, A., Brandimarte, L. & Hancock, J. How privacy’s past may shape its future. Science 375, 270–272 (2022). 35050645 · DOI
- Authors
- Porter, B., & Machery, E.
- Affiliations
- University of Pittsburgh (US)
- Corresponding author
- Porter, Brian
- ORCID
- Porter, Brian
- Venue
- Scientific Reports
- Published
- 2024-11-14
- Volume
- 14
- Issue
- 1
- Pages
- 26133-26133
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC11564748
- OpenAlex
- W4404347568
- Cited by
- 133
- References
- 20
- Citations by year
- 2026: 66 · 2025: 65 · 2024: 2
- FWCI
- 32.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Scientific Reports (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Aesthetic Perception and Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Poetry Analysis and Criticism
- Keywords
- AI-generated poetry, authorship attribution, non-expert readers, rhythm, aesthetic judgment, heuristics, generative AI
- MeSH terms
- Adult, Artificial Intelligence, Female, Humans, Male, Poetry as Topic, Writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
