Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 18 of the 36 references it cites were found in a registry or an index, and 2 are not checked yet; 17 of the 31 sources its claims cite were checked for support.

How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction

Open access
Asano, Yuki, Bastos, Marco · 2026 · Journal of Creative Communications

Abstract

This study examines the quality of generative and collaborative AI for fictional prose in Japanese. We rolled out an experiment followed by focus groups to elicit responses to 25 texts with the second half randomly presenting the original text, zero-shot-AI-generated text, AI-generated with prompt specification, and human-AI collaboration using multiple-shot and prompt specification. We found that human-AI collaboration had the highest perceived quality, followed by the original text, AI-generated with prompt specification, and finally zero-shot AI-generated content. While participants mentioned the relative absence of originality in human-AI-collaboration, this content was deemed more human-like than the original text due to the polished editing and selection resulting from inputting drafts, adding or replacing nouns, and rearranging sentences and dialogues. The results presented in this study offer a positive outlook on the use of Generative AI for creative writing, but they also highlight the limitations of these tools for developing highly original literary work.

Verified 2026-10-12 · 2 references not checked yet

Overview

CoChat Summary

TL;DR

This study examines Japanese readers’ rankings of 25 stories whose second halves varied across four conditions: original human writing, zero-shot AI generation, detailed-prompt AI generation, and human-AI collaboration. Human-AI collaboration received the highest mean quality rating but was rated lowest in originality, and the only robust quality contrast was that zero-shot AI generation was rated lower than collaboration.

Key findings

  • Human-AI collaboration (D) had the highest mean score at 145.0, followed by original human writing (A) at 139.6, detailed prompt specification (C) at 134.6, and zero-shot generation (B) at 130.7; Table 1 reports A = 139.9, B = 130.4, C = 134.7, D = 145.0.
  • The only statistically significant pairwise quality contrast was D vs. B (Cohen’s d = .56); D vs. A (d = .20) and D vs. C (d = .39) were small and not significant, while B vs. A was marginally significant.
  • Between-condition variance was .061 versus average within-condition variance of .643, and original human writing showed the highest variance (SD = 32.8) while human-AI collaboration showed the lowest (SD = 20.2), suggesting AI’s main effect is reduced variance rather than higher mean quality.
  • Human-AI collaboration was rated highest in perceived quality but lowest in originality; focus groups described it as polished, natural, and “safe” but lacking distinctive originality, while original human writing produced polarized responses.

Methods

  • The corpus comprised 25 Japanese stories, each with a shared human-authored first half and four alternative second halves: A original human-written text, B zero-shot ChatGPT-4 generation, C detailed prompt specification, and D human-AI collaboration with human editing; this yielded 100 story versions and over 73,000 Japanese syllables.
  • Sixty participants were recruited, with 58 valid responses analyzed; participants had backgrounds in creative writing, Japanese literature, or related fields, and 86.2% were native Japanese speakers. They ranked all four endings per story in randomized order using scores of 10, 7, 4, and 1, between July and August 2024.
  • Two focus groups followed in mid-August 2024 with 4 and 3 participants, respectively; discussions were transcribed, anonymized, and translated. Quantitative analysis used average scores, t-tests, and mixed-effects models with lme4 and ordinal packages in R.

Limitations

  • The paper states that some Version A excerpts were drawn from early in their source narratives, which may have introduced apparent incoherence where scenes skipped key plot developments; it recommends future studies use longer, more self-contained text samples.
  • The paper states that focus groups were small and drawn from the experimental sample; the two sessions also diverged on whether AI-generated content was distinguishable.
  • Evident from the text: the sample is not representative of general readers because participants were selected for creative writing or literature backgrounds, and the study is limited to Japanese fiction, short passages, and ChatGPT-4 at a specific time.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/09732586261463934

Cites

18 of 36 references verified0 citation errors · 16 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 7 Not valid · 9 Not checked · 54 Own results and interpretations

14 more claims

Create a free account to see all 80 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 25
  • It may state two separate claims as one. 50
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Show all 34 references

Metadata

Authors
Asano, Y., & Bastos, M.
Affiliations
  • City St George's, University of London (GB)
  • Department of Media, Culture and Creative Industries, City St George’s, University of London, UK
Venue
Journal of Creative Communications
Published
2026-08-30
ISSN
  • 0973-2586 (ISSN-L)
  • 0973-2594
OpenAlex
W7133554842
Cited by
0
References
34
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Creative Communications (publishedVersion, cc-by-nc)
  • AoIR Selected Papers of Internet Research (publishedVersion)
  • SSRN Electronic Journal (acceptedVersion)
  • City Research Online (City University London) (submittedVersion)
Funders
  • University College Dublin (R20825, R21650)
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (406504/2022-9, /2022-9)
Topics
  • Social Sciences › Law › Literary and Philosophical Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Artistic and Creative Research
Keywords
generative AI, human-AI collaboration, AI-generated text, reader perception, perceived quality, originality, creative writing, AI-assisted editing, stylistic expression

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction. https://app.cochat.ai/papers/10.1177/09732586261463934
The paper
Asano, Y., & Bastos, M. (2026). How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction. Journal of Creative Communications. https://doi.org/10.1177/09732586261463934

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.