CoChat
Sign Up
PreprintCited by 9Authorship Attribution and ProfilingArtificial Intelligence in GamesTopic Modeling

StoryScope: Investigating idiosyncrasies in AI fiction

Open access
Russell, Jenna, Rajendhran, Rishanth, Pham, Chau Minh, Iyyer, Mohit, Wieting, John·2026·arXiv
DOI 10.48550/arxiv.2604.03136arXiv 2604.03136
Open this paper in CoChat — free account

Abstract

As AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluated. While most existing work in this space focuses on identifying surface-level signatures of AI writing, we ask instead whether AI-generated stories can be distinguished from human ones without relying on stylistic signals, focusing on discourse-level narrative choices such as character agency and chronological discontinuity. We propose StoryScope, a pipeline that automatically induces a fine-grained, interpretable feature space of discourse-level narrative features across 10 dimensions. We apply StoryScope to a parallel corpus of 10,272 writing prompts, each written by a human author and five LLMs, yielding 61,608 stories, each ~5,000 words, and 304 extracted features per story. Narrative features alone achieve 93.2% macro-F1 for human vs. AI detection and 68.4% macro-F1 for six-way authorship attribution, retaining over 97% of the performance of models that include stylistic cues. A compact set of 30 core narrative features captures much of this signal: AI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonist' choices as more morally ambiguous and have increased temporal complexity. Per-model fingerprint features enable six-way attribution: for example, Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.

AI snapshot

Written from the paper’s full text.

TL;DR

StoryScope induces 304 interpretable discourse-level narrative features and shows that narrative features alone distinguish human from AI fiction at 93.2% macro-F1 and six-way authorship at 68.4% macro-F1. AI stories cluster in a shared narrative region and favor explicit themes, tidy causal chains, and linear plots, while human stories are more morally ambiguous, temporally complex, and rarer in narrative feature space.

Key findings

  • Binary human-vs-AI detection: Narrative features (257) reach 93.2% macro-F1 / .959 AUPRC; Narrative+Style (304) reaches 96.0% / .982; Style Only (39) 85.8% / .867; Core Only (30) 84.8% / .828; Core+Fingerprint (101) 91.1% / .934. Text baselines are stronger: ModernBERT 99.9%, Stylometric+XGB 99.8%, TF-IDF+XGB 99.7%, Binoculars 55.9%. After LAMP stylistic edits to 278 Gemini stories, narrative detection is 93.9% macro-F1 vs. 95.5% on unedited stories.
  • Six-way authorship attribution: Narrative features reach 68.4% macro-F1, Narrative+Style 77.3%, Style Only 60.4%, Core Only 46.5%, Core+Fingerprint 63.4%. Per-class narrative-only F1: Human 0.89, Claude 0.77, GPT 0.73, Gemini 0.60, DeepSeek 0.57, Kimi 0.55. Text baselines all report at least 99.5% macro-F1.
  • Narrative differences: AI stories over-explain themes (narrator thematic commentary 77% vs. 52% human), use dialogue for philosophical debate more often (59% vs. 34%), convey emotion through embodied cues more often (81% vs. 38%), favor protagonist-driven resolutions (69% vs. 46%), and have “no subplots” more often (79% vs. 57%). Human stories use explicit named intertextual references more often (47% vs. 24%), break the fourth wall more often (67% vs. 39%), address the reader directly more often (28% vs. 7%), and present morally ambivalent protagonists more often (59% vs. 38%).
  • Rarity and clustering: Human stories have higher mean rarity percentile than AI stories (0.71 vs. 0.49; Cohen’s d = 0.83); 24.7% of human stories fall in the top 10% rarest corpus-wide vs. 7.1% of AI stories. The human story is ranked rarest of all six versions 57.8% of the time vs. 16.7% by chance. Mean human-AI centroid distance is 1.6× mean AI-AI centroid distance (6.6 vs. 4.3), and human stories are more dispersed (mean distance to human centroid 33.2 vs. average AI radius 27.4).

Methods

  • Parallel corpus: 10,272 human-written stories from Books3, with writing prompts reverse-engineered via Gemini 2.5 Flash, then stories generated by five LLMs (Gemini 3 Flash, Kimi K2.5, DeepSeek V3.2, Claude Sonnet 4.6, GPT-5.4), yielding 61,608 stories averaging 4,753 words and 304 extracted features per story. The paper releases 10,272 prompts, 51,336 AI-generated narratives, and narrative features, but not human stories due to copyright.
  • Pipeline: Stories are converted into structured NarraBench-grounded templates using GPT-5.1; cross-source comparisons are run on a 600-story discovery pool over 100 prompts; feature discovery proposes 408 candidates, deduplicated via embedding clustering to 304 features across five response types. Feature assignment uses Gemini 3 Flash, then XGBoost classifiers with SHAP-based feature-role analysis identify 30 core and 75 fingerprint features.
  • Evaluation: Binary and six-way classifiers are trained on encoded feature vectors with prompt-level grouping; main text reports training on 52,434 stories (8,739 prompts) and a held-out test set of 1,377 prompts (8,262 stories). Baselines include ModernBERT, Stylometric+XGB, TF-IDF+XGB, and Binoculars. Additional analyses include length-matched evaluation, LAMP stylistic edits, topic analysis, and a memorization-risk screen.

Limitations

  • The paper states that human-written stories are not released due to copyright concerns, and that Books3 is used strictly for academic purposes with no endorsement for model training or commercial text generation. It also reports a memorization-risk audit flagging 352 of 50,672 generated-story comparisons (0.70%) as high risk, with a filtered rerun leaving main conclusions essentially unchanged.
  • The paper reports length-confound and topic checks: a length-matched test subset leaves Narrative macro-F1 unchanged at 93.2%, and a Kruskal-Wallis test finds no significant topic-wise differences (H = 4.69, p = 0.46). It does not present a dedicated limitations section.
  • Evident from the text: feature extraction and discovery rely on LLM judgments (GPT-5.1, Gemini 3 Flash), with human-model agreement κ = 0.84 and repeatability α = 0.90, so measurement error remains possible. The corpus uses Books3 short stories and reverse-engineered prompts, and the AI sources are specific model versions, so generalization to other human writing contexts, prompts, or future models is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2604.03136

Cites

75 of 75 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

10 of 25 cited sources checked · 4 supported · 1 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction · p. 2

    AI-generated text systematically overuses em-dashes, words like 'delve' and 'tapestry,' and other surface-level patterns that even simple classifiers detect reliably.

    “AI-generated text systematically overuses em-dashes, words like 'delve' and 'tapestry,' and other surfacelevel patterns that even simple classifiers detect reliably (Sun et al., 2025; Shaib et al., 2026).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Sun et al., 2025 source

      Section 3.1 (p. 6) finds model-specific word choices and characteristic phrases that a simple TF-IDF logistic-regression classifier identifies with high accuracy. But the paper does not establish overuse of em-dashes, “delve,” or “tapestry”; its classifiers distinguish among LLMs, rather than reliably detecting AI-generated text against human writing (Introduction, p. 1; Related Work, p. 9). (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites Shaib et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation
    § 1 Introduction · p. 2

    Discourse-level narrative features are far harder to 'humanize' than stylistic features, as changing them requires significant structural rewrites rather than simple post-hoc edits.

    “Discourse-level narrative features (e.g., plot structure, character agency, information revelation), which we refer to simply as narrative features throughout, are far harder to 'humanize,' as changing them requires significant structural rewrites rather than simple post-hoc edits (Namuduri et al., 2025).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Namuduri et al., 2025 source

      The Introduction (p. 1) distinguishes discourse flow from within-sentence patterns, and the analysis (p. 8) finds that LLM texts have more uniform discourse structures than human texts. The paper does not establish that narrative features are harder to humanize than stylistic features or that changing them requires structural rewrites rather than post-hoc edits. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Findingown data
    § 1 Introduction · p. 2

    Narrative features with style achieve a macro-F1 of 77.3% for six-way authorship attribution.

    “In the six-way authorship attribution task (identifying which source wrote a story), narrative features get a macro-F1 of 68.4% without style, compared to 77.3% with style.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.
  • Findingown data
    § 1 Introduction · p. 2

    Narrative differences between human and AI stories remain after editing stories for style, with macro-F1 93.9%.

    “and that these changes remain after editing stories for style (macro-F1 93.9%).”
    • Also stated in: § 4.2 Do narrative features still work after stylistic changes? · p. 8 · plain
      “After span-level artifact removal, the narrative model detects edited Gemini stories at 93.9% macro-F1 (AUPRC 0.988), compared to 95.5% macro-F1 (AUPRC 0.996) on the original unedited stories, a drop of only 1.6 points.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.
    • Rests on an assumption: The LAMP rewriting framework removes only stylistic artifacts and does not alter narrative structure.
  • Findingnone given
    § 2.2 STORYSCOPE interpretability · p. 5

    Not all features contribute equally to distinguishing sources.

    “Not all features contribute equally to distinguishing sources.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.
  • May combine two claims. 63
  • Classified as probably not a substantial claim. 21
  • The quote only partly states this claim. 9

  • May combine two claims.

    Study design and conductcitation
    § 2.1 STORYSCOPE · p. 3

    The template-based intermediate representation preserves narrative elements while abstracting away surface wording, forcing downstream stages to reason over narrative content rather than stylistic surface.

    “Templates convert prose into structured narrative fields, forcing downstream stages to reason over narrative content rather than stylistic surface.”
    • Also stated in: § 2.1 STORYSCOPE · p. 4 · plain
      “The templates serve as a controlled intermediate representation (Pham et al., 2024; Wang et al., 2023): by converting prose into structured fields, the subsequent comparison stage reasons over narrative content rather than stylistic surface”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Pham et al., 2024 source

      The Introduction and §3.2 describe generating topic labels and assigning them to documents, while Appendix J contains prompt templates for those tasks. Searches for narrative fields, an intermediate representation, and downstream reasoning over content rather than style found no passage addressing the claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites Wang et al., 2023 source

      The paper does not describe templates that convert prose into structured narrative fields. More importantly, Section 5.2 and Table 4 apply the proposed clustering method specifically to writing style, contrary to the claim that its templates force downstream stages to reason over content rather than style. (LLM, conf=0.76)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related work · p. 9

    LLM text exhibits systematic lexical and syntactic patterns.

    “LLM text exhibits systematic lexical and syntactic patterns (Shaib et al., 2026; Kobak et al., 2025; Shaib et al., 2025)”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites Shaib et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Kobak et al., 2025 source

      The Discussion’s Summary and Related work sections report systematic excess use of LLM-preferred words, supporting the lexical part of the claim. The paper does not establish systematic syntactic patterns; its analysis concerns word frequencies rather than sentence structure. Page numbers were not available in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites Shaib et al., 2025 source

      The Abstract (p. 1) says syntactic templates often appear in model outputs, supporting the claim about syntactic patterns in LLM text. Related Work (p. 9) mentions lexical patterns as features exploited by NLP models, but does not establish systematic lexical patterns in LLM-generated text. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related work · p. 10

    Constrained generation from human text fragments further blurs the authorship boundary.

    “constrained generation from human text fragments further blurs the authorship boundary (Pham et al., 2025)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Pham et al., 2025 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6 Related work · p. 10

    Prompt-based comparisons reveal that human stories contain more cultural nuance, emotional ambiguity, and unexpected twists than GPT-generated counterparts.

    “prompt-based comparisons reveal that human stories contain more cultural nuance, emotional ambiguity, and unexpected twists than GPT-generated counterparts (Beguš, 2024).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Beguš, 2024 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related work · p. 9

    Detecting partial AI edits to human text remains harder.

    “Detecting partial AI edits to human text remains harder (Thai et al., 2026; Chen et al., 2025; He et al., 2025).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Thai et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Chen et al., 2025 source
    • cites He et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related work · p. 10

    Injecting human-written stories as reward anchors during reinforcement learning improves small models' long-form story writing.

    “injecting human-written stories as reward anchors during reinforcement learning improves small models' long-form story writing (Rajendhran et al., 2026).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Rajendhran et al., 2026 source

  • May combine two claims.

    Findingown data
    § 4 Detecting AI from narrative features · p. 6

    Models built using only style features achieve 85.8% macro-F1, comparable to the core-only model's 84.8% macro-F1.

    “Models built using only the style features achieve comparable performance to the core-only model (85.8% and 84.8% macro-F1 respectively; Table 2), confirming that style remains a strong detection cue, especially in long-context settings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    The largest confused pair is Gemini and DeepSeek (222 and 207 stories), which dwarfs the most common human misclassification (Human to Kimi, 46).

    “the largest pair (Gemini ↔ DeepSeek, 222 and 207 stories) dwarfs the most common human misclassification (Human → Kimi, 46).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    Kimi has the fewest fingerprints and lowest F1, sitting at the generic center of the AI distribution with no distinctive narrative choices.

    “Kimi has the fewest fingerprints and lowest F1, sitting at the generic center of the AI distribution with no distinctive narrative choices.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 5 Pinpointing each source's writing style · p. 9

    DeepSeek front-loads crucial context that other sources leave until later.

    “DeepSeek front-loads crucial context that other sources leave until later.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    AI stories are more explicit and moralizing (roughly 20% higher on a 1-5 scale), with tighter thematic unity and more central moral questions.

    “AI stories are more explicit and moralizing (roughly 20% higher on a 1-5 scale), with tighter thematic unity and more central moral questions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 8

    GPT is the third most distinctive source, with 82.1% F1 with style and 73.0% without.

    “and GPT (82.1% / 73.0%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    Human stories are more dispersed: their mean distance to the human centroid is 22% greater than the average AI radius (33.2 vs. 27.4) and their median 10-nearest-neighbor radius is 1.13 times larger (33.1 vs. 29.2).

    “Human stories are also more dispersed: their mean distance to the human centroid is 22% greater than the average AI radius (33.2 vs. 27.4) and their median 10-nearest-neighbor radius is 1.13 × larger (33.1 vs. 29.2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    Narrators explicitly explain the story's theme 77% of the time in AI stories, versus 52% for humans.

    “Narrators explicitly explain the story's theme 77% of the time, versus 52% for humans: a grieving character's arc will typically end with the narrator stating the lesson learned.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingnone given
    § 5 Pinpointing each source's writing style · p. 9

    GPT centers on socially-oriented storytelling: gossip and rumor as a plot mechanism (64% vs. 44-55% for other sources), a tendency to frame stories as reflections on events from years or decades ago, and ensemble-heavy social networks matching human levels.

    “GPT centers on socially-oriented storytelling: gossip and rumor as a plot mechanism (64% vs. 44-55% for other sources), a tendency to frame stories as reflections on events from years or decades ago, and ensemble-heavy social networks matching human levels.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    Claude favors epilogues and avoids dream sequences, producing careful, consistent stories that favor quiet endings over 'avalanche' endings.

    “It favors epilogues and avoids dream sequences, producing careful, consistent stories that favor quiet endings over 'avalanche' endings.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    The five AI sources occupy overlapping regions of narrative feature space, separate from human authors.

    “The five AI sources occupy overlapping regions of narrative feature space, separate from human authors (Figure 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Do narrative features still work after stylistic changes? · p. 8

    After span-level artifact removal, the narrative model detects edited Gemini stories at 93.9% macro-F1 (AUPRC 0.988), compared to 95.5% macro-F1 (AUPRC 0.996) on original unedited stories, a drop of only 1.6 points.

    “After span-level artifact removal, the narrative model detects edited Gemini stories at 93.9% macro-F1 (AUPRC 0.988), compared to 95.5% macro-F1 (AUPRC 0.996) on the original unedited stories, a drop of only 1.6 points.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.
    • Rests on an assumption: The LAMP rewriting framework removes only stylistic artifacts and does not alter narrative structure.

  • May combine two claims.

    Findingnone given
    § 5 Pinpointing each source's writing style · p. 9

    Gemini produces the tidiest endings, extended denouements, and the bleakest settings (88% tagged bleak and oppressive).

    “Gemini produces the tidiest endings, extended denouements, and the bleakest settings (88% tagged bleak and oppressive).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § 7 Conclusion · p. 10

    Human stories show greater structural diversity and occupy a rarer, more dispersed region of narrative space.

    “while human stories show greater structural diversity and occupy a rarer, more dispersed region of narrative space.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Human stories from Books3 are representative of human-authored narratives.

  • May combine two claims.

    Findingown data
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    AI pays closer attention to physical environment and characters' inner mental states, showing emotion through bodies and environments rather than naming feelings directly.

    “AI pays closer attention to physical environment and characters' inner mental states, showing emotion through bodies and environments rather than naming feelings directly.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 8

    In the narrative-only model, human and Claude retain strong separability while DeepSeek, Gemini, and Kimi fall to 55.2-59.6% F1.

    “The gap widens in the narrative-only model: human and Claude retain strong separability while the bottom three models fall to 55.2-59.6%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § 5 Pinpointing each source's writing style · p. 9

    Claude's stories are defined by restraint: event intensity escalates less than in any other source, and narrative voice is the most uniform.

    “Its stories are defined by restraint: event intensity escalates less than in any other source, and narrative voice is the most uniform.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 8

    The narrative+style model achieves 77.3% macro-F1 for six-way attribution.

    “Narr. + Style | 304 | 77.3 | 77.3%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    AI favors single-track narratives with fewer loose ends; human stories are messier, with time jumps and disjointed causal chains.

    “AI favors single-track narratives with fewer loose ends; human stories are messier, with time jumps and disjointed causal chains-a human mystery might open at the funeral and spiral backward through decades, while AI tells the same story from first clue to the grand reveal.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingnone given
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    Humans use more time jumps, flashbacks and flash-forwards, and nonlinear structure to delay key revelations.

    “Humans use more time jumps, flashbacks and flash-forwards, and nonlinear structure to delay key revelations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingnone given
    § 5 Pinpointing each source's writing style · p. 9

    GPT subverts expectations more than other AI (41% vs. 27-36%) and leaves reconciliations ambiguous.

    “GPT subverts expectations more than other AI (41% vs. 27-36%) and leaves reconciliations ambiguous.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7 Conclusion · p. 10

    AI stories are systematically more thematically explicit, causally tidy, and temporally linear.

    “AI stories are systematically more thematically explicit, causally tidy, and temporally linear”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Human stories from Books3 are representative of human-authored narratives.

  • May combine two claims.

    Findingown data
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    AI resolutions favor internal understanding or acceptance (47% vs. 27%), whereas humans are more comfortable with ambiguous endings.

    “AI resolutions favor internal understanding or acceptance (47% vs. 27%), whereas humans are more comfortable with ambiguous endings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Findingown data
    § 5 Pinpointing each source's writing style · p. 9

    24.7% of human stories fall in the top 10% rarest stories corpus-wide, compared to 7.1% of AI stories; in the top 1%, 3.0% are human vs. 0.6% AI.

    “24.7% of all human stories fall in the top 10% rarest stories corpus-wide, compared to just 7.1% of AI stories (top 1%: 3.0% human vs. 0.6% AI).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § C Application of features · p. 18

    The remaining aspect-based misses were concentrated in a small number of harder-to-apply features, mostly in setting, whereas single-call application showed broad systematic dropout, especially in revelation and temporal-structure features.

    “The remaining aspect-based misses were concentrated in a small number of harder-to-apply features, mostly in setting, whereas single-call application showed broad systematic dropout, especially in revelation and temporal-structure features.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § E Dimension-level ablations · p. 20

    No dimension is individually sufficient, with the best single dimension being agents at 80.2% binary macro-F1.

    “No dimension is individually sufficient (best: agents at 80.2% binary macro-F1) and none is individually necessary (worst removal: agents at -1.2), so the signal is redundant across correlated dimensions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § C Application of features · p. 18

    Aspect-based application produced higher average coverage than single-call application over 12 story files, with average coverage rising from 68.4% to 95.4% of features.

    “Aspect-based application produced far more complete feature vectors: average coverage rose from 68.4% of features to 95.4% of features.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § H Per-story rarity analysis · p. 23

    Human stories are overrepresented in all tails relative to a 1/6 baseline, but AI stories are present throughout.

    “Human stories are overrepresented in all tails relative to a 1 6 baseline, but AI stories are present throughout.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given
    § I Core and fingerprint features · p. 23

    Table 14 lists the 20 features that reliably characterize AI writing in the binary task, ranked by a core score computed as mean SHAP multiplied by stability score and by one plus the absolute human-AI gap for the qualifying encoded column.

    “Table 14 lists the 20 features that reliably characterize AI writing in the binary task, ranked by a core score computed as mean SHAP multiplied by stability score and by one plus the absolute human-AI gap for the qualifying encoded column.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B STORYSCOPE details · p. 17

    The direct pipeline's features are style-heavy (humor usage, vocabulary register, allusion types, dominant imagery), while the template pipeline's features are structure-heavy (emotional arcs, relationship trajectories, event density, flashback usage).

    “The direct pipeline's features are style-heavy (humor usage, vocabulary register, allusion types, dominant imagery), while the template pipeline's are structure-heavy (emotional arcs, relationship trajectories, event density, flashback usage).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § H Per-story rarity analysis · p. 23

    Using the standard sample-size-weighted pooled SD over the 1,377 human and 6,885 AI test stories gives Cohen's d = 0.83, with sample SDs of 0.227 for human stories and 0.274 for pooled AI stories and a pooled SD of 0.267.

    “Using the standard sample-size-weighted pooled SD over the 1,377 human and 6,885 AI test stories gives Cohen's d =0.83, with sample SDs of 0.227 for human stories and 0.274 for pooled AI stories and a pooled SD of 0.267.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § F Memorization risk in story generation · p. 21

    In the filtered rerun after dropping high-risk prompts, the Narrative + Style model improves slightly from 96.00 to 96.06 macro-F1 in binary detection and from 77.29 to 77.56 macro-F1 in six-way attribution.

    “Dropping the high-risk prompts leaves the main detectability conclusions essentially unchanged. In the filtered rerun, the Narrative + Style model improves slightly from 96.00 to 96.06 macro-F1 in binary detection and from 77.29 to 77.56 macro-F1 in six-way attribution; the Narrative-only model shifts from 93.18 to 93.23 in binary and from 68.42 to 68.68 in multiclass (see Table 10).”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    AI writing is elevated relative to human writing on thematic over-determination features, including thematic explicitness and moralizing (AI mean 3.94 vs human 3.28), moral/philosophical weighting, thematic unity, narratorial thematic commentary, dialogue function as philosophical debate, and reference explicitness.

    “AI-elevated: Thematic over-determination s”
    • Also stated in: § J Prompts · p. 26 · plain
      “Thematic Explicitness &Moralizing | 3.28 | 3.94 | - 0.65 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § H Per-story rarity analysis · p. 23

    Human stories are shifted toward higher rarity, with mean percentile 0.71 vs. 0.49 for AI and AUC = 0.73.

    “Human stories are shifted toward higher rarity (mean percentile 0.71 vs. 0.49 for AI, AUC = 0.73).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § B STORYSCOPE details · p. 17

    A parallel Gemini 3 Flash run permits fewer, larger batches but produces substantially fewer features than the GPT-5.1 outputs.

    “A parallel Gemini 3 Flash run permits fewer, larger batches but produces substantially fewer features, so we retain the GPT-5.1 outputs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § E Dimension-level ablations · p. 20

    No dimension is individually necessary, with the worst removal being agents at -1.2.

    “No dimension is individually sufficient (best: agents at 80.2% binary macro-F1) and none is individually necessary (worst removal: agents at -1.2), so the signal is redundant across correlated dimensions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § H Per-story rarity analysis · p. 23

    The rarest test-set tail is mixed: the top 1% contains 42 human and 41 AI stories; the top 5% contains 180 human vs. 234 AI; the top 10% contains 340 human vs. 487 AI stories across five models.

    “The rarest test-set tail is mixed: the top 1% contains 42 human and 41 AI stories; the top 5% contains 180 human vs. 234 AI; the top 10% contains 340 human vs. 487 AI stories across five models (Table 13).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B STORYSCOPE details · p. 17

    Feature deduplication reduced the taxonomy from 408 to 304 features (25.5% reduction) and merged 65 multi-feature clusters.

    “We keep the feature nearest each cluster centroid as the representative, reducing the taxonomy from 408 to 304 features (25.5% reduction) and merging 65 multi-feature clusters.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data
    § I Core and fingerprint features · p. 25

    Table 15 lists 13 core human-characterizing features, with arrows indicating the specific option value elevated for human authors.

    “Table 15: Core human-characterizing features (13), Arrow ( → ) indicates the specific option value elevated for human authors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § H Per-story rarity analysis · p. 23

    At the prompt level, the human story is rarer than all five AI alternatives 57.8% of the time, with a mean margin of 0.96 raw distance units.

    “At the prompt level, the human story is rarer than all five AI alternatives 57.8% of the time (mean margin=0.96 raw distance units).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § C Application of features · p. 19

    Aggregate inter-run agreement across the five runs is Krippendorff's α (nominal) = 0.90, with mean pairwise Cohen's κ = 0.89.

    “Aggregate inter-run agreement across the five runs is Krippendorff's α (nominal) = 0.90, mean pairwise Cohen's κ = 0.89.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § Distribution of story lengths across all stories · p. 18

    For the strict narrative variant used in the final numbers, the exclusion set contains 47 features, leaving 257 features in the strict non-style taxonomy used by the final narrative runs.

    “For the strict narrative variant used in the final numbers, the exclusion set contains 47 features. This leaves 257 features in the strict non-style taxonomy used by the final narrative runs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § F Memorization risk in story generation · p. 21

    In the filtered rerun after dropping high-risk prompts, the Narrative-only model shifts from 93.18 to 93.23 macro-F1 in binary and from 68.42 to 68.68 macro-F1 in multiclass.

    “Dropping the high-risk prompts leaves the main detectability conclusions essentially unchanged. In the filtered rerun, the Narrative + Style model improves slightly from 96.00 to 96.06 macro-F1 in binary detection and from 77.29 to 77.56 macro-F1 in six-way attribution; the Narrative-only model shifts from 93.18 to 93.23 in binary and from 68.42 to 68.68 in multiclass (see Table 10).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § C Application of features · p. 19

    Minimal thinking caused a 0.45% reduction in performance on the binary task and an 11.61% increase in performance on the six-way attribution task.

    “On a smaller test of full vs. minimal thinking, the minimal thinking caused a 0.45% reduction in performance on the binary task and an 11.61% increase in performance on the six-way attribution task.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    Human writing is elevated relative to AI writing on reader engagement, including fourth-wall permeability (human mean 0.67 vs AI 0.39) and direct reader address (0.28 vs 0.07).

    “Human-elevated: Reader engagement”
    • Also stated in: § J Prompts · p. 26 · plain
      “Fourth-Wall Permeability o | 0.67 | 0.39 | +0.28 |”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    Human writing is elevated relative to AI writing on intertextual richness, including explicit named reference (human 47% vs AI 24%) and a balanced mix of reference explicitness.

    “Human-elevated: Intertextual richness”
    • Also stated in: § J Prompts · p. 26 · plain
      “Intertextual Strategy → explicit named reference | 47% | 24% | +23 |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    AI writing is elevated relative to human writing on structural streamlining features, including causal chain continuity (AI mean 4.20 vs human 3.92), spatial granularity, agency in resolution as protagonist choice, character introduction as external description, subplot integration as no subplots, resolution mode as internal understanding, opening spatial grounding, and pre-threat character投资.

    “AI-elevated: Structural streamlining”
    • Also stated in: § J Prompts · p. 26 · plain
      “Causal Chain Continuity s | 3.92 | 4.20 | - 0.28 |”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    Human writing is elevated relative to AI writing on temporal complexity, including depth of recontextualization after surprise (human mean 3.28 vs AI 2.95), chronological discontinuity, nonlinear framing for delayed disclosure, and anachrony intensity.

    “Human-elevated: Temporal complexity”
    • Also stated in: § J Prompts · p. 26 · plain
      “Depth of Recontextualization After Surprise s | 3.28 | 2.95 | +0.34 |”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    AI writing is elevated relative to human writing on sensory and embodied performativity features, including emotional expression as embodied (AI 81% vs human 38%), setting as psychological mirror, environmental and ecological emphasis, sensory modalities as olfactory, sensory density, and depth of interior access.

    “AI-elevated: Sensory &embodied performativity”
    • Also stated in: § J Prompts · p. 26 · plain
      “Emotional Expression → embodied | 38% | 81% | - 42 |”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § J Prompts · p. 26

    Human writing is elevated relative to AI writing on narrative diversity, including location variety scope (human mean 1.34 vs AI 1.08), dialogue-to-narration proportion, subplot integration as thematically parallel (42% vs 21%), moral polarity as ambivalent/mixed (59% vs 38%), and emotional expression as explicit labels (29% vs 8%).

    “Human-elevated: Narrative diversity”
    • Also stated in: § J Prompts · p. 26 · plain
      “Location Variety Scope o | 1.34 | 1.08 | +0.26 |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4 Detecting AI from narrative features · p. 6

    Most automatic AI detection models have higher performance on longer text.

    “Most automatic AI detection models have higher performance on longer text (Bao et al., 2024; Xu et al., 2025b).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Bao et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Xu et al., 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 3 Experiments · p. 5

    All evaluations use prompt-level grouping to prevent train/test leakage.

    “All evaluations use prompt-level grouping to prevent train/test leakage.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.1 What core narrative choices separate human from AI writing? · p. 7

    Human writing acknowledges its audience as a co-participant; AI writes as though no one is watching.

    “Human writing acknowledges its audience as a co-participant (e.g., an aside to 'you, dear reader'); AI writes as though no one is watching.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Interpretationown data
    § 4.2 Do narrative features still work after stylistic changes? · p. 8

    Narrative features are largely orthogonal to the surface prose artifacts that LAMP targets: editing out clichéd phrasing or purple prose does not alter the structural narrative choices that drive the classifier.

    “This near-zero effect indicates that narrative features are largely orthogonal to the surface prose artifacts that LAMP targets: editing out clichéd phrasing or purple prose does not alter the structural narrative choices (causal linearity, thematic explicitness, sensory over-description) that drive our classifier.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.
    • Rests on an assumption: The LAMP rewriting framework removes only stylistic artifacts and does not alter narrative structure.

  • May combine two claims.

    Interpretationown data
    § 4 Detecting AI from narrative features · p. 7

    The human-AI boundary lives in a compact subset of narrative decisions, while fingerprint features contribute model-specific separability.

    “Our results suggest that the human-AI boundary lives in a compact subset of narrative decisions, while fingerprint features contribute model-specific separability.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.
    • Rests on an assumption: The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.
    • Rests on an assumption: The five LLMs used are representative of AI fiction generation.

  • May combine two claims.

    Premisecitation
    § 4.2 Do narrative features still work after stylistic changes? · p. 8

    Style is the most obvious surface-level cue distinguishing AI generations from human writing, and style-based detectors are known to be brittle to paraphrasing and light editing.

    “Style is the most obvious surface-level cue distinguishing AI generations from human writing, and style-based detectors are known to be brittle to paraphrasing and light editing (Saakyan et al., 2026).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Saakyan et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § D Experimental setup · p. 20

    The final split layout is 7,383 train / 1,405 val / 1,384 test prompts; hyperparameters are tuned on the validation split, final models are retrained on train+val (8,788 prompts; 52,707 stories), and all final results are reported on the held-out test set (1,384 prompts, 8,301 stories).

    “The final split layout is 7,383 train / 1,405 val / 1,384 test prompts. We tune hyperparameters on the validation split, then retrain each final model on train+val (8,788 prompts; 52,707 stories) and report all final results on the held-out test set (1,384 prompts, 8,301 stories).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § Distribution of story lengths across all stories · p. 18

    A style-dependence audit was conducted over all 304 taxonomy features, prompting GPT-5.4 to rate each as high, medium, or low style dependence.

    “We conducted a style-dependence audit over all 304 taxonomy features, prompting GPT-5.4 (OpenAI, 2026) to rate each as high , medium , or low style dependence.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites OpenAI, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data
    § C Application of features · p. 19

    The reliability of Gemini 3 Flash (minimal thinking) as the production feature extractor was evaluated via a repeated-measures design yielding 300 total extraction outputs and, for fully present items, 600 within-item run pairs.

    “Weevaluate the reliability of Gemini 3 Flash (minimal thinking) as the production feature extractor via a repeated-measures design. This yields 300 total extraction outputs and, for fully present items, 600 within-item run pairs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related work · p. 10

    Narrative theory provides structural grounding for generation and understanding.

    “Narrative theory provides structural grounding for generation and understanding (Liu et al., 2026a)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Liu et al., 2026a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductreasoning
    § F Memorization risk in story generation · p. 21

    The memorization analysis measures output overlap rather than formal training-set membership, so it is treated as a sensitivity analysis rather than proof that a model saw a specific story during pretraining.

    “We measure output overlap rather than formal training-set membership, so we treat it as a sensitivity analysis rather than proof that a model saw a specific story during pretraining.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § C Application of features · p. 18

    The aspect-based variant preserves near-complete coverage by restricting each call to one narrative dimension, while later dimensions are omitted more often in the single-call setup.

    “Later dimensions are omitted more often in the single-call setup, while the aspect-based variant preserves near-complete coverage by restricting each call to one narrative dimension.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § B STORYSCOPE details · p. 17

    Source identities are anonymized in all LLM-facing prompts throughout the pipeline, and presentation order is randomized in pairwise template comparisons to reduce positional bias while keeping comparisons reproducible.

    “Throughout the pipeline, source identities are anonymized in all LLMfacing prompts. Within pairwise template comparisons, presentation order is randomized to reduce positional bias while keeping the comparisons reproducible.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § G Additional analysis · p. 22

    The length-confound audit re-evaluates the frozen final binary classifiers on a decile-stratified length-matched test subset with 2,754 stories (1,377 human, 1,377 AI) and nearly identical medians (human 4,973 vs. AI 4,951 words).

    “and re-evaluate them on a decile-stratified length-matched test subset with 2,754 stories (1,377 human, 1,377 AI) and nearly identical medians (human 4,973 vs. AI 4,951 words).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § G Additional analysis · p. 22

    Topic shifts the point estimates somewhat, especially for multiclass attribution, but there is no strong evidence that detectability changes systematically by topic.

    “So topic shifts the point estimates somewhat, especially for multiclass attribution, but we do not find strong evidence that detectability changes systematically by topic.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § E Dimension-level ablations · p. 20

    AI-generated fiction has a recognizable narrative profile: more linear plots, tighter causal structure, more explicit thematic framing, and less diverse character and social structure often co-occur.

    “This is expected, since AI-generated fiction has a recognizable narrative profile: e.g., more linear plots, tighter causal structure, more explicit thematic framing, and less diverse character and social structure often co-occur.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6 Related work · p. 10

    Fine-tuning on literary corpora produces preferred text but activates verbatim memorization.

    “Fine-tuning on literary corpora produces preferred text (Chakrabarty et al., 2026) but activates verbatim memorization (Liu et al., 2026b)”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Chakrabarty et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Liu et al., 2026b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § H Per-story rarity analysis · p. 22

    Per-story rarity is computed as the mean Euclidean distance to a story's k = 25 nearest neighbors in the pooled train+val narrative space; rarity percentiles are computed against the train+val rarity distribution; and all effect sizes are computed over individual stories, not per-prompt summaries.

    “To operationalize the originality definition from §1, we compute per-story rarity as the mean Euclidean distance to a story's k = 25 nearest neighbors in the pooled train+val narrative space. Rarity percentiles are computed against the train+val rarity distribution: a story at the 90th percentile sits in a sparser region of narrative space than 90% of all train+val stories. All effect sizes here a”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § J Prompts · p. 28

    Each story is generated from a standalone narrative prompt in the Books3-derived prompt set, and the same prompt text is given to all six sources.

    “We do not use a single writing template; each story is generated from a standalone narrative prompt in the Books3-derived prompt set, and the same prompt text is given to all six sources.”
    Quote in the paperSubstantial claim
75 of 75

  • The paper relies on this without support.

    The human-written stories in Books3 are representative of human-authored fiction and are correctly attributed to human authors.

    “We extract 10,272 human-written stories from Books3 (Presser, 2020).”

  • The paper relies on this without support.

    The writing prompts reverse-engineered from human stories faithfully capture the underlying premise, so that AI-generated stories are valid mirrors of the human stories.

    “To generate mirrored AI stories, we reverse-engineer writing prompts from each human story by prompting Gemini 2.5 Flash (Gemini Team, 2025a) to infer the underlying premise (Li et al., 2024).”

  • The paper relies on this without support.

    The five LLMs used are representative of AI fiction generation.

    “We generate stories from five LLMs: Gemini 3 Flash (Gemini Team, 2025b), Kimi K2.5 (Kimi Team, 2026), DeepSeek V3.2 (DeepSeek-AI, 2025), Claude Sonnet 4.6 (Anthropic, 2026), and GPT-5.4 (OpenAI, 2026).”

  • The paper relies on this without support.

    The LAMP rewriting framework removes only stylistic artifacts and does not alter narrative structure.

    “We test this using Chakrabarty et al. (2025)'s span-level rewriting framework (LAMP), which identifies and rewrites seven categories of AI writing artifacts (e.g., cliché, redundant exposition, purple prose) using 25 few-shot examples from professional writers.”

  • The paper relies on this without support.

    Human stories from Books3 are representative of human-authored narratives.

    “All stories extracted from Books3.”

  • The paper relies on this without support.

    The human corpus includes stories that are likely to be widely available on the public web and in Books3-adjacent collections.

    “Because the human corpus includes stories that are likely to be widely available on the public web and in Books3-adjacent collections, we run a conservative contamination-risk audit over the reverse-engineered outputs.”

  • The paper relies on this without support.

    Memorization could lead to the inclusion of style atypical to AI writing.

    “Memorization could lead to the inclusion of style atypical to AI writing.”

  • The paper relies on this without support.

    Qwen3.5-9B's BISAC code assignments accurately categorize the topics of the test stories.

    “We prompted Qwen3.5-9B (Alibaba Cloud, 2026) to give a Book Industry Standards and Communications (BISAC) code (Book Industry Study Group, 2024) given a story text, and apply this over the test set.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting training and test-set sizes

    Major

    The experiments section reports 8,739 training prompts and 1,377 test prompts, while the experimental setup reports 8,788 train-plus-validation prompts and 1,384 test prompts. The corresponding story counts also differ, so the paper gives incompatible sizes for its reported evaluations.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. Gemini Team. Gemini: A family of highly capable multimodal models, 2023. URL https: //arxiv.org/abs/2312.11805 .
  2. Gérard Genette. Narrative Discourse: An Essay in Method . Cornell University Press, Ithaca, NY, 1980.
  3. Kimi Team. Kimi K2.5: Visual agentic intelligence, 2026. URL https://arxiv.org/abs/2602. 02276 .
  4. Theodore Sturgeon. Tiny and the monster. In Isaac Asimov and Martin H. Greenberg (eds.), Isaac Asimov Presents The Great SF Stories 9 (1947) . DAW Books, 1983. Originally published in Astounding Science Fiction, May 1947.
  5. Ellis Paul Torrance. Torrance Tests of Creative Thinking: Norms-Technical Manual: Verbal Tests, Forms A and B; Figural Tests, Forms A and B . Personnel Press, 1966.
  6. Yiming Zhang, Harshita Diddee, Susan Holm, Hanchen Liu, Xinyue Liu, Vinay Samuel, Barry Wang, and Daphne Ippolito. NoveltyBench: Evaluating creativity and diversity in language models. In Second Conference on Language Modeling , 2025a. URL https: //openreview.net/forum?id=XZm1ekzERf .

Cited by

Sorted by citations

Authors
Russell, J., Rajendhran, R., Pham, C. M., Iyyer, M., & Wieting, J.
Affiliations
Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2604.03136
arXiv
2604.03136
OpenAlex
W7151072005
Semantic Scholar
83f711b4e0db299151df9a08608d34541f97eeae
Cited by
0
References
57
Type
posted content
Open access
Licence
Locations
arXiv (Cornell University) (public-domain)
Funders
National Science Foundation (2046248, 2312949)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
AI-generated fiction, authorship attribution, narrative characteristics, character agency, large language models, originality, narrative structure, moral ambiguity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review