CoChat
Sign Up
Cited by 63Creativity in Education and NeuroscienceTopic ModelingArtificial Intelligence in Games

The language of creativity: Evidence from humans and large language models

Open access
Orwig, William, Edenbaum, Emma R., Greene, Joshua D., Schacter, Daniel L.·2024·The Journal of Creative Behavior
DOI 10.1002/jocb.636PMC PMC11065427
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent developments in computerized scoring via semantic distance have provided automated assessments of verbal creativity. Here, we extend past work, applying computational linguistic approaches to characterize salient features of creative text. We hypothesize that, in addition to semantic diversity, the degree to which a story includes perceptual details, thus transporting the reader to another time and place, would be predictive of creativity. Additionally, we explore the use of generative language models to supplement human data collection and examine the extent to which machine‐generated stories can mimic human creativity. We collect 600 short stories from human participants and GPT‐3, subsequently randomized and assessed on their creative quality. Results indicate that the presence of perceptual details, in conjunction with semantic diversity, is highly predictive of creativity. These results were replicated in an independent sample of stories (n = 120) generated by GPT‐4. We do not observe a significant difference between human and AI‐generated stories in terms of creativity ratings, and we also observe positive correlations between human and AI assessments of creativity. Implications and future directions are discussed.

Integrity signals

Registry

Crossref · 10.1002/jocb.636

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Within GPT-3 stories, perceptual details were positively associated with creativity (b = .09, 95% CI [0.03, 0.15], t = 2.92, p = .004).

    “Additionally, we observed a positive association between the presence of perceptual details and creativity (b = .09,95% CI [0.03, 0.15], t = 2.92, p = .004).”
    Quote in the paperSubstantial claim
  • Findingown data

    Perceptual details were positively associated with creativity in the full sample (b = .12, 95% CI [0.08, 0.17], t = 5.40, p < .001), controlling for word count and prompt.

    “Additionally, we observed a positive association between the presence of perceptual details and creativity (b = .12,95% CI [0.08, 0.17], t = 5.40, p < .001).”
    Quote in the paperSubstantial claim
  • Findingown data

    Within GPT-3 stories, DSI was highly predictive of creativity (b = .18, 95% CI [0.11, 0.25], t = 4.90, p < .001).

    “Within the sample of GPT-3 stories, we observed that DSI was highly predictive of creativity (b = .18,95% CI [0.11, 0.25], t = 4.90, p < .001).”
    Quote in the paperSubstantial claim
  • Findingown data

    The model with DSI and perceptual details explains approximately half of the observed variance in creativity ratings (R2 = .49).

    “This model explains approximately half of the observed variance in creativity ratings (R 2 = .49).”
    Quote in the paperSubstantial claim
  • Findingown data

    Within human stories, DSI was highly predictive of creativity (b = .32, 95% CI [0.24, 0.39], t = 8.63, p < .001).

    “Within the sample of human stories, we observed that DSI was highly predictive of creativity (b = .32,95% CI [0.24, 0.39], t = 8.63, p < .001).”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 29
  • May combine two claims. 12

  • The quote only partly states this claim.

    Findingown data

    DSI was highly predictive of creativity in the full sample (b = .23, 95% CI [0.18, 0.27], t = 9.55, p < .001), controlling for word count and prompt.

    “Within the full sample, we observed that DSI was highly predictive of creativity (b = .23,95% CI [0.18, 0.27], t = 9.55, p < .001).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The model predicts that a story written by a human will have a creativity score 0.07 points higher than a story from GPT-3 of the same length and prompt.

    “Our model predicts that a story written by a human will have a creativity score 0.07 points higher than a story from GPT-3 of the same length and prompt.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When pooling human, GPT-3, and GPT-4 stories, GPT-4 stories tend to score lower in creativity than human stories, but the difference is not significant (t = -1.98, p = .05).

    “When pooling human, GPT-3 and GPT-4 stories, we found that compared to human stories, both GPT-3 (t = À1.66,p = .10)and GPT-4 (t = À1.98,p = .05)tend to score lower in creativity, though this difference was not significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When pooling human, GPT-3, and GPT-4 stories, GPT-3 stories tend to score lower in creativity than human stories, but the difference is not significant (t = -1.66, p = .10).

    “When pooling human, GPT-3 and GPT-4 stories, we found that compared to human stories, both GPT-3 (t = À1.66,p = .10)and GPT-4 (t = À1.98,p = .05)tend to score lower in creativity, though this difference was not significant.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To ensure that neither source of text was solely driving the effect, the analysis was repeated within sub-samples of human and GPT-3 stories.

    “To ensure that neither source of text was solely driving the effect, we repeated this analysis within sub-samples of human and GPT-3 stories.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Recent studies have shown impairment of perceptual memory retrieval following hippocampal damage.

    “Recent studies have shown impairment of perceptual memory retrieval following hippocampal damage (St-Laurent, Moscovitch, Jadd, & McAndrews, 2014;St-Laurent, Moscovitch, & McAndrews, 2016).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites St-Laurent, Moscovitch, Jadd, & McAndrews, 2014

      Not checked in the free run.

    • cites St-Laurent, Moscovitch, & McAndrews, 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationcitation

    This advancement surpasses earlier computational approaches, including those based solely on semantic distance, highlighting the multifaceted utility of large language models in both generating and evaluating creative content.

    “This advancement surpasses earlier computational approaches, including those based solely on semantic distance, highlighting the multifaceted utility of large language models in both generating and evaluating creative content.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    If future imagination and creative thinking are mutually supported by episodic processes, a logical extension is that creative stories will contain a high degree of episodic details.

    “If future imagination and creative thinking are mutually supported by episodic processes, a logical extension is that creative stories will contain a high degree of episodic details.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Raters were not informed that stories were generated from humans and AI.

    “Raters were not informed that stories were generated from humans and AI.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    A computerized version of the five-sentence creative story task was used to assess creative writing.

    “To assess creative writing, we used a computerized version of the five-sentence creative story task (Johnson et al., 2022;Prabhakaran, Green, & Gray, 2014).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Johnson et al., 2022

      Not checked in the free run.

    • cites Prabhakaran, Green, & Gray, 2014

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Current theories suggest that both semantic and episodic memory processes contribute to the emergence of creative ideas.

    “Current theories suggest that both semantic and episodic memory processes contribute to the emergence of creative ideas (Benedek, Beaty, Schacter, & Kenett, 2023).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Benedek, Beaty, Schacter, & Kenett, 2023

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    GPT-4 scoring was applied to each story independently, ensuring that the model evaluated them based solely on their content, without the influence of previous assessments or additional contextual information.

    “This scoring method was applied to each of the stories independently, ensuring that the model evaluated them based solely on their content, without the influence of previous assessments or additional contextual information.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Attention checks were included to ensure participant engagement.

    “Attention checks were included to ensure participant engagement.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    To capture a general assessment of creativity and ensure that no individual rater had too great an influence, 12 ratings were collected for each story.

    “To capture a general assessment of creativity and ensure that no individual rater had too great an influence, we collect 12 ratings for each story.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A posterior predictive check was performed to ensure that the model usefully mimics the observed data.

    “First, we performed a posterior predictive check to ensure that the model usefully mimics the observed data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Perceptual details, such as sights, sounds and smells, are a central feature of episodic memory.

    “Perceptual details, such as sights, sounds and smells, are a central feature of episodic memory (e.g., Brunel, Labeye, Lesourd, & Versace, 2009;Conway, 2001;Saive, Royet, & Plainly, 2014;Wheeler, Petersen, & Buckner, 2000).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Brunel, Labeye, Lesourd, & Versace, 2009

      Not checked in the free run.

    • cites Conway, 2001

      Not checked in the free run.

    • cites Wheeler, Petersen, & Buckner, 2000

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Informed consent was obtained from all participants prior to participation, with protocol approval from the Institutional Review Board of Harvard University.

    “Informed consent was obtained from all participants prior to participation, with protocol approval from the Institutional Review Board of Harvard University.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Perceptual details are particularly relevant to creative writing and may serve to illustrate the role of imagination in the construction of creative narratives.

    “We contend that perceptual details are particularly relevant to creative writing, and may serve to illustrate the role of imagination in the construction of creative narratives.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    LIWC performed an automated count for words falling into psychological categories defined by an integrated dictionary.

    “Stories were entered into the Linguistic Inquiry and Word Count program (LIWC), which performed an automated count for words falling into a variety of different psychological categories defined by an integrated dictionary (Boyd, Ashokkumar, Seraj, & Pennebaker, 2022;Pennebaker et al., 2001).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Boyd, Ashokkumar, Seraj, & Pennebaker, 2022

      Not checked in the free run.

    • cites Pennebaker et al., 2001

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The presence of perceptual details is a salient feature of creative narratives, serving to construct a rich scene and transport the reader to an alternative time and place.

    “We suggest that the presence of perceptual details is a salient feature of creative narratives, serving to construct a rich scene and transport the reader to an alternative time and place.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the context of poetry, GPT-3 generated text is indistinguishable from human poems.

    “Other studies have demonstrated that in the context of poetry, GPT-3 generated text is indistinguishable from human poems (K€ obis & Mossink, 2021;Liao, Wang, Liu, & Jiang, 2019).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites K€ obis & Mossink, 2021

      Not checked in the free run.

    • cites Liao, Wang, Liu, & Jiang, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Provoking sensory details with descriptions of smells, tastes, sights, and sounds may allow readers to more deeply engage with a narrative.

    “Provoking sensory details with descriptions of smells, tastes, sights, and sounds may allow readers to more deeply engage with a narrative.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data

    The findings from this study have implications for the assessment of verbal creativity and shed light on the potential role of large language models in augmenting human data collection for creative tasks.

    “The findings from this study have implications for the assessment of verbal creativity and shed light on the potential role of large language models in augmenting human data collection for creative tasks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    DSI is a well-established and validated measure of semantic diversity.

    “We assessed the extent to which a story connects divergent ideas using DSI: a well-established and validated measure of semantic diversity (Johnson et al., 2022).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Johnson et al., 2022

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    The present study contributes to research on cognitive processes underlying creative thinking.

    “In conclusion, the present study contributes to the growing body of research on the cognitive processes that underlie creative thinking.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    In creative writing, episodic memory may enable individuals to retrieve and incorporate specific sensory details, leading to more descriptive narratives.

    “In the context of creative writing, episodic memory may enable individuals to retrieve and incorporate specific sensory details, leading to more descriptive narratives.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Relatively less is known about the contributions of episodic memory to creativity.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    The measure of episodic detail used by van Genugten et al. (2022) is distinct from our assessment of perceptual detail.

    “While the findings of van Genugten, Beaty, Madore, and Schacter (2022) did not show a reliable association between episodic detail and originality of stories, their measure of episodic detail is distinct from our assessment of perceptual detail.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Flexibility of semantic memory structure contributes to creative thinking.

    “Applications of network science have led to a deeper understanding how flexibility of semantic memory structure contribute to creative thinking (Abraham, 2014;Kenett, 2018).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Abraham, 2014

      Not checked in the free run.

    • cites Kenett, 2018

      Not checked in the free run.

    • Rests on an assumption: Semantic memory can be represented as a network structure with related concepts stored nearby in a semantic space.

  • May combine two claims.

    Interpretationreasoning

    The findings are consistent with the framework and suggest a possible role for episodic memory processes in generating perceptual details in creative narratives.

    “Our findings are consistent with this framework and suggest a possible role for episodic memory processes in generating perceptual details in creative narratives.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Relatively less is known about the contributions of episodic memory to creativity.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Perceptual detail can evoke vivid images and sensory experiences in the reader's mind, producing a more immersive and transportive reading experience.

    “Perceptual detail refers to the richness and specificity of sensory information in a text, and can be used to evoke vivid images and sensory experiences in the reader's mind that produce a more immersive and transportive reading experience.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The choice of prompt can greatly influence the quality and content of generated stories, especially when using higher temperatures in generative language models.

    “We recognize that the choice of prompt can greatly influence the quality and content of the generated stories, especially when using higher temperatures in generative language models.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Creative writing in both humans and GPT-3/GPT-4 makes use of perceptual details that are similar to sensory details typically ascribed to episodic memory in humans, as well as novel semantic associations.

    “While we do not wish to claim that these large language models have an experience of episodic remembering as humans do, we suggest that creative writing in both humans and GPT-3/GPT-4 make use of perceptual details that are similar to sensory details typically ascribed to episodic memory in humans, as well as novel semantic associations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Relatively less is known about the contributions of episodic memory to creativity.

  • May combine two claims.

    Interpretationown data

    The present study shows that semantic diversity and perceptual details are constituent features of creative writing in short stories written by humans and large language models.

    “The present study highlights constituent features of creative writing, specifically the role of semantic diversity and perceptual details in short stories written by humans and large language models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Semantic memory plays a relatively larger role when creative thinking relies primarily on conceptual information, whereas episodic memory processes are more relevant to tasks requiring complex imagination involving mental simulations and construction of scenes.

    “Benedek, Beaty, Schacter, and Kenett (2023) contend that semantic memory could play a relatively larger role when creative thinking relies primarily on conceptual information, whereas episodic memory processes could be more relevant to tasks and strategies requiring more complex forms of imagination involving mental simulations and construction of scenes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Benedek, Beaty, Schacter, and Kenett (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Studies have finetuned a variety of LLMs to predict human creativity ratings in divergent thinking and problem-solving tasks.

    “These studies finetuned a variety of LLMs to predict human creativity ratings in divergent thinking tasks and problemsolving tasks.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Luchini et al., 2023

      Not checked in the free run.

    • cites Organisciak, Acar, Dumas, & Berthiaume, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Both semantic and episodic memory play a crucial role in the creative process.

    “A proposed framework for understanding the role of memory in creative ideation (Benedek, Beaty, Schacter, & Kenett, 2023) suggests that both semantic and episodic memory play a crucial role in the creative process.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Benedek, Beaty, Schacter, & Kenett, 2023

      Not checked in the free run.

36 of 36

  • The paper relies on this without support.

    The role of perceptual details in creative cognition has not yet been systematically examined.

    “Despite the growing literature on episodic contributions to creativity, the role of perceptual details in creative cognition has not yet been systematically examined.”

  • The paper relies on this without support.

    Semantic memory can be represented as a network structure with related concepts stored nearby in a semantic space.

    “Semantic memory can be represented as a network structure, where related concepts are stored nearby in a semantic space.”

  • The paper relies on this without support.

    Relatively less is known about the contributions of episodic memory to creativity.

    “Relatively less is known about the contributions of episodic memory to creativity.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Much research on GenAI and its promise for creativity tends to be about singular, relatively short tasks that are mostly unconstrained.

    “Much research on GenAI and its promise for creativity tends to be about singular, relatively short tasks that are mostly unconstrained (e.g., write a poem on some topic and in a certain style, generate alternative uses, provide a list of ideas).”

    From The Impact of Generative AI on Creativity in Software Development: A Research Agenda (2025) · cites this paper as [98]

    The paper’s opening discussion describes GenAI creativity studies using the Alternative Uses Task and poetry, supporting the examples of brief, stand-alone creative tasks. It does not establish that most research uses mostly unconstrained tasks; its own Methods describe a five-sentence story task requiring three specified words. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Empirical evidence on the creativity of LLM outputs is mixed.

    “Empirical evidence on the creativity of LLM outputs is mixed.”

    From The Impact of Generative AI on Creativity in Software Development: A Research Agenda (2025) · cites this paper as [98]

  • Some studies find LLMs performing at the same level as humans on creative thinking tasks.

    “Various studies find LLMs perform-ing at the same level as humans on creative thinking tasks. For instance, the creativity of human and GenAI produced stories was found to exhibit no significant differences [98].”

    From The Impact of Generative AI on Creativity in Software Development: A Research Agenda (2025) · cites this paper as [98]

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(28, 3731) = 7.38reported p < .001 · recomputed p < .001

    “F (28, 3731) = 7.38, p < .001”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Semantic diversity is said to explain over half the variance, exceeding the full model

    Minor

    The full model includes semantic diversity alongside perceptual details and controls, yet explains 49% of the variance. Semantic diversity alone cannot explain over half of the variance accounted for by that same full model.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Pennebaker J.W., 2001, Linguistic inquiry and word count (LIWC): LIWC2001
  2. 10.1073/pnas.97.20.1112 · DOI

Cited by

Sorted by citations

Related

Authors
Orwig, W., Edenbaum, E. R., Greene, J. D., & Schacter, D. L.
Affiliations
Harvard University (US)
Corresponding author
Orwig, William
ORCID
Venue
The Journal of Creative Behavior
Publisher
Wiley
Published
2024-01-11
Volume
58
Issue
1
Pages
128-136
ISSN
  • 0022-0175 (ISSN-L)
  • 2162-6057
DOI
10.1002/jocb.636
PMC
PMC11065427
OpenAlex
W4390740352
Semantic Scholar
4679d083fb4a90dd1bdeb5f86a6d8a84b9968f1e
Cited by
56
References
45
Citations by year
2026: 20 · 2025: 28 · 2024: 8
FWCI
22.78
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • The Journal of Creative Behavior (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute on Aging (AG008441)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords
verbal creativity, creative texts, perceptual detail, semantic diversity, computational linguistics, large language models, GPT-3, GPT-4

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review