Cited by 10Humor Studies and ApplicationsComplex Network Analysis TechniquesMisinformation and Its Impacts

Competition Dynamics in the Meme Ecosystem

Open access
Ford, Trenton W., Krohn, Rachel, Weninger, Tim·2023·ACM Transactions on Social Computing

Abstract

Creating and sharing memes is a common modality of online social interaction. Because of the prevalence of memes, an abundance of research focuses on understanding how memes are shared and perceived within their online social environment and in what ways they differ from other modalities of online communication. In the present work, we present findings that suggest a relationship between how memes are introduced, shared, and diminish in online social systems and the behavior of biological species within an ecosystem. Framed by this perspective, we borrow ecological methodologies and concepts like resource availability, speciation, and competition to study human attention, creativity, and sharing dynamics (respectively) over a large collection of memes shared on Reddit. Specifically, we find that the population of memes has scaled almost exactly with the total amount of content created over the past decade. We find a consequence of limited human attention in the face of a growing number of memes is that the diversity of memes has decreased at the community level, albeit slightly, in the same period. A further consequence is that the average life span of memes has decreased dramatically, which is further evidence of an increase in competition and a decrease in a meme species’ primary resource: human attention. From this work, we have found reasonable preliminary support for linking memes to species and, thus, meme research to ecological research. We believe that future research should work to strengthen this relationship and transfer more methodologies to meme and information research from ecology.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper treats text memes as ecological species and finds that, on Reddit from 2010–2020, meme activity remains a stable fraction of platform content even as Reddit grows. It also finds slightly decreasing community-level meme diversity and dramatically shorter meme life spans, suggesting intensified competition for limited human attention.

Key findings

  • Collective attention to text memes is stationary after normalizing for Reddit growth: Reddit Text mean normalized meme frequency has Pearson R = +0.03, p < 0.01, and ADF S = −4.73, p ≈ 7.17e−5; Google Trends shows Pearson R = +0.02, p < 0.01, but ADF S = −2.55, p ≈ 0.10.
  • Community-level meme diversity decreased slightly: Simpson diversity fell about 0.48% yearly and Shannon about 0.12% yearly, with Pearson R = −0.63 for Simpson and R = −0.47 for Shannon.
  • Meme life spans shrank over time: average life span decreased with α = 0.005 (Pearson R = −0.85, p < 0.01), α = 0.01 (R = −0.77, p < 0.01), and α = 0.02 (R = −0.59, p < 0.01).
  • Meme “beachhead” subreddits show turnover: Kendall’s τ for consecutive-year innovation rankings trended downward until 2017/2018, then reversed, indicating that highly innovative subreddits are not consistently innovative over time.

Methods

  • Collected 352 popular text-based memes from KnowYourMeme in January 2020, ranging from one to eight word tokens, plus all Reddit posts and comments from January 2010 to January 2020 via PushShift; used Google Trends via GTAB for Web-scale corroboration.
  • Normalized meme frequencies using a background language model of 5,000 randomly sampled Reddit tokens; computed monthly Shannon and Simpson diversity indexes per subreddit, meme life spans using thresholds α = 0.005, 0.01, and 0.02, and mean reciprocal rank for subreddit meme beachheads.
  • Used Pearson correlation, Augmented Dickey-Fuller unit root tests, and Kendall’s τ rank correlation to assess trends and stationarity.

Limitations

  • Stated: the KnowYourMeme dataset likely underrepresents newer memes because popularity is cumulative; the study is text-only and uses exact text matching, omitting visual memes and meme mutations; sampling intervals cannot be guaranteed; and the work assumes rather than proves a meme–species relationship.
  • Stated: Reddit–Google Trends correlation was only moderate across memes (median Pearson R = 0.499; 61% of pairs had R > 0.3), limiting confidence in Web-scale generalization.
  • Evident: life-span results depend on the chosen α threshold, and the authors removed memes still active at data collection end to avoid recency bias, which may affect the life-span estimates.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3596213

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 21 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The average relative peak frequency for all memes grouped by peak year shows no change in peak dynamics over time.

    “Overall, there appears to be no change in peak dynamics over time.”
    Quote in the paperSubstantial claim
  • Findingown data

    For the 352 text-based memes, Google Trends and Reddit meme popularity trends matched relatively well.

    “We found that Google Trends and Reddit meme popularity trends matched relatively well.”
    Quote in the paperSubstantial claim
  • Findingown data

    The peak use cumulative distribution of the memes shows a delay.

    “As expected, the peak use CDF shows a delay, and the peak use of the memes is relatively uniform.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For Reddit Text, there is almost no association between time and the normalized meme frequency (Pearson R = +0.03, p-value < 0.01).

    “and for Reddit Text (Pearson R = +0.03,p-value < 0.01).”
    Quote in the paperSubstantial claim
  • Findingown data

    As years progressed, the top contributing subreddits became less consistent.

    “However, as years progressed, the top contributing subreddits became less consistent.”
    Quote in the paperSubstantial claim
  • May combine two claims. 14
  • Classified as probably not a substantial claim. 20
  • May combine two claims.

    Findingown data

    The average life span of memes has decreased dramatically, and average lifecycles have become shorter as more memes are created.

    “A further consequence is that the average life span of memes has decreased dramatically, which is further evidence of an increase in competition and a decrease in a meme species' primary resource: human attention.”
    • Also stated in: § 1 INTRODUCTION · plain
      “Finally, we find that (RQ3) as more memes have been created, their average lifecycles have become shorter”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Only the Shannon diversity index averages are stationary; Simpson diversity is non-stationary.

    “Using ADF along with AIC (Akaike information criterion) to optimize test parameters, we find that only for the Shannon diversity index averages is the distribution stationary.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Both Simpson and Shannon diversity indexes show a negative correlation between average community diversity and time with significant p-values.

    “For both Simpson and Shannon diversity indexes, Pearson's test statistic shows a negative correlation between diversity and time with significant p-values.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The peak use of the memes is relatively uniform.

    “As expected, the peak use CDF shows a delay, and the peak use of the memes is relatively uniform.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

    Findingown data

    In the Reddit data, more posts, comments, and memes are being made at an accelerating pace year after year.

    “At first glance, Figure 2 appears to show that our data support these claims: more posts, comments, and memes are being made at an accelerating pace year after year.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Confidence intervals for average Simpson and Shannon diversity are very small, typically about 0.1%.

    “Confidence intervals are present in Figure 4, but they are very small: typically about 0.1%.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    The unigram background model's behavior closely mirrors the growth of Reddit but is one order of magnitude larger.

    “This background behavior closely mirrors the growth of Reddit but is one order of magnitude larger.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Reddit continues to grow.

    “although Reddit continues to grow”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The difference between peak and baseline frequency remains relatively stable and does not change substantially from year to year.

    “The difference between the peak and the baseline frequency (i.e., frequency before and after the peak) remains relatively stable, nor does it change substantially from year to year.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The number of active memes has increased, mirroring Reddit's overall growth.

    “As expected, the number of active memes has increased, mirroring Reddit's overall growth.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The background model B(t) accurately controls for Reddit's growth when normalizing meme frequencies.
  • May combine two claims.

    Findingown data

    Meme life spans have decreased significantly over 10 years, with average life span decreasing under multiple thresholds (α = 0.005: Pearson R = -0.85, p < 0.01; α = 0.01: Pearson R = -0.77, p < 0.01; α = 0.02: Pearson R = -0.59, p < 0.01).

    “Finally, we have shown that meme life spans have decreased significantly.”
    • Also stated in: § 8 CONCLUSION · plain
      “Over 10 years, the average life span has decreased (α = 0.005: Pearson R = -0.85,p-value <0.01; α = 0.01: Pearson R = -0.77,p-value < 0.01; α = 0.02: Pearson R = -0.59,p-value <0.01)”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    There has been an increase in competition and a decrease in a meme species' primary resource: human attention.

    “which is further evidence of an increase in competition and a decrease in a meme species' primary resource: human attention.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: When the meme marketplace overflows with content—an excess of supply—the content must necessarily compete for consumer attention.
    • Rests on an assumption: Human attention is a limited resource.
  • May combine two claims.

    Premisecitation

    Beginning in June 2017, Reddit removed the default subreddits, including /r/pics, /r/funny, and many other most innovative subreddits, and introduced /r/popular, a mix of posts from various subreddits.

    “beginning in June 2017, Reddit removed the default subreddits, which included /r/pics, /r/funny, and many of the other most innovative subreddits, and instead introduced /r/popular, which was a mix of posts from various subreddits.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous work has shown that innovation and technological development is accelerating.

    “Previous work has shown that innovation and technological development is accelerating.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [41]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [29]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Shannon's and Simpson's diversity indexes take into account both the number of unique species present and the population of each species; Simpson's weights dominance more heavily, while Shannon's weights richness and evenness more heavily.

    “These indexes take into account both the number of unique species present, as well as the population of each species.However, Simpson's considers the dominance of a species more heavily, whereas Shannon's considers the richness and evenness of present species more heavily.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study received a research exemption from the University of Notre Dame Institutional Review Board.

    “We sought and received a research exemption from the University of Notre Dame Institutional Review Board.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    The costs historically associated with the production and consumption of information have decreased dramatically, whereas the cognitive cost of consumption appears to be increasing.

    “Due, in part, to the rise of online media platforms, the costs historically associated with the production and consumption of information have decreased dramatically, whereas the cognitive cost of consumption appears to be increasing [24].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Recent work suggests that collective attention dynamics are accelerating.

    “Recent work suggests that these dynamics are accelerating [26].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Ecological research focuses on environments hosting new species because they can host exotic species along with high levels of species diversity.

    “Ecological research pays special attention to the environments in which species exist; the communities and biomes that host new species are often the focus of study because they can host exotic species along with high levels of species diversity [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    New content is becoming viral faster and staying viral for less time.

    “As a result, new content is becoming viral faster and staying viral for less time.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    The detection and analysis of highly innovative communities are of great interest in this line of work, as these communities are essential contributors to online culture.

    “The detection and analysis of highly innovative communities are of great interest in this line of work, as these communities are essential contributors to online culture [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In online social systems, early empirical evidence suggests that social innovations are accelerating.

    “In online social systems, the early empirical evidence suggests that a similar pattern exists: that social innovations are accelerating [17,26,37,38].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [17]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [26]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [37]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [38]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Memes can reasonably map onto an ecological framing, and ecological questions and techniques can produce meaningful results in meme research.

    “This analysis not only shows that memes can reasonably map onto an ecological framing, but that we can borrow questions and techniques directly from ecology to drive inquiry in meme research and produces meaningful results.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductreasoning

    The methodology of ranking subreddits by order of first meme appearance reasonably controls uncertainty about which community was first, second, and so forth.

    “Under these circumstances, we elect to rank subreddits based on the order in which a meme first appears; this methodology reasonably controls the uncertainty about which community was first, second, and so forth, to articulate the innovation.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: A single meme may have multiple beachheads within Reddit before it becomes widespread, and each beachhead may contribute to the meme's eventual popularity.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Both Shannon and Simpson diversity indexes are generally used together to capture the complete picture of diversity.

    “Generally, both indexes are used together to capture the complete picture of diversity.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Exchanging ideas via memes has become one of the most popular and effective methods of sharing ideas in online social systems.

    “Exchanging ideas via memes has become one of the most popular and effective methods of sharing ideas in online social systems [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    This work does not prove a relationship between memes and an ecological counterpart.

    “this work does not prove a relationship between memes and an ecological counterpart; it simply assumed that such a relationship existed and analyzed memes as if so.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Memes can be treated as species or as exhibiting species-like behaviors at the individual, species, population, community, and ecosystem levels.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Because the research makes no claims about individual users and presents population-level and environment-level trends, it does not expose users to direct assertions about their behavior.

    “Further, we believe that because this research makes no claims about individual users and presents findings about population-level and environment-level trends, we do not expose users to direct assertions about their behavior.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Long-lasting memes have transcended their origins and evolved from memes into accepted slang, maintaining a consistent presence even as Reddit grows and other memes come and go.

    “These long-lasting memes have transcended their origins and evolved from memes into accepted slang, maintaining a consistent presence even as Reddit grows, and other memes come and go.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    Much of online meme culture, especially recently, is conveyed through visual depictions of memes.

    “Much of the online meme culture, especially recently, is conveyed through visual depictions of memes [46].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The analysis and results are based on aggregations of user-level information.

    “The analysis and results presented in this article are based on aggregations of userlevel information.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Study design and conductnone given

    Specific examples highlighted in the research are without sufficient context to reasonably reassociate them with any particular use.

    “Any specific examples highlighted within the research are without sufficient context to reasonably reassociate them with any particular use.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The focus on textual artifacts of memes may limit the generalizability of the findings.

    “Our focus on the textual artifacts of memes may also limit the generalizability of our findings.”
    Quote in the paperNot a substantial claim
33 of 33
  • The authors state this as an assumption.

    Memes are species and meme species are similar to biological species.

    “We begin with the assumption that memes are species and that meme species are similar to biological species.”
  • The authors state this as an assumption.

    The spaces in which memes exist are ecosystems.

    “We consider the spaces in which these memes exist as ecosystems.”
  • The paper relies on this without support.

    Memes exist within a habitat with many other meme and non-meme species.

    “Memes exist within a habitat with many other meme and non-meme species.”
  • The paper relies on this without support.

    Many species are active simultaneously, so they must coexist and compete for resources within a shared habitat.

    “Because many species are active simultaneously, they must coexist and compete for resources within a shared habitat.”
  • The paper relies on this without support.

    When the meme marketplace overflows with content—an excess of supply—the content must necessarily compete for consumer attention.

    “When the meme marketplace overflows with content-an excess of supply-the content must necessarily compete for consumer attention.”
  • The paper relies on this without support.

    Human attention is a limited resource.

    “We find a consequence of limited human attention in the face of a growing number of memes is that the diversity of memes has decreased at the community level, albeit slightly, in the same period.”
  • The authors state this as an assumption.

    Memes in online social systems can be modeled as species within a physical environment.

    “The foundation of the present work is that memes in online social systems map to species within a physical environment.”
  • The paper relies on this without support.

    Reddit is a natural habitat for memes to flourish.

    “Given its size, popularity, and abundance of well-defined communities, Reddit is a natural habitat for memes to flourish.”
  • The paper relies on this without support.

    The collected dataset of 352 text memes is varied and large enough to support the analysis and the conclusions drawn from it.

    “Despite the limitations, we believe that the dataset is varied and large enough to support our analysis and the conclusions we draw from it.”
  • The paper relies on this without support.

    A large segment of online meme culture is dedicated to visual meme forms (images, gifs, and videos).

    “A large segment of online meme culture is dedicated to visual meme forms: images, gifs, and videos.”
  • The paper relies on this without support.

    A given habitat has limited resources shared between all resident species, limiting the total population of the habitat.

    “This is consistent with real-world ecology dynamics: a given habitat has limited resources, which must be shared between all resident species, limiting the total population of the habitat.”
  • The paper relies on this without support.

    As the number of habitats increases, each habitat becomes more suitable for a particular type of species, leading to lower overall diversity in these ecosystems.

    “To further underscore the ecological relationship: as the number of habitats increases, each habitat becomes more suitable for a particular type of species, leading to lower overall diversity in these ecosystems.”
  • The paper relies on this without support.

    A single meme may have multiple beachheads within Reddit before it becomes widespread, and each beachhead may contribute to the meme's eventual popularity.

    “For this reason, it is necessary to consider that a single meme may have multiple beachheads within Reddit before it becomes widespread and that each of these beachheads may contribute to the meme's eventual popularity.”
  • The paper relies on this without support.

    In Web data, user viewing or engagement with a meme can only be observed through sharing or liking/upvoting it.

    “Unfortunately, in the context of the Web, we have little data on whether users view or engage with a meme except by sharing or or liking/upvoting it.”
  • The paper relies on this without support.

    Changes in meme behavior can be used to infer changes in human attention with respect to memes on Reddit.

    “Looking at changes in meme behavior, we can make determinations about changes in human attention with respect to memes on the platform.”
  • The paper relies on this without support.

    The background model B(t) accurately controls for Reddit's growth when normalizing meme frequencies.

    “The life span is also determined using normalized meme frequencies to control for Reddit growth.”
  • The authors state this as an assumption.

    Memes can be treated as species or as exhibiting species-like behaviors at the individual, species, population, community, and ecosystem levels.

    “We began with the assumption that memes are species”
    “Assuming that memes exhibit species-like behaviors at the individual, species, population, community, and ecosystem levels”
    “it simply assumed that such a relationship existed”
  • The paper relies on this without support.

    Textual representations of visual memes follow proportionately.

    “We expect textual representations of visual memes to follow proportionately, but we cannot validate this claim in the current work.”
  • The paper relies on this without support.

    The text of memes is frequently used without the image in humorous or sarcastic contexts.

    “The text of this meme and others like it are frequently used without the image in humorous or sarcastic contexts.”
19 of 19

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Figure 5 assigns colors to two different groups of subreddits

    Minor

    The main text says the colors identify the top 10 subreddits across all 10 years, while the Figure 5 caption says they identify those most innovative in 2019.

  • Conclusion conflicts with unchanged peak width

    Minor

    The Figure 8 caption says the width of memes' primary popularity peak has not changed, whereas the conclusion says memes spend less time at peak popularity.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Herbert A. Simon. 1969. Designing Organizations for an Information-Rich World. Brookings Institute Lecture.

Cited by

Sorted by citations

Authors
Ford, T. W., Krohn, R., & Weninger, T.
Affiliations
  • William & Mary (US)
  • Rose–Hulman Institute of Technology (US)
  • University of Notre Dame (US)
Venue
ACM Transactions on Social Computing
Published
2023-10-26
Volume
6
Issue
3-4
Pages
1-19
ISSN
  • 2469-7818 (ISSN-L)
  • 2469-7826
OpenAlex
W3126752836
Cited by
10
References
54
Citations by year
2026: 2 · 2025: 3 · 2023: 1 · 2022: 4
FWCI
1.70
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Social Computing (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rose-Hulman Scholar (Rose–Hulman Institute of Technology) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Defense Advanced Research Projects Agency (W911NF192027)
Topics
  • Social Sciences › Psychology › Social Psychology › Humor Studies and Applications
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
memes, competition dynamics, human attention, speciation, resource availability, online social interaction, Reddit, information ecology, creativity, content sharing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-28.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review