CoChat
Sign Up
PreprintCited by 2Information Retrieval and Search BehaviorWeb visibility and informetricsTopic Modeling

C-SEO Bench: Does Conversational SEO Work?

Open access
Puerto, Haritz, Gubri, Martin, Green, Tommaso, Oh, Seong Joon, Yun, Sangdoo·2025·arXiv
DOI 10.48550/arxiv.2506.11097arXiv 2506.11097
Open this paper in CoChat — free account

Abstract

Large Language Models (LLMs) are transforming search engines into Conversational Search Engines (CSE). Consequently, Search Engine Optimization (SEO) is being shifted into Conversational Search Engine Optimization (C-SEO). We are beginning to see dedicated C-SEO methods for modifying web documents to increase their visibility in CSE responses. However, they are often tested only for a limited breadth of application domains; we do not know whether certain C-SEO methods would be effective for a broad range of domains. Moreover, existing evaluations consider only a single-actor scenario where only one web document adopts a C-SEO method; in reality, multiple players are likely to competitively adopt the cutting-edge C-SEO techniques, drawing an analogy from the dynamics we have seen in SEO. We present C-SEO Bench, the first benchmark designed to evaluate C-SEO methods across multiple tasks, domains, and number of actors. We consider two search tasks, question answering and product recommendation, with three domains each. We also formalize a new evaluation protocol with varying adoption rates among involved actors. Our experiments reveal that most current C-SEO methods are not only largely ineffective but also frequently have a negative impact on document ranking, which is opposite to what is expected. Instead, traditional SEO strategies, those aiming to improve the ranking of the source in the LLM context, are significantly more effective. We also observe that as we increase the number of C-SEO adopters, the overall gains decrease, depicting a congested and zero-sum nature of the problem. Our code and data are available at https://github.com/parameterlab/c-seo-bench and https://huggingface.co/datasets/parameterlab/c-seo-bench.

AI snapshot

Written from the paper’s full text.

TL;DR

C-SEO Bench evaluates conversational search engine optimization methods across two tasks, six domains, and varying adoption rates. Most current C-SEO methods are ineffective or harmful to citation ranking, while traditional SEO and document position in the LLM context dominate.

Key findings

  • Across 54 cases, only three statistically significant positive ranking improvements appear: LLM Guidance on retail and video games, and Content Improvement on retail (p < 0.05, Holm-Bonferroni). No method is effective for question answering or for Haiku 3.5.
  • Many C-SEO methods significantly reduce rankings: Statistics decreases rankings in 19 of 24 evaluated settings; on Haiku 3.5 product recommendation, 26 of 30 cases show significant negative effects, and on o4-mini question answering, 19 of 30 cases do so.
  • Traditional SEO/in-context position is far more effective: placing a document first yields best-SEO gains of 2.77 ± 2.31 (Retail), 1.89 ± 2.32 (Games), 1.60 ± 2.04 (Books), 0.87 ± 1.35 (Web), 0.70 ± 1.64 (News), and 1.54 ± 2.07 (Debate) on GPT-4o-mini.
  • As adoption increases, average gains per adopter decrease, indicating a congested, zero-sum dynamic; LLM Guidance has the highest AUC among the best two C-SEO methods.

Methods

  • Benchmark covers two tasks: product recommendation (Retail: 500 queries/5,000 docs; Video Games: 436/4,360; Books: 249/2,245) and question answering (Web: 300/1,500; News: 294/2,375; Debate: 142/880), totaling more than 1.9k queries and 16k documents.
  • Ten C-SEO methods are tested: eight from Aggarwal et al. (Authoritative, Statistics, Citations, Fluency, Unique Words, Technical Terms, Simple Language, Quotes) plus Content Improvement and LLM Guidance. C-SEO transformations use gpt-4o-mini-2024-07-18; the conversational search engine is run with gpt-4o-mini-2024-07-18, claude-3-5-haiku-20241022, o3, and o4-mini.
  • Evaluation measures rank improvement Δ = baseline rank − post-C-SEO rank, uses right-tailed Wilcoxon signed-rank tests with Holm-Bonferroni correction, varies adoption rate α from 0 to 1 with up to ten actors, and reports AUC via the trapezoidal rule.

Limitations

  • Stated: conclusions apply exclusively to white-hat C-SEO, not black-hat methods; the paper does not examine interplay between traditional SEO and C-SEO, uses commercial proprietary models, does not test documents that do not fit the LLM context, and is restricted to English-language content.
  • Evident from the setup: adoption simulations use a maximum of ten actors and random target selection for unilateral adoption, so competitive dynamics are modeled at a small scale and may not capture larger real-world adoption patterns.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2506.11097

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

11 of 12 cited sources checked · 2 supported · 2 flagged · accuracy not yet measured

  • Premisecitation
    § Abstract · p. 1

    Existing evaluations consider only a single-actor scenario where only one web document adopts a C-SEO method.

    “Moreover, existing evaluations consider only a single-actor scenario where only one web document adopts a C-SEO method”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “current datasets and benchmarks focus on single applications with only a single actor adopting a C-SEO method [Aggarwal et al., 2024, Pfrommer et al., 2024, Kumar and Lakkaraju, 2024, Nestaas et al., 2025].”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Aggarwal et al., 2024 source

      Section 2.2.2 describes an evaluation that optimizes one source website per query. But Section 5.2, “Optimization of Multiple Websites,” explicitly reports a separate evaluation that optimizes all source contents simultaneously. Thus this paper's evaluations are not limited to the claimed single-actor scenario. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites Pfrommer et al., 2024 source
    • cites Kumar and Lakkaraju, 2024 source

      Section 3 (Strategic Text Sequence; page unavailable) says the strategic text sequence is inserted into one target product’s information among ten products, supporting a single-actor description of this paper’s evaluation. The paper does not establish that existing evaluations generally use only this scenario. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Interpretationown data
    § 1 Introduction · p. 2

    Traditional SEO methods remain essential for CSE, contrary to prior beliefs.

    “These results (illustrated in Figure 1) suggest that traditional SEO methods, those that improve retrieval ranking , remain essential for CSE, in contrary to prior beliefs [Aggarwal et al., 2024].”
    • Also stated in: § 1 Introduction · p. 3 · boosted
      “SEO remains essential. We show that the ranking of the documents in the LLM context is much more influential than any current C-SEO method to determine the document importance in the LLM response.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Aggarwal et al., 2024 source

      Section 2.2 says traditional SEO methods are not directly applicable to generative engines, and the Search Engine Optimization section says traditional strategies will not apply in that setting. Section 4 also finds that keyword stuffing offers little to no improvement. The paper therefore makes the contrary point rather than saying traditional SEO remains essential for CSE. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Findingreasoning
    § 6.2 Main Results · p. 9

    The paper does not observe the same effectiveness of C-SEO methods as Aggarwal et al. [2024].

    “While the results from Aggarwal et al. [2024] show some initial optimism about C-SEO methods, we do not observe the same effectiveness.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Aggarwal et al., 2024 source
  • Premisecitation
    § Abstract · p. 1

    Existing C-SEO methods are often tested only for a limited breadth of application domains.

    “However, they are often tested only for a limited breadth of application domains; we do not know whether certain C-SEO methods would be effective for a broad range of domains.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “Yet, current datasets and benchmarks focus on single applications with only a single actor adopting a C-SEO method [Aggarwal et al., 2024, Pfrommer et al., 2024, Kumar and Lakkaraju, 2024, Nestaas et al., 2025].”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Aggarwal et al., 2024 source

      The Search Engine Optimization section describes traditional SEO methods, and Section 5.1 says domain-specific optimizations are often applied. Neither section says existing C-SEO methods are often tested across only a limited range of domains or that their broader effectiveness is unknown; searches for those testing claims found no supporting passage. (LLM, conf=0.74)

      Experimental — accuracy not yet measured
    • cites Pfrommer et al., 2024 source

      The Introduction (p. unavailable) says this study specifically considers consumer products, and §4 Dataset (p. unavailable) describes tests across 50 product categories within that domain. This supports a limited application-domain scope for the cited method, but the paper does not establish that existing C-SEO methods are often tested with similarly limited breadth. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Kumar and Lakkaraju, 2024 source

      Section 4 (Experiments) tests the proposed method on two coffee-machine products, and the Conclusion describes the study as being in the context of e-commerce. This supports a limited-domain example, but the paper does not establish that existing C-SEO methods are often tested across only a narrow range of domains. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Interpretationreasoning
    § 6.2 Main Results · p. 9

    The differences between the paper's results and Aggarwal et al. [2024] are due to the metrics used.

    “While the results from Aggarwal et al. [2024] show some initial optimism about C-SEO methods, we do not observe the same effectiveness. The differences are due to the metrics used.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Aggarwal et al., 2024 source

      Section 4 (Results; page unavailable) reports improvements from GEO methods, and Section 3.4 (Evaluation Metrics; page unavailable) identifies the metrics used. The paper does not compare its results with the citing paper’s results or establish that different metrics caused the discrepancy. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • May combine two claims. 15
  • Classified as probably not a substantial claim. 14
  • The quote only partly states this claim. 4

  • May combine two claims.

    Interpretationcitation
    § 6.2 Main Results · p. 9

    Aggarwal et al.'s results with position-adjusted word count show a general decrease in scores, implicitly indicating that C-SEO methods do not generally improve citation ranking.

    “They also show the results with a position-adjusted word count to consider citation rankings. However, their results with this metric show a general decrease in the scores, implicitly indicating that the C-SEO methods do not generally improve citation ranking.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Aggarwal et al., 2024 source

      The Results and Table 5 report that Keyword Stuffing, a classical SEO method, scores below the no-optimization baseline on Position-Adjusted Word Count; the paper also says such methods often perform worse. But Section 2.2.1 defines that metric as combining cited word count with position, so a lower score does not establish that citation ranking itself declined. The paper does not show a general decrease for all methods. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given
    § 5 C-SEO Methods · p. 7

    The paper proposes two new C-SEO methods: Content Improvement and LLM Guidance.

    “In addition, we propose to benchmark two new C-SEO methods:”
    • Also stated in: § 5 C-SEO Methods · p. 7 · plain
      “1. Content Improvement : The combination of all the eight transformations above into a single holistic text improvement method.”
    • Also stated in: § 5 C-SEO Methods · p. 7 · plain
      “2. LLM Guidance : Inspired by the LLMs.txt standard 4 , this novel prompt-based method generates a markdown summary that is concatenated at the beginning of the original document to guide the LLM about its content.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given
    § Abstract · p. 1

    C-SEO Bench includes two search tasks, question answering and product recommendation, with three domains each.

    “We consider two search tasks, question answering and product recommendation, with three domains each.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “In particular, we focus on two common tasks for current CSE applications, answering web questions and product recommendations.”
    • Also stated in: § 4 C-SEO Bench · p. 5 · plain
      “C-SEO Bench is a benchmark designed to evaluate C-SEO methods across two tasks: product recommendation and question answering . Each task spans multiple domains to capture various real-world use cases and evaluate domain-specific effects.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 6.2 Main Results · p. 9

    The overall average effect of C-SEO methods is close to zero, but with high variance.

    “As a result, the overall average effect is close to zero, but with high variance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.2 Main Results · p. 8

    LLM guidance on the retail data has a positive boost in 26.2% of the cases and a negative boost in 12.8% of the cases.

    “For example, the LLM guidance on the retail data has a positive boost in 26.2% of the cases and a negative boost in 12.8% of the cases.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.2 Main Results · p. 8

    When C-SEO methods change rankings, the shifts tend to be relatively large and both positive and negative, partially canceling out.

    “When changes do occur, they tend to cause relatively large shifts. But, these shifts are both positive and negative, and partially cancel each other out.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 5 C-SEO Methods · p. 7

    The paper benchmarks ten C-SEO methods total.

    “Overall, we benchmark a total of ten methods.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data
    § Abstract · p. 1

    Traditional SEO strategies aiming to improve retrieval ranking are significantly more effective than C-SEO methods in improving citation ranking.

    “Instead, traditional SEO strategies, those aiming to improve the ranking of the source in the LLM context, are significantly more effective.”
    • Also stated in: § 1 Introduction · p. 2 · hedged
      “These results (illustrated in Figure 1) suggest that traditional SEO methods, those that improve retrieval ranking , remain essential for CSE, in contrary to prior beliefs [Aggarwal et al., 2024].”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data
    § 6.3 Importance of Traditional SEO · p. 9

    Figure 4 shows the ranking improvement of documents after positioning them in the ith position in the LLM context.

    “Figure 4 shows the ranking improvement of documents after positioning them in the i th position in the LLM context.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.1 Datasets Construction · p. 6

    C-SEO Bench contains more than 1.9k queries and 16k documents.

    “In total, our C-SEO benchmark contains more than 1.9k queries and 16k documents.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given
    § Abstract · p. 1

    The paper introduces an evaluation protocol with varying adoption rates among involved actors.

    “We also formalize a new evaluation protocol with varying adoption rates among involved actors.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “Lastly, we propose a new evaluation framework where multiple actors employ C-SEO methods concurrently.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Content providers' interaction can be modeled as a noncooperative game.

  • May combine two claims.

    Findingown data
    § 6.2 Main Results · p. 9

    These gains are only significant on the retail domain for content improvement, and only on the retail and video games domains for LLM guidance.

    “But these gains are only significant on the retail domain for content improvement, and only on the retail and video games domains for LLM guidance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.4 C-SEO Behaves as a Zero-Sum Game · p. 9

    Retail and video games are the two domains with the highest gains.

    “for the best two C-SEO methods ( i.e. , LLM guidance and content improvement) in the two domains with the highest gains ( i.e. , retail and video games).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingnone given
    § D Other Results · p. 15

    Best SEO is always statistically significant on the o4-mini model.

    “Best SEO is always statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 6.4 C-SEO Behaves as a Zero-Sum Game · p. 9

    LLM guidance and content improvement are the best two C-SEO methods.

    “for the best two C-SEO methods ( i.e. , LLM guidance and content improvement) in the two domains with the highest gains ( i.e. , retail and video games).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingnone given
    § D Other Results · p. 15

    None of the LLM Document Optimization methods achieve significant gains on the o4-mini model.

    “None of the LLM Document Optimization methods achieve significant gains.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 6.4 C-SEO Behaves as a Zero-Sum Game · p. 9

    While early adopters experience significant gains, these gains decrease steadily as more players adopt the same method.

    “We observe that while early adopters experience significant gains, these gains decrease steadily as more players adopt the same method.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 6.1 Experimental Setup · p. 8

    In unilateral adoption experiments, the paper randomly selects a document to apply a C-SEO method so that SEO ranking does not impact evaluation of C-SEO.

    “In the experiments with unilateral adoption (i.e., only one player improving its document), we randomly select a document to apply a C-SEO method. In this way, the ranking from SEO does not impact the evaluation of C-SEO.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation
    § 4.2 Evaluation · p. 6

    The paper adjusts p-values using the Holm-Bonferroni correction to prevent the multiple comparison problem.

    “We adjust the p-values using the HolmBonferroni correction [Holm, 1979] to prevent the multiple comparison problem and to be able to compare multiple C-SEO methods within each domain.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Holm, 1979

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 6.3 Importance of Traditional SEO · p. 9

    The last three positions lead to performance degradation because many documents would be moved to worse positions than their original ones.

    “Since target documents are selected randomly, the last three positions (i.e., 8, 9, and 10) lead to performance degradation because many documents would be moved to worse positions than their original ones.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 4.2 Evaluation · p. 6

    The paper uses the right-tailed Wilcoxon signed-rank test to assess statistical significance of rank improvement.

    “We access the statistical significance of the rank improvement using the right-tailed Wilcoxon signed-rank test.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 1 Introduction · p. 2

    C-SEO will not replace SEO, but will complement it.

    “Hence, we believe C-SEO will not replace SEO, but will complement it.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning
    § Abstract · p. 1

    It is unknown whether certain C-SEO methods would be effective for a broad range of domains.

    “we do not know whether certain C-SEO methods would be effective for a broad range of domains.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisereasoning
    § Abstract · p. 1

    In reality, multiple players are likely to competitively adopt cutting-edge C-SEO techniques.

    “in reality, multiple players are likely to competitively adopt the cutting-edge C-SEO techniques, drawing an analogy from the dynamics we have seen in SEO.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “Moreover, due to this increasing adoption of CSE, more actors ( e.g. , online sellers, or content providers) are concurrently applying C-SEO methods, creating a competitive dynamic similar to that of SEO.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data
    § 7 Conclusion · p. 10

    The dataset is restricted to English-language content, which may reduce the generalizability of the findings to other linguistic or cultural contexts.

    “Lastly, our dataset is restricted to English-language content, which may reduce the generalizability of our findings to other linguistic or cultural contexts.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 7 Conclusion · p. 10

    The paper's conclusions apply exclusively to white-hat C-SEO methods, leaving black-hat C-SEO methods outside scope.

    “Nevertheless, our conclusions apply exclusively to white-hat C-SEO methods, leaving black-hat C-SEO methods outside our scope.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 6.4 C-SEO Behaves as a Zero-Sum Game · p. 10

    C-SEO methods benefit early adopters, but their advantage decreases over time, eventually converging toward zero at full adoption.

    “As a result, while C-SEO methods benefit early adopters, their advantage decreases over time, eventually converging toward zero at full adoption.”
    Quote in the paperSubstantial claim
27 of 27

  • The authors state this as an assumption.

    The product recommendation task uses a cold start setting, where no information about the user is available.

    “We assume a cold start setting, where no information about the user is available.”

  • The authors state this as an assumption.

    Content providers' interaction can be modeled as a noncooperative game.

    “We model the interaction among content providers as a noncooperative game.”

  • The paper relies on this without support.

    In real scenarios, user queries are unknown.

    “However, their method assumes knowing the user query beforehand, which limits its applicability to real scenarios, where user queries are unknown.”

  • The paper relies on this without support.

    CSE systems consist of two steps: retrieval and generation.

    “CSE systems usually consist of two steps: retrieval and generation.”

  • The paper relies on this without support.

    The C-SEO methods evaluated in the benchmark are representative of current C-SEO methods.

    “Our experiments reveal that current C-SEO methods are not only largely ineffective, but can also have a significant negative impact, as they often decrease the ranking of documents.”

  • The paper relies on this without support.

    The two tasks and six domains of the benchmark reflect the most prevalent real-world use cases of conversational search engines.

    “spanning two tasks and six domains that reflect the most prevalent real-world use cases of conversational search engines.”

  • The paper relies on this without support.

    The simulated competitive game between content creators captures the dynamics of real-world C-SEO adoption.

    “We simulate C-SEO adoption as a competitive game between content creators to better understand how these methods behave under partial or full adoption.”

  • The paper relies on this without support.

    The position of a document in the LLM context is a valid proxy for traditional SEO retrieval ranking.

    “traditional SEO tries to improve the ranking of a document in the retrieval component and consequently in the LLM context.”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • C-SEO Bench evaluates ten white-hat C-SEO strategies across six domains and two tasks (product recommendation and question answering).

    “C-SEO Bench (Puerto et al., 2025), which evaluates ten white-hat C-SEO strategies across six domains and two tasks (product recommendation and question answering).”

    From GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization (2026) · cites this paper as Puerto et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Second SEO position is not significant in every domain

    Minor

    The caption says at least the first two positions have significant boosts in all domains, but Table 6 gives the second position in Debate a p-value of 0.0634, above 0.05.

  • Prior benchmark is described as single-adopter but listed as many-adopter

    Minor

    The text includes Nestaas et al. among benchmarks limited to one adopting actor, while Table 1 lists its number of adopters as "Many."

2 of 2

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 1

  1. Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics , 6(2):65-70, 1979. ISSN 03036898, 14679469. URL http://www.jstor.org/stable/ 4615733 .

Cited by

Sorted by citations

Related

Authors
Puerto, H., Gubri, M., Green, T., Oh, S. J., & Yun, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2506.11097
arXiv
2506.11097
OpenAlex
W6910487551
Cited by
0
References
0
FWCI
0.00
Type
conference paper
Language
en
Open access
Licence
Locations
  • TUbilio (Technical University of Darmstadt) (acceptedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
search engine optimization, large language models, benchmark, document ranking, question answering, product recommendation, zero-sum games

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review