CoChat
Sign Up
PreprintCited by 0Information Retrieval and Search BehaviorDigital Marketing and Social MediaAI in Service Interactions

What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce

Open access
Allouah, Amine, Besbes, Omar, Figueroa, Josué D, Kanoria, Yash, Kumar, Akshit·2025·arXiv
DOI 10.48550/arxiv.2508.02630arXiv 2508.02630
Open this paper in CoChat — free accountOpen PDF

Abstract

Online marketplaces will be transformed by autonomous AI agents acting on behalf of consumers. Rather than humans browsing and clicking, AI agents can parse webpages or leverage APIs to view, evaluate and choose products. We investigate the behavior of AI agents using ACES, a provider-agnostic framework for auditing agent decision-making. We reveal that agents can exhibit choice homogeneity, often concentrating demand on a few ``modal'' products while ignoring others entirely. Yet, these preferences are unstable: model updates can drastically reshuffle market shares. Furthermore, randomized trials show that while agents have improved over time on simple tasks with a clearly identified best choice, they exhibit strong position biases -- varying across providers and model versions, and persisting even in text-only "headless" interfaces -- undermining any universal notion of a ``top'' rank. Agents also consistently penalize sponsored tags while rewarding platform endorsements, and sensitivities to price, ratings, and reviews vary sharply across models. Finally, we demonstrate that sellers can respond: a seller-side agent making simple, query-conditional description tweaks can drive significant gains in market share. These findings reveal that agentic markets are volatile and fundamentally different from human-centric commerce, highlighting the need for continuous auditing and raising questions for platform design, seller strategy and regulation.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces ACES, a provider-agnostic sandbox for auditing AI shopping agents, and uses randomized experiments to show that frontier models exhibit model-dependent market shares, strong position biases, and badge sensitivities that persist across interfaces and prompts. It also shows seller-side AI description edits can causally shift market share, while model upgrades act as demand shocks.

Key findings

  • Baseline rationality improves sharply across generations: older models often missed small differences (e.g., 1% price discount failures >63% for Claude Sonnet 3.5 and 25% for GPT-4o; +0.1 rating failures 28.7% for Claude Sonnet 4, 15.1% for GPT-4.1, 71.7% for GPT-4o), while the latest models almost never make mistakes, with rating-test error rates below 1.7%.
  • Market shares are strongly model-dependent and volatile: for fitness watches, Claude Sonnet 4 selects Fitbit Inspire 45% of the time versus about 25% for GPT-4.1 and Gemini 2.5 Flash; after upgrades, Fitbit Inspire rises to 77% with Claude Opus 4.5 but falls to 6% with GPT-5.1. In iPhone 16 Pro covers, GPT-4.1 selects Mikeke 62.6% of the time, whereas GPT-5.1 selects ESR 95% and Mikeke only 5%.
  • Position biases are large, model-specific, and persist: GPT-4.1 and GPT-5.1 show almost opposite position preferences; for Claude Sonnet 4, moving a product from bottom-right (4.5% selection) to the top row in columns 2 or 3 yields a 5-fold increase, and in the headless setting moving from position 8 (4.1%) to position 1 yields a 7-fold increase. Sponsored tags are penalized and Overall Pick is rewarded: from a 10% baseline, sponsored falls to 8.9%/8.0%/7.9% and Overall Pick rises to 24.3%/19.9%/42.6% for Claude Sonnet 4/GPT-4.1/Gemini 2.5 Flash; headless results are similar (sponsored 5.4%/1.8%/8.9%; Overall Pick 58.4%/55.6%/72.7%).
  • Seller-side AI description edits can shift demand: a single iteration produced statistically significant market-share gains in 33% of experiments, with average gains of +3.66 p.p. (Claude Sonnet 4), +8.37 p.p. (GPT-4.1), +14.79 p.p. (Gemini 2.5 Flash), +7.38 p.p. (Claude Opus 4.5), +14.89 p.p. (GPT-5.1), and +0.32 p.p. (Gemini 3.0 Pro Preview, not significant). The office lamp category showed gains up to +80.4 p.p. for GPT-5.1.

Methods

  • ACES pairs a provider-agnostic browser-use vision-language-model agent with a programmable mock e-commerce app; each episode opens the app, captures a grid screenshot, queries the model with a generic prompt, and records the declared choice.
  • Experiments cover eight product categories with eight products each, using randomized positions, prices, ratings, review counts, and tags (Sponsored, Overall Pick, Only X Remaining); market shares use 1,000 experiments per category, choice-behavior estimates use 500 scenarios per category, and seller-response experiments use 200 trials per category.
  • Main analysis fits conditional logit models to estimate causal sensitivities to position, badges, price, rating, and reviews for six focal models: Claude Sonnet 4, GPT-4.1, Gemini 2.5 Flash, Claude Opus 4.5, GPT-5.1, and Gemini 3.0 Pro Preview; robustness checks include headless/API JSON lists and prompt variations.

Limitations

  • The paper states its workflow is simplified: no product-detail pages, one-shot product selection, generic prompts, and a mock app that omits aspects of full-funnel shopping; it also notes the study is phenomenological/descriptive and does not calibrate against a human baseline.
  • The seller-response study assumes only one seller uses an AI agent and tries only one AI-generated recommendation, so it does not capture repeated or competitive seller optimization.
  • Evident from the text: the assortment is limited to eight categories and eight Amazon-derived products each, inference settings use low reasoning/temperature 1.0, some choice sets were discarded due to technical issues or rate limits, and Gemini 2.5 Flash headless experiments used only 100 experiments per category due to latency.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2508.02630

Cites

Statistics

Nothing to recompute.

Claims the paper makes

8 of 18 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Findingown data
    § 1 Introduction · p. 4

    Market shares can drastically change when models are upgraded.

    “the market shares can drastically change when models are upgraded.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.
  • Findingown data
    § 1 Introduction · p. 5

    In five out of six AI buying models tested, there was a statistically significant increase, averaged across product categories, in market share for a randomly chosen focal product.

    “In five out of the six AI buying models that we tested, we found statistically significant increase (averaged across product categories) in market share for a randomly chosen focal product”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.
  • Findingown data
    § 5.2 Our Findings · p. 18

    A sponsored tag reduces the likelihood of selection: a product with a baseline selection probability of 10% falls to 8.9% under Claude Sonnet 4, 8.0% under GPT-4.1, and 7.9% under Gemini 2.5 Flash.

    “A sponsored tag reduces the likelihood of selection (a product with a baseline selection probability of 10% falls to 8.9% under Claude Sonnet 4, 8.0% under GPT-4.1, and 7.9% under Gemini 2.5 Flash)”
    Quote in the paperSubstantial claim
  • Findingown data
    § 6.3 Drivers of Improvement · p. 24

    In the toilet paper category with GPT-5.1, the seller agent enriched a sparse description by adding high-value sensory and functional keywords (e.g., '2-Ply', 'Septic-Safe', 'Unscented'), resulting in a +15.5 p.p. increase.

    “For instance, in the toilet paper category with GPT5.1, the seller agent enriched a sparse description by adding high-value sensory and functional keywords (e.g., '2-Ply', 'Septic-Safe', 'Unscented'), resulting in a +15 . 5 p.p. increase.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.
  • Findingown data
    § 5.2 Our Findings · p. 17

    All three models display a sizeable response to where a product appears on the page.

    “All three models display a sizeable response to where a product appears on the page.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 26
  • May combine two claims. 82
  • The quote only partly states this claim. 3

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1.1 Related Literature · p. 8

    Recent empirical evidence suggests badges can meaningfully change clicks and add-to-cart probabilities, and complementary work studies badge longevity, uniformity, bias across marketplaces, and causal effects of badges in online communities.

    “Recent empirical evidence suggests badges can meaningfully change clicks and add-to-cart probabilities [29]; complementary work studies badge longevity, uniformity, and bias across marketplaces, and the causal effects of badges in online communities [3, 22, 28].”
    Quote in the paperNot a substantial claim3 of 4 sources checked
    • cites [29] [29] Markus Lill, Nastasia Gallitz, Lucas Stich, and Martin Spann. Product badges and consumer choice on digital platforms. Available at SSRN 4935668 , 2024. (Cited on pages 7, 8, 18, and 30) - source
    • cites [3] [3] Archit Bansal, Kunal Banerjee, and Abhijnan Chakraborty. These deals won't last! longevity, uniformity and bias in product badge assignment in e-commerce platforms. arXiv preprint arXiv:2204.12552 , 2022. (Cited on page 8) - source
    • cites [22] [22] Nicole Immorlica, Greg Stoddard, and Vasilis Syrgkanis. Social status and badge design. arXiv preprint arXiv:1312.2299 , 2013. (Cited on page 8) - source

      Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [28] [28] Tomasz Kusmierczyk and Manuel Gomez-Rodriguez. Harnessing natural experiments to quantify the causal effect of badges. arXiv preprint arXiv:1707.08160 , 2017. (Cited on page 8) - source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 1.1 Related Literature · p. 7

    WebArena introduces reproducible, long-horizon web tasks over functional sites, with baseline agents achieving low end-to-end success relative to humans.

    “WebArena introduces reproducible, longhorizon web tasks over functional sites (e.g., e-commerce, forums), with baseline agents achieving low end-to-end success relative to humans [50].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [50] [50] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854 , 2024. (Cited on page 7) - source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A.1 Business Goal & Context · p. 37

    The shopping agent workflow follows a common design pattern that is well-evaluated in the literature and is already productionized.

    “This shopping agent workflow follows a common design pattern that is well-evaluated in the literature [30, 47], and is already productionized [1, 21].”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [30] [30] Yougang Lyu, Xiaoyu Zhang, Lingyong Yan, Maarten de Rijke, Zhaochun Ren, and Xiuying Chen. Deepshop: A benchmark for deep research shopping agents. arXiv preprint arXiv:2506.02839 , 2025. (Cited on pages 8 and 37) - source

      The Introduction and “Web agents for task automation” section describe established web-agent approaches and prior benchmarks, supporting the point that shopping-agent workflows have been studied. But the paper says evaluations of complex, real-world shopping scenarios remain limited, so it does not establish that this workflow is already well evaluated. The production claim is attached to different citations [1, 21], not this source. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [47] [47] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In Advances in Neural Information Processing Systems , volume 35, pages 20744-20757, 2022. (Cited on pages 8 and 37) - source

      Section 3 describes a shopping-agent workflow of searching, inspecting products, choosing options, and buying; the Discussion reports an empirical evaluation of agents using that workflow. The paper does not establish that it is a common design pattern across the literature. The separate productionization claim is attached to different citations, not this source. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [1] [1] Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906 , 2025. (Cited on pages 8 and 37) - source

      The Background and “Hierarchical Methods for Computer Use Agents” sections describe hierarchical planning, execution, and grounding as an established approach, and the paper evaluates its Agent S2 implementation on computer-use benchmarks. The paper does not establish that the particular shopping-agent workflow follows that pattern or has been evaluated. The production claim is attached to different citations [1, 21], not this source. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [21] [21] Siyuan Hu, Mingyu Ouyang, Difei Gao, and Mike Zheng Shou. The dawn of gui agent: A preliminary case study with claude 3.5 computer use. arXv preprint arXiv:2411.10323 , 2024. (Cited on pages 37 and 38) - source

      Section 4.2.1 describes a shopping task in which the agent searches Amazon, filters headphones by budget, and adds an item to the cart. The paper presents this as a preliminary case study; it does not establish that the workflow is a well-evaluated design pattern or that it has been productionized. No page number is available in our copy. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    The scarcity tag effect is weakly negative or statistically indistinguishable from zero.

    “the scarcity tag effect is weakly negative or statistically indistinguishable from zero”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    For Claude Sonnet 4 and GPT-4.1, moving from the second to the first row is worth roughly a doubling of price; for Gemini 2.5 Flash the same move is worth only about +17%.

    “For Claude Sonnet 4 and GPT-4.1, moving from the second to the first row is worth roughly a doubling of price, whereas for Gemini 2.5 Flash the same move is worth only about +17%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.1 Assessing Temporal Model Drift and Impact of Model Finalization · p. 25

    For Gemini 2.5 Flash, signs are directionally stable, but the magnitude change for the top-row coefficient is statistically significant and increases over time, indicating that the preference of Gemini 2.5 Flash for the top row increases.

    “For Gemini 2.5 Flash, signs are directionally stable, but the magnitude change for the top-row coefficient is statistically significant and increases over time, indicating that the preference of Gemini 2.5 Flash for the top row increases (see Table 14).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 4

    Frontier models from August 2025 sometimes miss the cheapest item when one price is only slightly lower; GPT-4.1 fails approximately 9% of the time.

    “While frontier models from August 2025 sometimes miss the cheapest item when one price is only slightly lower (e.g., GPT-4.1 fails approximately 9% of the time)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 4

    For the fitness watch category, Claude Sonnet 4 selects the Fitbit Inspire model with probability 45%, while GPT-4.1 and Gemini 2.5 Flash select it only about 25% of the time.

    “for the fitness watch category, Claude Sonnet 4 selects the Fitbit Inspire model with probability 45%, while GPT-4.1 and Gemini 2.5 Flash select it only about 25% of the time”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Findingown data
    § 6.3 Drivers of Improvement · p. 24

    The optimized descriptions are not significantly shorter; rather, they structurally prioritize high-relevance tokens.

    “The optimized descriptions are not significantly shorter; rather, they structurally prioritize high-relevance tokens.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 6

    Model finalization, the post training from Gemini 2.5 Flash Preview to the official release, substantially reallocated market share across products and inverted the model's position bias from the bottom row to the top row.

    “The post training that took place while transitioning from Gemini 2.5 Flash Preview to the official release substantially reallocated market share across products and inverted the model's position bias from the bottom row to the top row (Section D).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Findingown data
    § 7.2.2 Our Findings · p. 27

    GPT-4.1 shows a Position 1 premium but otherwise a non-monotone profile with local lifts at Positions 3 and 5 in the headless setting.

    “GPT-4.1 shows a Position 1 premium but otherwise a non-monotone profile with local lifts at Positions 3 and 5.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.2 Our Findings · p. 22

    With GPT-5.1 as seller agent, average change in market share is +7.38 p.p. for Claude Opus 4.5, +14.89 p.p. for GPT-5.1, and +0.32 p.p. for Gemini 3.0 Pro Preview; the first two are statistically significant, the last is not.

    “Claude Opus 4.5 | +7 . 38 (1 . 00) | Yes”
    • Also stated in: § 6.2 Our Findings · p. 22 · plain
      “GPT-5.1 | +14 . 89 (1 . 05) | Yes”
    • Also stated in: § 6.2 Our Findings · p. 22 · plain
      “Gemini 3.0 Pro Preview | +0 . 32 (1 . 22) | No”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.

  • May combine two claims.

    Findingown data
    § 7.1 Assessing Temporal Model Drift and Impact of Model Finalization · p. 25

    The deprecation of Gemini 2.5 Flash Preview and its replacement by Gemini 2.5 Flash acted as a significant exogenous demand shock: it inverted position biases (shifting from a negative to a positive top-row bias) and caused substantial swings in product-level market shares (e.g., flipping the modal product in the office lamp and toilet paper categories).

    “We find that this upstream update acted as a significant exogenous demand shock: it inverted position biases (shifting from a negative to a positive top-row bias) and caused substantial swings in product-level market shares (e.g., flipping the modal product in the office lamp and toilet paper categories), highlighting the volatility sellers face when underlying buyer models are updated.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Our Findings · p. 13

    AI agents tend to collapse the demand distribution onto a smaller set of 'modal products' in some categories (e.g., stapler, iPhone covers, toothpaste, office lamp), and this behavior appeared more pronounced in the latest models.

    “AI agents tend to collapse the demand distribution onto a smaller set of 'modal products' in some categories (for example, stapler, iPhone covers, toothpaste, office lamp), and this behavior appeared to be more pronounced in the latest models (compare iPhone 16 Pro Cover and washing machines in Figures 5 and 6).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.2 Our Findings · p. 22

    With GPT-4.1 as seller agent, average change in market share for the focal product is +3.66 p.p. for Claude Sonnet 4, +8.37 p.p. for GPT-4.1, and +14.79 p.p. for Gemini 2.5 Flash, all statistically significant.

    “Claude Sonnet 4 | +3 . 66 (1 . 33) | Yes”
    • Also stated in: § 6.2 Our Findings · p. 22 · plain
      “GPT-4.1 | +8 . 37 (1 . 31) | Yes”
    • Also stated in: § 6.2 Our Findings · p. 22 · plain
      “Gemini 2.5 Flash | +14 . 79 (1 . 36) | Yes”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 6

    The estimated market-share increases for the focal product were +3.66 (1.33) p.p. for Claude Sonnet 4, +8.37 (1.31) p.p. for GPT-4.1, +14.79 (1.36) p.p. for Gemini 2.5 Flash, +7.38 (1.0) p.p. for Claude Opus 4.5, +14.89 (1.05) p.p. for GPT-5.1, and +0.32 (1.22) p.p. for Gemini 3 Pro Preview.

    “+3.66 (1.33) percentage points (p.p.) for Claude Sonnet 4, +8.37 (1.31) p.p. for GPT-4.1, +14.79 (1.36) p.p. for Gemini 2.5 Flash, +7.38 (1.0) p.p. for Claude Opus 4.5, +14.89 (1.05) p.p. for GPT-5.1, and +0.32 (1.22) p.p. for Gemini 3 Pro Preview.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    All three models assign negative weight to price and positive weights to ratings and the number of reviews.

    “All three models assign negative weight to price and positive weights to ratings and the number of reviews.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    For a product with a baseline selection probability of 10%, a +0.1 increase in rating lifts the probability to 15.4% with Claude Sonnet 4, 20.3% with GPT-4.1, and 16.0% with Gemini 2.5 Flash.

    “For example, consider a product with a baseline selection probability of 10% , an +0 . 1 increase in rating lifts the probability to 15 . 4% , 20 . 3% and 16 . 0% with Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Model Upgrades and Bias Patterns · p. 20

    Aversion to sponsored tags and the rewarding of platform endorsement persists after the model upgrade.

    “Similarly, one can see that aversion to sponsored tags, and the rewarding of platform endorsement persists after the upgrade.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 17

    For Claude Sonnet 4, moving a product selected 4.5% of the time at the bottom right corner to the top row in the second or third column leads to a 5-fold increase in selection rate; moving to the top left corner yields half of that increase.

    “For example, for Claude Sonnet 4, moving a product which is selected 4.5% of the time at the bottom right corner to the top row in the second or third column leads to a 5-fold increase in selection rate. Interestingly, moving to the top left corner would only yield half of that increase.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.2 Our Findings · p. 13

    Failure rates decline steadily with newer model releases; the latest models (Claude Sonnet 4/Opus 4.5, GPT-4.1/5.1, Gemini 2.0/2.5 Flash/3 Pro Preview) exhibit near-zero error.

    “We find that failure rates decline steadily with newer releases; the latest models-Claude Sonnet 4/Opus 4.5, GPT-4.1/5.1, Gemini 2.0/2.5 Flash/3 Pro Preview-exhibit near-zero error.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    An increment of +0.1 in ratings allows a seller to increase price by a quarter with Gemini 2.5 Flash, a third with Claude Sonnet 4, and 67% with GPT-4.1.

    “An increment of +0.1 in ratings, allows a seller to increase their price by a quarter with Gemini 2.5 Flash and a third with Claude Sonnet 4 while this more than doubles to 67% with GPT-4.1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Model Upgrades and Bias Patterns · p. 20

    For OpenAI's GPT model, GPT-4.1's preferred slot is exactly the least preferred by its successor GPT-5.1, and vice-versa.

    “In the case of OpenAI's GPT model, GPT-4.1 preferred slot is exactly is the least preferred by its successor GPT 5.1, and vice-versa.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.2 Our Findings · p. 13

    In price tests with one cheaper listing and a 1% discount, failure rates exceed 63% for Claude Sonnet 3.5 and 25% for GPT-4o; the latest models almost never make a mistake.

    “For example, in price tests with one cheaper listing, older models often missed small discounts (e.g., with 1% discount, failures exceed 63% for Claude Sonnet 3.5 and 25% for GPT-4o), while the latest models almost never make a mistake.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 4

    The market share for the Fitbit Inspire jumped from 45% with Claude Sonnet 4 to 77% with Claude Opus 4.5, while it fell from 25% to only 6% with GPT-5.1.

    “the market share for the Fitbit Inspire jumped (from 45% with Claude Sonnet 4) to 77% with Claude Opus 4.5, while it fell (from 25%) to only 6% with GPT-5.1.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 6

    Adding a mention that the buyer is price-sensitive steepens the agent's price sensitivity as intended, but significant position effects remain.

    “while adding a mention that the buyer is 'price-sensitive' steepens the agent's price sensitivity as intended, significant position effects remain”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The generic default prompt elicits baseline model-prior behavior that is not pre-solved by over-specifying the target item.
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • The quote only partly states this claim.

    Findingown data
    § 1 Introduction · p. 4

    The latest models Claude Opus 4.5, GPT-5.1, and Gemini 3 Pro Preview almost never make any mistake on the tested tasks.

    “the latest models (Claude Opus 4.5, GPT-5.1, and Gemini 3 Pro Preview) almost never make any mistake.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 3.2 Our Findings · p. 13

    In rating tests, a +0.1 bump on a dense 5-star scale caused failure rates of 28.7% for Claude Sonnet 4 and 15.1% for GPT-4.1 in August 2025, but error rates almost disappeared with state-of-the-art models in December 2025 (below 1.7%).

    “In rating tests, a +0.1 bump (on a dense 5-star scale) still confused state of the art models in August 2025 (e.g., failure rates of 28 . 7% for Claude Sonnet 4 and 15 . 1% for GPT-4.1) but such error rates almost disappeared with state of the art models in December 2025 (with error rates below 1.7%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 4

    Tiny rating advantages of +0.1 can sometimes trip models, with failure rates ranging from 0% for Gemini 2.0 Flash to 16% for GPT-4.1 and 71.7% for GPT-4o.

    “tiny rating advantages ( +0 . 1 ) can sometimes trip models (ranging from 0% failures for Gemini 2.0 Flash to 16% for GPT-4.1 and 71.7% for GPT-4o)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 4

    For some categories, market shares concentrate on a few select products and some brands are never selected; in the staplers category, Amazon Basics dominates whereas Arrow is never selected.

    “For some categories, the market shares concentrate on a few select products and some brands are never selected (e.g., in the staplers category, Amazon Basics dominates whereas Arrow in never selected; see Figure 1)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 6.2 Our Findings · p. 22

    In the remaining 33% of cases, a one-shot modification produces large gains: for Claude Sonnet 4 +25.8 p.p. in office lamp; for GPT-4.1 +7.1 p.p. in fitness watch, +30.3 p.p. in office lamp, +16.6 p.p. in toothpaste; for Gemini 2.5 Flash +6.5 p.p. in mousepad, +52.0 p.p. in office lamp, +29.1 p.p. in toothpaste, +12.0 p.p. in washing machine; for Claude Opus 4.5 +41.0 p.p. in office lamp; for GPT-

    “In the remaining 33% cases, a one-shot modification produces large gains: for Claude Sonnet 4 we observe +25 . 8 p.p. in office lamp; for GPT-4.1, +7 . 1 p.p. in fitness watch, +30 . 3 p.p. in office lamp, and +16 . 6 p.p. in toothpaste; for Gemini 2.5 Flash, +6 . 5 p.p. in mousepad, and +52 . 0 p.p. in office lamp, +29 . 1 p.p. in toothpaste and +12 . 0 p.p. in washing machine. Similarly, for the”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 6

    In a headless API setting providing only a ranked JSON list of product data and no images, three focal models showed similar biases, including strong, model-specific position effects that persist under prompt variations.

    “We replicated our experiments on three focal models in a 'headless' API setting-providing agents only with a ranked JSON list of product data and no images-and once again found similar biases, e.g., strong, model-specific position effects, that persist even under prompt variations (Section 7.2).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The generic default prompt elicits baseline model-prior behavior that is not pre-solved by over-specifying the target item.
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims. The quote only partly states this claim.

    Findingown data
    § Abstract · p. 1

    There is a consistent and clear improvement trend in instruction following and economic rationality across model generations.

    “while agents have improved over time on simple tasks with a clearly identified best choice”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 7.2.2 Our Findings · p. 27

    Gemini 2.5 Flash peaks at Position 3 and is comparatively neutral or negative elsewhere in the headless setting.

    “Gemini 2.5 Flash peaks at Position 3 and is comparatively neutral or negative elsewhere.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 5

    Position biases are large; position causes large changes in the selection rate.

    “position biases are large (i.e., position causes large changes in the selection rate)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The generic default prompt elicits baseline model-prior behavior that is not pre-solved by over-specifying the target item.
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 5

    Across the six models, AI agents react positively to higher average ratings and to a larger number of ratings, and are sensitive to price.

    “Across the six models, AI agents react positively to higher average ratings and to a larger number of ratings, and are sensitive to price.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The generic default prompt elicits baseline model-prior behavior that is not pre-solved by over-specifying the target item.
    • Rests on an assumption: The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 17

    Horizontal column patterns vary sharply across models: GPT-4.1 strongly favors the first column; Claude Sonnet 4 largely ignores the first column and prefers the two middle columns; Gemini 2.5 Flash tilts toward the third and fourth columns.

    “However, the horizontal (column) patterns vary sharply across models. GPT-4.1 strongly favors the first column; Claude Sonnet 4, in contrast, largely ignores the first column and prefers the two middle columns; and Gemini 2.5 Flash tilts toward the third and fourth columns.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    Doubling the number of reviews allows a price increase of 19% with Claude Sonnet 4, 37% with GPT-4.1, and 17% with Gemini 2.5 Flash.

    “Finally, doubling of reviews allows a 19% , 37% and 17% increase in price with Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Model Upgrades and Bias Patterns · p. 20

    Position biases persist but are very different when models are upgraded; for every upgrade the position bias changed.

    “The presence of position biases persist but these are very different when models are upgraded. This is best seen in Figure 7, which depicts the probability of selecting a particular product as a function of its position. For every upgrade the position bias changed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Our Findings · p. 14

    In the iPhone 16 Pro cover category, GPT-4.1 selects Mikeke most often (62.6%), whereas GPT-5.1 selects ESR most often (95% of the time) and Mikeke only 5% of the time.

    “For example, GPT-4.1 selects Mikeke most often (62.6%) in the iPhone 16 Pro cover (cf. Figure 5(a)), whereas GPT-5.1 selects ESR most often (95% of the time) and Mikeke is only selected 5% of the time (cf. Figure 6(a)).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2 Our Findings · p. 18

    The Overall Pick tag produces price headroom of +92% for Claude Sonnet 4, +65% for GPT-4.1, and +138% for Gemini 2.5 Flash.

    “The Overall Pick tag produces very large lifts-e.g., +92% for Claude Sonnet 4, +65% for GPT-4.1 and +138% price headroom for Gemini 2.5 Flash.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.3.1 UI Setting. · p. 28

    Under the 'ignore position & prioritize price' prompt, rating weights fall and the number of reviews coefficients fall; the Overall Pick premium collapses to near zero for Claude Sonnet 4 and GPT-4.1 and shrinks markedly for Gemini 2.5 Flash.

    “As price becomes central, other attributes downshift: rating weights fall and so do the number of reviews coefficients; the Overall Pick premium collapses to near zero for Claude Sonnet 4 and GPT-4.1 and shrinks markedly for Gemini 2.5 Flash.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion and Discussion · p. 29

    Position effects are large and model-specific.

    “First, position effects are large and model-specific.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 7.3.1 UI Setting. · p. 28

    Under the 'ignore position & prioritize price' prompt, a 10% price decrease raises choice odds by about 81% for Claude Sonnet 4, 165% for GPT-4.1, and 101% for Gemini 2.5 Flash.

    “Interpreting these in odds terms, a 10% price decrease raises choice odds by about 81% for Claude Sonnet 4, 165% for GPT-4.1, and 101% for Gemini 2.5 Flash.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B.1.2 Main Results. · p. 44

    Models exhibit perceptual limitations, treating all products as identical and failing to recognize differences in price or rating, particularly when the difference is subtle (e.g., a 1% price reduction).

    “Perceptual Limitations. The model treated all products as identical, failing to recognize differences in price or rating. This type of error was particularly prevalent when the difference was subtle, for e.g., in the case when the price of one listing was reduced by 1% .”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.3.2 Headless Setting. · p. 29

    Prompting the model to 'ignore position' attenuates-but does not eliminate position effects in the headless setting.

    “Prompting the model to 'ignore position' attenuates-but does not eliminate position effects.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.3.2 Headless Setting. · p. 29

    With the 'ignore position' prompt, position effects decline but remain statistically and economically meaningful for the top slots in the headless setting.

    “With the 'ignore position' prompt, position effects decline but remain statistically and economically meaningful for the top slots.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § D Model Finalization: The case of Gemini 2.5 Flash Preview to Gemini 2.5 Flash · p. 65

    When the buyer agent finalizes from Gemini 2.5 Flash Preview to Gemini 2.5 Flash, product-level selection shares move substantially, with frequent flips in the modal product.

    “Shares move substantially, with frequent flips in the modal product: e.g., in office lamp category the mode shifts from TORCHSTAR (Gemini 2.5 Flash Preview) to SUNMORY (Gemini 2.5 Flash); in toilet paper from Cottonelle to Angel Soft.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.2.2 Our Findings · p. 27

    All three models assign negative weight to price and positive weight to ratings and the number of reviews, directionally consistent with a rational shopper who prefers cheaper, better-rated, more widely reviewed products.

    “All three models assign negative weight to price and positive weight to ratings and the number of reviews-directionally consistent with a rational shopper who prefers cheaper, better-rated, more widely reviewed products.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: A rational shopper prefers cheaper, better-rated, more widely reviewed products.

  • May combine two claims.

    Findingown data
    § B.3.1 Case Study of Seller's Response to AI-assisted shopping · p. 54

    The increase in SUNMORY's market share with GPT-5.1 comes primarily at the expense of TORCHSTAR, whose share drops from 87.3% to 10.1%.

    “This comes primarily at the expense of the market leader, TORCHSTAR, whose share drops from 87 . 3% to 10 . 1% .”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Why these should look tastier to Claude: · p. 56

    For Gemini 3.0 Pro Preview, the market share of SUNMORY only increases from 0.0% to 7.1% despite the optimized description, whereas for GPT-5.1 the gain is massive.

    “For the case of Gemini 3.0 Pro Preview, the market share of SUNMORY only increases from 0 . 0% to 7 . 1% despite the optimized description, whereas for GPT-5.1 the gain is massive.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion and Discussion · p. 29

    These behaviors persist in a headless/API setting that exposes ranked JSON lists, under modified prompts, and across model generations.

    “Fourth, these behaviors persist in a headless/API setting that exposes ranked JSON lists, under modified prompts, and across model generations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion and Discussion · p. 29

    The Overall Pick tag lifts selection probability (a lot) while a Sponsored tag depresses it.

    “Second, the Overall Pick tag lifts selection probability (a lot) while a Sponsored tag depresses it.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § D Model Finalization: The case of Gemini 2.5 Flash Preview to Gemini 2.5 Flash · p. 65

    Gemini 2.5 Preview is roughly indifferent between Columns 3-4, whereas Gemini 2.5 Flash strictly prefers Column 3 over Column 4.

    “Both versions penalize Columns 1-2; Gemini 2.5 Preview is roughly indifferent between Columns 3-4, whereas Gemini 2.5 Flash strictly prefers Column 3 over Column 4.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7.3.1 UI Setting. · p. 28

    'Ignore Position and Prioritize price' prompt increases price sensitivity and reduces other sensitivities in the UI setting.

    “As price becomes central, other attributes downshift: rating weights fall and so do the number of reviews coefficients; the Overall Pick premium collapses to near zero for Claude Sonnet 4 and GPT-4.1 and shrinks markedly for Gemini 2.5 Flash.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B.3.3 Drivers of Improvement: Textual Analysis of Seller Interventions. · p. 59

    In several cases, the original description was factually complete but lacked the specific vocabulary used in the buyer's prompt, and the seller agent successfully injected these missing keywords.

    “In several cases, the original description was factually complete but lacked the specific vocabulary used in the buyer's prompt. The seller agent successfully 'injected' these missing keywords.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § D Model Finalization: The case of Gemini 2.5 Flash Preview to Gemini 2.5 Flash · p. 65

    Position biases change between Gemini 2.5 Flash Preview and Gemini 2.5 Flash: Gemini 2.5 Flash Preview exhibits a negative top-row bias, while Gemini 2.5 Flash's is positive.

    “Table 20 shows that position biases also change: Gemini 2.5 Flash Preview exhibits a negative top-row bias, while Gemini 2.5 Flash's is positive.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7.3.1 UI Setting. · p. 28

    For Gemini 2.5 Flash, there is a statistically significant attenuation in the coefficient of the top-row and column 2 while the bias for column 3 increases under the 'Ignore Position' prompt in the UI setting.

    “For Gemini 2.5 Flash, we observe that there is an (statistically significant) attenuation in the coefficient of the top-row and column 2 while the bias for column 3 increases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7.3.2 Headless Setting. · p. 29

    Under the 'ignore position and prioritize price' prompt, a 10% price increase reduces choice odds by roughly 46% for Claude Sonnet 4, 59% for GPT-4.1, and 53% for Gemini 2.5 Flash, compared to approximately 20% under the default prompt for all models.

    “For intuition, a 10% price increase reduces choice odds by roughly 46%-60% under the 'ignore position and prioritize price' prompt (Claude Sonnet 4 ∼ 46%, GPT-4.1 ∼ 59%, Gemini 2.5 Flash ∼ 53%), compared to approximately 20% under the default prompt for all the models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion and Discussion · p. 29

    Agents weigh price, ratings, and reviews in the expected directions, but magnitudes vary widely across models.

    “Third, agents weigh price, ratings, and reviews in the expected directions, but magnitudes vary widely across models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B.3.3 Drivers of Improvement: Textual Analysis of Seller Interventions. · p. 59

    Relocating 'Office' to the front of the SUNMORY office lamp title drove market share gains of +80.4 p.p. for GPT-5.1, +52.0 p.p. for Gemini 2.5 Flash, +41.0 p.p. for Claude Opus 4.5, +25.8 p.p. for Claude Sonnet 4, and +7.1 p.p. for Gemini 3.0 Pro Preview.

    “This simple reordering drove massive gains across the board: GPT-5.1 (+80.4 p.p.), Gemini 2.5 Flash (+52.0 p.p.), Claude Opus 4.5 (+41.0 p.p.), Claude Sonnet 4 (+25.8 p.p.), and even the otherwise resistant Gemini 3.0 Pro Preview (+7.1 p.p.).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.2.2 Our Findings · p. 27

    In the headless/API list, a Sponsored badge reduces choice probabilities even after controlling for rank and attributes, with selection falling from a 10% baseline to 5.4% for Claude Sonnet 4, 1.8% for GPT-4.1, and 8.9% for Gemini 2.5 Flash.

    “In the headless/API list, a Sponsored badge reduces choice probabilities even after controlling for rank and attributes: starting from a 10% baseline, selection falls to 5.4% for Claude Sonnet 4, 1.8% for GPT-4.1, and 8.9% for Gemini 2.5 Flash.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 7.3.1 UI Setting. · p. 28

    Under the 'ignore position & prioritize price' prompt, positional effects compress heterogeneously: for Claude Sonnet 4, Column 1 flips sign while for Gemini 2.5 Flash, the top-row coefficient flips positive to negative.

    “Positional effects compress heterogeneously: for Claude Sonnet 4, Column 1 flips signwhile for Gemini 2.5 Flash, the top-row coefficient flips positive to negative.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning
    § Why these should look tastier to Claude: · p. 55

    The recommended descriptions increase the density of technical specifications in the title.

    “Additionally, the recommended descriptions increase the density of technical specifications ('32W', '3000LM', 'Stepless Dimmable') in the title itself.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 2 ACES : Our Sandbox Environment · p. 12

    The simplified workflow enables a clean measurement of choice behavior.

    “Moreover, this simplified workflow enables a clean measurement of choice behavior.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 1.1 Related Literature · p. 9

    Our work views the AI agent not as a proxy for a human, but as a distinct delegated economic actor.

    “In contrast, our work views the AI agent not as a proxy for a human, but as a distinct delegated economic actor.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    AI tools for shopping assistance have recently become available to consumers and are quickly gaining adoption.

    “AI tools for shopping assistance have recently become available to consumers, and are quickly gaining adoption [13].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] [13] Jur Gaarlandt, Wesley Korver, Nathan Furr, and Andrew Shipilov. Ai agents are changing how people shop. here's what that means for brands. Harvard Business Review , 26, 2025. (Cited on pages 2 and 7) -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data
    § 5.2 Our Findings · p. 18

    Sponsored ads buy attention through placement but induce a credibility cost in the 'eyes' of AI agents.

    “Sponsored ads thus buy attention through placement but induce a credibility cost in the 'eyes' of AI agents, highlighting a trade-off that platform designers and sellers must weigh.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 Introduction · p. 2

    Autonomous AI agents can take over the shopping journey entirely, including product selection.

    “autonomous AI agents offer the capability to take over the shopping journey entirely including product selection.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 1 Introduction · p. 6

    Position and other effects are not driven only by visual processing from parsing a webpage, and are likely to persist under standardized agent protocols.

    “This suggests position and other effects are not driven (only) by visual processing from parsing a webpage, and are likely to persist under standardized agent protocols.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § 2 ACES : Our Sandbox Environment · p. 10

    The eight product categories were chosen to represent a mix of high- and low-ticket markets, cover a broad range of product types, and allow inclusion of items from both well-known and niche brands.

    “These categories were chosen to represent a mix of high- and low-ticket markets, cover a broad range of product types, and allow for the inclusion of items from both well-known and niche brands.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 2 ACES : Our Sandbox Environment · p. 10

    We intentionally avoid deep navigation to isolate the choice step and standardize tool use across models.

    “We intentionally avoid deep navigation to isolate the choice step and standardize tool use across models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation
    § 5.2 Our Findings · p. 18

    The strong positive response to overall pick implies that platform endorsements are treated as credible signals, echoing the behavior of human buyers.

    “the strong positive response to overall pick implies that platform endorsements are treated as credible signals-echoing the behavior of human buyers [29].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] [29] Markus Lill, Nastasia Gallitz, Lucas Stich, and Martin Spann. Product badges and consumer choice on digital platforms. Available at SSRN 4935668 , 2024. (Cited on pages 7, 8, 18, and 30) - source

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 Introduction · p. 2

    AI tools have the potential to significantly reduce search frictions by systematically scanning broader assortments than humanly possible.

    “AI tools have the potential to significantly reduce search frictions by systematically scanning broader assortments than humanly possible.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 7

    Biases that persist across many variations of the experiment mirror similar findings in other contexts, such as race and gender disparities in AI-based hiring.

    “our finding of biases which persist across many variations of our experiment mirrors similar findings in other contexts, e.g., race and gender disparities in AI-based hiring [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] [13] Jur Gaarlandt, Wesley Korver, Nathan Furr, and Andrew Shipilov. Ai agents are changing how people shop. here's what that means for brands. Harvard Business Review , 26, 2025. (Cited on pages 2 and 7) -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1.1 Related Literature · p. 8

    Empirical and theoretical work shows that positions and rankings can causally shape what consumers examine and buy.

    “Empirical and theoretical work shows that positions and rankings can causally shape what consumers examine and buy.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [41] [41] Raluca M Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science , 37(4):530-552, 2018. (Cited on pages 7, 8, and 30) -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [14] [14] Anindya Ghose, Panagiotis G Ipeirotis, and Beibei Li. Examining the impact of ranking on consumer behavior and search engine revenue. Management Science , 60(7):1632-1654, 2014. (Cited on pages 8 and 30) - source

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 Introduction · p. 6

    ACES provides the first rigorous, provider-agnostic framework for algorithmic economic auditing of AI shopping models.

    “ACES provides the first (to our knowledge) rigorous, provider-agnostic framework for this purpose.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data
    § 1 Introduction · p. 4

    Agentic shopping may induce possible market dominance patterns and may suppress intrinsic consumer preference dispersion.

    “raising questions about possible market dominance patterns induced by agentic shopping, which may suppress intrinsic consumer preference dispersion.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 1.1 Related Literature · p. 9

    We deliberately use generic prompts to elicit baseline behavior that is not pre-solved by over-specifying the target item.

    “We deliberately use generic prompts to elicit baseline behavior that is not pre-solved by over-specifying the target item.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given
    § 3.1 Experimental Setup · p. 12

    We deliberately avoid directing attention to a single attribute to mirror real multi-attribute trade-offs.

    “We deliberately avoid directing attention to a single attribute to mirror real multi-attribute trade-offs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 7.2.1 Experimental Setup · p. 26

    The headless setup is a deliberately simplified stress test of the ranking/metadata channel rather than a substitute for image-aware agents.

    “our headless setup is therefore a deliberately simplified stress test of the ranking/metadata channel rather than a substitute for image-aware agents.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Product images are an essential part of shopping.
    • Rests on an assumption: Production systems will incorporate product images even in API flows.

  • May combine two claims.

    Study design and conductreasoning
    § 5.1 Experimental Setup · p. 15

    Randomization of position, tags, price, rating, and number of reviews is exogenous to inherent product attractiveness, allowing causal estimates of the impact of platform levers and product attributes.

    “Given these variations are exogenous to the inherent attractiveness of the products, they allow us to obtain causal estimates of the impact of different platform levers like position, tags and product attributes like price, rating and number of reviews.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given
    § 3.1 Experimental Setup · p. 12

    The experiments are designed such that there is objectively only one product that satisfies the requirements of the test.

    “The experiments are designed such that there is objectively only one product that satisfies the requirements of the test.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § 2 ACES : Our Sandbox Environment · p. 12

    The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of selection behavior, though it omits aspects of full-funnel shopping.

    “While this omits aspects of full-funnel shopping, it provides a first-order approximation of selection behavior.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Study design and conductreasoning
    § 1 Introduction · p. 3

    ACES randomizes page layout, listing order, prices, ratings, reviews, and promotional badges, providing a realistic yet controlled setting to causally identify how platform levers and product attributes shape agent choices.

    “By randomizing page layout, listing order, prices, ratings, reviews, and promotional badges, ACES provides a realistic yet controlled setting to causally identify how platform levers and product attributes shape agent choices.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 1 Introduction · p. 4

    AI agents could drive drastic swings in market shares, and the direction of those swings varies across models.

    “AI agents could drive drastic swings in market shares and the direction of those swings vary across models.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.
    • Rests on an assumption: The eight product categories and eight products per category are representative of real e-commerce assortments.
    • Rests on an assumption: The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

  • May combine two claims.

    Interpretationown data
    § 6.2 Our Findings · p. 24

    AI-mediated markets are volatile and heterogeneous; a change that works for one model may backfire for another.

    “This underscores that AI-mediated markets are volatile and heterogeneous-a change that works for one model may backfire for another.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Only one seller uses an AI agent to optimize their product description.
    • The authors state this as an assumption: The seller tries only one AI-generated recommendation through one interaction with their AI agent.

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § B.1.1 Experimental Setup. · p. 40

    In the price-based rationality experiments, all products are identical except for price.

    “In both, all products are identical except for price.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 8 Conclusion and Discussion · p. 30

    Several patterns we document mirror directionally human behavior: ranking/position materially shape choices in digital markets, and platform endorsements/badges shift demand.

    “That said, several patterns we document mirror directionally human behavior-ranking/position materially shape choices in digital markets [14, 41], and platform endorsements/badges shift demand [29].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] [14] Anindya Ghose, Panagiotis G Ipeirotis, and Beibei Li. Examining the impact of ranking on consumer behavior and search engine revenue. Management Science , 60(7):1632-1654, 2014. (Cited on pages 8 and 30) - source
    • cites [41] [41] Raluca M Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science , 37(4):530-552, 2018. (Cited on pages 7, 8, and 30) -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [29] [29] Markus Lill, Nastasia Gallitz, Lucas Stich, and Martin Spann. Product badges and consumer choice on digital platforms. Available at SSRN 4935668 , 2024. (Cited on pages 7, 8, 18, and 30) - source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data
    § B.2.1 Experimental Details. · p. 44

    In the mock-app experiments, the assortment of the eight products is the same across experiments, while position, price, rating, number of reviews, and tags are randomly permuted.

    “Notice that the assortment of the eight products is the same across the two experiments. However the position of the eight products change across the two experiments along with their price, rating and number of reviews. Also notice that the Sponsored, Overall Pick and Only X Remaining (scarcity) tags are assigned to different products across the two experiments.”
    • Also stated in: § B.2.1 Experimental Details. · p. 45 · plain
      “Note that across the different experiments shown in Figure 16, the position of the products is randomly permuted along with random perturbation to their price, rating and number of reviews.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 7.3.2 Headless Setting. · p. 29

    Explicit instructions reduce, but do not purge, positional effects in the headless setting, echoing the VLM results.

    “This echoes the VLM results: explicit instructions reduce, but do not purge, positional effects.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 7.3.1 UI Setting. · p. 28

    Light personalization via prompting powerfully re-weights price, but position effects remain economically meaningful.

    “A key takeaway is that light personalization via prompting powerfully re-weights price, but position effects remain economically meaningful.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § Why these should look tastier to Claude: · p. 55

    The increased density of technical specifications likely aligns with the buying agents' internal heuristics that associate detailed technical metrics with higher product quality or suitability for a workspace environment.

    “This likely aligns with the buying agents' internal heuristics which may associate detailed technical metrics with higher product quality or suitability for a workspace environment.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § B.1.1 Experimental Setup. · p. 40

    The instruction-following tasks were specified such that there was a unique product that satisfied the query.

    “The tasks were specified in such a way that there was a unique product that satisfied the query.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 8 Conclusion and Discussion · p. 30

    Models display human-like regularities (endorsement salience, and sensible weights on price, ratings, and reviews).

    “yet models display human-like regularities (endorsement salience, and sensible weights on price, ratings, and reviews).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § B.1.1 Experimental Setup. · p. 41

    In the rating-based rationality experiments, except for the ratings, all other product attributes are the same.

    “Note that except for the ratings, all the other product attributes are the same.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § D Model Finalization: The case of Gemini 2.5 Flash Preview to Gemini 2.5 Flash · p. 66

    Model changes alter both attribute sensitivities and position bias magnitudes, so identical designs can yield different outcomes solely due to an upstream revision.

    “More broadly, because model changes alter both attribute sensitivities and position bias magnitudes, identical designs can yield different outcomes solely due to an upstream revision-reinforcing the need for ongoing auditing, disclosure of impactful updates, and diversification of strategies across agent ecosystems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 7.3.1 UI Setting. · p. 28

    Simple prompt engineering leaves largely unaffected position effects, so eliminating these biases requires design changes beyond simply prompting.

    “The key takeaway is that simple prompt engineering leaves largely unaffected position effects, so eliminating these biases requires design changes beyond simply prompting.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 7.2.2 Our Findings · p. 27

    Interpreted causally, agents discount advertising but treat platform endorsement signals as credible.

    “Interpreted causally (badges are randomized), agents discount advertising but treat platform endorsement signals as credible.”
    Quote in the paperSubstantial claim
99 of 99

  • The paper relies on this without support.

    Selection frequencies in ACES are a valid proxy for market share in a fully delegated world.

    “we estimate selection frequencies and treat those as a proxy for market share in a fully delegated world.”

  • The paper relies on this without support.

    The eight product categories and eight products per category are representative of real e-commerce assortments.

    “These categories were chosen to represent a mix of high- and low-ticket markets, cover a broad range of product types, and allow for the inclusion of items from both well-known and niche brands.”
    “we chose eight representative products (found based on public information on the Amazon website) which form our assortment of products.”

  • The paper relies on this without support.

    The simplified mock-app with a fixed 2x4 grid and no product-detail pages provides a first-order approximation of real shopping selection behavior.

    “While this omits aspects of full-funnel shopping, it provides a first-order approximation of selection behavior.”

  • The paper relies on this without support.

    The generic default prompt elicits baseline model-prior behavior that is not pre-solved by over-specifying the target item.

    “Initially, we use a single generic prompt (Figure 4) to elicit baseline (model-prior) behavior.”
    “We deliberately use generic prompts to elicit baseline behavior that is not pre-solved by over-specifying the target item.”

  • The paper relies on this without support.

    The lowest viable reasoning levels and temperature 1.0 yield a consistent sampling baseline representative of practical agent behavior.

    “we standardized the inference settings of all AI agents to the lowest viable reasoning levels.”
    “All models were configured with a temperature of 1.0 to maintain a consistent sampling baseline.”

  • The authors state this as an assumption.

    Only one seller uses an AI agent to optimize their product description.

    “Note that for simplicity in this study of seller response, we assume that only one seller uses an AI agent to optimize their product description, and the seller tries only one AI generated recommendation (through one interaction with their AI agent).”

  • The authors state this as an assumption.

    The seller tries only one AI-generated recommendation through one interaction with their AI agent.

    “Note that for simplicity in this study of seller response, we assume that only one seller uses an AI agent to optimize their product description, and the seller tries only one AI generated recommendation (through one interaction with their AI agent).”

  • The paper relies on this without support.

    Product images are an essential part of shopping.

    “We acknowledge that product images are an essential part of shopping and that production systems will incorporate them even in API flows”

  • The paper relies on this without support.

    Production systems will incorporate product images even in API flows.

    “We acknowledge that product images are an essential part of shopping and that production systems will incorporate them even in API flows”

  • The paper relies on this without support.

    A rational shopper prefers cheaper, better-rated, more widely reviewed products.

    “directionally consistent with a rational shopper who prefers cheaper, better-rated, more widely reviewed products.”

  • The authors state this as an assumption.

    The default prompt in both the mock-app UI and the headless JSON setting can be thought of as the model's prior.

    “Wecan think of the default prompt in both the mock-app UI (Fig. 4) and the headless JSON setting (Fig. 10) as the model's prior.”

  • The paper relies on this without support.

    Large Language Models process text sequentially, and tokens appearing early in the sequence often exert a disproportionate influence on the model's relevance scoring, particularly when matching against the user's query intent.

    “As Large Language Models process text sequentially, tokens appearing early in the sequence often exert a disproportionate influence on the model's relevance scoring, particularly when matching against the user's query intent ('office lamp').”

  • The paper relies on this without support.

    The seller agent can uncover the latent behavior of the buying model by customizing the response to the specific simulated market shares under a given buying model.

    “by customizing this for each buying model, we expect the seller agent to uncover the latent behavior of the buying model, which can help improve the description for the focal product.”

  • The paper relies on this without support.

    In a traditional market, brand perception and product utility provide a buffer against erratic swings in market share.

    “In a traditional market, brand perception and product utility provide a buffer against such erratic swings.”

  • The paper relies on this without support.

    Human markets have preferences that are generally more distributed and stable than algorithmic markets.

    “than human markets, where preferences are generally more distributed and stable.”
15 of 15

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • With generative engines, candidate content is read directly into a model's context window and ordered by the model's learned notion of relevance and quality, so optimization now acts on the content itself.

    “With generative engines, candidate content is read directly into a model's context window and ordered by the model's learned notion of relevance and quality, so optimization now acts on the content itself (Aggarwal et al., 2024; Allouah et al., 2025).”

    From E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (2025) · cites this paper as Allouah et al., 2025

    Section 6.3 (p. 24) reports that changing product descriptions to prioritize relevant terms improved their selection by AI buying agents, supporting the point that optimization acts on content. But Section 2 (p. 10) says the main experiments showed agents screenshots of product listings, not content read directly into a model’s context window; Section 7.2 (pp. 25–26) describes a separate JSON-list variant. The paper studies product choice rather than establishing that candidate content is ordered by a learned notion of relevance and quality. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Early findings indicate that signals for generative engines may diverge substantially from the classical SEO playbook.

    “Early findings indicate that these signals may diverge substantially from the classical SEO playbook (Aggarwal et al., 2024; Allouah et al., 2025).”

    From E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (2025) · cites this paper as Allouah et al., 2025

    Section 6.3 (p. 24) contrasts static product descriptions in traditional e-commerce with query-conditional optimization for AI buying agents, supporting the idea that optimization signals may differ. The paper studies shopping agents, however, and does not establish that the same divergence applies to generative engines generally or compare its findings with the classical SEO playbook as a whole. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Price-equivalent results are cited under the wrong table number

    Minor

    The discussion of Claude Sonnet 4, GPT-4.1, and Gemini 2.5 Flash directs readers to Table 11, but their price-equivalent trade-offs appear in Table 10. Table 11 covers the newer models.

  • Claude's position coefficient changes significantly under the ignore-position prompt

    Minor

    For Claude Sonnet 4, the September top-row coefficient falls from 1.250 to 1.039 under the ignore-position prompt. Using the reported standard errors, the difference is about 3.3 standard errors, contrary to the statement that it is not statistically significant.

  • Ignore-position prompt changes a reported attribute coefficient

    Minor

    Claude Sonnet 4's September review coefficient rises from 0.431 to 0.583 under the ignore-position prompt. The difference is about 4.6 standard errors using the reported values, contradicting the claim that product-attribute coefficients do not change significantly.

3 of 3

No citation lists for this paper

No DOI on record — the citation lists can’t be looked up.

Research works printed but not linked: 8

  1. [2][2] Mark Armstrong and John Vickers. A model of delegated project choice. Econometrica , 78(1):213-244, 2010. (Cited on page 9) -
  2. [11][11] MahsaDerakhshan, Negin Golrezaei, Vahideh Manshadi, and Vahab Mirrokni. Product ranking on online platforms. Management Science , 68(6):4024-4041, 2022. (Cited on page 8) -
  3. [12][12] Antoine D´ esir, Vineet Goyal, Danny Segev, and Chun Ye. Constrained assortment optimization under the markov chain-based choice model. Management Science , 66(2):698-721, 2020. (Cited on page 8) -
  4. [13][13] Jur Gaarlandt, Wesley Korver, Nathan Furr, and Andrew Shipilov. Ai agents are changing how people shop. here's what that means for brands. Harvard Business Review , 26, 2025. (Cited on pages 2 and 7) -
  5. [18][18] MohammadTaghi Hajiaghayi, Keivan Rezaei, and Suho Shin. Delegating to multiple agents. In Proceedings of the 24th ACM Conference on Economics and Computation (EC) , pages 861-885, 2023. (Cited on page 9) -
  6. [27][27] A G¨ urhan K¨ ok, Marshall L Fisher, and Ramnath Vaidyanathan. Assortment planning: Review of literature and industry practice. Retail supply chain management , pages 209-246, 2008. (Cited on page 8) -
  7. [32][32] Daniel McFadden. Conditional logit analysis of qualitative choice behavior. In Paul Zarembka, editor, Frontiers in Econometrics , pages 105-142. Academic Press, 1974. (Cited on pages 4 and 16) -
  8. [41][41] Raluca M Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science , 37(4):530-552, 2018. (Cited on pages 7, 8, and 30) -

Authors
Allouah, A., Besbes, O., Figueroa, J. D., Kanoria, Y., & Kumar, A.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2508.02630
arXiv
2508.02630
OpenAlex
W4417105883
Semantic Scholar
de1c84a1b5110773838f8a42a49509a7288ea772
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Social Sciences › Sociology and Political Science › Digital Marketing and Social Media
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
autonomous AI agents, online marketplaces, position bias, model dependence, sponsored content, market share dynamics, AI agent evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review