The DOI matches its DataCite record; 8 of the 8 references it cites were found in a registry or an index; none of the 25 sources its claims cite is checked for support yet.
ARC Prize 2025: Technical Report
Open accessAbstract
The ARC-AGI benchmark series serves as a critical measure of few-shot generalization on novel tasks, a core aspect of intelligence. The ARC Prize 2025 global competition targeted the newly released ARC-AGI-2 dataset, which features greater task complexity compared to its predecessor. The Kaggle competition attracted 1,455 teams and 15,154 entries, with the top score reaching 24% on the ARC-AGI-2 private evaluation set. Paper submissions nearly doubled year-over-year to 90 entries, reflecting the growing research interest in fluid intelligence and abstract reasoning. The defining theme of 2025 is the emergence of the refinement loop -- a per-task iterative program optimization loop guided by a feedback signal. Refinement loops come in a variety of forms, in particular evolutionary program synthesis approaches and application-layer refinements to commercial AI systems. Such refinement loops are also possible in weight space, as evidenced by zero-pretraining deep learning methods which are now achieving competitive performance with remarkably small networks (7M parameters). In parallel, four frontier AI labs (Anthropic, Google DeepMind, OpenAI, and xAI) reported ARC-AGI performance in public model cards in 2025, establishing ARC-AGI as an industry standard benchmark for AI reasoning. However, our analysis indicates that current frontier AI reasoning performance remains fundamentally constrained to knowledge coverage, giving rise to new forms of benchmark contamination. In this paper, we survey the top-performing methods, examine the role of refinement loops in AGI progress, discuss knowledge-dependent overfitting, and preview ARC-AGI-3, which introduces interactive reasoning challenges that require exploration, planning, memory, goal acquisition, and alignment capabilities.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ARC Prize 2025 reports that the top Kaggle score on the ARC-AGI-2 private evaluation set reached 24% at $0.20 per task, with 1,455 teams and 15,154 entries. The report identifies refinement loops—iterative per-task program optimization guided by feedback—as the defining theme, while arguing frontier reasoning remains constrained by knowledge coverage and previewing ARC-AGI-3.
Key findings
- ARC Prize 2025 Kaggle competition drew 1,455 teams and 15,154 entries; the top score reached 24% on the ARC-AGI-2 private evaluation set at $0.20 per task. Paper submissions rose to 90, up from 47 in 2024.
- Refinement loops emerged as the central theme, including evolutionary program synthesis and zero-pretraining deep learning. TRM (7M parameters) achieved 45% on ARC-AGI-1 and 8% on ARC-AGI-2; CompressARC (76K parameters) achieved 20% on the ARC-AGI-1 evaluation set, processing each puzzle in ~20 minutes on a single NVIDIA RTX 4070 GPU.
- Commercial application-layer refinement harnesses improved ARC-AGI-2 performance: Poetiq’s Gemini 3 Pro harness went from 31% at $0.81/task to 54% at $31/task, with comparable gains on Claude Opus 4.5 at ~$60/task. Four frontier labs (Anthropic, Google DeepMind, OpenAI, xAI) reported ARC-AGI performance in public model cards in 2025.
- The report argues current frontier reasoning is fundamentally constrained by knowledge coverage, creating new benchmark-contamination risks; evidence included Gemini 3 Deep Think using correct ARC color mappings even though the verification harness did not mention ARC tasks. ARC-AGI-3 is planned for early 2026 with hundreds of interactive environments requiring exploration, planning, memory, goal acquisition, and alignment.
Methods
- Technical report/survey of ARC Prize 2025: summarizes competition results, top-performing methods, paper submissions, and the ARC-AGI benchmark series.
- Data/sample: ARC-AGI-2 (400 public training tasks, 120 semi-private evaluation tasks, 120 private evaluation tasks), 1,455 Kaggle teams and 15,154 entries, 90 paper submissions, human validation (100% of tasks solved by at least two non-expert humans; each task attempted by 2–10 humans), and public model outputs/model cards.
- Main analysis: qualitative synthesis of refinement-loop approaches (evolutionary program synthesis, zero-pretraining deep learning, commercial harnesses), knowledge-dependent overfitting, and a preview of ARC-AGI-3; no controlled experimental design is reported for the report’s own claims.
Limitations
- Stated: the authors cannot determine whether the apparent ARC-AGI-1/ARC-AGI-2 overfitting is accidental or intentional, and cannot precisely quantify the magnitude of this effect.
- Stated: observed refinement harnesses are currently domain-specific; general-purpose reliability improvements require a verifier or environment capable of producing a feedback signal.
- Evident from the text: the report is a survey/technical report rather than a controlled study, so claims about refinement loops and knowledge coverage rest on selected examples, competition outcomes, and model-output snippets; human-validation and model-analysis protocols are not fully specified.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
26 Own interpretation · 19 Sources not found · 14 Own result · 8 No basis given · 3 Own method · 1 Not checked yet
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Public training tasks (400, imported from ARC-AGI-1) -Intended to demonstrate the task format and allow for learning the Core Knowledge priors.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“2. All material task-specific training occurs at test time.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“1.The resulting networks are extremely small relative to their ARC-AGI performance.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We verified a Gemini 3 Pro refinement harness implementation open-sourced by Poetiq, which improves performance on ARC-AGI-2 from a baseline of 31% accuracy at $0.81 per task to 54% accuracy at $31 per task.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“This includes early results in which AI systems produce novel scientific knowledge in fields with adequate knowledge coverage.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our LLM verification harness does not mention ARC-AGI tasks or color formats, yet the model employs correct ARC color mappings in its reasoning.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of reasoning outputs from commercial systems reveals self-corrective behavior: ... which fails the complete set requirement.This suggests the current solution might not fully satisfy the puzzle's constraints.I need to re-examine the box configuration and explore alternative arrangements ... (Claude Opus 4.5)”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the last quarter of 2025, we introduced a new leaderboard category termed”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Higher reasoning modes for these systems exhibit a strong correlation with increased reasoning token counts (longer programs), even when not strictly necessary for task completion.”
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The defining theme of 2025 is the emergence of the refinement loop -a per-task iterative program optimization loop guided by a feedback signal.”
- Also stated in: § 3 Program Refinement Loops · plain
“The central theme driving AGI progress in 2025 is the emergence of the refinement loop.”
- Also stated in: § 5 Conclusions · boosted
“The emergence of the refinement loop as a central theme represents a significant shift in approaches to abstract reasoning, evidenced by both zero-pretraining deep learning methods and commercial AI reasoning systems.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3 Program Refinement Loops · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“establishing ARC-AGI as an industry standard benchmark for AI reasoning.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“More recent work demonstrates early success with a version of this that does away with pretraining entirely, initializing a network from scratch for a specific task and fitting the weights to encode a task-solving program using only examples of that task.Examples include Liao et al. (19), Hierarchical Reasoning Models (HRM) (28) and Jolicoeur-Martineau (15).”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [28]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [15]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our human study establish that all tasks in ARC-AGI-2 are solvable by humans with no prior training.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Current AI reasoning performance is fundamentally related to model knowledge.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Liao derives how a standard Variational Autoencoder (VAE) loss with decoder regularization can substitute for combinatorial search to refine very small neural network programs.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As of 2025, with the advent of AI reasoning systems, task domains with the following two characteristics are reliably automatable, with no new science needed: 1. Sufficient task knowledge coverage exists in the pretraining corpus. 2. The task provides a verifiable feedback signal.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evidence of iterative refinement appears in commercial AI reasoning systems.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ARC Prize 2025 demonstrated continued open-source progress towards AGI, with the top Kaggle score reaching 24% on ARC-AGI-2 and significant growth in paper submissions (90, up from 47 in 2024).”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach still requires that the foundational model possess adequate knowledge coverage of the task domain.”
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These extended natural language programs enable more refinement through additional exploration and verification cycles.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The method operates by minimizing the description length of each task at test time, following the Minimum Description Length (MDL) principle.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“However, techniques such as GEPA (18) and DSPy (17) enable the development of general-purpose reliability improvements at the application layer, provided a verifier or environment capable of producing a feedback signal is available.”
What the sources say
- Not matchedcites [18]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [17]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ARC-AGI-1 and ARC-AGI-2 were designed to resist this style of overfitting by employing a private dataset for official scoring and verification.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 29
- It may state two separate claims as one. 30
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Kaggle competition attracted 1,455 teams and 15,154 entries, with the top score reaching 24% on the ARC-AGI-2 private evaluation set.”
- Also stated in: § 6.1 Acknowledgments · plain
“In total, 1,455 teams submitted 15,154 entries to the Kaggle competition, at levels similar to ARC Prize 2024.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Also stated in: § 6.1 Acknowledgments · plain
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The original ARC-AGI-1 private evaluation tasks were tested by two people who scored 97% and 98%, and collectively solved 100% of tasks.”
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“The Tiny Recursive Model (TRM) (Paper Award 1st Place, Jolicoeur-Martineau (15)), which builds upon the earlier Hierarchical Reasoning Model (HRM) (28), achieves 45% test accuracy on ARC-AGI-1 and 8% on ARC-AGI-2 with only a 7M parameter network.”
What the sources say
- Not matchedcites [15]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [28]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For ARC-AGI-2, after selection and validation, 100% of the tasks were solved by at least two (or more) independent, non-expert human testers drawn from the general public, with each task attempted by between 2 and 10 humans.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with the top score reaching 24% on the ARC-AGI-2 private evaluation set.”
- Also stated in: § 6.1 Acknowledgments · plain
“The top competition score reached a new state-of-the-art on the ARC-AGI-2 private dataset of 24% at a compute cost of $0.20 per task.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 6.1 Acknowledgments · plain
In the paper
“CompressARC (Paper Award 3rd Place, Liao and Gu ( 19)) uses only 76K parameters, yet achieves 20% on the ARC-AGI-1 evaluation set, processing each puzzle in approximately 20 minutes on a single NVIDIA RTX 4070 GPU.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Semi-Private Evaluation tasks (120) -Used for intermediate leaderboard scoring during competitions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Private Evaluation tasks (120) -Used for final competition scoring and verification.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In parallel, four frontier AI labs (Anthropic, Google DeepMind, OpenAI, and xAI) reported ARC-AGI performance in public model cards in 2025, establishing ARC-AGI as an industry standard benchmark for AI reasoning.”
- Also stated in: § 5 Conclusions · plain
“with four major AI labs (Anthropic, Google DeepMind, OpenAI, and xAI) reporting ARC-AGI performance on model cards.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Also stated in: § 5 Conclusions · plain
In the paper
“• No pretraining: Models are randomly initialized and trained only at test time.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Paper submissions nearly doubled year-over-year to 90 entries, reflecting the growing research interest in fluid intelligence and abstract reasoning.”
- Also stated in: § 1.1 Dataset Composition · plain
“The paper submission track saw significant growth, with 90 papers submitted, up from 47 in 2024.”
- Also stated in: § 5 Conclusions · plain
“significant growth in paper submissions (90, up from 47 in 2024).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 1.1 Dataset Composition · plain
In the paper
“• No branching search: The approach relies solely on gradient descent.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consider ARC-AGI-1 task #4cd1b7b2.Gemini 3 Pro used 96 reasoning tokens to solve this task, whereas Gemini 3 Deep Think employed 138,000 tokens.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ARC Prize 2025 ran from March 26, 2025, to November 3, 2025.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“• No dataset: One model trains on a single target task and produces one answer.”
What the sources say
- Not matchedcites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All ARC Prize 2025 winning solutions and papers are open-source, reproducible, and available at arcprize.org.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Currently, the refinement harnesses we observe are domain-specific.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The same refinement approach achieved comparable gains on Claude Opus 4.5, with accuracy rivaling Gemini 3 Pro but at about twice the cost per task (approximately $60 per task), as reported by Poetiq.”
What the sources say
- Not matchedcites Poetiq
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Supporting evidence for this knowledge-dependent reasoning emerged across multiple domains in 2025, including performance on ARC-AGI-2 (abstract reasoning), 2025 IMO Gold Medal achievement (mathematics), and 2025 ICPC 100% performance (competitive programming) -all driven by AI reasoning systems.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An important finding with frontier commercial models released in late 2025 (Gemini 3, Claude Opus 4.5, etc.) is that refinement loops can be implemented at the application layer to meaningfully improve task reliability, rather than relying solely on provider reasoning systems.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While not publicly released, these tasks have been exposed to commercial APIs and thus carry some risk of leakage.”
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Refinement loops come in a variety of forms, in particular evolutionary program synthesis approaches and application-layer refinements to commercial AI systems.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An important characteristic of ARC-AGI tasks is that they remain hard for AI systems, yet easy for humans.”
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Both approaches implement a two-phase refinement process.First, an exploration phase generates many candidate solutions.Second, a verification phase analyzes these programs to produce a feedback signal.This cycle repeats per task until the resulting program is fully refined and provides accurate answers for all training input/output pairs.”
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“Pang's approach follows a similar strategy but operates in Python and dynamically creates a program abstraction library to guide synthesis.”
What the sources say
- Not matchedcites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“ARC-AGI tasks do not require specialized world knowledge (e.g., historical facts) nor language to solve.”
What the sources say
- Not matchedcites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our analysis indicates that current frontier AI reasoning performance remains fundamentally constrained to knowledge coverage, giving rise to new forms of benchmark contamination.”
- Also stated in: § 4 AGI Progress & The Future of ARC-AGI · plain
“Current AI reasoning performance is fundamentally related to model knowledge.”
- Also stated in: § 5 Conclusions · plain
“The year revealed that AI reasoning performance remains fundamentally constrained by knowledge coverage, a characteristic distinct from human reasoning which is capable of extreme generalization.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Human reasoning capability is not bound to knowledge in the way AI reasoning is.
- Also stated in: § 4 AGI Progress & The Future of ARC-AGI · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such refinement loops are also possible in weight space, as evidenced by zero-pretraining deep learning methods which are now achieving competitive performance with remarkably small networks (7M parameters).”
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“These knowledge priors are acquired by children very early (typically before age four) and are universally shared by all humans.”
What the sources say
- Not matchedcites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Representatives of refinement loops include: • Deep learning with test-time training methods, where the program being refined is the weights of a pretrained model. • Zero-pretraining deep learning methods such as TRM. • Evolutionary Program Synthesis in either symbolic program space or natural language program space. • Test-time Chain-of-Thought optimization with feedback from a verifier model.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ARC-AGI-2 maintains the same task format and core principles as ARC-AGI-1 while incorporating more complex tasks with greater generalization difficulty, and normalizing the distribution of task difficulty.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The ARC-AGI public training tasks are designed to expose test-takers to all the Core Knowledge priors needed to solve ARC-AGI tasks.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- The authors state this as an assumption: ARC-AGI tasks assume only Core Knowledge as prior knowledge.
In the paper
“Berman's approach employs an evolutionary search harness that evolves ARC solution programs in natural language.”
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Test-time training is responsible for the top scores in both ARC Prize 2024 (the ARChitects) and 2025 (NVARC).”
What the sources say
- Not matchedcites [27]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [25]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Collecting domain knowledge and building verifiers is not cost-free.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“According to Sam Altman (OpenAI), approximately 7% of ChatGPT free users have engaged with”
What the sources say
- Not matchedcites [2]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“By this definition, we have not yet achieved AGI.”
What the sources say
- Not matchedcites [21]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means that even welldesigned benchmarks resistant to direct memorization can now be”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ARC-AGI has served as a valuable scientific indicator for AI reasoning progress, with four major AI labs (Anthropic, Google DeepMind, OpenAI, and xAI) reporting ARC-AGI performance on model cards.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They have demonstrated non-zero fluid intelligence and can adapt to tasks somewhat removed from their precise knowledge base when the foundational model is grounded in the broader domain.”
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although we assess that this new form of”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Recent work by Hsu reports an AI refinement loop using a generator-verifier architecture to produce novel results in quantum physics (12).”
What the sources say
- Not matchedcites [12]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We assert that this phenomenon is now occurring with ARC-AGI-1 and ARC-AGI-2 -accidentally or intentionally, although we cannot determine which.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A Chain-of-Thought can be interpreted as a natural language program that transforms one latent state into another.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This strongly suggests that ARC data are well-represented in the underlying model -sufficiently so to make correct ARC inferences based solely on the structure and format of 2D JSON arrays of integers.”
boostedstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the ARC-AGI-1 and ARC-AGI-2 format, we assess that the Grand Prize accuracy gap is now primarily bottlenecked by engineering, while the efficiency gap remains bottlenecked by fundamental science and new ideas.”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These task domains are substantially broader than those addressable by pure language models without reasoning capabilities.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“State-of-the-art scores are only reported on the Semi-Private and Private Evaluation task sets to reduce the risk of overfitting and data contamination.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The authors state this as an assumption.
ARC-AGI tasks assume only Core Knowledge as prior knowledge.
“The only assumed prior knowledge is Core Knowledge (7) -concepts such as objectness, basic topology, elementary integer arithmetic, etc.”
The paper relies on this without support.
Human reasoning capability is not bound to knowledge in the way AI reasoning is.
“This relationship warrants careful consideration, as human reasoning capability is not similarly bound to knowledge.”
“a characteristic distinct from human reasoning which is capable of extreme generalization.”
The paper relies on this without support.
The ARC-AGI-2 private evaluation set is free from leakage.
“This set is fully private and theoretically free from leakage.”
The paper relies on this without support.
AI automation is a function of the societal willingness to invest in the necessary talent, compute, and data resources.
“Presently, AI automation is a function of the societal willingness to invest in the necessary talent, compute, and data resources.”
The paper relies on this without support.
Many potentially automatable problems fall beyond current societal cost cutoffs.
“However, many potentially automatable problems fall beyond current societal cost cutoffs.”
Citations
Cites
Spelke, E. S., & Kinzler, K. D. (2007) · Developmental Science · doi:10.1111/j.1467-7687.2007.00569.x · Cited by 1,089
On the Measure of Intelligence
Chollet, F. (2019) · arXiv · doi:10.48550/arxiv.1911.01547 · Cited by 218
Self-citationARC Is a Vision Problem!Preprint
Hu, K., Cy, A., Qiu, L., Ding, X. D., Wang, R., Zhu, Y. E., Andreas, J., & He, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14761 · Cited by 19
Don't throw the baby out with the bathwater: How and why deep learning for ARCPreprint
Cole, J., & Osman, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.14276 · Cited by 10
Wang, G., Li, J., Sun, Y., Chen, X., Liu, C., Wu, Y., Lu, M., Song, S., & Yadkori, Y. A. (2025) · arXiv · doi:10.48550/arxiv.2506.21734 · Cited by 3
Rethinking Visual Intelligence: Insights from Video PretrainingPreprint
Acuaviva, P., Davtyan, A., Hassan, M., Stapf, S., Rahimi, A. M., Alahi, A., & Favaro, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24448 · Cited by 2
Vector symbolic algebras for the Abstraction and Reasoning Corpus
Joffe, I., & Eliasmith, C. (2026) · npj Unconventional Computing · doi:10.1038/s44335-026-00100-5
ARC-NCA: Towards Developmental Solutions to the Abstraction and Reasoning Corpus
Guichard, E., Reimers, F., Kvalsund, M., Lepperød, M., & Nichele, S. (2025) · ALIFE 2025: Ciphers of Life: Proceedings of the Artificial Life Conference 2025 · doi:10.1162/isal.a.833
Research works printed but not linked: 2
- Alexia Jolicoeur-Martineau. Less is More: Recursive Reasoning with Tiny Networks. https://arxiv. org/abs/2510.04871, 2025.
- Julien Pourcel, Clement Colas, and Pierre Oudeyer. Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI. https://openreview.net/pdf?id=z4IG090qt2, 2025.
Cited by
Sorted by citations
Loop the Loopies!Preprint
Gao, Z., Chen, Y., Xiao, Y. H., Yang, X., Tao, R., Zhou, J., & Dai, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16051 · Cited by 22
Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in MathematicsPreprint
Firsching, M., Lezeau, P., Mercuri, S., Horváth, M. Z., Dillies, Y., Sönne, C., Wieser, E., Zhang, F., Hubert, T., y Arcas, B. A., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.13171 · Cited by 15
VISTA: A Visual Harness for Reasoning in an Interactive WorldPreprint
Han, Q., Hu, K., Qiu, L., Wu, C., & He, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.02200 · Cited by 5
TopoBench: Benchmarking LLMs on Hard Topological ReasoningPreprint
Maniparambil, M., Hoehing, N., Kapuriya, J., Karuvally, A., Rushe, E., Ventresque, A., O'Connor, N. E., & Reid, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.12133 · Cited by 3
An Alternative Trajectory for Generative AIPreprint
Belova, M., Kansal, Y., Liang, Y., Xiao, J., & Jha, N. K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.14147 · Cited by 3
Metadata
- Authors
- Chollet, F., Knoop, M., Kamradt, G., & Landers, B.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2601.10904
- OpenAlex
- W7124746691
- Semantic Scholar
- 98d447f23324847d99bc3fad4de05ad8371b037a
- Cited by
- 18
- References
- 4
- Citations by year
- 2026: 18
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- arxiv.org/licenses/nonexclusive-distrib/1.0/
- Locations
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- few-shot generalization, fluid intelligence, abstract thinking, benchmark contamination, interactive reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
