ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
Open accessAbstract
The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI), introduced in 2019, established a challenging benchmark for evaluating the general fluid intelligence of artificial systems via a set of unique, novel tasks only requiring minimal prior knowledge. While ARC-AGI has spurred significant research activity over the past five years, recent AI progress calls for benchmarks capable of finer-grained evaluation at higher levels of cognitive complexity. We introduce ARC-AGI-2, an upgraded version of the benchmark. ARC-AGI-2 preserves the input-output pair task format of its predecessor, ensuring continuity for researchers. It incorporates a newly curated and expanded set of tasks specifically designed to provide a more granular signal to assess abstract reasoning and problem-solving abilities at higher levels of fluid intelligence. To contextualize the difficulty and characteristics of ARC-AGI-2, we present extensive results from human testing, providing a robust baseline that highlights the benchmark's accessibility to human intelligence, yet difficulty for current AI systems. ARC-AGI-2 aims to serve as a next-generation tool for rigorously measuring progress towards more general and human-like AI capabilities.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When a benchmark's accuracy saturates, the prevailing response is to retire it and replace it with a more challenging version.
“When a benchmark's accuracy saturates, it is often retired and replaced with a more challenging version.”
From Life After Benchmark Saturation: A Case Study of CORE-Bench (2026) · cites this paper as [12]
Each ARC-AGI-2 task is solved by roughly 75% of individual participants.
“each ARC-AGI-2 task is solved by roughly 75% of individual participants [Chollet et al. 2025b]”
From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Chollet et al. 2025b
Human evaluation studies with over 400 test-takers found that expert panels solve 100% of ARC-AGI-2 tasks, average individual test-takers score 60%, and each task is solved by approximately 75% of those who attempted it.
“Human evaluation studies with over 400 test-takers found that while expert panels solve 100% of tasks, average individual test-takers score 60%, with each task solved by approximately 75% of those who attempted it [Chollet et al. 2025b].”
From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Chollet et al. 2025b
These findings motivated ARC-AGI-2's design.
“These findings motivated ARC-AGI-2's design [Chollet et al. 2025b].”
From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Chollet et al. 2025b
ARC-AGI-2 preserves the ARC format but sharply increases difficulty via deeper multi-step compositionality, richer symbolic interpretation, context-dependent rule application, and explicit resistance to brute-force search.
“ARC-AGI-2 preserves the ARC format but sharply increases difficulty via deeper multi-step compositionality, richer symbolic interpretation, context-dependent rule application, and explicit resistance to brute-force search [Chollet et al. 2025b].”
From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Chollet et al. 2025b
The human baseline of 100% represents task-level solvability (every task solved by at least one person); average individual accuracy is 76% on ARC-AGI-1 and 60% on ARC-AGI-2.
“The human baseline of 100% represents tasklevel solvability (every task solved by at least one person); average individual accuracy is 76% on ARC-AGI-1 and 60% on ARC-AGI-2 [Chollet et al. 2025b].”
From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Chollet et al. 2025b
Cites
Still being checked.
Cited by
Sorted by citations
Shojaee, P., Mirzadeh, I., Alizadeh vahid, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-3603 · Cited by 47 · see citing works
Critique of impure reason: Unveiling the reasoning behaviour of medical large language models
Sim, S. Z. Y., & Chen, T. (2025) · eLife · doi:10.7554/elife.106187 · Cited by 24 · see citing works
Transforming agency: On the mode of existence of large language models
Barandiaran, X. E., & Almendros, L. S. (2025) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-025-10094-3 · Cited by 24 · see citing works
Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01939 · Cited by 14
“Additionally, the scope of our dataset could be expanded to encompass domains beyond mathematics, such as programming or more complex tasks like ARC-AGI (Chollet et al., 2025; Chollet, 2019).”
Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement
Ye, H., Jin, J., Xie, Y., Zhang, X., & Song, G. (2025) · arXiv · doi:10.48550/arxiv.2505.08245 · Cited by 8 · see citing works
Toward a woven literature: Open-source infrastructure for reproducible publishingPreprint
Karakuzu, A. (2025) · doi:10.55458/neurolibre.00041 · Cited by 3 · see citing works
A Comprehensive Behavioral Dataset for the Abstraction and Reasoning Corpus
LeGris, S., Vong, W. K., Lake, B. M., & Gureckis, T. M. (2025) · Scientific Data · doi:10.1038/s41597-025-05687-1 · Cited by 3 · see citing works
Toward a woven literature: Open-source infrastructure for networked scientific publishing
Karakuzu, A. (2025) · Aperture Neuro · doi:10.52294/001c.146072 · Cited by 2 · see citing works
EvoX: Meta-Evolution for Automated Discovery
Liu, S., Agarwal, S., Maheswaran, M., Cemri, M., Li, Z., Mang, Q., Naren, A., Boneh, E., Cheng, A., Pan, M. Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.23413 · Cited by 1 · see citing works
“ARC-AGI-2 Tasks [8] evaluate abstract and compositional reasoning across programmatic problem-solving instances.”
Life After Benchmark Saturation: A Case Study of CORE-Bench
Nadgir, N., Kapoor, S., Liu, K., Kirgis, P., Orona, M., Rabanser, S., Bayer, T., Shetty, A., Ling, Y., Chan-Sew, D., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.26158
“The prevailing response has been to retire these benchmarks in favor of more difficult successors; for example, ARC-AGI 1 progressing to ARC-AGI 2 and 3, MMLU to MMLU-Pro, HumanEval to HumanEval+, and SWE-bench to SWE-bench Pro [12, 21, 53, 35, 15].”
Al Applications for Public Health Systemic Factors
Wu, M. (2026) · Springer eBooks · doi:10.1007/978-3-032-15872-7_7
Rote Memorization or Intelligence: An Assessment of Inferential Reasoning in Large Language Models
Mehmood, R., Rehman, E., & Habib, M. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01262-7
Neuro-Symbolic pathways to AGI: compositional reasoning and trustworthy deployment
Hakim, S. B., Gharami, K., Wang, H. H., Adil, M., Velasquez, A., & Song, H. H. (2026) · Progress in Artificial Intelligence · doi:10.1007/s13748-026-00463-7
Twin: Playing an Unknown Game with a Test-Time Digital Twin
Skoutnev, A., Acharya, K., Longhitano, G., Udell, M., Ellis, K., & Drori, I. (2026) · arXiv · doi:10.48550/arxiv.2608.14490
The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning
Vahdati, S., Aioanei, A., Suresh, H., & Lehmann, J. (2026) · arXiv · doi:10.48550/arxiv.2603.13372
Meta$^n$: Recursive Self-Improvement through Emergent Depth
Kim, Z. M., Lee, Y.-J., Jwa, S., & Kang, D. (2026) · arXiv
“On ARC-AGI-2 [Chollet et al., 2026]’s held-out split, for instance, Meta n is the only system to solve any task at all, where both OpenEvolve and Gödel Agent solve none.”
Influential
- Authors
- Chollet, F., Knoop, M., Kamradt, G., Landers, B., & Pinkard, H.
- ORCID
- Pinkard, Henry
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2505.11831
- OpenAlex
- W4417300113
- Cited by
- 8
- References
- 0
- Citations by year
- 2026: 3 · 2025: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- fluid intelligence, factor G, AI agents, multi-agent coordination, reasoning systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
