Exploring the System 1 Thinking Capability of Large Reasoning Models
Open accessAbstract
This paper explores the system 1 thinking capability of Large Reasoning Models (LRMs), the intuitive ability to respond efficiently with minimal token usage. While existing LRMs rely on long-chain reasoning and excel at complex tasks, their system 1 thinking ability remains largely underexplored. This capability is essential as it reflects models' difficulty awareness and reasoning efficiency, both critical for real-world applications. We propose S1-Bench, a multi-domain, multilingual benchmark comprising model-simple system 1 questions. Our investigation of 28 LRMs reveals under-accuracy and inefficiency on system 1 problems. We find existing efficient reasoning methods either generalize poorly to simple questions or sacrifice performance for efficiency. Further exploration uncovers LRMs' early difficulty awareness accompanied by lower confidence, and shows that problem difficulty is implicitly encoded in hidden states. Code and the extended version of the paper with appendices are available at https://github.com/WYRipple/S1_Bench.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
S1-Bench [237] evaluates large reasoning models on straightforward tasks aligned with System 1 thinking, focusing on intuitive and fast reasoning rather than complex deliberative System 2 processes.
“S1-Bench [237] evaluates large reasoning models on straightforward tasks aligned with System 1 thinking, focusing on intuitive and fast reasoning rather than the more complex, deliberative processes associated with System 2.”
From Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025) · cites this paper as [237]
Cites
Token-Budget-Aware LLM Reasoning
Han, T., Wang, Z., Fang, C., Zhao, S., Ma, S., & Chen, Z. (2024) · arXiv · Cited by 31 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 2 · see citing works
“Beyond performing direct prompt-based reasoning, these fine-tuned models often deliver more promising performance when tackling complex reasoning challenges.”
Influential
- Authors
- Zhang, W., Nie, S., Zhang, X., Zhang, Z., & Liu, T.
- Affiliations
- Chinese Academy of Sciences (CN)
- Alibaba Group (Cayman Islands) (KY)
- Venue
- Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence
- Published
- 2026-09
- Pages
- 6073-6081
- OpenAlex
- W4415160745
- Cited by
- 0
- References
- 1
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- intuitive thinking, large reasoning models, reasoning efficiency, token efficiency, confidence calibration, hidden state representations, benchmark, multilingual benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
