CoChat
Sign Up
Cited by 82Topic ModelingNatural Language Processing TechniquesDomain Adaptation and Few-Shot Learning

s1: Simple test-time scaling

Open access
Muennighoff, Niklas, Yang, Zitong, Shi, Weijia, Li, Xiang Lisa, Fei-Fei, Li, Hajishirzi, Hannaneh, Zettlemoyer, Luke, Liang, Percy +2 more·2025·Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
DOI 10.18653/v1/2025.emnlp-main.1025
Open this paper in CoChat — free accountOpen PDF

Abstract

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes, Tatsunori Hashimoto. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2025.emnlp-main.1025

Cites

0 citation errors · 1 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Test-time scaling improves both reliability and accuracy, especially on tasks that require multistep reasoning, without additional training or fine-tuning.

    “Without requiring additional training or fine-tuning, test-time scaling improves both reliability and accuracy, especially on tasks that require multistep reasoning.”

    From Model and Task-Aware Test-Time Scaling Strategies for Large Language and Vision-Language Models in Medicine: Evaluation Study (2026) · cites this paper as [17]

    The Introduction and §4.2 Performance report that increasing test-time compute improves s1-32B’s results on reasoning benchmarks, including AIME24. Budget forcing is a test-time intervention, but the paper says s1-32B was first fine-tuned on 1,000 examples; it does not establish improved reliability as a separate outcome. Thus the claim is supported for accuracy on reasoning tasks, but not as an unqualified improvement without fine-tuning. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • A 32-billion parameter model, when afforded a reasoning budget of approximately 4,000 tokens, can match the performance of much larger models on complex math reasoning tasks.

    “Research has demonstrated that a 32-billion parameter model, when afforded a reasoning budget of approximately 4,000 tokens, can match the performance of much larger models on complex math reasoning tasks [18].”

    From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [18]

    The Abstract and Introduction report that s1-32B performs strongly on competition-math benchmarks, including outperforming o1-preview. But our copy does not establish that it matched larger models specifically with an approximately 4,000-token reasoning budget; the 4,000-token figure in the Figure 6 caption describes a rejection-sampling limit, not that performance comparison. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Test-time scaling-allocating extra token-budget during inference-offers a training-free alternative to increasing accuracy.

    “In contrast, test-time scaling—allocating extra token-budget during inference—offers a training-free alternative to increasing accuracy.”

    From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [18]

    The Budget forcing section describes allocating thinking tokens through a decoding-time intervention, and §4.2 reports improved AIME24 performance with more test-time compute. But the Introduction and §4.2 say the model was first fine-tuned on 1,000 samples. The intervention requires no further training at inference; the paper does not establish its reported accuracy gains as wholly training-free. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Exposing reasoning traces in plaintext leaves proprietary systems vulnerable to model distillation by competitors and risks unmasking internal safety and refusal mechanisms or revealing harmful information.

    “Exposing these reasoning traces in plaintext leaves proprietary systems highly vulnerable to model distillation by competitors (Muennighoff et al., 2025), and it risks unmasking internal safety and refusal mechanisms or revealing harmful information (Green et al., 2025;Mao et al., 2026).”

    From Stealing Reasoning Traces from Proprietary LLM APIs (2026) · cites this paper as Muennighoff et al., 2025

    The Distillation section says the authors used reasoning traces from other models to train s1, and the Discussion reports that 1,000 examples produced a competitive reasoning model. This supports the distillation concern, but the paper does not establish that exposing plaintext traces makes proprietary systems highly vulnerable to competitors. The safety and refusal-mechanism clause is attributed to other citations, not this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Test-time scaling has emerged as an effective method for enhancing LLM performance during inference without additional training.

    “To address these challenges, test-time scaling has recently emerged as an effective method for enhancing LLM performance during inference [17-20].”

    From Model and Task-Aware Test-Time Scaling Strategies for Large Language and Vision-Language Models in Medicine: Evaluation Study (2026) · cites this paper as [17]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 4

  1. Ilya Loshchilov and Frank Hutter. 2019. De- coupled weight decay regularization. Preprint, arXiv:1711.05101. · arXiv
  2. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian- guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. 2025. Wizardmath: Empowering mathemat- ical reasoning for large language models via rein- forced evol-instruct. Preprint, arXiv:2308.09583. · arXiv
  3. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, and 1 others. 2025. Humanity's last exam. Preprint, arXiv:2501.14249. · arXiv
  4. Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. 2023. Self-evaluation guided beam search for reasoning. Preprint, arXiv:2305.00633. · arXiv

Cited by

Sorted by citations

Related

Authors
Muennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Candes, E., & Hashimoto, T.
Affiliations
  • Stanford University (US)
  • University of Washington (US)
ORCID
Venue
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics
Published
2025
Pages
20286-20332
DOI
10.18653/v1/2025.emnlp-main.1025
OpenAlex
W4416036482
Cited by
76
References
0
Citations by year
2026: 61 · 2025: 15
FWCI
72.45
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
Funders
  • National Science Foundation (2338866, CAREER)
  • Office of Naval Research (ONR-N00014, N00014)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
test-time scaling, scaling laws, large language models, model size, inference efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review