CoChat
Sign Up
Preprint

gpt-oss-120b & gpt-oss-20b Model Card

OpenAI, :, Agarwal, Sandhini, Ahmad, Lama, Ai, Jason, Altman, Sam, Applebaum, Andy, Arbus, Edwin +119 more·2025·arXiv
DOI 10.48550/arxiv.2508.10925arXiv 2508.10925
Open this paper in CoChat — free account

Abstract

We present gpt-oss-120b and gpt-oss-20b, two open-weight reasoning models that push the frontier of accuracy and inference cost. The models use an efficient mixture-of-expert transformer architecture and are trained using large-scale distillation and reinforcement learning. We optimize the models to have strong agentic capabilities (deep research browsing, python tool use, and support for developer-provided functions), all while using a rendered chat format that enables clear instruction following and role delineation. Both models achieve strong results on benchmarks ranging from mathematics, coding, and safety. We release the model weights, inference implementations, tool environments, and tokenizers under an Apache 2.0 license to enable broad use and further research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2508.10925

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MXFP4, as used in GPT-OSS, adopts a microscaling-based representation.

    “For example, MXFP4, as used in GPT-OSS [100], adopts a microscaling-based representation [175].”

    From A Survey of Large Language Models (2026) · cites this paper as [100]

  • Modern architectures that approximate full attention with a sliding window or stack attention and RNN layers together are still less effective than full attention in using longer context to achieve better performance in language modeling.

    “Some modern architectures approximate full attention with a sliding window [1, 107], or stack attention and RNN layers together [91, 11]. However, these techniques are still less effective than full attention in using longer context to achieve better performance in language modeling.”

    From End-to-End Test-Time Training for Long Context (2025) · cites this paper as [1]

  • TrajectoryKit runs its agents on the substantially smaller open-weights model gpt-oss-20b, so its model capability does not vary with the depth of the agent.

    “Unlike the other two systems, TrajectoryKit runs its agents on a substantially smaller open-weights model gpt-oss-20b (OpenAI et al., 2025), so its model capability does not vary with the depth of the agent.”

    From Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research (2026) · cites this paper as OpenAI et al., 2025

  • Approaches using state-of-the-art open-source LLMs (Gemma2-27B, Mistral-7B, and GPT-OSS-20B) were used as baselines.

    “Besides, approaches using state-of-the-art open-source LLMs (Gemma2-27B57, Mistral-7B58, and GPT-OSS-20B59) were used as baselines as well.”

    From TimeX: Phenotype Onset Extraction from Clinical Narratives (2026) · cites this paper as [59]

  • Approaches such as learnable sink tokens, query-key normalization, and element-wise sigmoid gating have been proposed to mitigate attention sinks.

    “To mitigate this, approaches like learnable sink tokens, query-key normalization, and element-wise sigmoid gating have been proposed [99,100].”

    From A Survey of Large Language Models (2026) · cites this paper as [100]

    The Architecture section (p. 5) describes a learned bias in the softmax denominator as similar to attention sinks. It does not describe learnable sink tokens, query-key normalization, or element-wise sigmoid gating as proposed ways to mitigate attention sinks. Searches for those terms and for attention-sink mitigation found no supporting passage in our copy. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Research works printed but not linked: 20

  1. [1][1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, 'Attention is all you need,' in Proceedings of Advances in Neural Information Processing Systems , 2017. -
  2. [3][3] D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen, 'Gshard: Scaling giant models with conditional computation and automatic sharding,' arXiv preprint arXiv:2006.16668 , 2020. - · arXiv
  3. [4][4] N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al. , 'Glam: Efficient scaling of language models with mixture-of-experts,' in International conference on machine learning , pp. 5547-5569, PMLR, 2022. -
  4. [9][9] N. Shazeer, 'GLU variants improve transformer,' arXiv preprint arXiv:2002.05202 , 2020. - · arXiv
  5. [10][10] R. Child, S. Gray, A. Radford, and I. Sutskever, 'Generating long sequences with sparse transformers,' arXiv preprint arXiv:1904.10509 , 2019. - · arXiv
  6. [11][11] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. , 'Language models are few-shot learners,' NeurIPS , 2020. -
  7. [13][13] N. Shazeer, 'Fast transformer decoding: One write-head is all you need,' arXiv preprint arXiv:1911.02150 , 2019. - · arXiv
  8. [15][15] B. Peng, J. Quesnelle, H. Fan, and E. Shippole, 'YaRN: Efficient context window extension of large language models,' arXiv preprint arXiv:2309.00071 , 2023. - · arXiv
  9. [17][17] G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis, 'Efficient streaming language models with attention sinks,' arXiv preprint arXiv:2309.17453 , 2023. - · arXiv
  10. [18][18] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. , 'GPT-4o system card,' arXiv preprint arXiv:2410.21276 , 2024. - · arXiv
  11. [19][19] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. , 'Pytorch: An imperative style, high-performance deep learning library,' Advances in neural information processing systems , vol. 32, 2019. -
  12. [20][20] P. Tillet, H.-T. Kung, and D. Cox, 'Triton: an intermediate language and compiler for tiled neural network computations,' in Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages , pp. 10-19, 2019. -
  13. [23][23] S. Yao, N. Shinn, P. Razavi, and K. Narasimhan, ' τ -bench: A benchmark for tool-agent-user interaction in real-world domains,' arXiv preprint arXiv:2406.12045 , 2024. - · arXiv
  14. [25][25] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, 'Measuring massive multitask language understanding,' arXiv preprint arXiv:2009.03300 , 2020. - · arXiv
  15. [26][26] L. Phan, A. Gatti, Z. Han, N. Li, J. Hu, H. Zhang, C. B. C. Zhang, M. Shaaban, J. Ling, S. Shi, et al. , 'Humanity's last exam,' arXiv preprint arXiv:2501.14249 , 2025. - · arXiv
  16. [28][28] R. K. Arora, J. Wei, R. S. Hicks, P. Bowman, J. Quiñonero-Candela, F. Tsimpourlas, M. Sharman, M. Shah, A. Vallone, A. Beutel, et al. , 'HealthBench: Evaluating large language models towards improved human health,' arXiv preprint arXiv:2505.08775 , 2025. - · arXiv
  17. [29][29] M. Y. Guan, M. Joglekar, E. Wallace, S. Jain, B. Barak, A. Helyar, R. Dias, A. Vallone, H. Ren, J. Wei, H. W. Chung, S. Toyer, J. Heidecke, A. Beutel, and A. Glaese, 'Deliberative alignment: Reasoning enables safer language models,' arXiv preprint arXiv:2412.16339 , 2024. - · arXiv
  18. [30][30] E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, 'The instruction hierarchy: Training LLMs to prioritize privileged instructions,' arXiv preprint arXiv:2404.13208 , 2024. - · arXiv
  19. [31][31] A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P. Abbeel, J. Svegliato, S. Emmons, O. Watkins, et al. , 'A strongreject for empty jailbreaks,' arXiv preprint arXiv:2402.10260 , 2024. - · arXiv
  20. [32][32] A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman, 'BBQ: A hand-built bias benchmark for question answering,' arXiv preprint arXiv:2110.08193 , 2021. - · arXiv

Authors
OpenAI, :, Agarwal, S., Ahmad, L., Ai, J., Altman, S., Applebaum, A., Arbus, E., Arora, R. K., Bai, Y., Baker, B., Bao, H., Barak, B., Bennett, A., Bertao, T., Brett, N., Brevdo, E., Brockman, G., Bubeck, S., Chang, C., Chen, K., Chen, M., Cheung, E., Clark, A., Cook, D., Dukhan, M., Dvorak, C., Fives, K., Fomenko, V., Garipov, T., Georgiev, K., Glaese, M., Gogineni, T., Goucher, A., Gross, L., Guzman, K. G., Hallman, J., Hehir, J., Heidecke, J., Helyar, A., Hu, H., Huet, R., Huh, J., Jain, S., Johnson, Z., Koch, C., Kofman, I., Kundel, D., Kwon, J., Kyrylov, V., Le, E. Y., Leclerc, G., Lennon, J. P., Lessans, S., Lezcano-Casado, M., Li, Y., Li, Z., Lin, J., Liss, J., Lily, Liu, Liu, J., Lu, K., Lu, C., Martinovic, Z., McCallum, L., McGrath, J., McKinney, S., McLaughlin, A., Mei, S., Mostovoy, S., Mu, T., Myles, G., Neitz, A., Nichol, A., Pachocki, J., Paino, A., Palmie, D., Pantuliano, A., Parascandolo, G., Park, J., Pathak, L., Paz, C., Peran, L., Pimenov, D., Pokrass, M., Proehl, E., Qiu, H., Raila, G., Raso, F., Ren, H., Richardson, K., Robinson, D., Rotsted, B., Salman, H., Sanjeev, S., Schwarzer, M., Sculley, D., Sikchi, H., Simon, K., Singhal, K., Song, Y., Stuckey, D., Sun, Z., Tillet, P., Toizer, S., Tsimpourlas, F., Vyas, N., Wallace, E., Wang, X., Wang, M., Watkins, O., Weil, K., Wendling, A., Whinnery, K., Whitney, C., Wong, H., Yang, L., Yang, Y., Yasunaga, M., Ying, K., Zaremba, W., Zhan, W., Zhang, C., Zhang, B., Zhang, E., & Zhao, S.
Affiliations
Xiaoxuan
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2508.10925
arXiv
2508.10925
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review