Numerical Fragility in Transformers: A Layer-wise Theory for Risk Estimation and Selective Stabilization
Open accessAbstract
Low-precision execution can induce substantial forward discrepancies in Transformers even for fixed weights and input, yet these discrepancies are usually monitored only at the output and lack a layer-wise theoretical account. We develop a first-order decomposition of output mismatch into layer-local attention, LayerNorm, and residual-transport terms, and derive from it a practical causal risk estimator and a budgeted controller, Bound-Guided Selective Stabilization (BGSS). Controlled sweeps verify the predicted local sign, monotonicity, and transport structure. On GPT-2, the transport-aware combined predictor is positively correlated with FP32-reference mismatch in all $18$ runs and improves over a no-transport ablation in $17/18$ runs. Reference-patch attribution shows that the same score preserves useful layer ordering information (mean Spearman $0.362$). In budget-matched mitigation, BGSS outperforms random same-budget control in onset events ($10.67$ vs. $11.67$), final mismatch ($1.243\times 10^{-3}$ vs. $1.284\times 10^{-3}$), and worst-case mismatch ($3.14\times 10^{-3}$ vs. $8.49\times 10^{-3}$), while matching a risk-only same-budget controller on onset suppression and sharply reducing worst-case mismatch ($3.14\times 10^{-3}$ vs. $5.71\times 10^{-3}$). These results support a theory-to-algorithm account of Transformer numerical fragility in which finite-precision risk can be analyzed, estimated, localized, and selectively stabilized.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Neither Baek's nor Saratchandran and Lucey's work addresses the entropy-driven, sequence-length-dependent deterioration of the attention matrix itself, nor connects it to the Fisher-Rao geometry.
“Neither, however, addresses the entropy-driven, sequence-length-dependent deterioration of the attention matrix itself that we characterise here, nor connects it to the Fisher-Rao geometry of the attention distributions.”
From Curvature-Corrected Rotary Position Embeddings: An Entropy-Invariant Temperature for Mitigating Numerical-Rank Collapse in Long-Context Attention (2026) · cites this paper as [11]
Baek’s §4.2 analyzes finite-precision error in the attention output and softmax sensitivity, not entropy-driven, sequence-length-dependent deterioration of the attention matrix or its Fisher–Rao geometry. The paper does use sequence length in its notation and experiments, but the supplied copy does not establish the claim about Saratchandran and Lucey’s work, which cannot be checked here. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169 · cited 1× · see citing works
“…systems (Vaswani et al., 2017) are routinely executed in reduced precision to increase throughput and reduce memory cost (Micikevicius et al., 2018; Kalamkar et al., 2019; Dettmers et al., 2022; Yao et al., 2022; Frantar et al., 2022; Xiao et al., 2023; Lin et al., 2024; Bondarenko et al., 2023).”
Flagged first, then by citations
Research works printed but not linked: 18
- Ba, J. L., Kiros, J. R., and Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450. · arXiv
- Bondarenko, Y., Nagel, M., and Blankevoort, T. (2023). Quantizable transformers: Removing outliers by helping attention heads do nothing. In Advances in Neural Information Processing Systems, volume 36.
- Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. (2022). LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. In Advances in Neural Information Processing Systems, volume 35.
- Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. (2022). GPTQ: Accurate Post-Training Quantiza- tion for Generative Pre-trained Transformers. arXiv preprint arXiv:2210.17323. · arXiv
- Goldberg, D. (1991). What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1).
- He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recog- nition.
- Higham, N. J. (2002). Accuracy and Stability of Nu- merical Algorithms. SIAM, 2 edition. IEEE Computer Society (2019). Ieee standard for floating-point arithmetic. IEEE Std 754-2019. DOI: 10.1109/IEEESTD.2019.8766229. · DOI
- Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. (2024). AWQ: Activation-aware Weight Quanti- zation for On-Device LLM Compression and Accel- eration. In Proceedings of Machine Learning and Systems, volume 6.
- Meng, K., Bau, D., Andonian, A., and Belinkov, Y. (2022). Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems.
- Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., and Wu, H. (2018). Mixed precision training. In International Conference on Learning Representations.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems.
- Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. (2020). Investigating gender bias in language models using causal medi- ation analysis. In Advances in Neural Information Processing Systems, volume 33.
- Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. (2023). SmoothQuant: Accurate and Effi- cient Post-Training Quantization for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning.
- Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., and Liu, T.- Y. (2020). On layer normalization in the transformer architecture. In Proceedings of the 37th International Conference on Machine Learning.
- Yao, Z., Aminabadi, R. Y., Zhang, M., Wu, X., Li, C., and He, Y. (2022). ZeroQuant: Efficient and Af- fordable Post-Training Quantization for Large-Scale Transformers. In Advances in Neural Information Processing Systems, volume 35.
- Zafrir, O., Boudoukh, G., Izsak, P., and Wasserblat, M. (2019). Q8BERT: Quantized 8Bit BERT. In Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing at NeurIPS.
- Zhang, B. and Sennrich, R. (2019). Root mean square layer normalization. In Advances in Neural Informa- tion Processing Systems, volume 32. 3 seeds, for 18 completed runs and 96 monitored windows per run. E3 reuses these E2 runs, selects the top-8 highest-mismatch windows per run, and evaluates single-layer FP32 reference-patch effects against the same manual forward path; the mean recompute gap is 9.24 × 10 -7 . E5 uses stable FP32-master / FP16-shadow training on WikiText-2 train with sequence length 256, 256 monitored steps, 3 seeds, and a maximum of 24 controller actions for the budget-matched policies.
- The official code release is provided at the URL in Section 1 and is released under the MIT license. Secondary quantitative summaries. For E2, the transport-aware combined predictor improves Pearson correlation over the no-transport ablation by a mean of 0.163 and Spearman correlation by a mean of 0.139. For E3, the mean top-1 hit rate is 0.264, with mean top-3 and top-5 overlaps of 0.505 and 0.622, respectively. For E5, BGSS and the budget-matched baselines use identical average protected layer-steps (2742). Relative to the random same-budget controller, BGSS reduces mean onset events by 1.0 and reduces mean max mismatch by more than a factor of 2.7; relative to the risk-only same-budget controller, BGSS preserves the same mean onset event count while reducing mean max mismatch from 5.71 × 10 -3 to 3.14 × 10 -3 . 10 -6 10 -5 10 -4 10 -3 10 -2
Cited by
Sorted by citations
Mtetwa, J. T., Ogudo, K. A., & Pudaruth, S. (2026) · Mathematics · doi:10.3390/math14142637
- Authors
- Baek, J.
- Affiliations
- Oregon State University (US)
- Corresponding author
- Baek, Jinwoo
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2510.21770
- OpenAlex
- W7101859704
- Cited by
- 0
- References
- 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- transformer models, low-precision arithmetic, layer-wise analysis, attention mechanism, layer normalization, risk estimation, GPT-2, finite precision arithmetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
