Cross-Layer Reliability Evaluation and Efficient Hardening of Large Vision Transformers Models
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Vision Transformers (ViTs) are highly accurate Machine Learning (ML) models. However, their large size and complexity increase the expected error rate due to hardware faults. Measuring the error rate of large ViT models is challenging, as conventional microarchitectural fault simulations can take years to produce statistically significant data. This paper proposes a two-level evaluation based on data collected through more than 70 hours of neutron beam experiments and more than 600 hours of software fault simulation. We consider 12 ViT models executed in 2 NVIDIA GPU architectures. We first characterize the fault model in ViT's kernels to identify the faults more likely to propagate to the output. We then design dedicated procedures efficiently integrated into the ViT to locate and correct these faults. We propose Maximum corrupted Malicious values (MaxiMals), an experimentally tuned low-cost mitigation solution to reduce the impact of transient faults on ViTs. We demonstrate that MaxiMals can correct 90.7% of critical failures, with execution time overheads as low as 5.61%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Due to their large number of parameters, ViTs often suffer from high computational complexity [18] and an increased susceptibility to hardware-induced errors [19].”
From Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors (2025) · cites this paper as [19]
The cited source says what the claim attributes to it.
In the citing paper
“These trends are consistent with the data collected through more than 70 h of neutron beam experiments [19].”
From Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors (2025) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“0.0 to load the ViT and perform inferences on a batch of random images from the ImageNet dataset [19].”
InfluentialImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679 · cited 12×
“The average Critical SDC for SwinV2 models is reduced by 21.82%.”
InfluentialSwin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,484 · cited 12×
“The average Critical SDC for SwinV2 models is reduced by 21.82%.”
Influential(2022) · doi:10.1007/978-3-031-20053-3_27 · Cited by 1,060
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005 · cited 1×
“However, given the complexity of the models and the high number of parameters (which can exceed 1 trillion [3]), ViTs need to be executed on large hardware accelerators, such as Graphic Processing Units (GPUs).”
EVA-02: A visual representation for neon genesis
Fang, Y., Sun, Q., Wang, X., Huang, T., Wang, X., & Cao, Y. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105171 · Cited by 264 · cited 3×
“Thanks to their ability to learn a wide range of concepts from data, ViTs are especially useful for complex applications like autonomous driving [1] and industrial automation [2].”
Eliminating microarchitectural dependency from Architectural Vulnerability
Sridharan, V., & Kaeli, D. R. (2009) · 2009 IEEE 15th International Symposium on High Performance Computer Architecture · doi:10.1109/hpca.2009.4798243 · Cited by 182 · cited 1×
“The PVF is the probability of an injected fault propagating from the assembly instruction to the application output [21].”
Hoang, L.-H., Hanif, M. A., & Shafique, M. (2020) · Design, Automation, and Test in Europe · doi:10.23919/date48585.2020.9116571 · Cited by 161
Comparison of Combinational and Sequential Error Rates for a Deep Submicron Process
Mahatme, N. N., Jagannathan, S., Loveless, T. D., Massengill, L. W., Bhuva, B. L., Wen, S.-J., & Wong, R. (2011) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2011.2171993 · Cited by 149 · cited 1×
“Hardware accelerators for ViT employed on autonomous systems are susceptible to soft errors caused by faults induced by ionizing particles, such as high-energy neutrons [14].”
Research works printed but not linked: 2
- A. Dosovitskiy et al., "An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale," 2021.
- Z. Tu et al., "MaxViT: Multi-axis vision transformer," in ECCV, 2022, pp. 459--479.
Cited by
Reliability and Security of AI Hardware
Gnad, D. R. E., Gotthard, M., Krautter, J., Kritikakou, A. S., Meyers, V., Rech, P., Condia, J. E. R., Ruospo, A., Sánchez, E., dos Santos, F. F., et al. (2024) · IEEE European Test Symposium (ETS) · doi:10.1109/ets61313.2024.10567471 · Cited by 25
“Since the floating-point representation range is large [10], [55], [61]–[65], the corrupted values can go to infinity or even become Not a Number (NaN).”
BackgroundSelf-citationDemystifying the Resilience of Large Language Model Inference: An End-to-End Perspective
Sun, Y., Coalson, Z., Chen, S., Liu, H., Zhang, Z., Hong, S., Fang, B., & Yang, L. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759803 · Cited by 15
Special Session: Reliability Assessment Recipes for DNN Accelerators
Ahmadilivani, M. H., Bosio, A., Deveautour, B., dos Santos, F. F., Guerrero-Balaguera, J.-D., Jenihhin, M., Kritikakou, A. S., Sierra, R. L., Pappalardo, S., Raik, J., et al. (2024) · IEEE VLSI Test Symposium (VTS) · doi:10.1109/vts60656.2024.10538707 · Cited by 13
Self-citationSun, Y., Zhu, Z., Mulpuru, C., Gioiosa, R., Zhang, Z., Fang, B., & Yang, L. (2025) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3731545.3731570 · Cited by 11
Approximate Fault-Tolerant Neural Network Systems
Traiola, M., Pappalardo, S., Piri, A., Ruospo, A., Deveautour, B., Sánchez, E., Bosio, A., Saeedi, S., Carpegna, A., Göğebakan, A. B., et al. (2024) · IEEE European Test Symposium (ETS) · doi:10.1109/ets61313.2024.10567290 · Cited by 12
“A variety of approaches have been proposed, from the most realistic neutron beam exposure [64]–[68] to combining gate-level fault simulation accuracy with software fault injection’s speed and flexibility [69]–[72].”
BackgroundSelf-citation
Metadata
- Authors
- Roquet, L., Fernandes dos Santos, F., Rech, P., Traiola, M., Sentieys, O., & Kritikakou, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- TARAN, Univ Rennes, CNRS, Inria, IRISA UMR 6074, F-35000 Rennes, Rennes, Brittany, France
- University of Trento (IT)
- ORCID
- Venue
- Proceedings of the 61st ACM/IEEE Design Automation Conference
- Publisher
- ACM
- Published
- 2024-06-23
- Pages
- 1-6
- ISSN
- 0738-100X (ISSN-L)
- OpenAlex
- W4391801271
- Cited by
- 16
- References
- 22
- Citations by year
- 2026: 8 · 2025: 3 · 2024: 5
- FWCI
- 12.26
- Type
- conference paper
- Language
- en
- Open access
- green
- hal.science
- Locations
- ACM/IEEE Design Automation Conference (DAC) (publishedVersion)
- HAL (submittedVersion, cc-by)
- Funders
- Agence Nationale de la Recherche (ANR-21-CE25-0008, ANR-21-CE24-0015)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Radiation Effects in Electronics
- Physical Sciences › Computer Science › Hardware and Architecture › VLSI and Analog Circuit Testing
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- vision transformer, hardware fault tolerance, transient faults, fault mitigation, GPU reliability, fault propagation, error rate estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Cross-Layer Reliability Evaluation and Efficient Hardening of Large Vision Transformers Models. https://app.cochat.ai/papers/10.1145/3649329.3655688
- The paper
- Roquet, L., Fernandes dos Santos, F., Rech, P., Traiola, M., Sentieys, O., & Kritikakou, A. (2024). Cross-Layer Reliability Evaluation and Efficient Hardening of Large Vision Transformers Models. Proceedings of the 61st ACM/IEEE Design Automation Conference. https://doi.org/10.1145/3649329.3655688
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.