CoChat

Neutron-Induced Error Rate of Vision Transformer Models on GPUs

Open access
Dos Santos, Fernando Fernandes, Rech, Paolo, Kritikakou, Angeliki, Sentieys, Olivier · 2023 · 2023 23rd European Conference on Radiation and Its Effects on Components and Systems (RADECS)

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index; 2 of the 12 sources its claims cite were checked for support.

Abstract

Vision Transformers (ViTs) are the new trend to improve performance and accuracy of machine learning. Through neutron beam experiments we show that ViTs have a higher FIT rate than traditional models but similar error criticality.

Overview

CoChat Summary

TL;DR

In neutron beam experiments on an NVIDIA Quadro M2000 GPU, Vision Transformer models showed higher silent data corruption FIT rates than ResNet-50, while error criticality was similar. The smallest-patch ViT 16/224 had the lowest critical SDC rate among the tested ViTs.

Key findings

  • SDC FIT rates were 71.60 for ResNet 50, 101.81 for ViT 16/224, 372.36 for ViT 32/224, and 398.97 for ViT 32/384, so ViT configurations had similar or much higher SDC rates than ResNet 50.
  • Critical SDC rates were 15.91 for ResNet 50, 10.43 for ViT 16/224, 16.93 for ViT 32/224, and 14.91 for ViT 32/384; ViT 16/224 had the lowest critical SDC rate across tested configurations.
  • DUE rates were similar among the tested models; System Crash/Hang and Misaligned/Illegal memory accesses were the primary DUE sources, accounting for an average of 85.2% of DUEs.
  • ViT 16/224’s SDC rate was 3.66× lower than ViT 32/224 and 3.92× lower than ViT 32/384, attributed to its smaller patch size.

Methods

  • Controlled neutron beam experiments were performed at the ChipIR facility of the Rutherford Appleton Laboratory, UK, with a neutron flux of about 10^6 n/(cm^2/s), about 7–8 orders of magnitude higher than the terrestrial flux of 13 n/(cm^2×h); observed error rates were lower than 1 error per 700 executions, and FIT rates are presented with a Poisson distribution and 95% confidence interval.
  • The device under test was an NVIDIA Quadro M2000 GPU (Maxwell ISA, 28 nm TSMC CMOS, 4 GB DDR, 768 CUDA cores, 75 W, 796 MHz). Evaluated models were ResNet 50, ViT 16/224, ViT 32/224*, and ViT 32/384, using a 256-image ImageNet subset grouped into four batches of 64, with PyTorch 1.13.0 and CUDA 11.7.
  • Outputs were compared against golden values after each iteration; events were classified as Masked, DUE, or SDC, with SDCs split into Tolerable SDCs and Critical SDCs. Only the GPU core was exposed to the beam, while board DDR and power control circuitry were outside the beam spot.

Limitations

  • The paper states that further investigation is required to understand how Transformer characteristics such as patch size and encoder structure impact error criticality, and whether the observed trend is consistent across other ViT models such as ViT-G/14, SwinV2-G, and DaViT-G.
  • The study evaluates only one GPU model and uses a 256-image ImageNet subset for efficiency; only the GPU core is irradiated, not the DDR or power control circuitry.
  • The text makes evident an ambiguity: the results paragraph reports DUE rates with the same numeric values as the Critical SDC rates, which may confuse the reported DUE figures.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/radecs59069.2023.10767034

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 The architecture of the Transformer model used in the experiments. The input image is divided into a sequence of linear patches and passed through the Encoder, which consists of a series of operations, including normalization, MLP, and the attention mechanism. The figure was extracted from [1].
Fig. 1 The architecture of the Transformer model used in the experiments. The input image is divided into a sequence of linear patches and passed through the Encoder, which consists of a series of operations, including normalization, MLP, and the attention mechanism. The figure was extracted from [1].From the PDF
Fig. 2 The experimental hardware setup comprises GPUs aligned along the ChipIR beamline, and motherboards control the GPUs. The software setup involves a Python-based server that manages GPUs within the beam room.
Fig. 2 The experimental hardware setup comprises GPUs aligned along the ChipIR beamline, and motherboards control the GPUs. The software setup involves a Python-based server that manages GPUs within the beam room.From the PDF
Fig. 3 Tolerable and Critical SDC and DUE FIT rates for ResNet 50 and 3 ViT configurations (16/224, 32/224, and 32/384).
Fig. 3 Tolerable and Critical SDC and DUE FIT rates for ResNet 50 and 3 ViT configurations (16/224, 32/224, and 32/384).From the PDF

Verification

Claims the paper makes

0 Valid · 2 Not valid · 5 Not checked · 22 Own results and interpretations

9 more claims

Create a free account to see all 29 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 7
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Citations

Cites

Show all 6 references

Research works printed but not linked: 8

  1. Khan, 2022, ACM Comput. Surv.
  2. Oliveira, 2016, IEEE Trans Comput
  3. Sullivan, IEEE/ACM International Symposium on Microarchitecture
  4. Santos, 2019, IEEE Trans. Reliab.
  5. Libano, 2019, IEEE Trans. Nucl. Sci.
  6. Junior, 2022, IEEE Trans. Nucl. Sci.
  7. Vaswani, 2017, CoRR
  8. 2006, JEDEC, JEDEC Standard, Tech. Rep.

Cited by

See all 4

Metadata

Authors
Dos Santos, F. F., Rech, P., Kritikakou, A., & Sentieys, O.
Affiliations
  • Univ. Rennes,Inria,France
  • University of Trento (IT)
  • Institut national de recherche en sciences et technologies du numérique (FR)
Venue
2023 23rd European Conference on Radiation and Its Effects on Components and Systems (RADECS)
Publisher
IEEE
Published
2023-09-25
Pages
1-4
OpenAlex
W4392278819
Cited by
4
References
4
Citations by year
2026: 2 · 2025: 2
FWCI
1.96
Type
conference paper
Language
en
Open access
Locations
  • European Conference on Radiation and Its Effects on Components and Systems (publishedVersion)
  • HAL (submittedVersion, cc-by)
Funders
  • European Commission (HORIZON2020, 101008126, 899546)
  • Agence Nationale de la Recherche (ANR-21-CE25-0008, ANR-21-CE24-0015)
  • Région Bretagne (899546)
Topics
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Radiation Effects in Electronics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
vision transformer, neutron-induced errors, GPU reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Neutron-Induced Error Rate of Vision Transformer Models on GPUs. https://app.cochat.ai/papers/10.1109/radecs59069.2023.10767034
The paper
Dos Santos, F. F., Rech, P., Kritikakou, A., & Sentieys, O. (2023). Neutron-Induced Error Rate of Vision Transformer Models on GPUs. 2023 23rd European Conference on Radiation and Its Effects on Components and Systems (RADECS). https://doi.org/10.1109/radecs59069.2023.10767034

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.