Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 4 of the 4 sources its claims cite were checked for support.

PTQ4ViT: Post-training Quantization for Vision Transformers with Twin Uniform Quantization

Open access
Yuan, Zhihang, Xue, Chenhao, Chen, Yiqi, Wu, Qiang, Sun, Guangyu · 2022 · Lecture notes in computer science

Abstract

The twin uniform quantization method is proposed to reduce the quantization error on activation values after softmax and GELU functions and to use a Hessian guided metric to evaluate different scaling factors.

Overview

CoChat Summary

TL;DR

PTQ4ViT is a post-training quantization framework for vision transformers that uses twin uniform quantization for post-softmax and post-GELU activations and a Hessian-guided metric to select scaling factors. It achieves near-lossless ImageNet classification accuracy at 8-bit quantization, with less than 0.5% top-1 drop on ViT, DeiT, and Swin models.

Key findings

  • At 8-bit quantization, PTQ4ViT achieves less than 0.5% accuracy drop across tested models, e.g. ViT-B/384 drops 0.17% (86.00 → 85.82) and Swin-B/384 drops 0.04% (86.44 → 86.39), while base PTQ drops more than 1% on some vision transformers.
  • At 6-bit quantization, PTQ4ViT reduces average accuracy drop to 2.1% versus 9.8% for base PTQ; e.g. DeiT-B/224 reaches 80.25% vs 81.80% FP32, and ViT-B/384 reaches 83.34% vs 86.00% FP32.
  • PTQ4ViT outperforms EasyQuant and Liu et al. at 8-bit and 6-bit using only 32 calibration images versus 1024: DeiT-S/224 W8A8 79.47% vs 76.59% (EasyQuant) and 77.47% (Liu), and W6A6 76.28% vs 73.26% and 74.58%.
  • Ablation shows both components matter: on ViT-S/224, using Hessian-guided metric alone improves 6-bit accuracy by 6.96%, and using twin uniform quantization without Hessian guidance drops 8-bit accuracy to 79.25%, even below base PTQ’s 80.47%.

Methods

  • The paper proposes twin uniform quantization with two ranges R1 and R2 and separate scaling factors, using a data format where the most significant bit flags the range and constraining ∆R2 = 2^m ∆R1 so ranges can be aligned with shift operations on CPUs/GPUs.
  • It proposes a Hessian-guided metric based on a diagonal Fisher information approximation to select scaling factors layer-by-layer, replacing MSE, cosine distance, and Pearson correlation coefficient, which the authors find inaccurate on vision transformers.
  • Experiments use ImageNet classification with ViT, DeiT, and Swin models, 32 randomly selected calibration images, quantization of weights and inputs for fully connected layers and matrix multiplication inputs, and no quantization of softmax or normalization layers.

Limitations

  • The paper states that at 4-bit quantization PTQ4ViT performs poorly: DeiT-B W4A4 reaches 60.91%, and with bias correction 64.39%, still lower than Liu et al.’s mixed-precision W4A4 result of 75.94%; it concludes mixed-precision is important for lower bit-widths.
  • The paper states the Hessian-guided metric is closer to task loss than other metrics but still has a gap with task loss.
  • Evident from the text: evaluation is limited to ImageNet classification on ViT/DeiT/Swin; the method assumes the floating-point network’s prediction approximates ground truth and uses a diagonal Fisher approximation, and no actual hardware latency or energy measurements are reported despite hardware-efficiency claims.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/978-3-031-19775-8_12

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 1 Not valid · 0 Not checked · 58 Own results and interpretations

40 more claims

Create a free account to see all 63 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 6
  • It may state two separate claims as one. 14

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 29 references

Research works printed but not linked: 1

  1. Banner, R., Nahshan, Y., Soudry, D.: Post training 4-bit quantization of convolutional networks for rapid-deployment. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8–14 December 2019. Vancouver, BC, Canada, pp. 7948–7956 (2019). https://proceedings.neurips.cc/paper/2019/hash/c0a62e133894cdce435bcb4a5df1db2d-Abstract.html

Cited by

See all 163

Metadata

Authors
Yuan, Z., Xue, C., Chen, Y., Wu, Q., & Sun, G.
Affiliations
Peking University (CN)
Corresponding author
Sun, Guangyu
Venue
Lecture notes in computer science
Published
2022
Pages
191-207
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
OpenAlex
W4313069943
Cited by
163
References
29
Citations by year
2026: 45 · 2025: 61 · 2024: 36 · 2023: 19 · 2022: 2
FWCI
10.70
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (B18001, 62032001, 92064006, 61832020)
  • Higher Education Discipline Innovation Project (B 18001)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
Keywords
post-training quantization, vision transformer, neural network compression, activation distribution, calibration, 8-bit quantization, ImageNet classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: PTQ4ViT: Post-training Quantization for Vision Transformers with Twin Uniform Quantization. https://app.cochat.ai/papers/10.1007/978-3-031-19775-8_12
The paper
Yuan, Z., Xue, C., Chen, Y., Wu, Q., & Sun, G. (2022). PTQ4ViT: Post-training Quantization for Vision Transformers with Twin Uniform Quantization. Lecture notes in computer science. https://doi.org/10.1007/978-3-031-19775-8_12

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.