AxBy-ViT: Reconfigurable Approximate Computation Bypass for Vision Transformers
Abstract
Vision Transformers, evolved from the attention based transformer architectures, have obtained elevated attention as a competitive alternative to convolutional neural networks in computer vision learning tasks. Although showing promising accuracy, vision transformers face challenges due to large model size and intensive computation workload with millions of parameters, inhibiting their practical deployment on resource constrained platforms.In this paper, we investigate trivial computations - a set of computations the results of which can be determined without actual computations - in vision transformers. Typical examples include multiplication with 0, +1/-1 and addition with 0. Existing studies have leveraged the existence of trivial computations to save energy or accelerate execution. Based on this, we present AxBy-ViT, a reconfigurable approximate computation bypass scheme for vision transformers that leverages approximate matching of bit representations of operands. Case study on vision transformers using the ImageNet dataset shows that with 2.4% accuracy loss, AxBy-ViT can match up to 28.69% of the operand for computation bypass. The code repository of AxBy-ViT can be found at https://github.com/VU-DETAIL/AxBy-ViT.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Different blocks in ViT have different error tolerance.
“Thus, different blocks in ViT have different error tolerance.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
On increasing θ and K, the error increases exponentially.
“On increasing θ and K , the error increases exponentially.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
On setting θ to 121, MHA and MLP show an accuracy drop below 3%, but input-embedding and FC show a larger accuracy drop.
“On setting θ to 121, MHA and MLP show an accuracy drop below 3%, but input-embedding and FC show a larger accuracy drop.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
Exactly matching the operands with 0 or ±1 finds the scope of trivialization in only 0.1% of the operands.
“They note that exactly matching the operands with 0 or ± 1 finds the scope of trivialization in only 0.1% of the operands.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
The operands in input-embedding, MHA, MLP and FC computations show Gaussian distribution such that more than 50% operands are concentrated around zero.
“They note that the operands in input-embedding, MHA, MLP and FC computations show Gaussian distribution such that more than 50% operands are concentrated around zero.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
Two approximate matching techniques increase the scope of trivialization by 1000×.
“They propose two approximate matching techniques that increase this scope by 1000 × .”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
A small fraction of values in input-embedding and FC are ±1.
“Also, a small fraction of values in input-embedding and FC are ± 1.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [240]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154 · see citing works
- Authors
- Ma, D., Qin, X., & Jiao, X.
- Affiliations
- Villanova University (US)
- Venue
- 2022 23rd International Symposium on Quality Electronic Design (ISQED)
- Publisher
- IEEE
- Published
- 2022-04-06
- Pages
- 1-5
- OpenAlex
- W4283714288
- Cited by
- 1
- References
- 2
- Citations by year
- 2023: 1
- FWCI
- 0.40
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Symposium on Quality Electronic Design (ISQED) (publishedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- vision transformer, approximate computing, reconfigurable hardware, energy efficiency, resource-constrained deployment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
