CoChat
Sign Up
Cited by 957Advanced Neural Network ApplicationsVisual Attention and Saliency DetectionDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.

EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention

Open access
Liu, Xinyu, Peng, Houwen, Zheng, Ningxin, Yang, Yuqing, Hu, Han, Yuan, Yixuan·2023·2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52729.2023.01386
Open this paper in CoChat — free accountOpen PDF

Abstract

Vision transformers have shown great success due to their high model capabilities. However, their remarkable performance is accompanied by heavy computation costs, which makes them unsuitable for real-time applications. In this paper, we propose a family of high-speed vision transformers named Efficient ViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations, especially the tensor reshaping and element-wise functions in MHSA. Therefore, we design a new building block with a sandwich layout, i.e., using a single memory-bound MHSA between efficient FFN layers, which improves memory efficiency while enhancing channel communication. Moreover, we discover that the attention maps share high similarities across heads, leading to computational redundancy. To address this, we present a cascaded group attention module feeding attention heads with different splits of the full feature, which not only saves computation cost but also improves attention diversity. Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy. For instance, our EfficientViT-M5 surpasses MobileNetV3-Large by 1.9% in accuracy, while getting 40.4% and 45.2% higher throughput on Nvidia V100 GPU and Intel Xeon CPU, respectively. Compared to the recent efficient model MobileViT-XXS, EfficientViT-M2 achieves 1.8% superior accuracy, while running$5.8\times/3.7\times$faster on the GPU/CPU, and$7.4\times faster$when converted to ONNX format. Code and models are available at here.

Overview

Integrity signals

Registry

Crossref · 10.1109/cvpr52729.2023.01386

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 84 references

Research works printed but not linked: 7

  1. gong, 2022, NASVit: Neural architecture search for efficient vision transformers with gradient conflict aware supernet training, ICLRE
  2. beyer, 0, Are we done with imagenet?
  3. bai, 2019, Onnx Open neural network exchange
  4. wightman, 2019, pytorch-image-models
  5. ba, 2016, Layer normalization
  6. 2021, Coremltools Use coremltools to convert machine learning models from third-party libraries to the core ml format
  7. vasu, 2022, An improved one millisecond mobile backbone

Cited by

Sorted by citations

See all 957

Related

Metadata

Authors
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y.
Affiliations
  • Chinese University of Hong Kong (HK)
  • Microsoft (United States) (US)
ORCID
Venue
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2023-06
Pages
14420-14430
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52729.2023.01386
OpenAlex
W4386076539
Semantic Scholar
9a83aeadc8db65fb6da39ec977360541cddaff5c
Cited by
956
References
84
Citations by year
2026: 289 · 2025: 431 · 2024: 225 · 2023: 11
FWCI
102.10
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Innovation and Technology Commission (ITS/100/20)
  • Innovation and Technology Fund
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
EfficientViT, multi-head self-attention, channel communication, real-time image classification, computational efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review