Vision Transformers: From Semantic Segmentation to Dense Prediction
Open accessThe DOI matches its Crossref record; 102 of the 109 references it cites were found in a registry or an index, and 6 are not checked yet; 8 of the 23 sources its claims cite were checked for support.
Abstract
The emergence of vision transformers (ViTs) in image classification has shifted the methodologies for visual representation learning. In particular, ViTs learn visual representation at full receptive field per layer across all the image patches, in comparison to the increasing receptive fields of CNNs across layers and other alternatives (e.g., large kernels and atrous convolution). In this work, for the first time we explore the global context learning potentials of ViTs for dense visual prediction (e.g., semantic segmentation). Our motivation is that through learning global context at full receptive field layer by layer, ViTs may capture stronger long-range dependency information, critical for dense prediction tasks. We first demonstrate that encoding an image as a sequence of patches, a vanilla ViT without local convolution and resolution reduction can yield stronger visual representation for semantic segmentation. For example, our model, termed as SEgmentation TRansformer (SETR), excels on ADE20K (50.28% mIoU, the first position in the test leaderboard on the day of submission) and performs competitively on Cityscapes. However, the basic ViT architecture falls short in broader dense prediction applications, such as object detection and instance segmentation, due to its lack of a pyramidal structure, high computational demand, and insufficient local context. For tackling general dense visual prediction tasks in a cost-effective manner, we further formulate a family of Hier…
Verified 2026-11-19 · 6 references not checked yet
Overview
CoChat Summary
TL;DR
This paper extends Vision Transformers from image classification to dense prediction, first introducing SETR for semantic segmentation and then a Hierarchical Local-Global (HLG) Transformer backbone for general dense tasks. It reports state-of-the-art or competitive results on ADE20K, Cityscapes, ImageNet, and COCO while noting high computational and pretraining costs.
Key findings
- SETR replaces the FCN with a ViT encoder using global self-attention at every layer; the paper reports new state of the art on ADE20K (50.28%) and Pascal Context (55.83%), competitive Cityscapes results, and first place on the ADE20K test server leaderboard at submission day.
- On ADE20K, SETR-MLA achieves 48.64% mIoU single-scale and 50.3% multi-scale; with BEiT pretraining, SETR-MLA-BEiT reaches 53.1% mIoU multi-scale. On Cityscapes validation, SETR-PUP reaches 79.34% mIoU at 80k; on Cityscapes test, SETR-PUP reaches 81.08% and 81.64% with coarse data.
- HLG Transformers achieve ImageNet-1K Top-1 of 81.1% (HLG-Tiny), 82.3% (HLG-Small), 83.6% (HLG-Medium), and 84.1% (HLG-Large). On COCO, HLG-Large reaches 47.7 AP with RetinaNet detection and 49.1 APb / 44.2 APm with Mask R-CNN instance segmentation.
- Ablations show pretraining is critical: randomly initialized SETR-PUP gives only 42.27% mIoU on Cityscapes. Hierarchical local attention brings a 0.64% increment, and step-by-step HLG components improve ImageNet Top-1 from 79.2% to 82.3%.
Methods
- Design: SETR treats an image as a sequence of patches, uses a standard Transformer encoder with global self-attention, and introduces three decoders—Naive upsampling, Progressive UPsampling (PUP), and Multi-Level feature Aggregation (MLA). HLG adds a four-stage pyramidal backbone with local window attention, dilated local attention, global attention, parameter/computation sharing, and DWMLP.
- Data: semantic segmentation on Cityscapes (5000 finely annotated images, 19,998 coarse images, 19 classes), ADE20K (20,210/2,000/3,352 images, 150 classes), and Pascal Context; classification on ImageNet-1K (1.28M training, 50K validation, 1,000 classes) and ImageNet-1K V2; detection and instance segmentation on COCO 2017 (118K training, 5K validation).
- Analysis: mean IoU and pixel accuracy for segmentation, Top-1 accuracy for classification, AP/APb/APm for detection and instance segmentation; comparisons against FCN, Semantic FPN, ViT/DeiT/BEiT, Swin, PVT, and others, plus ablations on decoders, pretraining, auxiliary loss, window embedding, and local attention.
Limitations
- The paper states that single-stage Transformers suffer from high computational and pretraining costs, and that Vision Transformers incur higher FLOPs, time, and GPU memory usage than convolutional models; randomly initialized SETR-PUP gives only 42.27% mIoU, showing pretraining is critical.
- The paper notes that all its models use output stride 16 due to GPU memory constraints, whereas the dilated FCN baseline uses output stride 8.
- No dedicated limitations section is present; evident from reported results, HLG-Large has lower FPS (221) and higher memory (31.6GB) than Swin-B (278 FPS, 25.2GB) in Table 8.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 3 Not valid · 9 Not checked · 98 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Even with DeiT Touvron et al. (2022) to reduce computation costs, the results of SETR paired with DeiT show a significant decline.”
What the sources say
- Supportedcites Touvron et al. (2022) matched to “DeiT III: Revenge of the ViT” (2022) source
The citation is attached to DeiT, identifying the method used in the citing authors’ SETR experiment. The source discusses DeiT in Related Work (p. 4) and reports a DeiT-S segmentation comparison in Section 4.4.3 (p. 19). The reported decline in SETR results is the citing authors’ own observation, not a finding attributed to this source. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Nonetheless, none of these can fully eliminate the limitation of FCN architecture in receptive field.”
What the sources say
- Not checked — no full text of the sourcecites Peng et al. (2017) matched to “Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network” (2017) source
- Partly supportedcites Chen et al. (2017) matched to “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs” (2018) sourceExperimental — accuracy not yet measured
Section 3.1 (p. 5) describes a trade-off between a small field of view for accurate localization and a large one for context, which supports the idea that enlarging it does not solve every problem. But the paper says atrous convolution can arbitrarily enlarge a filter’s field of view; it does not conclude that none of the listed remedies can fully eliminate an FCN receptive-field limitation, and it does not assess the attention methods in the claim. (LLM, conf=0.76)
- Not checked — no full text of the sourcecites Zhao et al. (2017) matched to “Pyramid Scene Parsing Network” (2017) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, employing superior pre-training techniques on 'TBase' such as DeiT Touvron et al. (2022) yields even better performance for these smaller Vision Transformer models on Cityscapes compared to their larger counterparts.”
What the sources say
- Partly supportedcites Touvron et al. (2022) matched to “DeiT III: Revenge of the ViT” (2022) sourceExperimental — accuracy not yet measured
Section 4.4.2 (p. 18) reports that the paper's ViT training recipe improves the FLOPs–accuracy trade-off in semantic segmentation, but those experiments use ADE20k, not Cityscapes. Table 11 (p. 19) does not establish that a T-Base model outperforms larger ViTs; its reported ViT-L segmentation scores exceed those of ViT-B. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“their model still follows the conventional FCN design in that spatial resolution of feature maps is reduced progressively.”
What the sources say
- Supportedcites Wang et al. (2020) matched to “Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation” (2020) source
Section 3.2 (p. 7) describes a ResNet-based architecture with a strided stem and striding between stages, yielding feature maps at one-sixteenth of the input resolution. The authors remove the stride from the final stage, but the backbone still progressively reduces spatial resolution. (LLM, conf=0.89)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and its performance is on par with the best results reported so far.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
72 more claims
Create a free account to see all 117 claims and to verify the 3 references still open.
- It may state two separate claims as one. 28
- The quote only partly states this claim. 2
- Probably not a substantial claim. 14
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 4.1 Single-stage Transformer Setup · p. 9
“| SETR- PUP | 21K | T-Large | 318M | 1083G | 18 | 3582M | 78.39 | 79.34 |”
§ 4.1 Single-stage Transformer Setup · p. 9
“| SETR- PUP | 21K | T-Large | 426G | 70 | 48.58 | 1083G | 6 | 78.39 |”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Image classification details · p. 11
“The number of training iteration is set to 40K for Cityscapes.”
§ Image classification details · p. 11
“| SETR- PUP-HLG-Large (80k, MS) | 1K | HLG-Large | 100M | 791G | 82.9 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ SETR variants · p. 10
“| SETR- MLA-BEiT (160k, SS) | 21K | ViT-Large | 311M | 318G | 53.0 |”
§ 4.1 Single-stage Transformer Setup · p. 9
“| SETR- MLA-BEiT | 21K | T-Large | 368G | 80 | 53.0 | 972G | 7 | 80.20 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.3 Vision Transformer Pretraining · p. 12
“HLG-Small with 34.2M parameters achieves 82.3% accuracy”
§ 4.4 Results of Semantic Segmentation · p. 13
“| HLG-Small | 224 2 | 24.2M | 4.7G | 82.3 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Single-stage Transformer Setup · p. 9
“| SETR- MLA | 21K | T-Large | 311M | 972G | 20 | 3353M | 76.65 | 77.24 |”
§ 4.1 Single-stage Transformer Setup · p. 9
“| SETR- MLA | 21K | T-Large | 368G | 80 | 48.64 | 972G | 7 | 76.65 |”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 13×
“Method #Params AP AP50 AP75 APS APM APL ResNet18 [4] 21.”
InfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“In this work, we alternatively rethink the semantic segmentation task from a different perspective.”
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“S INCE the introduction of Vision Transformers (ViTs) to image classification [1], the landscape of visual representation learning has gradually shifted away from CNNs [2], [3], [4], [5], [6].”
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · cited 1×
“Many recent efforts have been focused on addressing the limited receptive field/context modeling problem in FCN.”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · cited 2×
“Inspired by this phenomenal success, for the first time we expand the application of ViTs from image classification [1] to more challenging dense prediction ( e.g. , semantic segmentation).”
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 3×
“AANet [41] mixes convolution and self-attention for backbone training.”
InfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 8×
“Inspired by this phenomenal success, for the first time we expand the application of ViTs from image classification [1] to more challenging dense prediction (e.”
InfluentialMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 1×
“…73.93 FCN (40k, MS) [60] ResNet-101 75.14 FCN (80k, SS) [60] ResNet-101 75.52 FCN (80k, MS) [60] ResNet-101 76.61 PSPNet [14] ResNet-101 78.50 DeepLab-v3 [75] (MS) ResNet-101 79.30 NonLocal [15] ResNet-101 79.10 CCNet [16] ResNet-101 80.20 GCNet [76] ResNet-101 78.10 Axial-DeepLab-XL [18]”
InfluentialImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“The ImageNet-1K dataset [79] contains 1.”
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · cited 1×
“This scheme is however limited to short range interaction within each individual window.”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 12×
“Despite the success of ViTs [1] on coarse classification tasks, more cost-effective variants are necessary for more challenging dense prediction tasks.”
InfluentialFeature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 3×
“We first evaluate the image classification task using our HLG Trans-formers as the backbone.”
InfluentialHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 1×
“This plain local attention can be achieved by non-overlapping window division with focus on information exchange between spatially adjacent patches (Figure 4 left).”
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 2×
“Recently, ViT [1] is the first work to show that a pure Transformer based image classification model can achieve the state-of-the-art.”
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 4×
“Window embedding operations can be implemented by depth-wise convolution [56], average pooling or max pooling with kernel size and stride equal to the window size.”
InfluentialEncoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · cited 2×
“This leads to different variants of the encoder-decoder structures [9, 28, 29] for multi-level feature fusion.”
BackgroundEnd-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · cited 2×
“DETR [45] and the following deformable version utilize Trans-former for object detection where Transformer is appended inside the detection head.”
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
U-Net: Convolutional Networks for Biomedical Image SegmentationPreprint
Ronneberger, O., Fischer, P., & Brox, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04597 · Cited by 17,820
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · cited 6×
“…(40k, SS) [60] ResNet-101 73.93 FCN (40k, MS) [60] ResNet-101 75.14 FCN (80k, SS) [60] ResNet-101 75.52 FCN (80k, MS) [60] ResNet-101 76.61 PSPNet [14] ResNet-101 78.50 DeepLab-v3 [75] (MS) ResNet-101 79.30 NonLocal [15] ResNet-101 79.10 CCNet [16] ResNet-101 80.20 GCNet [76] ResNet-101 78.10…”
BackgroundMethodInfluentialAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 3×
“Evaluation metric: Following the standard evaluation protocol [61], the metric of mean Intersection over Union (mIoU) averaged over all classes is reported.”
InfluentialWang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · cited 4×
“Indeed, there are a number of remedies introduced, e.g. , manipulating the convolution operation (large kernel sizes [11], atrous convolutions [12, 13], and image/feature pyramids [14]), integrating attention modules [15– 17], and decomposed attention [18].”
BackgroundResultInfluentialLiu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 1×
“STTR [46] and LSTR [47] adopt Transformer for disparity estimation and lane shape prediction respectively.”
Dual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351 · cited 1×
“DANet [34] embeds both spatial attention and channel attention.”
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 9×
“For example, PVT [49] first introduces spatial reduction attention in a pyramid architecture.”
InfluentialLearning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422 · cited 1×
“Apart from these pure Transformer based models, there are also CNN-Transformer hybrid ones.”
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 2×
“STTR [46] and LSTR [47] adopt Transformer for disparity estimation and lane shape prediction respectively.”
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008
Self-citationTransformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045 · cited 4×
“In contrast, our prediction model keeps the same spatial resolution throughout and thus represents a step-change in model design.”
InfluentialBiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_20 · Cited by 2,968 · cited 2×
“Crucially, it has been playing a timing, critical role in forming the recent surge of research on contextual visual representation learning [19–23].”
Unified Perceptual Parsing for Scene UnderstandingPreprint
Xiao, T.-T., Liu, Y., Zhou, B.-L., Jiang, Y.-N., & Sun, J. (2018) · European Conference on Computer Vision · doi:10.48550/arxiv.1807.10221 · Cited by 2,735
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,484
CvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,444
Self-citationPvt v2: Improved baselines with pyramid vision transformer
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0274-8 · Cited by 2,439 · cited 1×
“With a linear embedding layer applied to the flattened pixel vectors of every patch, we then obtain a sequence of feature embedding vectors as the input to a ViT.”
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1×
“While the predictions of FCN are relatively coarse, several CRF/MRF [25], [26], [27] based approaches are developed to help refine the coarse predictions.”
Self-citationGCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond
Cao, Y., Xu, J., Lin, S., Wei, F., & Hu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00246 · Cited by 2,307
Unified Perceptual Parsing for Scene Understanding
Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_26 · Cited by 2,201 · cited 2×
“The segmentation result is obtained by Progressive UPsampling (PUP).”
InfluentialCrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
Chen, C.-F. R., Fan, Q., & Panda, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00041 · Cited by 2,074 · cited 1×
“CrossViT [51] instead leverages interaction between large-patch and small-patch branches.”
BackgroundSemantic Understanding of Scenes Through the ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., & Torralba, A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1140-0 · Cited by 1,831 · cited 1×
“ADE20K: [62] is a challenging scene parsing benchmark with 150 fine-grained semantic concepts.”
Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network
Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.189 · Cited by 1,750 · cited 2×
“Nonetheless, none of these can fully eliminate the limitation of FCN architecture in receptive field.”
DenseASPP for Semantic Segmentation in Street Scenes
Yang, M., Yu, K., Zhang, C., Li, Z., & Yang, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00388 · Cited by 1,674 · cited 2×
“It is shown that ViT can learn stronger long-range dependency information critical for semantic segmentation in unconstrained scene images [9, 10], achieving superior semantic segmentation performances.”
InfluentialGoing Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Panoptic Feature Pyramid Networks
Kirillov, A., Girshick, R., He, K., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00656 · Cited by 1,460 · cited 2×
“The segmentation result is obtained by Progressive UPsampling (PUP).”
The Role of Context for Object Detection and Semantic Segmentation in the Wild
Mottaghi, R., Chen, X., Liu, X., Cho, N.-G., Lee, S.-W., Fidler, S., Urtasun, R., & Yuille, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.119 · Cited by 1,438
Zhang, X. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_79 · Cited by 1,434
Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.00112 · Cited by 1,394 · cited 1×
“First, though convolution is completely removed in [18] as in our SETR, their model still follows the conventional FCN design in that spatial resolution of feature maps is reduced progressively.”
BEiT: BERT Pre-Training of Image Transformers
Bao, H., Dong, L., Piao, S., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2106.08254 · Cited by 1,392
Object-Contextual Representations for Semantic Segmentation
Yuan, Y., Chen, X., & Wang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_11 · Cited by 1,285
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · cited 1×
“…the Trans-formers output feature by 16 × in one-shot, on ADE20K val set (Table 4 and 5 method outperforms the FCN counterparts by a large margin, verifying that the advantage largely comes from the proposed sequence-to-sequence representation learning strategy rather than bigger pre-training data.”
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q., Vaswani, A., & Shlens, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,205 · cited 1×
“AANet [41] mixes convolution and self-attention for backbone training.”
PSANet: Point-wise Spatial Attention Network for Scene Parsing
Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C. C., Lin, D., & Jia, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_17 · Cited by 1,192 · cited 1×
“PSANet [33] develops the pointwise spatial attention module for dynamically capturing the long range context.”
BackgroundThe Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174
Exploring Self-Attention for Image Recognition
Zhao, H., Jia, J., & Koltun, V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01009 · Cited by 1,002 · cited 3×
“SAN [44] explores two types of self-attention modules.”
InfluentialQuantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.385 · Cited by 995 · cited 1×
“Similar to [98], a recursion perspective into this problem is applied.”
Stand-Alone Self-Attention in Vision Models
Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019) · arXiv · doi:10.48550/arxiv.1906.05909 · Cited by 977
CoAtNet: Marrying Convolution and Attention for All Data Sizes
Dai, Z., Liu, H., Le, Q. V., & Tan, M. (2021) · arXiv · doi:10.48550/arxiv.2106.04803 · Cited by 925
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic SegmentationPreprint
Wang, H., Zhu, Y.-K., Green, B., Adam, H., Yuille, A., & Chen, L.-C. (2020) · European Conference on Computer Vision · doi:10.48550/arxiv.2003.07853 · Cited by 880
Twins: Revisiting the Design of Spatial Attention in Vision Transformers
Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., & Shen, C. (2021) · arXiv · doi:10.48550/arxiv.2104.13840 · Cited by 817 · cited 1×
“Our contributions with SETR are summarized as follows: (1) As an exemplar dense prediction problem, we explore the potentials of self-attention mechanism for visual representation learning, offering an alternative to the dominating FCN design.”
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., & Chen, L.-C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_7 · Cited by 809 · cited 15×
“On this dataset we can now compare with the closely related Axial-DeepLab [18, 78] which aims to use an attention-alone model but still follows the basic structure of FCN.”
BackgroundMethodInfluentialDo ImageNet Classifiers Generalize to ImageNet?
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019) · arXiv · doi:10.48550/arxiv.1902.10811 · Cited by 790
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782
Touvron, H., Cord, M., & Jégou, H. (2022) · European Conference on Computer Vision · doi:10.48550/arxiv.2204.07118 · Cited by 733
Semantic Image Segmentation via Deep Parsing Network
Liu, Z., Li, X., Luo, P., Loy, C.-C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.162 · Cited by 675 · cited 1×
“While the predictions of FCN are relatively coarse, several CRF/MRF [25], [26], [27] based approaches are developed to help refine the coarse predictions.”
Rethinking Spatial Dimensions of Vision Transformers
Heo, B., Yun, S., Han, D., Chun, S., Choe, J., & Oh, S. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01172 · Cited by 640
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
Cheng, B., Collins, M. D., Zhu, Y., Liu, T., Huang, T. S., Adam, H., & Chen, L.-C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01249 · Cited by 635
A Real-Time Algorithm for Signal Analysis with the Help of the Wavelet Transform
Holschneider, M., Kronland-Martinet, R., Morlet, J., & Tchamitchian, P. (1989) · Inverse Problems and Theoretical Imaging · doi:10.1007/978-3-642-97177-8_28 · Cited by 633
Local Relation Networks for Image Recognition
Hu, H., Zhang, Z., Xie, Z., & Lin, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00356 · Cited by 570
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2014) · arXiv · doi:10.48550/arxiv.1409.0575 · Cited by 567
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2016) · arXiv · doi:10.48550/arxiv.1610.02357 · Cited by 538
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Wei, Y., Huang, L., Shi, H., Liu, W., & Huang, T. S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3007032 · Cited by 524
Neighborhood Attention Transformer
Hassani, A., Walton, S., Li, J., Li, S., & Shi, H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00599 · Cited by 482
OCNet: Object Context Network for Scene Parsing
Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00916 · Cited by 453
Touvron, H., Cord, M., & Jégou, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20053-3_30 · Cited by 409
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03144 · Cited by 380
Axial Attention in Multidimensional Transformers
Ho, J., Kalchbrenner, N., Weissenborn, D., & Salimans, T. (2019) · arXiv · doi:10.48550/arxiv.1912.12180 · Cited by 370
A Real-Time Algorithm for Signal Analysis with the Help of the Wavelet Transform
Holschneider, M., Kronland-Martinet, R., Morlet, J., & Tchamitchian, P. (1990) · inverse problems and theoretical imaging · doi:10.1007/978-3-642-75988-8_28 · Cited by 339 · cited 1×
“Indeed, there are a number of remedies introduced, e.g. , manipulating the convolution operation (large kernel sizes [11], atrous convolutions [12, 13], and image/feature pyramids [14]), integrating attention modules [15– 17], and decomposed attention [18].”
End-to-end Lane Shape Prediction with Transformers
Liu, R., Yuan, Z., Liu, T., & Xiong, Z. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00374 · Cited by 338 · cited 1×
“STTR [46] and LSTR [47] adopt Transformer for disparity estimation and lane shape prediction respectively.”
MethodRevisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers
Li, Z., Liu, X., Drenkow, N., Ding, A., Creighton, F. X., Taylor, R. H., & Unberath, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00614 · Cited by 335 · cited 3×
“Beyond semantic segmentation, we further apply the proposed HLG Transformers to image classification and object detection problems, demonstrating the potentials of serving as a versatile ViT backbone.”
InfluentialMulti-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding
Zhang, P., Dai, X., Yang, J., Xiao, B., Yuan, L., Zhang, L., & Gao, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00299 · Cited by 330
Self-citationLocalViT: Analyzing Locality in Vision TransformersPreprint
Li, Y., Zhang, K., Cao, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.05707 · Cited by 279
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01105 · Cited by 255
BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · arXiv · doi:10.48550/arxiv.1808.00897 · Cited by 193
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182
Dynamic Graph Message Passing Networks
Zhang, L., Xu, D., Arnab, A., & Torr, P. H. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00378 · Cited by 128 · cited 2×
“DGMN [36] builds a dynamic graph message passing network for scene modeling and it can significantly reduce the computational complexity.”
Self-citationAll the attention you need: Global-local, spatial-channel attention for image retrieval
Song, C. H., Han, H. J., & Avrithis, Y. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00051 · Cited by 116
Dual Graph Convolutional Network for Semantic Segmentation
Li, Z., Li, X., Arnab, A., Yang, K., Tong, Y., & Torr, P. H. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06121 · Cited by 106 · cited 1×
“Recently, a number of state-of-the-art meth-ods [16, 36, 52] suggest that combing FCN with attention mechanism is a more effective strategy for learning long-range contextual information.”
Self-citationRegionViT: Regional-to-Local Attention for Vision Transformers
Chen, C.-F., Panda, R., & Fan, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02689 · Cited by 92 · cited 1×
“Same as recent ViTs [19, 21–24], we also borrow the pyramidal structure from CNNs [4, 6] for enabling the use of smaller image patches.”
All the attention you need: Global-local, spatial-channel attention for image retrieval
Song, C., Han, H. J., & Avrithis, Y. (2021) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.48550/arxiv.2107.08000 · Cited by 75
ConTNet: Why not use convolution and transformer at the same time?
Yan, H., Li, Z., Li, W., Wang, C., Wu, M., & Zhang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13497 · Cited by 57
Local-to-Global Self-Attention in Vision TransformersPreprint
Li, J., Yan, Y., Liao, S., Yang, X., & Shao, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04735 · Cited by 50
Global Aggregation then Local Distribution in Fully Convolutional Networks
Li, X., Li, Z., You, A., Yang, M., Yang, K., & Tong, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07229 · Cited by 40 · cited 2×
“Crucially, it has been playing a timing, critical role in forming the recent surge of research on contextual visual representation learning [19–23].”
Visual Parser: Representing Part-whole Hierarchies with TransformersPreprint
Sun, S., Yue, X., Bai, S., & Torr, P. H. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05790 · Cited by 34
Self-citationZhang, Z., Zhang, H., Zhao, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12723 · Cited by 4
Quantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · arXiv · doi:10.48550/arxiv.2005.00928
Research works printed but not linked: 8
- Bao H, Dong L, Piao S, Wei F (2022) Beit: Bert pretraining of image transformers. In: International Conference on Learning Representations
- Chen CF, Panda R, Fan Q (2022) Regionvit: Regionalto-local attention for vision transformers. In: International Conference on Learning Representations
- Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2015) Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: International Conference on Learning Representations
- Larsson G, Maire M, Shakhnarovich G (2016) Fractalnet: Ultra-deep neural networks without residuals. In: International Conference on Learning Represen-
- Li J, Yan Y, Liao S, Yang X, Shao L (????) Localto-global self-attention in vision transformers. arxiv 2021. arXiv preprint arXiv:210704735
- Li Y, Zhang K, Cao J, Timofte R, Van Gool L (2021a) Localvit: Bringing locality to vision transformers. arXiv preprint
- Veliˇ ckovi´ c P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International Conference on Learning Representations
- Zhang Z, Zhang H, Zhao L, Chen T, Pfister T (2021b) Aggregating nested transformers. arXiv preprint
Cited by
Hybrid Quantum–Classical Neural Networks for Efficient MNIST Binary Image Classification
Ranga, D., Prajapat, S., Akhtar, Z., Kumar, P., & Vasilakos, A. V. (2024) · Mathematics · doi:10.3390/math12233684 · Cited by 51
“ViT is computationally expensive and lacks specialization to smaller datasets [55,56].”
Ghribi, F., & Hamdaoui, F. (2025) · The Visual Computer · doi:10.1007/s00371-025-04122-1 · Cited by 30
SATR: A Structure-Affinity Attention-Based Transformer Encoder for Spine Segmentation
Xie, H., Huang, Z., Leung, F. H. F., et al. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635612 · Cited by 7
“This first pure transformer was applied directly to sequences of image patches and obtained comparable and even better re-sults in some tasks than CNNs, such as semantic segmentation [19, 20].”
MethodNoroozi, M., Arabi, H., & Karimian, A. (2024) · National and International Iranian Conference on Biomedical Engineering (ICBME) · doi:10.1109/icbme64381.2024.10895718 · Cited by 7
“• SETR [20] has a relatively lower dice coefficient of 0.9745 ± 2.215 e -05 but an improved accuracy value of 0.9649 ± 1.967 e -05 , making it meaningful for medical image segmentation, including handling different complexities related to cardiac anatomy.”
InfluentialKewenig, V. N., Lampinen, A. K., Nastase, S. A., Edwards, C., D'Elascombe, Q., Rechardt, A., Skipper, J. I., & Vigliocco, G. (2024) · Research Square · doi:10.21203/rs.3.rs-3913308/v1 · Cited by 6
Metadata
- Authors
- Zhang, L., Lu, J., Zheng, S., Zhao, X., Zhu, X., Fu, Y., Xiang, T., Feng, J., & Torr, P. H. S.
- Affiliations
- Fudan University (CN)
- University of Surrey (GB)
- University of Oxford (GB)
- Corresponding author
- Zhang, Li
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2024-07-16
- Volume
- 132
- Issue
- 12
- Pages
- 6142-6162
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W4400677548
- Cited by
- 23
- References
- 89
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 4
- FWCI
- 3.20
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- International Journal of Computer Vision (publishedVersion)
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- vision transformer, semantic segmentation, long-range dependencies, object detection, instance segmentation, image classification, visual representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.