The DOI matches its Crossref record; 71 of the 88 references it cites were found in a registry or an index; 8 of the 27 sources its claims cite were checked for support.
Vision Transformers for Dense Prediction
Open accessAbstract
We introduce dense prediction transformers, an architecture that leverages vision transformers in place of convolutional networks as a backbone for dense prediction tasks. We assemble tokens from various stages of the vision transformer into image-like representations at various resolutions and progressively combine them into full-resolution predictions using a convolutional decoder. The transformer backbone processes representations at a constant and relatively high resolution and has a global receptive field at every stage. These properties allow the dense prediction transformer to provide finer-grained and more globally coherent predictions when compared to fully-convolutional networks. Our experiments show that this architecture yields substantial improvements on dense prediction tasks, especially when a large amount of training data is available. For monocular depth estimation, we observe an improvement of up to 28% in relative performance when compared to a state-of-the-art fully-convolutional network. When applied to semantic segmentation, dense prediction transformers set a new state of the art on ADE20K with 49.02% mIoU. We further show that the architecture can be fine-tuned on smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets the new state of the art. Our models are available at https://github.com/intel-isl/DPT.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
32 Own result · 23 Own interpretation · 11 No basis given · 10 No full text of the sources · 3 Supported · 2 Partly supported · 1 Supported so far · 1 Own method · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“Specifically, we use the recently proposed vision transformer (ViT) [11] as a backbone architecture.”
§ Vision Transformers for Dense Prediction > 1. Introduction
What the sources say
- Supportedcites [11] 11. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. source
Section 3.1 (p. 3) introduces and describes the Vision Transformer (ViT) architecture. The citation identifies the ViT used as the citing authors’ backbone; their choice to use it is not a claim attributed to this source. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Virtually all existing architectures for dense prediction are based on convolutional networks [6,31,34,42,49,50,53].”
§ Vision Transformers for Dense Prediction > 1. Introduction
What the sources say
- Partly supportedcites [6] 6. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. DeepLab: Semantic im- age segmentation with deep convolutional nets, atrous con- volution, and fully connected crfs. TPAMI, 40(4):834-848, 2018. sourceExperimental — accuracy not yet measured
The Introduction (p. 1) says convolutional models such as PixelCNN had received much more attention for pixel-by-pixel image generation. It does not establish that virtually all architectures across the broader category of dense prediction are convolutional; it also discusses recurrent models and proposes a self-attention alternative (Introduction, p. 2). (LLM, conf=0.78)
- Not checked — no full text of the sourcecites [31] 31. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional networks for biomedical image segmen- tation. In MICCAI, 2015. source
- Supportedcites [34] 34. Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional networks for semantic segmentation. CVPR, 2015. source
The paper is an example of a convolutional-network architecture for dense prediction: its Abstract (p. 1) describes predicting a depth map, and §3.1.2 (p. 3) identifies its convolutional layers and target-depth output. As one of several citations, it supports the sentence by exemplifying such an architecture, rather than establishing the prevalence claim on its own. (LLM, conf=0.88)
- Could not verifycites [42] 42. Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recogni- tion. TPAMI, 2020. sourceExperimental — accuracy not yet measured
Section 2.1 (p. 8) describes transformer language-model architectures, not architectures for dense prediction. Searches for dense prediction, convolutional networks, and pixel-wise segmentation found no passage addressing whether dense-prediction architectures are predominantly convolutional. (LLM, conf=0.78)
- Not matchedcites [49] 49. Fisher Yu and Vladlen Koltun. Multi-scale context aggrega- tion by dilated convolutions. In ICLR, 2016.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [50] 50. Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object- contextual representations for semantic segmentation. In ECCV, 2020. source
- Not checked — no full text of the sourcecites [53] 53. Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We notice that the biggest gains in performance for zero-shot transfer were achieved for datasets that feature dense, high-resolution evaluations [15,55,59].”
§ Vision Transformers for Dense Prediction > Additional information
What the sources say
- Not checked — no full text of the sourcecites [15] 15. Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? The KITTI vision benchmark suite. In CVPR, 2012. source
- Supportedcites [55] 55. D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012. source
The citation identifies MPI-Sintel as one of the datasets in the citing authors’ comparison; the performance gains are the citing authors’ observation, not a result attributed to this paper. Design Decisions and Comparison to Middlebury (p. 5) describes Sintel’s higher-resolution frames, and The Sintel Data Set (p. 9) describes its flow fields and pixel-level error evaluation. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [59] 59. Thomas Schöps, Johannes L. Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and An- dreas Geiger. A multi-view stereo benchmark with high- resolution images and multi-camera videos. In CVPR, 2017. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Note that ResNext101-WSL was pretrained on a billion-scale corpus of weakly supervised data [25] in addition to ImageNet pretraining.”
§ Vision Transformers for Dense Prediction > Skip connections.
What the sources say
- Partly supportedcites [25] 25. Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. In ECCV, 2018. sourceExperimental — accuracy not yet measured
Section 2.3 (p. 4) identifies the models as ResNeXt-101, and Section A.5 (p. 23) says they were pretrained on billions of Instagram images using hashtags as labels. But it describes subsequent training on ImageNet as fine-tuning, not additional ImageNet pretraining; Section 3.2 (p. 12) also distinguishes Instagram-pretrained models from ImageNet-pretrained baselines. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While ignoring the token yields good performance, projection provides slightly better performance on average.”
§ Vision Transformers for Dense Prediction > Skip connections.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Adding the token, on the other hand, yields worse performance than simply ignoring it.”
§ Vision Transformers for Dense Prediction > Skip connections.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both DPT variants significantly outperform the state of the art.”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that both architectures have similar latency to MiDaS (Table 9).”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ViT-Large outperforms all other backbones but is also almost three times larger than ViT-Base and ViT-Hybrid.”
§ Vision Transformers for Dense Prediction > Skip connections.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We again see that DPT can provide strong performance even on smaller datasets.”
§ Vision Transformers for Dense Prediction > ADE20K.
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 13
- The quote only partly states this claim. 1
Every source the paper cites for this claim was checked and supports it.
In the paper
“In particular, it has recently been demonstrated that a direct application of token-based transformer architectures that have been successful in NLP can yield competitive performance on image classification [11].”
§ Vision Transformers for Dense Prediction > 2. Related Work
What the sources say
- Supportedcites [11] 11. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. source
The Introduction (p. 1) says image patches are treated like NLP tokens and fed to a standard Transformer. The Conclusion (p. 9) reports that this approach matches or exceeds the state of the art on many image-classification datasets when coupled with pre-training on large datasets. (LLM, conf=0.93)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“A key insight of this work was that, like transformer models in NLP, vision transformers need to be paired with a sufficient amount of training data to realize their potential.”
§ Vision Transformers for Dense Prediction > 2. Related Work
What the sources say
- Supportedcites [11] 11. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. source
The Introduction (p. 1) says ViT performs modestly on mid-sized datasets and does not generalize well with insufficient data, but attains excellent results when pre-trained at sufficient scale. Section 4.3 (p. 6) reports that larger ViT models show their full benefit only with the largest pre-training dataset tested. The Conclusion (p. 9) also describes the approach as working well when coupled with pre-training on large datasets. (LLM, conf=0.94)
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly, the ViT-Hybrid architecture extracts features at 1 16 the input resolution, which is twice as high as the lowest-resolution features that are commonly used with convolutional backbones.”
§ Vision Transformers for Dense Prediction > 3. Architecture
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It contains about 1.4 million images and is, to the best of our knowledge, the largest training set for monocular depth estimation that has ever been compiled.”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When applied to semantic segmentation, dense vision transformers set a new state of the art on ADE20K with 49.02% mIoU.”
§ Vision Transformers for Dense Prediction > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that the DPT tends to produce cleaner and finer-grained delineations of object boundaries and that the predictions are also in some cases less cluttered.”
§ Vision Transformers for Dense Prediction > ADE20K.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DPT can better reconstruct fine details while also improving global coherence in areas that are challenging for the convolutional architecture (for example, large homogeneous regions or relative depth arrangement across the image).”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average relative improvement over the best published architecture, MiDaS, is more than 23% for DPT-Hybrid and 28% for DPT-Large.”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DPT-Hybrid achieves this with a comparable network capacity (Table 9), while DPT-Large is about 3 times larger than MiDaS.”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe the tendency that attention is spatially more localized close to the reference token in shallow layers (leftmost columns), whereas deeper layers (rightmost columns) frequently attend across the whole image.”
§ Vision Transformers for Dense Prediction > Additional information
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ViT-Base has comparable performance to ResNext101-WSL, while ViT-Hybrid and ViT-Large improve performance even though they have been pretrained on significantly less data.”
§ Vision Transformers for Dense Prediction > Skip connections.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“While we observe a general trend of an improvement in per-class IoU in comparison to the baseline [51], we do not observe a strong pattern across classes.”
§ Vision Transformers for Dense Prediction > Additional information
What the sources say
- Not checked — no full text of the sourcecites [51] 51. Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Muller, R. Manmatha, Mu Li, and Alexander Smola. ResNeSt: Split- attention networks. arXiv preprint arXiv:2004.08955, 2020.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“While we observe a general trend of an improvement in per-class IoU in comparison to the baseline [51], we do not observe a strong pattern across classes.”
§ Vision Transformers for Dense Prediction > Additional information
What the sources say
- Not checked — no full text of the sourcecites [51] 51. Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Muller, R. Manmatha, Mu Li, and Alexander Smola. ResNeSt: Split- attention networks. arXiv preprint arXiv:2004.08955, 2020.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe more details and also better global depth arrangement in DPT predictions when compared to the fullyconvolutional baseline.”
§ Vision Transformers for Dense Prediction > Additional information
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The transformer backbone processes representations at a constant and relatively high resolution and has a global receptive field at every stage.”
§ Vision Transformers for Dense Prediction > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As long as the image size is divisible by p, the embedding procedure can be applied and will produce a varying number of image tokens N p .”
§ Vision Transformers for Dense Prediction > 3. Architecture
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike fully-convolutional networks, the vision transformer backbone foregoes explicit downsampling operations after an initial image embedding has been computed and maintains a representation with constant dimensionality throughout all processing stages.”
§ Vision Transformers for Dense Prediction > 1. Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since transformers are known to realize their full potential only when an abundance of training data is available, monocular depth estimation is an ideal task to test the capabilities of DPT.”
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is in stark contrast to convolutional networks, which progressively increase their receptive field as features pass through consecutive convolution and downsampling layers.”
§ Vision Transformers for Dense Prediction > 3. Architecture
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DPT-Large performs slightly worse, likely because of the significantly smaller dataset compared to our previous experiments.”
§ Vision Transformers for Dense Prediction > ADE20K.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“DeIT trains the ViT architecture with a more data-efficient pretraining procedure.”
§ Vision Transformers for Dense Prediction > Skip connections.
What the sources say
- Not checked — no full text of the sourcecites [38] 38. Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through at- tention. arXiv preprint arXiv:2012.12877, 2020. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Note that the DeIT-Base architecture is identical to ViT-Base, while DeIT-Base-Dist introduces an additional distillation token, which we ignore in the Reassemble operation.”
§ Vision Transformers for Dense Prediction > Skip connections.
What the sources say
- Not checked — no full text of the sourcecites [38] 38. Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through at- tention. arXiv preprint arXiv:2012.12877, 2020. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similar to prior work on transformers, DPT unfolds its full potential when trained on large-scale datasets.”
§ Vision Transformers for Dense Prediction > 5. Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that results for DPT and Mi-DaS are computed at the same input resolution (384 pixels).”
§ Vision Transformers for Dense Prediction > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The choice of backbone architecture has a large influence on the capabilities of the overall model, because information lost in the encoder cannot be recovered in the decoder.
“However, it is widely recognized that the choice of backbone architecture has a large influence on the capabilities of the overall model, as any information that is lost in the encoder is impossible to recover in the decoder.”
The paper relies on this without support.
Feature resolution and granularity are critical for dense prediction, where the architecture should ideally resolve features at or close to the input resolution.
“While feature resolution and granularity may not matter for some tasks, such as image classification, they are critical for dense prediction, where the architecture should ideally be able to resolve features at or close to the resolution of the input image.”
The paper relies on this without support.
Convolutions are linear operators with a limited receptive field.
“Convolutions, by definition, are linear operators that have a limited receptive field.”
The paper relies on this without support.
Downsampling intermediate representations is necessary to keep memory consumption at levels feasible with existing computer architectures.
“Downsampling the intermediate representations is necessary to keep memory consumption at levels that are feasible with existing computer architectures.”
The paper relies on this without support.
The transformer backbone maintains a constant feature resolution.
“Since the transformer backbone maintains a constant feature resolution, it is not clear at which points in the backbone features should be tapped.”
The paper relies on this without support.
Performance of fully-convolutional architectures suffers heavily when performing inference at an input resolution significantly different from the training resolution.
“Performance thus suffers heavily when performing inference at an input resolution that is significantly different from the training resolution.”
The paper relies on this without support.
Transformer encoders have a global receptive field in every layer.
“Transformer encoders, on the other hand, have a global receptive field in every layer.”
The paper relies on this without support.
The reduced meta-dataset datasets provide high-quality ground truth.
“We choose these datasets since they provide high-quality ground truth.”
The paper relies on this without support.
Convolutional architectures offer natural points of interest for passing features from the encoder to the decoder, namely before or after downsampling of the representation.
“Convolutional architectures offer natural points of interest for passing features from the encoder to the decoder, namely before or after downsampling of the representation.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Vision transformers, which are a deep learning approach to encoding low-dimensional input into a high dimensional feature space, have established new frontiers in depth estimation compared to convolutional neural networks (Ranftl et al., 2021).”
From Very high resolution canopy height maps from RGB imagery using self-supervised vision transformer and convolutional decoder trained on aerial lidar (2024) · cites this paper as Ranftl et al., 2021
In the citing paper
“However, the Transformer framework does not involve dimensionality reduction10,12, thus keeping all attention layer traceable to the original input features13, thus making the models interpretable.”
From Transformer for one stop interpretable cell type annotation (2023) · cites this paper as [10]
In the citing paper
“For example, the change of dimensions and non-linear aggregation of features throughout the autoencoder’s deep processing stages leads to untraceable and uninterpretable latent space and loss of information and feature resolution9,10.”
From Transformer for one stop interpretable cell type annotation (2023) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Vision Transformers for Dense Prediction > 4.1. Monocular Depth Estimation
“Our architecture matches or improves state-of-the-art performance on both datasets in all metrics.”
§ Vision Transformers for Dense Prediction > 5. Conclusion
“| BTS [20] | 0.956 | 0.993 | 0.998 | 0.059 2.756 | 0.096 | | DPT-Hybrid 0.959 | 0.995 | 0.999 | 0.062 2.573 | 0.092 |”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 4×
“We perform a similar experiment with the hybrid architecture in Table 6 (bottom), where R0 and R1 refer to us-ing features from the first and second downsampling stages of the ResNet50 embedding network.”
MethodInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · cited 3×
“These include training at higher input resolution (if the computational budget permits), dilated convolutions [51] to rapidly increase the receptive field without downsampling, appropriately-placed skip connections from multiple stages of the encoder to the decoder [33], or, more recently, by connecting multiresolutio…”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“The encoder is frequently based on an image classification network, also called the backbone, that is pretrained on a large corpus such as ImageNet [9].”
InfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“Following work in NLP, the ViT additionally adds a special token that is not grounded in the input image and serves as the final, global image representation which is used for classification.”
BackgroundMethodInfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 8×
“On a high level, the vision transformer (ViT) [11] operates on a bag-of-words representation of the image [38].”
BackgroundMethodResultInfluentialFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · cited 2×
“Virtually all existing architectures for dense prediction are based on convolutional networks [6, 33, 36, 44, 51, 52, 55].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Research works printed but not linked: 3
- yuan, 2020, Objectcontextual representations for semantic segmentation, ECCV
- godard, 2017, Unsupervised monocular depth estimation with leftright consistency, CVPR
- casser, 2019, Unsupervised learning of depth and ego-motion: A structured approach, AAAI
Cited by
Sorted by citations
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs
Ding, X., Zhang, X., Han, J., & Ding, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01166 · Cited by 1,592
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00987 · Cited by 1,296
“Following MiDaS [5, 45], we use the DPT [46] decoder for depth regression.”
MethodDINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078
“5, the segmentation training in this experiment took 28 hours on 16 V100 GPUs.”
MethodLarge Selective Kernel Network for Remote Sensing Object Detection
Li, Y., Hou, Q., Zheng, Z., Cheng, M., Yang, J. J., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01540 · Cited by 1,022
“Research [17,45,55, 72, 85] has demonstrated that the large receptive field is a key factor in their success.”
BackgroundZero-1-to-3: Zero-shot One Image to 3D Object
Liu, R., Wu, R., Hoorick, B. V., Tokmakov, P., Zakharov, S., & Vondrick, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00853 · Cited by 1,130
“Since MCC requires depth input, we use MiDaS [42, 41] off-the-shelf for depth estimation.”
MethodEgo4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630
“[185] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.”
BackgroundA survey of the vision transformers and their CNN-transformer based variants
Khan, A., Rauf, Z., Sohail, A., Khan, A. R., Asif, H., Asif, A., & Farooq, U. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10595-0 · Cited by 533
Tolan, J., Yang, H.-I., Nosarzewski, B., Couairon, G., Vo, H. V., Brandt, J., Spore, J., Majumdar, S., Haziza, D., Vamaraju, J., et al. (2024) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113888 · Cited by 419
TTST: A Top- k Token Selective Transformer for Remote Sensing Image Super-Resolution
Xiao, Y., Yuan, Q., Jiang, K., He, J., Lin, C., & Zhang, L. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3349004 · Cited by 432
“Recent studies [55], [56], [57] have demonstrated the effectiveness of large kernel convolutions in improving the performance of vision tasks.”
BackgroundRepurposing Diffusion-Based Image Generators for Monocular Depth Estimation
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R. C., & Konrad Schindler, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00907 · Cited by 595
“The step from CNNs to vision transformers has further boosted performance, as evidenced by DPT [36] and Omnidata [13].”
InfluentialSwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection
Liu, Z., Tan, Y., He, Q., & Xiao, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3127149 · Cited by 388
“DPT [69] assembles tokens from multiple stages of the vision transformer and progressively combines them into full-resolution predictions using a convolutional decoder.”
MethodTang, L., Zhang, H., Xu, H., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101870 · Cited by 418
Zhang, N., Nex, F., George Vosselman, A., & Kerle, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01778 · Cited by 442
“With the rise of vision transformer (ViT) [8] recent work applied it to various computer vision tasks [4, 16, 29, 31, 34], and achieved promising results.”
MethodDaViT: Dual Attention Vision Transformers
Ding, M., Xiao, B., Codella, N., Luo, P., Wang, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_5 · Cited by 465
Neural Window Fully-connected CRFs for Monocular Depth Estimation
Yuan, W., Gu, X., Dai, Z., Zhu, S., & Tan, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00389 · Cited by 485
“Since the state-of-theart performance on NYUv2 dataset has been saturated for a while, some methods have begun to use additional data to pretrain the model and then finetune it on NYUv2 training set [10, 28].”
BackgroundMethodMulti-class Token Transformer for Weakly Supervised Semantic Segmentation
Xu, L., Ouyang, W., Bennamoun, M., Boussaïd, F., & Xu, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00427 · Cited by 331
“…to model long-range dependencies of long sequences in the field of NLP. Recently, transformer models have been adapted to accommodate a wide variety of vision tasks [18], such as image classification [10], saliency detection [27] and semantic segmentation [30], achieving promising performances.”
BackgroundSegGPT: Towards Segmenting Everything In Context
Wang, X., Zhang, X., Cao, Y., Wang, W., Shen, C., & Huang, T. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00110 · Cited by 310
“…witnessed great progress in developing more accurate and faster algorithms for various segmentation tasks, such as foreground segmentation [45], interactive segmentation [55, 38], semantic segmentation [36, 32, 58, 43], instance segmentation [20, 13, 4, 52], and panoptic segmentation [26, 7, 10].”
MethodSemantic segmentation using Vision Transformers: A survey
Thisanke, H., Deshan, C., Chamith, K., Seneviratne, S., Vidanaarachchi, R., & Herath, D. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106669 · Cited by 288
Grounding Image Matching in 3D with MASt3R
Leroy, V., Cabon, Y., & Revaud, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73220-1_5 · Cited by 323
Ru, L., Zhan, Y., Yu, B., & Du, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01634 · Cited by 306
“Later variants show ViT also benefits downstream vision tasks, such as semantic segmentation [49, 9, 58], depth estimation [33], and video understanding [5].”
BackgroundPixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
Charatan, D., Li, S. L., Tagliasacchi, A., & Sitzmann, V. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01840 · Cited by 420
“In monocular depth estimation, state-of-the-art models rely on sophisticated scale-invariant depth losses [11, 13, 33, 34].”
BackgroundMonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer
Zhao, C., Zhang, Y., Poggi, M., Tosi, F., Guo, X., Zhu, Z. D., Huang, G., Tang, Y. S., & Mattoccia, S. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00077 · Cited by 330
“The popularity of ViTs has also reached supervised depth estimation as well [38, 26], yet being not adopted for self-supervised monocular depth estimation.”
BackgroundTransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers
Ding, Y., Yuan, W., Zhu, Q., Zhang, H., Liu, X., Wang, Y., & Liu, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00839 · Cited by 287
“Transformer [25] has been widely used in natural language processing due to its effectiveness and efficiency, and has drawn increasing attention from the computer vision community recently [2, 8, 17, 20, 21].”
MethodAdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01199 · Cited by 262
“Given an input image, AdaViT learns to adaptively derive policies on which patches , self-attention heads and transformer blocks to use or activate in the transformer backbone conditioned on the input image, encouraging us-ing less computation while maintaining the classification accuracy.”
MethodStripformer: Strip Transformer for Fast Image Deblurring
Tsai, F.-J., Peng, Y., Lin, Y., Tsai, C.-C., & Lin, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19800-7_9 · Cited by 300
Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans
Eftekhar, A., (Sasha), A. S., Malik, J., & Zamir, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01061 · Cited by 289
“Sometimes, MTL papers take mix datasets for a “single” task and consider each dataset as a different task [37, 47, 35, 46].”
BackgroundMethodInfluentialHu, M., Yin, W., Zhang, C., Cai, Z., Long, X., Chen, H., Wang, K., Yu, G., Shen, C., & Shen, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3444912 · Cited by 423
“Existing depth estimation methods are categorized into learning metric depth [17], [18], [19], [20], relative depth [21], [22], [23], [24], and affine-invariant depth [25], [26], [27], [28], [29].”
InfluentialSparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
Wang, G., Chen, Z., Loy, C. C., & Liu, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00832 · Cited by 321
“On the other hand, we use pre-trained depth DPT [41] to estimate the depth of I and crop a local patch d dpt with the same spatial location as d r .”
MethodInfluentialCTNet: Context-Based Tandem Network for Semantic Segmentation
Li, Z., Sun, Y., Zhang, L., & Tang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132068 · Cited by 273
“Recently, some methods [41], [42], [43], [44] apply the transformer idea to the backbone network for improving the network semantic space and the segmentation performance.”
MethodMultiMAE: Multi-modal Multi-task Masked Autoencoders
Bachmann, R., Mizrahi, D., Atanov, A., & Zamir, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_20 · Cited by 268
ColonFormer: An Efficient Transformer Based Method for Colon Polyp Segmentation
Duc, N. T., Nguyen, T.-O., Thủy, N. T. T., Tran, M., & Sang, D. V. (2022) · IEEE Access · doi:10.1109/access.2022.3195241 · Cited by 254
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds
He, C., Li, R., Li, S., & Zhang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00823 · Cited by 257
“Recently, Transformer [38] has demonstrated its great success in many computer vision tasks such as image classification [6, 37], 2D object detection [1, 19], and other dense prediction tasks [30, 48].”
BackgroundNoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior
Bian, W., Wang, Z., Li, K., & Bian, J.-W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00405 · Cited by 247
“To handle the limitation of large camera motion, we seek help from monocular depth estimation [22, 28, 29, 51].”
MethodXCiT: Cross-Covariance Image TransformersPreprint
El-Nouby, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., Verbeek, J. J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09681 · Cited by 316
“In addition, Zheng et al. [83] and Ranftl et al. [54] study problems like semantic segmentation and monocular depth estimation with the quadratic self-attention operation.”
BackgroundLi, Z., Chen, Z., Liu, X., & Jiang, J. (2023) · Machine Intelligence Research · doi:10.1007/s11633-023-1458-0 · Cited by 280
Nascimento, E. G. S., Melo, T. A. C., & Moreira, D. M. (2023) · Energy · doi:10.1016/j.energy.2023.127678 · Cited by 223
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
Yin, W., Zhang, C., Chen, H., Cai, Z., Yu, G., Wang, K., Chen, X., & Shen, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00830 · Cited by 305
Foundation models in robotics: Applications, challenges, and the future
Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y., Song, S., Kapoor, A., Hausman, K., et al. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241281508 · Cited by 238
“NeRF creates high-quality 3D reconstructions of scenes and objects from a set of 2D images captured from different viewpoints (without the need for explicit depth information).”
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields
Zhou, S., Chang, H., Jiang, S., Fan, Z., Zhu, Z., Xu, D., Chari, P., You, S. O., Wang, Z., & Kadambi, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02048 · Cited by 317
“Utilizing the image feature encoder with the DPT architecture [33] and text encoders from CLIP [32], LSeg extends text-image associations to a 2D pixel-level granularity.”
MethodGaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
Chen, Y., Chen, Z., Zhang, C., Wang, F., Yang, X., Wang, Y., Cai, Z., Yang, L., Liu, H., & Lin, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02029 · Cited by 242
“For aligning the coordinate system of θ y with θ , the depth of I My is first estimated using the technique from [38].”
Muhammad, K., Hussain, T., Ullah, H., Ser, J. D., Rezaei, M., Kumar, N., Hijji, M., Bellavista, P., & de Albuquerque, V. H. C. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3207665 · Cited by 232
“For instance, Ranftl et al. [162] establish an unprecedented state-of-the-art level of performance by introducing vision transformers in a semantic segmentation domain.”
BackgroundTransformer for one stop interpretable cell type annotation
Chen, J., Xu, H., Tao, W., Chen, Z., Zhao, Y., & Han, J.-D. J. (2023) · Nature Communications · doi:10.1038/s41467-023-35923-4 · Cited by 189
Cross-Modal Transformers for Infrared and Visible Image Fusion
Park, S. H., Vien, A. G., & Lee, C. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3289170 · Cited by 219
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization
Li, J., Zhang, J., Bai, X. P., Zheng, J. Y., Ning, X., Zhou, J. W., & Gu, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01963 · Cited by 227
“The first group [11, 31] is to extract accurate but sparse depth values from reliable point clouds, and the second [14, 36, 40, 42, 55] distills depth knowledge from current powerful monocular depth estimators [29, 30].”
BackgroundMethodFSGS: Real-Time Few-Shot View Synthesis Using Gaussian Splatting
Zhu, Z., Fan, Z., Jiang, Y., & Wang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72933-1_9 · Cited by 303
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
Tang, J., Wang, T., Zhang, B., Zhang, T., Yi, R., Ma, L., & Chen, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02086 · Cited by 308
“To be specific, we utilize an off-the-shelf single-view depth estimator [35] to estimate the depth d for the input image.”
MethodUniDepth: Universal Monocular Metric Depth Estimation
Piccinelli, L., Yang, Y.-H., Sakaridis, C., Segù, M., Li, S., Gool, L. V., & Yu, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00963 · Cited by 207
“While existing MMDE methods [2, 13, 15, 33, 34, 36, 50] have demonstrated remarkable accuracy across different benchmarks, they require training and testing on datasets with similar camera intrinsics and scene scales.”
MethodCrackFormer Network for Pavement Crack Segmentation
Liu, H., Yang, J., Miao, X., Mertz, C., & Kong, H. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3266776 · Cited by 201
“DPT [41], [43] assembles tokens from various stages of the vision transformer into image-like representations at various resolutions and progressively combine them into full-resolution predictions using a convolutional decoder, and uses transformer backbone to process representations at a constant and relatively high…”
BackgroundLight Field Image Super-Resolution With Transformers
Liang, Z., Wang, Y., Wang, L., Yang, J., & Zhou, S. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3146798 · Cited by 191
“Recently, Transformers have been demonstrated effective in modeling positional and long-range correlations, and were applied to various computer vision tasks such as image classification [19], object detection [20], [21], semantic segmentation [22], depth estimation [23] and super-resolution [24], [25].”
MethodDuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation
Tang, F., Xu, Z., Huang, Q., Wang, J., Hou, X., Su, J., & Liu, J. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8469-5_27 · Cited by 246
LSKNet: A Foundation Lightweight Backbone for Remote Sensing
Li, Y., Li, X., Dai, Y., Hou, Q., Liu, L., Liu, Y., Cheng, M., & Yang, J. Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02247-9 · Cited by 237
“Research [55–59] has demonstrated that the large receptive field is a key factor in their success.”
FCN-Transformer Feature Fusion for Polyp Segmentation
Sanderson, E., & Matuszewski, B. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-12053-4_65 · Cited by 237
“However, transformer-based architectures [24,36,33,34,32] have recently become popular for semantic segmentation and shown superior performance over FCN-based alternatives.”
BackgroundMethodAttention Attention Everywhere: Monocular Depth Prediction with Skip Attention
Agarwal, A., & Arora, C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00581 · Cited by 193
“Following the test protocol of [24, 9], and without additional training data, our method improves the Absolute relative error by 5.”
MethodUnleashing Text-to-Image Diffusion Models for Visual Perception
Zhao, W., Rao, Y., Liu, Z., Liu, B., Zhou, J., & Lu, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00527 · Cited by 249
“The compared methods include self-supervised pre-training (MAE [17]) and supervised pre-training (Swin [30] and ConvNeXt [31]).”
MethodP3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior
Patil, V., Sakaridis, C., Liniger, A., & Gool, L. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00166 · Cited by 177
Low-Light Image Enhancement via Structure Modeling and Guidance
Xu, X., Wang, R., & Lu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00954 · Cited by 177
“We adopt the DPT model [35], which is trained on ADE20K [72] and MIX 6 [35], to extract the normallight data’s segmentation maps and depth maps.”
MethodVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
GA-Nav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments
Guan, T., Kothandaraman, D., Chandra, R., et al. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3187278 · Cited by 155
“…self-attention is popular in neural network design due to its superior performance over prior state-of-the-art methods in segmentation [15], [11], image classification [32], and object detection [28], [33], [34], but the quadratic nature of its computational complexity has been an issue for…”
BackgroundMobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110
“For instance, a ViT-based segmentation network (Ranftl et al., 2021) learns about 345 million parameters and achieves similar performance as the CNN-based network, DeepLabv3 (Chen et al., 2017), with 59 million parameters.”
MethodDDP: Diffusion Model for Dense Visual Prediction
Ji, Y., Chen, Z., Xie, E., Hong, L., Liu, X., Liu, Z., Lü, T., Li, Z., & Luo, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01987 · Cited by 201
“Numerous methods have emerged and achieved tremendous progress, and these advances can be roughly divided to: multi-scale feature aggregation [8, 9, 76], high-capacity backbone [78, 89, 54] and powerful decoder head [69, 87, 18, 36].”
CompletionFormer: Depth Completion with Convolutions and Vision Transformers
Zhang, Y., Guo, X., Poggi, M., Zhu, Z. D., Huang, G., & Mattoccia, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01777 · Cited by 158
“Tasks related to 3D vision have also benefited from the enriched modeling capability of Transformer, such as stereo matching [13,15], supervised [14,30] and unsupervised monocular depth estimation [48], optical flow [10,34] and also depth completion [31].”
BackgroundBinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation
Li, Z., Wang, X., Liu, X., & Jiang, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3416065 · Cited by 301
“So far, there have been numerous mainstream supervised methods formulating MDE as per-pixel regression, such as DAV [23], DPT [24] and TransDepth [25] (Fig.”
MethodInfluential3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification
Lu, D., Xie, Q., Gao, K., Xu, L., & Li, J. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3198836 · Cited by 129
“There also exist several vision Transformer variants [30], [31], [32], [33] which explored ways to better model local features, such as replacing the predefined positional embedding or constructing connections between multiple tokens.”
BackgroundSource-free Depth for Object Pop-out
Wu, Z., Paudel, D. P., Fan, D.-P., Wang, J., Wang, S., Demonceaux, C., Timofte, R., & Gool, L. V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00101 · Cited by 157
“We choose the state-of-the-art DPT model [50] with frozen weights as our source-free depth network, which offers us promising depth at the target.”
BackgroundMethodDeep Digging into the Generalization of Self-Supervised Monocular Depth Estimation
Bae, J., Moon, S., & Im, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25090 · Cited by 142
“We additionally use a special token t s following the work (Ranftl, Bochkovskiy, and Koltun 2021).”
MethodNeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors
Deng, C., Jiang, C. M., Qi, C. R., Yan, X., Zhou, Y., Guibas, L. J., & Anguelov, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01977 · Cited by 144
Rethinking Low-Light Enhancement via Transformer-GAN
Yang, S., Zhou, D., Cao, J., & Guo, Y. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3167331 · Cited by 119
Revealing the Dark Secrets of Masked Image Modeling
Xie, Z., Geng, Z., Hu, J., Zhang, Z., Hu, H., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01391 · Cited by 151
“To compare with the previous approaches [64, 38], we set the maximum range as 10m for NYUv2 and 80m for KITTI.”
MethodResultSajjadi, M. S. M., Meyer, H., Pot, E., Bergmann, U., Greff, K., Radwan, N., Vora, S., Lucic, M., Duckworth, D., Dosovitskiy, A., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00613 · Cited by 143
“and video processing, where they have meanwhile achieved state of the art in several tasks [18], including inherently geometric applications such as depth estimation [29], point cloud processing [51], and generative geometry-free view synthesis [32].”
BackgroundiDisc: Internal Discretization for Monocular Depth Estimation
Piccinelli, L., Sakaridis, C., & Yu, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02057 · Cited by 144
“State-of-the-art (SotA) methods typically involve convolutional networks [12, 13, 24] or, since the advent of vision Transformer [11], transformer architectures [3, 41, 54, 59].”
BackgroundUSOD10K: A New Benchmark Dataset for Underwater Salient Object Detection
Hong, L., Wang, X., Zhang, G., & Zhao, M. L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3266163 · Cited by 131
“Cross-Modality Fusion The estimated depth maps of underwater images generated by DPT [19] cannot always be guaranteed high quality.”
InfluentialMonocular Depth Estimation: A Thorough Review
Arampatzakis, V., Pavlidis, G. P., Mitianoudis, N., & Papamarkos, N. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330944 · Cited by 135
“[39]whoproposedamodelconnecting12datasets:ReD-Web[40],DIML[41],Movies[42],MegaDepth,WSVD[43], TartanAir [44], HR-WSI [45], ApolloSpace [46], BlendedMVS, IRS [47], KITTI, and NYU-v2.”
BackgroundMethodInfluentialLocalBins: Improving Depth Estimation by Learning Local Distributions
Bhat, S. F., Alhashim, I., & Wonka, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19769-7_28 · Cited by 141
CDTrans: Cross-domain Transformer for Unsupervised Domain AdaptationPreprint
Xu, T., Chen, W., Wang, P., Wang, F., Li, H., & Jin, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06165 · Cited by 220
“…for transformers to help with ViT training; many other ViT variants (Yuan et al., 2021a; Wang et al., 2021; Han et al., 2021a; Chen et al., 2021a; Ranftl et al., 2021; Liu et al., 2021) are proposed from then, which achieve promising performance compared with its counterpart CNNs for both image…”
BackgroundEnriched CNN-Transformer Feature Aggregation Networks for Super-Resolution
Yoo, J., Kim, T., Lee, S., Kim, S. H., Lee, H., & Kim, T. H. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00493 · Cited by 120
“Since then, many follow-up studies have been introduced for various vision tasks, such as image classification [53, 61], object detection [4, 67], and dense prediction [44, 62].”
BackgroundA CNN-Transformer Hybrid Model Based on CSWin Transformer for UAV Image Object Detection
Lu, W., Lan, C., Niu, C., Liu, W., Lyu, L., Shi, Q., & Wang, S. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3234161 · Cited by 113
“Ranftl et al. [62] constructed a dense prediction transformer to perform dense object predictions by upsampling the low-resolution images to obtain high-resolution images.”
BackgroundRSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution
Geng, Z., Liang, L., Ding, T., & Zharkov, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01692 · Cited by 121
“Due to transformer’s strong ability of learning long dependencies between different image regions, follow-up work using variants of ViT refreshes the stateof-the-art results in many applications, such as segmentation [41, 62], object detection [58, 56] and depth estimation [35].”
MethodIs it Time to Replace CNNs with Transformers for Medical Images?Preprint
Matsoukas, C., Haslum, J. F., Söderberg, M., & Smith, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09038 · Cited by 216
“The use of vision transformers in the natural imaging domain has exploded recently, with applications ranging from classification [11], to object detection [3] and segmentation [21].”
MethodViTs for SITS: Vision Transformers for Satellite Image Time Series
Tarasiou, M., Chavez, E., & Zafeiriou, S. P. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01004 · Cited by 150
“For image recognition, [32, 69] re-introduce some of the inductive biases that made CNNs successful in vision, leading to improved performances without the need for long pre-training schedules, [43, 59] employ Transformers for dense prediction, [11,58,71] for object detection and [3,36,68] for video processing.”
BackgroundLLMFormer: Large Language Model for Open-Vocabulary Semantic Segmentation
Shi, H., Dao, S. D., & Cai, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02171-y · Cited by 111
CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow
Weinzaepfel, P., Lucas, T., Leroy, V., Cabon, Y., Arora, V., Brégier, R., Csurka, G., Antsfeld, L., Chidlovskii, B., & Revaud, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01647 · Cited by 110
“We finetune our pre-trained model, referred to as CroCo v2, with this improved cross-view completion scheme on stereo matching and optical flow using a Dense Prediction Transformer (DPT) [59] head.”
MethodSequential Modeling Enables Scalable Learning for Large Vision Models
Bai, Y., Geng, X., Mangalam, K., Bar, A., Yuille, A., Darrell, T., Malik, J., & Efros, A. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02157 · Cited by 192
“Some data types, e.g. semantic segmentation maps [100], edge maps [79], depth [66] and normal images [6], are already represented this way.”
BackgroundSC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic Scenes
Sun, L., Bian, J.-W., Zhan, H., Yin, W., Reid, I. D., & Shen, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3322549 · Cited by 119
“Table IX shows the ablation study results on DDAD dataset, where we also include DPT [44] and ResNet-50 version of LeReS.”
BackgroundMethodMulti-Frame Self-Supervised Depth with Transformers
Guizilini, V., Ambruş, R., Chen, D., Zakharov, S., & Gaidon, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00026 · Cited by 108
“Similarly, [39] uses patch-wise attention over convolutional features, and [50] eliminates convolutional encoding by proposing a fully attention-based backbone.”
MethodDist-YOLO: Fast Object Detection with Distance Estimation
Vajgl, M., Hurtík, P., & Nejezchleba, T. (2022) · Applied Sciences · doi:10.3390/app12031354 · Cited by 104
“The depth map can be entirely independent between distinct images [8] or preserve the consistency [9].”
UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation
Yang, L., Zhao, Z. W., & Zhao, H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3528453 · Cited by 132
“Formally, suppose a semantic segmentation model f is composed of an encoder g (e.g., ResNet [51], DINOv2 [47]) and a decoder h (e.g., ASPP [3], DPT [108]).”
TPRNet: camouflaged object detection via transformer-induced progressive refinement network
Zhang, Q., Ge, Y., Zhang, C., & Bi, H. (2022) · The Visual Computer · doi:10.1007/s00371-022-02611-1 · Cited by 96
“DPT [36] assembles tokens from multiple stages of the vision transformer and progressively combines them into full-resolution predictions using a convolutional decoder.”
MethodVision Transformer for NeRF-Based View Synthesis from a Single Input Image
Lin, K.-E., Lin, Y.-C., Lai, W., Lin, T.-Y., Shih, Y., & Ramamoorthi, R. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00087 · Cited by 120
“Variants based on self-attention have achieved stateof-the-art performance in many NLP [3, 9] and vision [10, 36, 44] tasks.”
BackgroundMethodInfluentialDeep multi-scale pyramidal features network for supervised video summarization
Khan, H., Hussain, T., Khan, S. U., et al. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121288 · Cited by 94
NeuralFeels with neural fields: Visuotactile perception for in-hand manipulation
Suresh, S., Qi, H., Wu, T., Fan, T., Pineda, L., Lambeta, M., Malik, J., Kalakrishnan, M., Calandra, R., Kaess, M., et al. (2024) · Science Robotics · doi:10.1126/scirobotics.adl0628 · Cited by 98
Towards Zero-Shot Scale-Aware Monocular Depth Estimation
Guizilini, V., Vasiljevic, I., Chen, D., Ambruş, R., & Gaidon, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00847 · Cited by 100
“Differently from traditional architectures for monocular depth estimation [19, 23, 46, 37], ZeroDepth follows [28] and decouples the encoding and decoding stages, which enables the decoding of estimates from embeddings that were not encoded.”
BackgroundMethodFayad, I., Ciais, P., Schwartz, M., Wigneron, J., Baghdadi, N. N., Truchis, A. D., d’Aspremont, A., Frappart, F., Saatchi, S. S., Sean, E., et al. (2023) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113945 · Cited by 101
“The decoder layers are inspired by the decoder layers presented in the study of Ranftl et al., 2021.”
FSNet: Focus Scanning Network for Camouflaged Object Detection
Ze, S., Kang, X., Wei, X., Liu, H., Dian, R., & Shutao Li, A. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3266659 · Cited by 92
“1) Transformer Encoder: Vision transformer (ViT) [35] is a popular and widely used model for dense prediction tasks [42].”
MethodOmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion
Li, Y., Guo, Y., Yan, Z., Huang, X., Duan, Y., & Ren, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00282 · Cited by 96
“Originally proposed in natural language processing [33], the transformer architecture has since been widely used in computer vision tasks such as image classification [10], depth estimation [26], object detection [2], and semantic segmentation [27, 40].”
Method3D Common Corruptions and Data Augmentation
Kar, O. F., Yeo, T., Atanov, A., & Zamir, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01839 · Cited by 106
“By training with larger and more diverse data with Omnidata, the DPT performance improves.”
MethodInfluential360MonoDepth: High-Resolution 360° Monocular Depth Estimation
Rey-Area, M., Yuan, M., & Richardt, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00374 · Cited by 93
“This closely matches the 384×384 training resolution used by MiDaS v2/v3 [55, 56], for which we use the authors’ implementation.”
MethodResultInfluentialPredictive Uncertainty Estimation for Camouflaged Object Detection
Zhang, Y., Zhang, J., Hamidouche, W., & Olivier Déforges, A. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3287137 · Cited by 91
“Note that we use hybrid-ViT-based backbone [65] as an example, and by replacing it with pure ResNets, we obtain the other two models as shown in Table I.”
MethodNeuralLift-360: Lifting an in-the-Wild 2D Photo to A 3D Object with 360° Views
Xu, D., Jiang, Y., Wang, P., Fan, Z., Wang, Y., & Wang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00435 · Cited by 112
“Under the guidance of monocular depth estimation networks [47, 48], the 2D image is firstly back-projected to a 3D data format (e.”
MethodContinuous 3D Perception Model with Persistent State
Wang, Q., Zhang, Y., Holynski, A., Efros, A. A., & Kanazawa, A. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00983 · Cited by 240
“Extensive research [7, 27, 42, 55, 58, 59, 85, 92] focuses on estimating monocular depth.”
Transformers for Vision: A Survey on Innovative Methods for Computer Vision
Palanisamy, B., Hassija, V., Chatterjee, A., Mandal, A., Chakraborty, D., Pandey, A., Chalapathi, G. S. S., & Kumar, D. (2025) · IEEE Access · doi:10.1109/access.2025.3571735 · Cited by 83
Metadata
- Authors
- Ranftl, R., Bochkovskiy, A., & Koltun, V.
- Affiliations
- Intel Labs
- Intel (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 12159-12168
- OpenAlex
- W3136635488
- Cited by
- 2,373
- References
- 88
- Citations by year
- 2026: 480 · 2025: 607 · 2024: 586 · 2023: 431 · 2022: 225 · 2021: 44
- FWCI
- 104.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- IEEE/CVF International Conference on Computer Vision (ICCV)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- vision transformer, dense prediction, monocular depth estimation, semantic segmentation, convolutional decoder, global receptive field, Pascal Context, NYUv2, KITTI, ADE20K
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
