Multiscale vision transformers
Open accessAbstract
We present Multiscale Vision Transformers (MViT) for video and image recognition, by connecting the seminal idea of multiscale feature hierarchies with transformer models. Multiscale Transformers have several channel-resolution scale stages. Starting from the input resolution and a small channel dimension, the stages hierarchically expand the channel capacity while reducing the spatial resolution. This creates a multiscale pyramid of features with early layers operating at high spatial resolution to model simple low-level visual information, and deeper layers at spatially coarse, but complex, high-dimensional features. We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10× more costly in computation and parameters. We further remove the temporal dimension and apply our model for image classification where it outperforms prior work on vision transformers. Code is available at: https://github.com/facebookresearch/SlowFast.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206 · see citing works
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · see citing works
Self-citationRandom Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617 · see citing works
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537 · see citing works
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223 · see citing works
Zhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01595 · Cited by 2,847 · see citing works
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.213 · Cited by 2,842 · see citing works
Self-citationViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767 · see citing works
Koenderink, J. J. (1984) · Biological Cybernetics · doi:10.1007/bf00336961 · Cited by 2,452 · cited 1× · see citing works
“In a parallel development, the computer vision community developed multiscale processing, sometimes called “pyramid” strategies, with Rosenfeld and Thurston [85], Burt and Adelson [8], Koenderink [61], among the key papers.”
MethodZero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · see citing works
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,152 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104 · see citing works
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., & Hu, S.-M. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0229-5 · Cited by 2,100 · see citing works
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084 · see citing works
Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
Qiu, Z., Yao, T., & Mei, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.590 · Cited by 1,944 · see citing works
Is Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Medical Transformer: Gated Axial-Attention for Medical Image Segmentation
Valanarasu, J. M. J., Oza, P., Hacihaliloglu, I., & Patel, V. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87193-2_4 · Cited by 1,566 · see citing works
Relation Networks for Object Detection
Hu, H., Gu, J., Zhang, Z., Dai, J., & Wei, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00378 · Cited by 1,524 · see citing works
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471 · see citing works
Reformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · see citing works
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q. V., Vaswani, A., & Shlens, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,320 · see citing works
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,262 · see citing works
Self-citationTransReID: Transformer-based Object Re-Identification
He, S., Luo, H., Wang, P., Wang, F., Li, H., & Jiang, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01474 · Cited by 1,244 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
Temporal Relational Reasoning in Videos
Zhou, B., Andonian, A., Oliva, A., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_49 · Cited by 1,104 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,098 · see citing works
Generating Long Sequences with Sparse Transformers
Child, R., Gray, S., Radford, A., & Sutskever, I. (2019) · arXiv · doi:10.48550/arxiv.1904.10509 · Cited by 1,063 · see citing works
Understanding Back-Translation at Scale
Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1045 · Cited by 1,044 · see citing works
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Gu, C., Sun, C., Ross, D. A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan, S., Toderici, G. D., Ricco, S., Sukthankar, R., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00633 · Cited by 1,029 · see citing works
Self-citation(2020) · International Conference on Machine Learning · Cited by 1,012 · see citing works
Exploring Self-Attention for Image Recognition
Zhao, H., Jia, J., & Koltun, V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01009 · Cited by 1,002 · see citing works
Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition
Fukushima, K., & Miyake, S. (1982) · Lecture Notes in Biomathematics · doi:10.1007/978-3-642-46466-9_18 · Cited by 991 · see citing works
Stand-Alone Self-Attention in Vision Models
Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019) · arXiv · doi:10.48550/arxiv.1906.05909 · Cited by 977 · see citing works
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867 · see citing works
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Jiang, H., Sun, D., Jampani, V., Yang, M., Miller, E. G., & Kautz, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00938 · Cited by 850 · see citing works
Edge and Curve Detection for Visual Scene Analysis
Rosenfeld, A., & Thurston, M. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223290 · Cited by 792 · see citing works
Videos as Space-Time Region Graphs
Wang, X., & Gupta, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_25 · Cited by 786 · see citing works
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution
Chen, Y., Fan, H., Xu, B., Yan, Z., Kalantidis, Y., Rohrbach, M., Shuicheng, Y., & Feng, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00353 · Cited by 779 · see citing works
Self-citationEnd-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740 · see citing works
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · cited 1× · see citing works
“Recent works [100, 60, 17, 94, 20, 16, 67] reduce the quadratic attention complexity to make transformers more efficient for natural language processing applications, which is complementary to our approach.”
BackgroundVideo Classification With Channel-Separated Convolutional Networks
Tran, D., Wang, H., Feiszli, M., & Torresani, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00565 · Cited by 642 · see citing works
Local Relation Networks for Image Recognition
Hu, H., Zhang, Z., Xie, Z., & Lin, S. C.-F. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00356 · Cited by 633 · see citing works
Multi-modal Transformer for Video Retrieval
Gabeur, V., Sun, C., Alahari, K., & Schmid, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_13 · Cited by 615 · see citing works
TEA: Temporal Excitation and Aggregation for Action Recognition
Li, Y., Ji, B., Shi, X., Zhang, J., Kang, B., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00099 · Cited by 592 · see citing works
Self-citationThe Laplacian Pyramid as a Compact Image Code
Burt, P. J., & Adelson, E. H. (1987) · Elsevier eBooks · doi:10.1016/b978-0-08-051581-6.50065-9 · Cited by 564 · see citing works
Neimark, D., Bar, O., Zohar, M., & Asselmann, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00355 · Cited by 551 · see citing works
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers
Wang, H., Zhu, Y., Adam, H., Yuille, A., & Chen, L.-C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00542 · Cited by 547 · see citing works
Visual Transformers: Token-based Image Representation and Processing for Computer VisionPreprint
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03677 · Cited by 529 · see citing works
Self-citationSTM: SpatioTemporal and Motion Encoding for Action Recognition
Jiang, B., Wang, M., Gan, W., Wu, W., & Yan, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00209 · Cited by 496 · see citing works
VideoLSTM convolves, attends and flows for action recognition
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., & Snoek, C. G. M. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.011 · Cited by 485 · see citing works
Multimodal Transformer With Multi-View Visual Representation for Image Captioning
Yu, J., Li, J., Zhou, Y., & Huang, Q. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2947482 · Cited by 474 · see citing works
Long-Term Feature Banks for Detailed Video Understanding
Wu, C.-Y., Feichtenhofer, C., Fan, H., He, K., Krähenbühl, P., & Girshick, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00037 · Cited by 454 · see citing works
Self-citationObject-Centric Learning with Slot AttentionPreprint
Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., & Kipf, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15055 · Cited by 399 · see citing works
Receptive fields of optic nerve fibres in the spider monkey
Hubel, D. H., & Wiesel, T. N. (1960) · The Journal of Physiology · doi:10.1113/jphysiol.1960.sp006596 · Cited by 366 · see citing works
VirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01101 · Cited by 350 · see citing works
A Short Note on the Kinetics-700-2020 Human Action Dataset
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.10864 · Cited by 334 · see citing works
Toward Transformer-Based Object DetectionPreprint
Beal, J., Kim, E. S., Tzeng, E., Park, D. H., Zhai, A., & Kislyuk, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09958 · Cited by 255 · see citing works
Actor-Centric Relation Network
Sun, C., Shrivastava, A., Vondrick, C., Murphy, K. P., Sukthankar, R., & Schmid, C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_20 · Cited by 235 · see citing works
Timeception for Complex Action Recognition
Hussein, N., Gavves, E., & Smeulders, A. W. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00034 · Cited by 228 · see citing works
Rethinking Spatiotemporal Feature Learning For Video Understanding.
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. P. (2017) · Cited by 205 · see citing works
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182 · see citing works
Augment your batch: Improving generalization through instance repetition
Hoffer, E., Ben-Nun, T., Hubara, I., Giladi, N., Hoefler, T., & Soudry, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00815 · Cited by 166 · see citing works
Sparse Sinkhorn AttentionPreprint
Tay, Y., Bahri, D., Liu, Y., Metzler, D., & Juan, D.-C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11296 · Cited by 162 · see citing works
MotionSqueeze: Neural Motion Feature Learning for Video Understanding
Kwon, H., Kim, M., Kwak, S., & Cho, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_21 · Cited by 147 · see citing works
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language ProcessingPreprint
Dai, Z., Lai, G., Yang, Y., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03236 · Cited by 139 · see citing works
LambdaNetworks: Modeling Long-Range Interactions Without Attention
Bello, I. (2021) · arXiv · doi:10.48550/arxiv.2102.08602 · Cited by 129 · see citing works
CPTR: Full Transformer Network for Image CaptioningPreprint
Liu, W., Chen, S., Guo, L., Zhu, X., & Liu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10804 · Cited by 128 · see citing works
Training Vision Transformers for Image Retrieval
El-Nouby, A., Neverova, N., Laptev, I., & Jégou, H. (2021) · arXiv · doi:10.48550/arxiv.2102.05644 · Cited by 127 · see citing works
Fast and Accurate Model Scaling
Dollár, P., Singh, M., & Girshick, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00098 · Cited by 116 · see citing works
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech RecognitionPreprint
Chen, C.-F., Fan, Q., Mallinar, N., Sercu, T., & Feris, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03848 · Cited by 93 · see citing works
Do We Really Need Explicit Position Encodings for Vision TransformersPreprint
Chu, X., Zhang, B., Tian, Z., Wei, X. K., & Xia, H. (2021) · arXiv (Cornell University) · Cited by 89 · see citing works
Fan, H., Murrell, T., Wang, H., Alwala, K. V., Li, Y., Li, Y., Xiong, B., Ravi, N., Li, M., Yang, H., et al. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3478329 · Cited by 50 · see citing works
Self-citationHow to Start Training: The Effect of Initialization and ArchitecturePreprint
Hanin, B., & Rolnick, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01719 · Cited by 47 · see citing works
TransPose: Towards Explainable Human Pose Estimation by TransformerPreprint
Yang, S., Quan, Z., Nie, M., & Yang, W. (2020) · arXiv (Cornell University) · Cited by 40 · see citing works
SpecTr: Spectral Transformer for Hyperspectral Pathology Image SegmentationPreprint
Yun, B., Wang, Y., Chen, J., Wang, H., Shen, W., & Li, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03604 · Cited by 36 · see citing works
Linear Attention Mechanism: An Efficient Attention for Semantic SegmentationPreprint
Li, R., Su, J., Duan, C., & Zheng, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14902 · Cited by 25 · see citing works
Transformer is All You Need: Multimodal Multitask Learning with a Unified TransformerPreprint
Hu, R., & Singh, A. (2021) · arXiv (Cornell University) · Cited by 23 · see citing works
Flagged first, then by citations
6 of the 111 references couldn’t be matched to a known paper.
Research works printed but not linked: 19
- [3][3] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv:1607.06450 , 2016. 4 - · arXiv
- [9][9] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. Endto-end object detection with transformers. In Proc. ECCV , pages 213-229. Springer, 2020. 2 -
- [10][10] Joao Carreira, Eric Noland, Andras Banki-Horvath, Chloe Hillier, and Andrew Zisserman. A short note about Kinetics600. arXiv:1808.01340 , 2018. 2 - · arXiv
- [22][22] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proc. CVPR , 2009. 6, 9, 14 -
- [25][25] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020. 1, 2, 3, 4, 5, 6, 9, 10, 13 - · arXiv
- [34][34] Shanghua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip HS Torr. Res2net: Anew multi-scale backbone architecture. IEEE PAMI , 2019. 2 -
- [35][35] Rohit Girdhar, Joao Carreira, Carl Doersch, and Andrew Zisserman. Video action transformer network. In Proc. CVPR , 2019. 2 -
- [36][36] Ross Girshick. Fast R-CNN. In Proc. ICCV , 2015. 14 -
- [37][37] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training ImageNet in 1 hour. arXiv:1706.02677 , 2017. 13, 14 - · arXiv
- [43][43] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. In Proc. ICCV , 2017. 14 -
- [47][47] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer-based object reidentification. arXiv preprint arXiv:2102.04378 , 2021. 2 - · arXiv
- [64][64] Hung Le, Doyen Sahoo, Nancy F Chen, and Steven CH Hoi. Multimodal transformer networks for end-toend video-grounded dialogue systems. arXiv preprint arXiv:1907.01166 , 2019. 2 - · arXiv
- [70][70] Ji Lin, Chuang Gan, and Song Han. Temporal shift module for efficient video understanding. In Proc. ICCV , 2019. 14 -
- [79][79] Zhaofan Qiu, Ting Yao, and Tao Mei. Learning spatiotemporal representation with pseudo-3d residual networks. In Proc. ICCV , 2017. 2 -
- [91][91] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proc. CVPR , 2015. 13 -
- [97][97] Jeya Maria Jose Valanarasu, Poojan Oza, Ilker Hacihaliloglu, and Vishal M Patel. Medical transformer: Gated axialattention for medical image segmentation. arXiv preprint arXiv:2102.10662 , 2021. 2 - · arXiv
- [103][103] Yuqing Wang, Zhaoliang Xu, Xinlong Wang, Chunhua Shen, Baoshan Cheng, Hao Shen, and Huaxia Xia. Endto-end video instance segmentation with transformers. arXiv preprint arXiv:2011.14503 , 2020. 2 - · arXiv
- [109][109] Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokensto-token vit: Training vision transformers from scratch on imagenet. arXiv preprint arXiv:2101.11986 , 2021. 2 - · arXiv
- [110][110] Zhenxun Yuan, Xiao Song, Lei Bai, Wengang Zhou, Zhe Wang, and Wanli Ouyang. Temporal-channel transformer for 3d lidar-based video object detection in autonomous driving. arXiv preprint arXiv:2011.13628 , 2020. 2 - · arXiv
Cited by
Sorted by citations
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494 · see citing works
Self-citationHan (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682 · see citing works
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595 · see citing works
Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294 · see citing works
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I. S., & Xie, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01548 · Cited by 1,791 · see citing works
(2022) · doi:10.1007/978-3-031-20053-3_27 · Cited by 1,060 · see citing works
Self-citationExploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20077-9_17 · Cited by 903 · see citing works
Self-citationMViTv2: Improved Multiscale Vision Transformers for Classification and Detection
Li, Y., Wu, C.-Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00476 · Cited by 837 · see citing works
Self-citationBeyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks
Guo, M.-H., Liu, Z.-N., Mu, T., & Hu, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3211006 · Cited by 681 · see citing works
UniFormer: Unifying Convolution and Self-Attention for Visual Recognition
Li, K., Wang, Y., Zhang, J., Gao, P., Song, G., Liu, Y., Li, H., & Qiao, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3282631 · Cited by 636 · see citing works
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · see citing works
Self-citationScaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning
Chen, R. J., Chen, C., Li, Y., Chen, T. Y., Trister, A. D., Krishnan, R. G., & Mahmood, F. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01567 · Cited by 621 · see citing works
Self-citationEarly Convolutions Help Transformers See BetterPreprint
Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 565 · see citing works
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587 · see citing works
Self-citationGenerative artificial intelligence: a systematic review and applications
Sengar, S. S., Hasan, A. B., Kumar, S., & Carroll, F. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20016-1 · Cited by 608 · see citing works
GroupViT: Semantic Segmentation Emerges from Text Supervision
Xu, J., Mello, S. D., Liu, S., Byeon, W., Breuel, T. M., Kautz, J., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01760 · Cited by 475 · see citing works
ActionFormer: Localizing Moments of Actions with Transformers
Zhang, C.-L., Wu, J., & Li, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_29 · Cited by 471 · see citing works
Self-citationVideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909 · see citing works
SimVP: Simpler yet Better Video Prediction
Gao, Z., Tan, C., Wu, L., & Li, S. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00317 · Cited by 430 · see citing works
Touvron, H., Cord, M., & Jégou, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20053-3_30 · Cited by 409 · see citing works
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition
Jiao, J., Tang, Y.-M., Lin, K.-Y., Gao, Y., Ma, A. J., Wang, Y., & Zheng, W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3243616 · Cited by 412 · see citing works
Li, J., Chen, J., Tang, Y., Wang, C., Landman, B. A., & Zhou, S. K. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102762 · Cited by 426 · see citing works
MPViT: Multi-Path Vision Transformer for Dense Prediction
Lee, Y., Kim, J., Willette, J., & Hwang, S. J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00714 · Cited by 388 · see citing works
Expanding Language-Image Pretrained Models for General Video Recognition
Ni, B., Peng, H., Chen, M., Zhang, S., Meng, G., Fu, J., Xiang, S., & Ling, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_1 · Cited by 361 · see citing works
VideoMamba: State Space Model for Efficient Video Understanding
Li, K., Li, X., Wang, Y., He, Y., Wang, Y., Wang, L., & Qiao, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73347-5_14 · Cited by 378 · see citing works
Edgenext: Efficiently amalgamated CNN-transformer architecture for mobile vision applications
Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S. W., Anwer, R. M., & Shahbaz Khan, F. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25082-8_1 · Cited by 407 · see citing works
PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer
Yu, Z., Shen, Y., Shi, J.-G., Zhao, H., Torr, P., & Zhao, G.-Y. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00415 · Cited by 349 · see citing works
Multiview Transformers for Video Recognition
Yan, S., Xiong, X., Arnab, A., Lu, Z., Zhang, M., Sun, C., & Schmid, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00333 · Cited by 312 · see citing works
P2T: Pyramid Pooling Transformer for Scene Understanding
Wu, Y.-H., Liu, Y., Zhan, X., & Cheng, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3202765 · Cited by 317 · see citing works
VOLO: Vision Outlooker for Visual Recognition
Yuan, L., Hou, Q., Jiang, Z., Feng, J., & Yan, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206108 · Cited by 309 · see citing works
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Shaker, A., Maaz, M., Rasheed, H., Khan, S., Yang, M., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01598 · Cited by 299 · see citing works
EdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers
Pan, J., Bulat, A., Tan, F., Zhu, X., Dudziak, Ł., Li, H., Tzimiropoulos, G., & Martínez, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_18 · Cited by 302 · see citing works
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
Xiong, Y., Varadarajan, B., Wu, L., Xiang, X., Xiao, F., Zhu, C., Dai, X., Wang, D., Sun, F., Iandola, F., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01525 · Cited by 304 · see citing works
ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond
Zhang, Q., Xu, Y., Zhang, J., & Tao, D. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-022-01739-w · Cited by 465 · see citing works
Humans in 4D: Reconstructing and Tracking Humans with Transformers
Goel, S., Pavlakos, G., Rajasegaran, J., Kanazawa, A., & Malik, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01358 · Cited by 283 · see citing works
Self-citationKim, J., Kim, H., Kim, H., Lee, D., & Yoon, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11223-9 · Cited by 334 · see citing works
CTNet: Context-Based Tandem Network for Semantic Segmentation
Li, Z., Sun, Y., Zhang, L., & Tang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132068 · Cited by 269 · see citing works
Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation
Gu, J., Kwon, H., Wang, D., Ye, W., Li, M., Chen, Y., Lai, L., Chandra, V., & Pan, D. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01178 · Cited by 270 · see citing works
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01199 · Cited by 262 · see citing works
FAST-VQA: Efficient End-to-End Video Quality Assessment with Fragment Sampling
Wu, H., Chen, C., Hou, J., Liao, L., Wang, A., Sun, W., Yan, Q., & Lin, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20068-7_31 · Cited by 264 · see citing works
Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning
Yao, T., Pan, Y., Li, Y., Ngo, C., & Mei, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_19 · Cited by 269 · see citing works
Self-citationScale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning
Reed, C., Gupta, R., Li, S., Brockman, S., Funk, C., Clipp, B., Keutzer, K., Candido, S., Uyttendaele, M., & Darrell, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00378 · Cited by 276 · see citing works
A systematic review of deep learning techniques for plant diseases
Paçal, İ., Kunduracıoğlu, İ., Alma, M. H., Deveci, M., Kadry, S., Nedoma, J., Slaný, V., & Martínek, R. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10944-7 · Cited by 263 · see citing works
Transformer-Based Visual Segmentation: A Survey
Li, X., Ding, H., Yuan, H., Zhang, W., Pang, J., Cheng, G., Chen, K., Liu, Z., & Loy, C. C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3434373 · Cited by 272 · see citing works
Nascimento, E. G. S., Melo, T. A. C., & Moreira, D. M. (2023) · Energy · doi:10.1016/j.energy.2023.127678 · Cited by 223 · see citing works
Frozen CLIP Models are Efficient Video Learners
Lin, Z., Geng, S., Zhang, R., Gao, P., de Melo, G., Wang, X., Dai, J., Qiao, Y., & Li, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_23 · Cited by 213 · see citing works
Anticipative Video Transformer
Girdhar, R., & Grauman, K. L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01325 · Cited by 214 · see citing works
BEVT: BERT Pretraining of Video Transformers
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Jiang, Y.-G., Zhou, L., & Yuan, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01432 · Cited by 234 · see citing works
Omnivore: A Single Model for Many Visual Modalities
Girdhar, R., Singh, M., Ravi, N., van der Maaten, L., Joulin, A., & Misra, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01563 · Cited by 212 · see citing works
Adaptive Token Sampling for Efficient Vision Transformers
Fayyaz, M., Koohpayegani, S. A., Jafari, F., Sengupta, S., Joze, H. R. V., Sommerlade, E., Pirsiavash, H., & Gall, J. (2021) · European Conference on Computer Vision · doi:10.1007/978-3-031-20083-0_24 · Cited by 307 · see citing works
S2-MLP: Spatial-Shift MLP Architecture for Vision
Yu, T., Li, X., Cai, Y., Sun, M., & Li, P. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00367 · Cited by 220 · see citing works
Conformer: Local Features Coupling Global Representations for Recognition and Detection
Peng, Z., Guo, Z., Huang, W., Wang, Y., Xie, L., Jiao, J., Tian, Q. C., & Ye, Q. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243048 · Cited by 190 · see citing works
Yao, T., Li, Y., Pan, Y., Wang, Y., Zhang, X.-P. S., & Mei, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268446 · Cited by 187 · see citing works
Self-citationSelva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 177 · see citing works
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Wu, C.-Y., Li, Y., Mangalam, K., Fan, H., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01322 · Cited by 176 · see citing works
Self-citationSpatio-temporal Relation Modeling for Few-shot Action Recognition
Thatipelli, A., Narayan, S., Khan, S. H., Anwer, R. M., Khan, F. S., & Ghanem, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01933 · Cited by 166 · see citing works
Kunduracıoğlu, İ., & Paçal, İ. (2024) · Journal of Plant Diseases and Protection · doi:10.1007/s41348-024-00896-z · Cited by 167 · see citing works
Lightweight transformer image feature extraction network
Zheng, W., Lu, S., Yang, Y., Yin, Z., & Yin, L. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1755 · Cited by 167 · see citing works
A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3392941 · Cited by 177 · see citing works
Self-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155 · see citing works
Shen, W., Peng, Z., Wang, X., Wang, H., Cen, J., Jiang, D., Xie, L., Yang, X., & Tian, Q. C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3246102 · Cited by 153 · see citing works
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Li, K., Wang, Y., Li, Y., Wang, Y., He, Y., Wang, L., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01826 · Cited by 143 · see citing works
Self-citationA Survey on Video Action Recognition in Sports: Datasets, Methods and Applications
Wu, F. L., Wang, Q., Bian, J., Ding, N., Lu, F., Cheng, J. S., Dou, D., & Xiong, H. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3232034 · Cited by 155 · see citing works
Yu, Z., Shen, Y., Shi, J.-G., Zhao, H., Cui, Y., Zhang, J., Torr, P., & Zhao, G.-Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01758-1 · Cited by 161 · see citing works
KVT: k-NN Attention for Boosting Vision Transformers
Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., & Jin, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_17 · Cited by 142 · see citing works
Video Frame Interpolation Transformer
Shi, Z., Xu, X., Liu, X., Chen, J., & Yang, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01696 · Cited by 135 · see citing works
Active Learning by Feature Mixing
Parvaneh, A., Abbasnejad, E., Teney, D., Haffari, R., van den Hengel, A., & Shi, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01192 · Cited by 134 · see citing works
Recurring the Transformer for Video Action Recognition
Yang, J., Dong, X., Liu, L., Zhang, C., Shen, J., & Yu, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01367 · Cited by 137 · see citing works
Wu, W., Wang, X., Luo, H., Wang, J., Yang, Y., & Ouyang, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00640 · Cited by 131 · see citing works
AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing
Jiang, J., Streli, P., Qiu, H., Fender, A. R., Laich, L., Snape, P., & Holz, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_26 · Cited by 128 · see citing works
ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer
Guo, P., Mei, Y., Zhou, J., Jiang, S., & Patel, V. M. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3314747 · Cited by 129 · see citing works
Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining
Chen, X., Pan, J., & Dong, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02421 · Cited by 128 · see citing works
SVFormer: Semi-supervised Video Transformer for Action Recognition
Xing, Z., Dai, Q., Han, H., Chen, J., Wu, Z., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01804 · Cited by 126 · see citing works
Three Things Everyone Should Know About Vision Transformers
Touvron, H., Cord, M., El-Nouby, A., Verbeek, J. J., & Jeǵou, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_29 · Cited by 129 · see citing works
CycleMLP: A MLP-Like Architecture for Dense Visual Predictions
Chen, S., Xie, E., Ge, C., Chen, R., Liang, D., & Luo, P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3303397 · Cited by 276 · see citing works
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Wasim, S. T., Naseer, M., Khan, S., Khan, F. S., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02206 · Cited by 125 · see citing works
Towards multimodal foundation models in molecular cell biology
Cui, H., Tejada-Lapuerta, A., Brbić, M., Sáez-Rodríguez, J., Cristea, S., Goodarzi, H., Lotfollahi, M., Theis, F. J., & Wang, B. (2025) · Nature · doi:10.1038/s41586-025-08710-y · Cited by 131 · see citing works
Multi-Modal Feature Pyramid Transformer for RGB-Infrared Object Detection
Zhu, Y.-H., Sun, X., Wang, M., & Huang, H. (2023) · IEEE transactions on intelligent transportation systems (Print) · doi:10.1109/tits.2023.3266487 · Cited by 130 · see citing works
TallFormer: Temporal Action Localization with a Long-Memory Transformer
Cheng, F., & Bertasius, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_29 · Cited by 123 · see citing works
Selective Structured State-Spaces for Long-Form Video Understanding
Wang, J., Zhu, W., Wang, P., Yu, X., Liu, L. L., Omar, M., & Hamid, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00618 · Cited by 121 · see citing works
Self-supervised Video Transformer
Ranasinghe, K., Naseer, M., Khan, S. H., Khan, F. S., & Ryoo, M. S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00289 · Cited by 123 · see citing works
Video Understanding With Large Language Models: A Survey
Tang, Y., Bi, J., Xu, S., Song, L., Liang, S., Wang, T., Zhang, D., An, J., Lin, J., Zhu, R., et al. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3566695 · Cited by 174 · see citing works
Yu, Q., Wang, H., Qiao, S., Collins, M. D., Zhu, Y., Adam, H., Yuille, A., & Chen, L.-C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19818-2_17 · Cited by 117 · see citing works
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Pan, X., Ye, T., Xia, Z., Song, S., & Huang, G. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00207 · Cited by 117 · see citing works
ActionCLIP: Adapting Language-Image Pretrained Models for Video Action Recognition
Wang, M., Xing, J., Mei, J., Liu, Y., & Jiang, Y. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3331841 · Cited by 434 · see citing works
Unsupervised video anomaly detection via normalizing flows with implicit latent features
Cho, M., Kim, T., Kim, W. J., Cho, S., & Lee, S. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108703 · Cited by 111 · see citing works
MARLIN: Masked Autoencoder for facial video Representation LearnINg
Cai, Z., Ghosh, S., Stefanov, K., Dhall, A., Cai, J., Rezatofighi, H., Haffari, R., & Hayat, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00150 · Cited by 113 · see citing works
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wu, W., Sun, Z., & Ouyang, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25386 · Cited by 112 · see citing works
TubeR: Tubelet Transformer for Video Action Detection
Zhao, J., Zhang, Y., Li, X., Chen, H., Shuai, B., Xu, M., Liu, C., Kundu, K., Xiong, Y., Modolo, D., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01323 · Cited by 101 · see citing works
Revisiting Weakly Supervised Pre-Training of Visual Perception Models
Singh, M., Gustafson, L., Adcock, A., De Freitas Reis, V., Gedik, B., Kosaraju, R. P., Mahajan, D., Girshick, R., Dollar, P., & Van Der Maaten, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00088 · Cited by 99 · see citing works
ROAD: The Road Event Awareness Dataset for Autonomous Driving
Singh, G., Akrigg, S., Maio, M. D., Fontana, V., Alitappeh, R. J., Khan, S., Saha, S., Jeddisaravi, K., Yousefi, F., Culley, J., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3150906 · Cited by 99 · see citing works
STIP-GCN: Space-time interest points graph convolutional network for action recognition
Yenduri, S., Chalavadi, V., & Mohan, C. K. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891959 · Cited by 98 · see citing works
Deep Frequency Filtering for Domain Generalization
Lin, S., Zhang, Z., Huang, Z., Lü, Y., Lan, C., Chu, P., You, Q., Wang, J., Liu, Z., Parulkar, A., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01135 · Cited by 96 · see citing works
Long Movie Clip Classification with State-Space Video Models
Islam, M. M., & Bertasius, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_6 · Cited by 95 · see citing works
Yang, M., Chengke, W., Guo, Y., Jiang, R., Zhou, F., Zhang, J., & Yang, Z. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104703 · Cited by 95 · see citing works
ShadowFormer: Global Context Helps Shadow Removal
Guo, L., Huang, S., Liu, D., Cheng, H., & Wen, B. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25148 · Cited by 95 · see citing works
DropKey for Vision Transformer
Li, B.-N., Hu, Y.-H., Nie, X.-C., Han, C.-Y., Jiang, X.-J., Guo, T.-D., & Liu, L.-Q. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.02174 · Cited by 81 · see citing works
Neighbourhood Representative Sampling for Efficient End-to-End Video Quality Assessment
Wu, H., Chen, C., Liao, L., Hou, J., Sun, W., Yan, Q., Gu, J., & Lin, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3319332 · Cited by 98 · see citing works
Snyder, Q., Jiang, Q., & Tripp, E. E. (2024) · Signal Processing · doi:10.1016/j.sigpro.2024.109683 · Cited by 95 · see citing works
Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition
Han, M., Zhang, D. J., Wang, Y., Yan, R., Yao, L., Chang, X., & Qiao, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00300 · Cited by 91 · see citing works
- Authors
- Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C.
- Affiliations
- Meta (Israel) (IL)
- Meta (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 6804-6815
- OpenAlex
- W3202698164
- Cited by
- 1,407
- References
- 111
- Citations by year
- 2026: 147 · 2025: 345 · 2024: 380 · 2023: 349 · 2022: 176 · 2021: 10
- FWCI
- 74.81
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video recognition, image classification, feature pyramid, spatial resolution reduction, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
