The DOI matches its Crossref record; 73 of the 78 references it cites were found in a registry or an index.
AutoFormer: Searching Transformers for Visual Recognition
Open accessAbstract
Recently, pure transformer-based models have shown great potentials for vision tasks such as image classification and detection. However, the design of transformer networks is challenging. It has been observed that the depth, embedding dimension, and number of heads can largely affect the performance of vision transformers. Previous models configure these dimensions based upon manual crafting. In this work, we propose a new one-shot architecture search framework, namely AutoFormer, dedicated to vision transformer search. AutoFormer entangles the weights of different blocks in the same layers during supernet training. Benefiting from the strategy, the trained supernet allows thousands of subnets to be very well-trained. Specifically, the performance of these subnets with weights inherited from the supernet is comparable to those retrained from scratch. Besides, the searched models, which we refer to AutoFormers, surpass the recent state-of-the-arts such as ViT and DeiT. In particular, AutoFormer-tiny/small/base achieve 74.7%/81.7%/82.4% top-1 accuracy on ImageNet with 5.7M/22.9M/53.7M parameters, respectively. Lastly, we verify the transferability of AutoFormer by providing the performance on downstream benchmarks and distillation experiments. Code and models are available at https://github.com/microsoft/Cream.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The methods include pruning [40, 41], Quantization [42-44], Knowledge Distillation [45] and Neural Architecture Search [46].”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [46]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
(2014) · Cited by 33,419
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Research works printed but not linked: 2
- li, 2020, Blockwisely supervised neural architecture search with knowledge distillation, CVPR
- huang, 2019, Ccnet: Crisscross attention for semantic segmentation, ICCV
Cited by
Sorted by citations
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer
Ma, J., Tang, L., Fan, F., Huang, J., Mei, X., & Ma, Y. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105686 · Cited by 1,557
Transformers in Time Series: A Survey
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., & Sun, L. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/759 · Cited by 1,260
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01386 · Cited by 956
Self-citationTinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 521
Self-citationA survey of the vision transformers and their CNN-transformer based variants
Khan, A., Rauf, Z., Sohail, A., Khan, A. R., Asif, H., Asif, A., & Farooq, U. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10595-0 · Cited by 533
Rethinking and Improving Relative Position Encoding for Vision Transformer
Wu, K., Peng, H., Chen, M., Fu, J., & Chao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00988 · Cited by 419
Self-citationA review of convolutional neural network architectures and their optimizations
Cong, S., & Zhou, Y. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10213-5 · Cited by 419
Rethinking Vision Transformers for MobileNet Size and Speed
Li, Y., Ju, H., Wen, Y., Evangelidis, G., Salahi, K., Wang, Y., Tulyakov, S., & Ren, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01549 · Cited by 394
CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention
Wang, W., Chen, W., Qiu, Q., Chen, L., Wu, B., Lin, B., He, X., & Liu, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3341806 · Cited by 348
SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning
Kong, Z., Dong, P., Ma, X., Meng, X., Niu, W., Sun, M., Shen, X., Yuan, G., Ren, B., Tang, H., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20083-0_37 · Cited by 169
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154
MiniViT: Compressing Vision Transformers with Weight Multiplexing
Zhang, J., Peng, H., Wu, K., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01183 · Cited by 135
Self-citationHeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
Dong, P., Sun, M., Lu, A., Xie, Y., Liu, K., Kong, Z., Meng, X., Li, Z., Lin, X., Fang, Z., et al. (2023) · 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071047 · Cited by 128
Lu, Z., Cheng, R., Jin, Y., Tan, K. C., & Deb, K. (2023) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3233364 · Cited by 129
Neural Architecture Search for Transformers: A Survey
Chitty-Venkata, K. T., Emani, M., Vishwanath, V., & Somani, A. K. (2022) · IEEE Access · doi:10.1109/access.2022.3212767 · Cited by 118
GT-HAD: Gated Transformer for Hyperspectral Anomaly Detection
Lian, J., Wang, L., Sun, H., & Huang, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3355166 · Cited by 118
Li, C., Tang, T., Wang, G., Peng, J., Wang, B., Liang, X., & Chang, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01206 · Cited by 111
Jiang, W., Liu, B., Liang, Y., Gao, H., Lin, P., Zhang, D., & Hu, G. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.122155 · Cited by 108
GLiT: Neural Architecture Search for Global and Local Image Transformer
Chen, B., Li, P., Li, C., Li, B., Bai, L., Chen, L., Sun, M., Yan, J., & Ouyang, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00008 · Cited by 100
Less Is More: Pay Less Attention in Vision Transformers
Pan, Z., Zhuang, B., He, H., Liu, J., & Cai, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20099 · Cited by 95
Convolution-Transformer for Image Feature Extraction
Yin, L., Wang, L., Lu, S., Wang, R., Yang, Y., Yang, B., Liu, S., Alsanad, A., AlQahtani, S. A., Yin, Z., et al. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2024.051083 · Cited by 89
ViTAS: Vision Transformer Architecture Search
Su, X. Q., You, S., Xie, J., Zheng, M., Wang, F., Qian, C., Zhang, C., Wang, X., & Xu, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_9 · Cited by 94
A Survey on Efficient Training of Transformers
Zhuang, B., Liu, J., Pan, Z., He, H., Weng, Y., & Shen, C. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/764 · Cited by 78
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
Chavan, A., Shen, Z., Liu, Z., Liu, Z., Cheng, K.-T., & Xing, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00488 · Cited by 72
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Vision Transformers
Li, C., Wang, G., Wang, B., Liang, X., Li, Z., & Chang, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3194044 · Cited by 74
Global Vision Transformer Pruning with Hessian-Aware Saliency
Yang, H., Yin, H., Shen, M., Molchanov, P. A., Li, H. H., & Kautz, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01779 · Cited by 92
Liu, D., Zhao, L., Wang, Y., & Kato, J. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109550 · Cited by 65
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks
Rao, Y., Liu, Z., Zhao, W., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3263826 · Cited by 75
A survey of model compression strategies for object detection
Lyu, Z., Yu, T., Pan, F., Zhang, Y., Luo, J., Zhang, D., Chen, Y., Zhang, B., & Li, G. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17192-x · Cited by 65
Training-free Transformer Architecture Search
Zhou, Q., Sheng, K., Zheng, X., Li, K., Sun, X., Tian, Y., Chen, J., & Ji, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01062 · Cited by 64
Neural Architecture Search Benchmarks: Insights and Survey
Chitty-Venkata, K. T., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · IEEE Access · doi:10.1109/access.2023.3253818 · Cited by 64
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
Xu, G., Hao, Z., Luo, Y., Hu, H., An, J., & Mao, S. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3315138 · Cited by 68
Efficient Multimodal Fusion via Interactive Prompting
Li, Y., Quan, R., Zhu, L., & Yang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00256 · Cited by 60
“In this section, we experiment with automatic fusion structure search via AutoFormer [5].”
MethodNoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers
Liu, Y., Yang, H., Dong, Z., Keutzer, K., Du, L., & Zhang, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01946 · Cited by 60
Pruning Self-Attentions Into Convolutional Layers in Single Path
He, H., Cai, J., Liu, J., Pan, Z., Zhang, J., Tao, D., & Zhuang, B. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355890 · Cited by 59
You, H., Xiong, Y., Dai, X., Wu, B., Zhang, P., Fan, H., Vajda, P., & Lin, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01387 · Cited by 67
Multi-scale convolution enhanced transformer for multivariate long-term time series forecasting
Li, Y., Xu, Y., Xu, Y., Li, X., & Zhang, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106745 · Cited by 63
Kim, G., Choi, J. G., & Lim, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108475 · Cited by 59
Taloma, R. L., Cuomo, F., Comminiello, D., & Pisani, P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11093-7 · Cited by 71
Res-trans networks for lung nodule classification
Liu, D., Liu, F., Tie, Y., Qi, L., & Wang, F. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02576-5 · Cited by 51
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm
Zhang, J., Li, X., Wang, Y., Wang, C., Yang, Y., Liu, Y., & Tao, D. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02034-6 · Cited by 57
Satellite Video Object Tracking Based on Location Prompts
Wang, J., Liu, F., Jiao, L., Gao, Y., Wang, H., Li, L., Chen, P., Liu, X., & Li, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3358549 · Cited by 46
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data
Nguyen, H. H., Blaschko, M. B., Saarakkala, S. S., & Tiulpin, A. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3312524 · Cited by 43
Two-stage fine-grained image classification model based on multi-granularity feature fusion
Xu, Y., Wu, S., Yang, M., Yang, M., Wu, Z., Yao, Y., & Wei, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110042 · Cited by 41
Wang, R., Bai, Q., Ao, J., Zhou, L., Xiong, Z., Wei, Z., Zhang, Y., Ko, T., & Li, H. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-10269 · Cited by 40
BViT: Broad Attention-Based Vision Transformer
Li, N., Chen, Y., Li, W., Ding, Z., Zhao, D., & Nie, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264730 · Cited by 40
STDAN: Deformable Attention Network for Space-Time Video Super-Resolution
Wang, H., Xiang, X., Tian, Y., Yang, W., & Liao, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3243029 · Cited by 36
Dynamic Mixture of Counter Network for Location-Agnostic Crowd Counting
Wang, M., Cai, H., Dai, Y., & Gong, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00025 · Cited by 35
Automated Progressive Learning for Efficient Training of Vision Transformers
Li, C., Zhuang, B., Wang, G., Liang, X., Chang, X., & Yang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01216 · Cited by 38
Ying, W., Zheng, Q., Wu, Y., Yang, K., Zhou, Z., Chen, J., & Ye, Z. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110869 · Cited by 32
ADNNet: Attention-based deep neural network for Air Quality Index prediction
Wu, X., Gu, X., & See, K. W. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125128 · Cited by 37
AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search
Lee, J., & Ham, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00563 · Cited by 41
TS-Fastformer: Fast Transformer for Time-series Forecasting
Lee, S., Hong, J., Liu, L., & Choi, W. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3630637 · Cited by 33
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery
Wei, Z., Dong, P., Hui, Z., Li, A., Li, L., Lu, M., Pan, H., & Li, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29511 · Cited by 30
Tang, Z., Zhang, J., Hu, M., Ning, Z., Shi, J., Zhai, R., Liu, C., Zhang, J., & Wang, G. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131923 · Cited by 30
DeepWind: a heterogeneous spatio-temporal model for wind forecasting
Wang, B., Shi, J., Tan, B., Ma, M., Hong, F., Yu, Y., & Li, T. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111385 · Cited by 28
NViT: Vision Transformer Compression and Parameter Redistribution.Preprint
Yang, H., Yin, H., Molchanov, P. A., Li, H., & Kautz, J. (2021) · arXiv (Cornell University) · Cited by 28
KMT-PLL: K-Means Cross-Attention Transformer for Partial Label Learning
Fan, J., Huang, L., Gong, C., You, Y., Gan, M., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3347792 · Cited by 28
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Geng, X., Wang, Z., Chen, C., Xu, Q., Xu, K., Jin, C., Gupta, M., Yang, X., Chen, Z., Sabry Aly, M. M., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3394494 · Cited by 26
Ma, S., Nie, J., Guan, S., He, Z., & Gao, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3443910 · Cited by 25
Convolution-Enhanced Vision Transformer Network for Smoke Recognition
Cheng, G., Zhou, Y., Gao, S., Li, Y., & Yu, H. (2023) · Fire Technology · doi:10.1007/s10694-023-01378-8 · Cited by 25
ViT-MVT: A Unified Vision Transformer Network for Multiple Vision Tasks
Xie, T., Dai, K., Jiang, Z., Li, R., Mao, S., Wang, K., & Zhao, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3342141 · Cited by 26
TFormer: A Transmission-Friendly ViT Model for IoT Devices
Lu, Z., Ding, C., Juefei-Xu, F., Boddeti, V. N., Wang, S., & Yang, Y. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3222765 · Cited by 24
Poly-PC: A Polyhedral Network for Multiple Point Cloud Tasks at Once
Xie, T., Wang, S., Wang, E. K., Yang, L., Jiang, Z., Zhang, X., Dai, K., Li, R., & Cheng, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00125 · Cited by 24
Elegans-AI: How the connectome of a living organism could model artificial neural networks
Bardozzo, F., Terlizzi, A., Simoncini, C., Lió, P., & Tagliaferri, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127598 · Cited by 24
A multi-task spatio-temporal fusion network for offshore wind power ramp events forecasting
Song, W., Yan, J., Han, S., Liu, S., Wang, H., Dai, Q., Huo, X., & Yongqian, L. I. U. (2024) · Renewable Energy · doi:10.1016/j.renene.2024.121774 · Cited by 23
Chen, X., Liu, C., Xia, H., & Chi, Z. (2024) · Control Engineering Practice · doi:10.1016/j.conengprac.2024.106165 · Cited by 24
Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search
Guo, B., Chen, T., He, S., Liu, H., Xu, L., Ye, P., & Chen, J. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548149 · Cited by 23
A survey of visual neural networks: current trends, challenges and opportunities
Feng, P., & Tang, Z. (2022) · Multimedia Systems · doi:10.1007/s00530-022-01003-8 · Cited by 23
Joint Graph Learning and Matching for Semantic Feature Correspondence
Liu, H., Wang, T., Li, Y., Lang, C., Jin, Y., & Ling, H. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.109059 · Cited by 22
Self-citationTang, C., Zhang, L. L., Jiang, H., Xu, J., Cao, T., Zhang, Q., Yang, Y., Wang, Z., & Yang, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00536 · Cited by 27
Knowledge Guided Evolutionary Transformer for Remote Sensing Scene Classification
Zhao, J., Jiao, L., Wang, C., Liu, X., Liu, F., Li, L., Ma, M., & Yang, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3407138 · Cited by 23
TimeRAG: Boosting LLM Time Series Forecasting via Retrieval-Augmented Generation
Yang, S., Wang, D., Zheng, H., & Jin, R. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889933 · Cited by 53
PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
Zheng, N., Jiang, H., Zhang, Q., Han, Z., Ma, L., Yang, Y., Yang, F., Zhang, C., Qiu, L., Yang, M., et al. (2023) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3600006.3613139 · Cited by 30
Lu, X., Suganuma, M., & Okatani, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00116 · Cited by 23
An overview of transformers for video anomaly detection
Dilek, E., & Dener, M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11218-1 · Cited by 28
Multi-Dimensional Vision Transformer Compression via Dependency Guided Gaussian Process Search
Hou, Z., & Kung, S.-Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00411 · Cited by 20
NAS-PED: Neural Architecture Search for Pedestrian Detection
Tang, Y., Liu, M., Li, B., Wang, Y., & Ouyang, W. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3507918 · Cited by 21
Shen, Z., Liu, Z., & Xing, E. P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_42 · Cited by 22
SRDD: a lightweight end-to-end object detection with transformer
Zhu, Y., Xia, Q., & Jin, W. (2022) · Connection Science · doi:10.1080/09540091.2022.2125499 · Cited by 18
Training-Free Transformer Architecture Search With Zero-Cost Proxy Guided Evolution
Zhou, Q., Sheng, K., Zheng, X., Li, K., Tian, Y., Chen, J., & Ji, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3378781 · Cited by 21
Dual Cross-Attention Transformer Networks for Temporal Predictive Modeling of Industrial Process
Wang, J., Xie, Y., Xie, S., & Chen, X. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3385820 · Cited by 18
Sun, H., He, X., Li, H., Kong, J., Qiao, M., Cheng, X., Li, P., Zhang, J., Liu, R., & Shang, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127347 · Cited by 18
MDL-NAS: A Joint Multi-domain Learning Framework for Vision Transformer
Wang, S., Xie, T., Cheng, J., Zhang, X., & Liu, H. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.01924 · Cited by 12
Gao, S., Liu, Y., Wang, J., Wang, Z., Xu, W., Yue, R., Cui, R., Liu, Y., & Fan, X. (2024) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2024.110349 · Cited by 19
HybridBranchNet: A novel structure for branch hybrid convolutional neural networks architecture
Parcham, E., & Fateh, M. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.05.025 · Cited by 17
GAIA-Universe: Everything is Super-Netify
Peng, J., Chang, Q., Yin, H., Bu, X., Sun, J., Xie, L., Zhang, X., Tian, Q. C., & Zhang, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3276392 · Cited by 16
Multiscale Spatial-Channel Transformer Architecture Search for Remote Sensing Image Change Detection
Zhang, M., Liu, L., Lei, Z., Ma, K., Feng, J., Liu, Z., & Jiao, L. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3347765 · Cited by 16
Zhang, S., Zhu, C., & Guo, X. (2024) · Energies · doi:10.3390/en17091996 · Cited by 16
Yuan, L., Wang, H., & Zhang, F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125588 · Cited by 16
UFO: Unified Feature Optimization
Xi, T., Sun, Y., Yu, D., Li, B., Peng, N., Zhang, G., Zhang, X., Wang, Z., Chen, J., Wang, J., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_27 · Cited by 16
A comprehensive review of lightweight deep learning models for edge computing with future directions
Lai, N., Dewi, D. A., Maidin, S. S., Xiao, W., Zhao, S., & Hu, Q. (2026) · Discover Computing · doi:10.1007/s10791-026-10021-3 · Cited by 22
Cai, K., Zhang, X., Zhang, M., Ge, Q., Li, S., Qiao, B., & Liu, Y. (2023) · Sustainable Environment Research · doi:10.1186/s42834-023-00175-w · Cited by 15
A frequency domain-based machine learning architecture for short-term wave height forecasting
Zhan, K., Li, C., & Zhu, R. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.115844 · Cited by 15
MoBoo: Memory-Boosted Vision Transformer for Class-Incremental Learning
Ni, B., Nie, X., Zhang, C., Xu, S., Zhang, X., Meng, G., & Xiang, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3417431 · Cited by 16
Auto-Spikformer: Spikformer architecture search
Che, K.-W., Zhou, Z., Niu, J., Ma, Z., Fang, W., Chen, Y.-Q., Shen, S., Yuan, L., & Tian, Y. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1372257 · Cited by 18
Aeroelastic force prediction via temporal fusion transformers
Montoya, M. C., Mishra, A., Verma, S., Mures, O. A., & Rubio-Medrano, C. E. (2024) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.13381 · Cited by 16
Remote Sensing Video Tracking: Current Status, Challenges, and Future
Liu, F., Wang, J., Jiao, L., Zhang, J., Wang, H., Li, S., Li, L., Chen, P., Liu, X., Ma, W., et al. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3573572 · Cited by 19
Metadata
- Authors
- Chen, M., Peng, H., Fu, J., & Ling, H.
- Affiliations
- Stony Brook University (US)
- Microsoft Research Asia (China) (CN)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 12250-12260
- OpenAlex
- W3204801262
- Cited by
- 323
- References
- 78
- Citations by year
- 2026: 28 · 2025: 75 · 2024: 103 · 2023: 79 · 2022: 33 · 2021: 5
- FWCI
- 17.99
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- vision transformer, image classification, transferability, knowledge distillation, Autoformer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
