Cited by 324Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMachine Learning and Data Classification

The DOI matches its Crossref record; 73 of the 78 references it cites were found in a registry or an index.

AutoFormer: Searching Transformers for Visual Recognition

Open access

Chen, Minghao, Peng, Houwen, Fu, Jianlong, Ling, Haibin·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv48922.2021.01205

Open this paper in CoChat — free account Open PDF

Abstract

Recently, pure transformer-based models have shown great potentials for vision tasks such as image classification and detection. However, the design of transformer networks is challenging. It has been observed that the depth, embedding dimension, and number of heads can largely affect the performance of vision transformers. Previous models configure these dimensions based upon manual crafting. In this work, we propose a new one-shot architecture search framework, namely AutoFormer, dedicated to vision transformer search. AutoFormer entangles the weights of different blocks in the same layers during supernet training. Benefiting from the strategy, the trained supernet allows thousands of subnets to be very well-trained. Specifically, the performance of these subnets with weights inherited from the supernet is comparable to those retrained from scratch. Besides, the searched models, which we refer to AutoFormers, surpass the recent state-of-the-arts such as ViT and DeiT. In particular, AutoFormer-tiny/small/base achieve 74.7%/81.7%/82.4% top-1 accuracy on ImageNet with 5.7M/22.9M/53.7M parameters, respectively. Lastly, we verify the transferability of AutoFormer by providing the performance on downstream benchmarks and distillation experiments. Code and models are available at https://github.com/microsoft/Cream.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.01205

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Untitled work
(2014) · Cited by 33,419
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889

Show all 78 references

Research works printed but not linked: 2

li, 2020, Blockwisely supervised neural architecture search with knowledge distillation, CVPR
huang, 2019, Ccnet: Crisscross attention for semantic segmentation, ICCV

Sorted by citations

Items 101–200 of 324

AttnZero: Efficient Attention Discovery for Vision Transformers
Li, L., Wei, Z., Dong, P., Luo, W., Xue, W., Liu, Q., & Guo, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72652-1_2 · Cited by 14
Multi-stage framework using transformer models, feature fusion and ensemble learning for enhancing eye disease classification
AlMohimeed, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-16415-5 · Cited by 13
A New Automatic Hyperparameter Recommendation Approach Under Low-Rank Tensor Completion Framework
Deng, L., & Xiao, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195658 · Cited by 13
DATA: Domain-Aware and Task-Aware Self-supervised Learning
Chang, Q., Peng, J., Xie, L., Sun, J., Yin, H., Tian, Q. C., & Zhang, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00961 · Cited by 12
Optimizing the Structures of Transformer Neural Networks Using Parallel Simulated Annealing
Trzciński, M., Łukasik, S., & Gandomi, A. H. (2024) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2024-0015 · Cited by 12
Traffic Load Simulation for Long-Span Bridges Using a Transformer Model Incorporating In-Lane Transverse Vehicle Movements
Dong, Y., Pan, Y., Wang, D., & Chen, A. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3452106 · Cited by 13
Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis
Xie, T., Zhang, H., Yang, L., Wang, K., Dai, K., Li, R., & Zhao, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3331223 · Cited by 11
Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting
Wang, M., Chen, W., & Chen, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29483 · Cited by 11
SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
Niu, W., Sanim, M. M. R., Shu, Z., Guan, J., Shen, X., Yin, M., Agrawal, G., & Ren, B. (2024) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3620666.3651384 · Cited by 19
Multimodal transformer for early alarm prediction
Strem, N., Dhami, D. S., Schmidt, B., & Kersting, K. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109643 · Cited by 12
ISCDFuse: Interval sampling correlation driven visual state space models for multimodal image fusion
Zhang, L., Wang, L., Wu, Y., Chen, M., Zheng, D., & Cai, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130329 · Cited by 11
Self-citation
Boosting UAV Tracking With Voxel-Based Trajectory-Aware Pre-Training
Li, S., Fu, C., Lu, K., Zuo, H., Li, Y., & Feng, C. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3236583 · Cited by 11
Hyperscale Hardware Optimized Neural Architecture Search
Li, S., Andersen, G., Chen, T., Cheng, L., Grady, J., Huang, D., Le, Q. V., Li, A., Li, X. S., Li, Y., et al. (2023) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3582016.3582049 · Cited by 11
Anomaly Detection in Time Series: Current Focus and Future Challenges
Arslan, F., Javaid, A., Awan, M. D. Z., & Ebad-ur-Rehman (2023) · Artificial intelligence · doi:10.5772/intechopen.111886 · Cited by 13
CO-Net: Learning Multiple Point Cloud Tasks at Once with A Cohesive Network
Xie, T., Wang, K., Lu, S., Zhang, Y., Dai, K., Li, X., Xu, J., Wang, K., Zhao, L., Zhang, X., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00326 · Cited by 11
Hardware-friendly compression and hardware acceleration for transformer: A survey
Huang, S., Tang, E., Li, S., Ping, X., & Chen, R. (2022) · Electronic Research Archive · doi:10.3934/era.2022192 · Cited by 14
ATNAS: Automatic Termination for Neural Architecture Search
Sakamoto, K., Ishibashi, H., Sato, R., Shirakawa, S., Akimoto, Y., & Hino, H. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.07.011 · Cited by 10
Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
Li, Z., Lu, A., Xie, Y., Kong, Z., Sun, M., Tang, H., Xue, Z. J., Dong, P., Ding, C., Wang, Y., et al. (2024) · International Conference on Supercomputing · doi:10.1145/3650200.3656622 · Cited by 12
SceneFormer: Neural Architecture Search of Transformers for Remote Sensing Scene Classification
Tong, L., Liu, J., & Du, B. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3555207 · Cited by 14
AnchorFormer: Differentiable anchor attention for efficient vision transformer
Shan, J., Wang, J., Zhao, L., Cai, L., Zhang, H., & Liritzis, I. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.07.016 · Cited by 18
A dual quantum image feature extraction method: PSQIFE
Su, J., Lu, S., & Li, L. (2022) · IET Image Processing · doi:10.1049/ipr2.12561 · Cited by 10
Expediting Large-Scale Vision Transformer for Dense Prediction Without Fine-Tuning
Yuan, Y., Liang, W., Ding, H., Liang, Z., Zhang, C., & Hu, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327511 · Cited by 14
Dense Vision Transformer Compression with Few Samples
Zhang, H., Zhou, Y., & Wang, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01498 · Cited by 9
DTSNet: Dynamic Transformer Slimming for Efficient Vision Recognition
Xiao, W., Li, X., Hu, L., Hao, Y., & Chen, M. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3607796 · Cited by 9
Self-citation
ShiftNAS: Improving One-shot NAS via Probability Shift
Zhang, M., Yu, X., Zhao, H., & Ou, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00544 · Cited by 9
Multi-tailed vision transformer for efficient inference
Wang, Y., Du, B., Wang, W., & Xu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106235 · Cited by 14
TVT: Training-Free Vision Transformer Search on Tiny Datasets
Wei, Z., Pan, H., Li, L., Dong, P., & Li, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78169-8_24 · Cited by 9
Driver Drowsiness Detection Using Vision Transformer
Azmi, M. M. B. M., & Zaman, F. H. K. (2024) · IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE) · doi:10.1109/iscaie61308.2024.10576317 · Cited by 8
Efficient feature selection for pre-trained vision transformers
Huang, L., Zeng, J., Yu, M., Ding, W., Bai, X., & Wang, K. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104326 · Cited by 8
Efficient perturbation-aware distinguishing score for zero-shot neural architecture search
Huang, J., Xue, B., Sun, Y., Zhang, M., & Yen, G. G. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113447 · Cited by 8
Auto-Points: Automatic Learning for Point Cloud Analysis With Neural Architecture Search
Wang, K., Xie, T., Zhang, X., Jiang, Z., Yang, L., Zhang, H., Li, X., Ren, Y., Yu, H., Li, J., et al. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3304892 · Cited by 8
Enhancing Anomaly Detection for Cultural Heritage via Long Short-Term Memory with Attention Mechanism
Wu, Y., Dong, Y., Shan, Z., Meng, X., He, Y., Jia, P., & Lu, D. (2024) · Electronics · doi:10.3390/electronics13071254 · Cited by 9
GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion
Zhao, G., Hu, Z., Feng, S., Wang, Z., & Wu, H. Y. (2024) · Remote Sensing · doi:10.3390/rs16173246 · Cited by 8
Solutions of Feature and Hyperparameter Model Selection in the Intelligent Manufacturing
Wang, C.-Y., Huang, C.-Y., & Chiang, Y.-H. (2022) · Processes · doi:10.3390/pr10050862 · Cited by 8
SuperTickets: Drawing Task-Agnostic Lottery Tickets from Supernets via Jointly Architecture Searching and Parameter Pruning
You, H., Li, B., Sun, Z., Ouyang, X., & Lin, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_40 · Cited by 8
Dense Attention: A Densely Connected Attention Mechanism for Vision Transformer
Li, N., Chen, Y., & Zhao, D. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191462 · Cited by 8
Transformer‐Enhanced Traffic Load Simulation for Wear Evaluation of Bridge Expansion Joint
Dong, Y., Pan, Y., Wang, D., & Chen, A. (2024) · Structural Control and Health Monitoring · doi:10.1155/2024/6631877 · Cited by 8
Progressive Multi-Task Anti-Noise Learning and Distilling Frameworks for Fine-Grained Vehicle Recognition
Liu, D. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3420151 · Cited by 11
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
Xu, X., Li, C., Chen, Y., Chang, X., Liu, J., & Wang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8388-9_3 · Cited by 11
FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer
Chang, C.-C., Sung, Y.-Y., Yu, S., Huang, N.-C., Marculescu, D., & Wu, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00247 · Cited by 7
Parameter-less Pareto local search for multi-objective neural architecture search with the Interleaved Multi-start Scheme
Phan, Q. M., & Luong, N. H. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101573 · Cited by 7
RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation
Nath, U., Wang, Y.-C., Turaga, P. K., & Yang, Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02133-4 · Cited by 7
PRX-Change: Enhancing remote sensing change detection through progressive feature refinement and Cross-Attention interaction
Tan, H., He, L., Du, W., Liu, H., Chen, H., Zhang, Y., & Yang, H. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104008 · Cited by 7
DRL-ECG-HF: Deep reinforcement learning for enhanced automated diagnosis of heart failure with imbalanced ECG data
Zhao, B., Gao, Z., Liu, X., Zhang, Z., Xiao, W., & Zhang, S. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107680 · Cited by 7
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets
Guo, Y. F., Chen, Y., Zheng, Y., Chen, Q., Zhao, P., Huang, J., Chen, J., & Tan, M. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00219 · Cited by 7
Dynamic Partitioning of Graphs Based on Multivariate Blood Glucose Data—A Graph Neural Network Model for Diabetes Prediction
Li, J., Jiang, X., & Wang, K. (2024) · Electronics · doi:10.3390/electronics13183727 · Cited by 6
A Multi-Step Furnace Temperature Prediction Model for Regenerative Aluminum Smelting Based on Reversible Instance Normalization-Convolutional Neural Network-Transformer
Dai, J., Ling, P., Shi, H., & Liu, H. (2024) · Processes · doi:10.3390/pr12112438 · Cited by 6
A transformer-based architecture for wave height forecasting within rectangular moonpool and its generalization performance study
Wang, X., Ma, N., Gu, X., & Shi, Q. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.120313 · Cited by 6
BoostedDim attention: A novel data-driven approach to improving LiDAR-based lane detection
Patil, O., Nair, B. B., Soni, R., Thayyilravi, A., & Manoj, C. R. (2024) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2024.102887 · Cited by 6
Multi-indicator based multi-objective evolutionary algorithm with application to neural architecture search
Ajani, O. S., Darlan, D., Ivan, D. F., & Mallipeddi, R. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02300-6 · Cited by 6
Data Level Lottery Ticket Hypothesis for Vision Transformers
Shen, X., Kong, Z., Qin, M., Dong, P., Yuan, G., Meng, X., Tang, H., Ma, X., & Wang, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/153 · Cited by 10
Sweet Gradient matters: Designing consistent and efficient estimator for Zero-shot Architecture Search
Yang, L., Fu, Y.-X., Lu, S.-N., Sun, Z.-H., Mei, J.-L., Zhao, W.-X., & Hu, Y. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.012 · Cited by 8
Automated Dominative Subspace Mining for Efficient Neural Architecture Search
Chen, Y., Guo, Y. F., Liao, D., Lv, F., Song, H., Kwok, J. T.-Y., & Tan, M. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3395463 · Cited by 11
A Knowledge Transfer-Based Personalized Human–Robot Interaction Control Method for Lower Limb Exoskeletons
Yang, M., Tian, D., Li, F., Chen, Z., Zhu, Y., Shang, W., Zhang, L., & Wu, X. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3479239 · Cited by 7
Ant colony optimization of hyper-parameters in multi-head attention layer for time series forecasting
Aliyari, M., Droguett, E. L., Barabady, J., & Ayele, Y. Z. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113562 · Cited by 6
Toward Better Accuracy-Efficiency Tradeoffs for Oriented SAR Ship Object Detection
Ju, M., Niu, B., Li, M., Mao, T., & Jin, S.-N. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3555330 · Cited by 5
BossNAS Family: Block-Wisely Self-Supervised Neural Architecture Search
Li, C., Lin, S., Tang, T., Wang, G., Li, M., Liang, X., & Chang, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3529517 · Cited by 8
A transformer-based deep neural network long-term forecasting of solar irradiance and temperature: A multivariate multistep multitarget approach
Baghaei, I., Shirazizadeh, A., Ahmadi, R., Zahedi, A., & Mirhosseini, M. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114517 · Cited by 5
Anomaly Detection - Recent Advances, AI and ML Perspectives and Applications
Krishna Parimala, V. (2023) · Artificial intelligence · doi:10.5772/intechopen.110988 · Cited by 5
Hierarchical Joint Graph Learning and Multivariate Time Series Forecasting
Kim, J., Lee, H., Yu, S., Hwang, U., Jung, W., & Yoon, K. (2023) · IEEE Access · doi:10.1109/access.2023.3325041 · Cited by 6
DNA Family: Boosting Weight-Sharing NAS With Block-Wise Supervisions
Wang, G., Li, C., Yuan, L., Peng, J., Xian, X., Liang, X., Chang, X., & Lin, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3335261 · Cited by 6
PHD-NAS: Preserving helpful data to promote Neural Architecture Search
Lu, S., Hu, Y. H., Yang, L., Mei, J., Sun, Z., Tan, J., & Song, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127646 · Cited by 5
Vision Transformers Inference Acceleration Based on Adaptive Layer Normalization
Keddous, F. E., Llanza, A., Shvai, N., & Nakib, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2024.128524 · Cited by 9
USH: an efficient real-time distracted driving detection model
Wang, H., & Li, Y. (2025) · Journal of Real-Time Image Processing · doi:10.1007/s11554-025-01745-4 · Cited by 5
Improving Vision Transformers with Nested Multi-head Attentions
Peng, J., Li, C., Zhao, Y., Lin, Y., Fang, X., & Gong, J. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00330 · Cited by 5
Fairness-Aware Vision Transformer via Debiased Self-Attention
Qiang, Y., Li, C., Khanduri, P., & Zhu, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_20 · Cited by 8
PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference
Li, Y., Tang, C., Meng, Y., Fan, J., Chai, Z., Ma, X., Wang, Z., & Zhu, W. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3605239 · Cited by 4
Integrating communication networks with reinforcement learning and big data analytics for optimizing carbon capture and utilization strategies
Li, A., Liu, R., & Yi, S. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.08.100 · Cited by 4
Deployable mixed-precision quantization with co-learning and one-time search
Wang, S., Zhang, Z., Ai, G., & Cheng, J. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106812 · Cited by 4
LLAFN-Generator: Learnable linear-attention with fast-normalization for large-scale image captioning
Yang, X., Tian, X., Wu, J., Yang, X., Ma, S., Qi, X., & Hou, Z. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104088 · Cited by 4
DeepClair: Utilizing Market Forecasts for Effective Portfolio Selection
Choi, D., Kim, J., Gim, M., Lee, J.-H., & Kang, J. (2024) · International Conference on Information and Knowledge Management · doi:10.1145/3627673.3680008 · Cited by 6
Adaptive search for broad attention based vision transformers
Li, N., Chen, Y., & Zhao, D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128696 · Cited by 4
GCPNet: Gradient-aware channel pruning network with bilateral coupled sampling strategy
Zhang, Z., Cao, C., Zheng, F., Sun, T., & Zhao, L.-J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126104 · Cited by 4
3D neural architecture search to optimize segmentation of plant parts
Saeed, F., Tan, C., Liu, T., & Li, C. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.100776 · Cited by 4
AutoViT: Achieving Real-Time Vision Transformers on Mobile via Latency-aware Coarse-to-Fine Search
Kong, Z., Xu, D., Li, Z., Dong, P., Tang, H., Wang, Y., & Mukherjee, S. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02480-w · Cited by 5
Remaining Useful Life Prediction of SiC MOSFETs Using the Autoformer-RELM Model
Luo, W., Liu, Y., Pan, Y., & Bai, L. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3579735 · Cited by 7
Time-Series Forecasting Through Contrastive Learning with a Two-Dimensional Self-attention Mechanism
Jiang, L., Zhang, F., Zhang, M., & Zhang, C. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8082-6_12 · Cited by 4
Traffic flow prediction method for the intelligent marine meteorological sensor network
Hou, T., Xing, H., Gu, W., Liang, X., Wang, X., & Liu, Y. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.118296 · Cited by 4
SDRM: A truncated SVD-based dimensionality reduction approach to efficient edge inference in Multi-head Attention Networks
Du, Y., Wang, X., Chen, H., Zhang, C., Cao, J., & Buyya, R. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113946 · Cited by 3
Surrogate-assisted multi-objective covariance matrix adaptation evolution strategies
Ajani, O. S., Adedigba, A. P., Veluvolu, K. C., & Mallipeddi, R. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113728 · Cited by 3
Self-supervised transformer architecture search via multi-level masked information fusion
Yan, C., Chang, X., Zhang, W., Li, Z., Yao, L., Luo, M., & Feng, T. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103640 · Cited by 3
Deep Learning Strategies for Semantic Segmentation in Robot-Assisted Radical Prostatectomy
Sibilano, E., Delprete, C., Marvulli, P. M., Brunetti, A., Marino, F., Lucarelli, G. T., Battaglia, M., & Bevilacqua, V. (2025) · Applied Sciences · doi:10.3390/app151910665 · Cited by 3
LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting
Chuang, Y.-N., Li, S., Yuan, J., Wang, G., Lai, K.-H., Sui, S., Yu, L., Ding, S., Chang, C.-Y., Reyes, A. C., et al. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3787470.3787475 · Cited by 16
Non-stationary multi-scale prediction model based on Patch Time Series Transformer for multi-step coal price forecasting
Ding, S., Du, B., Sun, K., Xu, J., Ning, H., & Liu, Z. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114184 · Cited by 3
Neural Architecture Search-Based Global–Local Vision Mamba for Palm-Vein Recognition
Qin, H., Fu, Y., Jing, C., El‐Yacoubi, M. A., Gao, X., & Xi, F. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3679936 · Cited by 3
A novel transformer-based method for predicting air absorbed dose rates in nuclear radiation environmental monitoring
Cao, Y., Liu, Z., Niu, Y., & Liu, X. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19870 · Cited by 3
NAS-SE: Designing A Highly-Efficient In-Situ Neural Architecture Search Engine for Large-Scale Deployment
Wan, Q., Wang, L., Wang, J., Song, S. L., & Fu, X. (2023) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3613424.3614265 · Cited by 3
Comparison between transformer, informer, autoformer and non-stationary transformer in financial market
Wu, Y. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/29/20230874 · Cited by 3
On Efficient Object-Detection NAS for ADAS on Edge devices
Gupta, D., Lee, R. D., & Wynter, L. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00183 · Cited by 4
MPNAS: Multimodal Sentiment Analysis Pruning via Neural Architecture Search
Zhang, B., Ren, A., Zhang, Z., Duan, M., Liu, D., Tan, Y., & Zhong, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887670 · Cited by 3
ACL: Adaptive Edge-Cloud Collaborative Learning for Heterogeneous Devices With Unlabeled Local Data
Zhang, Z., Zhao, D. Y., Liu, R., Yao, Y., Li, X., & Ma, H. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3553971 · Cited by 5
Kernel-Level Energy-Efficient Neural Architecture Search for Tabular Dataset
La, H.-L., & Ha, P. H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6005-6_12 · Cited by 3
Vision Generalist Model: A Survey
Wang, Z., Rao, Y., Sun, S., Liu, X., Wei, Y., Yu, X., Liu, Z., Wang, Y., Liu, H., Zhou, J. N., et al. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02502-7 · Cited by 3
ConvDarts: a fast and exact convolutional algorithm selector for deep learning frameworks
Bai, L., Ji, W., Li, Q., Yao, X., Wei, X., & Zhu, W. (2023) · CCF Transactions on High Performance Computing · doi:10.1007/s42514-023-00167-7 · Cited by 3
Detection of Basal Cell Carcinoma in Whole Slide Images
Xu, H., Wang, D., Sowmya, A., & Katz, I. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43987-2_26 · Cited by 3
Enhancing spatiotemporal predictive learning: an approach with nested attention module
Wang, S., & Ren, H. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02318-7 · Cited by 3
Long-term wind power forecasting with series decomposition and spatio-temporal graph neural network
Yang, Y., Chen, J., Zheng, W., & Zhang, F. (2024) · International Journal of Green Energy · doi:10.1080/15435075.2024.2382351 · Cited by 3
Wind Farm Meteorological Prediction Model Based on Frequency Domain Feature Extraction Fusion Mechanism
Liao, Y., Gao, Z., & Li, X. (2024) · IEEE Access · doi:10.1109/access.2024.3450569 · Cited by 3
GRU-TSMixers: Sleep Apnea and Hypopnea Detection Based on Multi Scale MLP-Mixers
Huang, Z., & He, K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650636 · Cited by 3
Temporal Forecasting for IT Power Demand of Data Center
Wang, Y. D., Yan, Y., Liu, W., Chen, S., Wang, Z., Ding, Z., & Wei, X. (2024) · IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia) · doi:10.1109/icpsasia61913.2024.10761478 · Cited by 3

Authors: Chen, M., Peng, H., Fu, J., & Ling, H.
Affiliations: Stony Brook University (US)
Microsoft Research Asia (China) (CN)
ORCID: Chen, Minghao
Peng, Houwen
Fu, Jianlong
Ling, Haibin
Venue: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2021-10
Pages: 12250-12260
DOI: 10.1109/iccv48922.2021.01205
OpenAlex: W3204801262
Cited by: 323
References: 78
Citations by year: 2026: 28 · 2025: 75 · 2024: 103 · 2023: 79 · 2022: 33 · 2021: 5
FWCI: 17.99
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords: vision transformer, image classification, transferability, knowledge distillation, Autoformer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

AutoFormer: Searching Transformers for Visual Recognition

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks