Knowledge Distillation in Object Detection: A Survey from CNN to Transformer
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Deep learning models, especially for object detection have gained immense popularity in computer vision. These models have demonstrated remarkable accuracy and performance, driving advancements across various applications. However, the high computational complexity and large storage requirements of state-of-the-art object detection models pose significant challenges for deployment on resource-constrained devices like mobile phones and embedded systems. Knowledge Distillation (KD) has emerged as a prominent solution to these challenges, effectively compressing large, complex teacher models into smaller, efficient student models. This technique maintains good accuracy while significantly reducing model size and computational demands, making object detection models more practical for real-world applications. This survey provides a comprehensive review of KD-based object detection models developed in recent years. It offers an in-depth analysis of existing techniques, highlighting their novelty and limitations, and explores future research directions. The survey covers the different distillation algorithms used in object detection. It also examines extended applications of knowledge distillation in object detection, such as improvements for lightweight models, addressing catastrophic forgetting in incremental learning, and enhancing small object detection. Furthermore, the survey also delves into the application of knowledge distillation in other domains such as image classification, semantic segmentation, 3D reconstruction, and document analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Subsequently, one-stage detectors such as YOLO [46] and SSD [48] prioritized real-time performance, while enhancements like Feature Pyramid Networks (FPNs) [45] improved multi-scale detection.”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“The field has advanced significantly, evolving from classical methods to modern deep learning-based approaches [42–61].”
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 1×
“Subsequently, one-stage detectors such as YOLO [46] and SSD [48] prioritized real-time performance, while enhancements like Feature Pyramid Networks (FPNs) [45] improved multi-scale detection.”
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · cited 1×
“Extensions such as Mask R-CNN [49] added instance segmentation capabilities, and EfficientDet [44] further optimized accuracy–efficiency trade-offs through compound scaling.”
Single-Shot Refinement Neural Network for Object Detection
Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S. Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00442 · Cited by 1,678
Hybrid Task Cascade for Instance Segmentation
Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00511 · Cited by 1,587
RepPoints: Point Set Representation for Object Detection
Yang, Z., Liu, S., Hu, H., Wang, L., & Lin, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00975 · Cited by 1,307 · cited 1×
“In Scale-aware Knowledge Distillation (ScaleKD) [182], the authors introduce a framework designed to improve the performance of small object detectors [43,47,52,183,184] through scale-aware knowledge distillation.”
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224
Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018) · arXiv · doi:10.48550/arxiv.1805.04770 · Cited by 1,212
Deepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks
Wang, S., Clark, R., Wen, H., & Trigoni, N. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989236 · Cited by 970
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
A Comprehensive Overhaul of Feature Distillation
Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., & Choi, J. Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00201 · Cited by 706
Bounding Box Regression With Uncertainty for Accurate Object Detection
He, Y., Zhu, C., Wang, J., Savvides, M., & Zhang, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00300 · Cited by 623
Continual learning of context-dependent processing in neural networks
Zeng, G., Chen, Y., Cui, B., & Yu, S. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0080-x · Cited by 309 · cited 1×
“Kang et al. [194] addresses the issue of catastrophic forgetting in incremental learning [195]—specifically object detection as little work has been done in this domain [117,196]—by proposing a method that uses knowledge distillation both within class as well as between class.”
Localization Distillation for Dense Object Detection
Zheng, Z., Ye, R., Wang, P., Ren, D., Zuo, W., Hou, Q., & Cheng, M.-M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00919 · Cited by 220 · cited 4×
“…ICD[118] Thisapproachhashighcomplexityandcomputationalcost,especiallyduringtraining LD[172] Improvesdenseobjectdetectionbytransferringfine-grainedlocalizationknowledge…”
InfluentialTowards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
Allen-Zhu, Z., & Li, Y. (2020) · arXiv · doi:10.48550/arxiv.2012.09816 · Cited by 178 · cited 1×
“We examine different distillation algorithms, including response-based [1,71–74], feature-based [75–89], relation-based [90–102], task-specific [103,104], and self-distillation [90,105].”
Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation
Feng, T., Wang, M., & Yuan, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00921 · Cited by 132
Rocket Launching: A Universal and Efficient Framework for Training Well-Performing Light Net
Zhou, G., Fan, Y., Cui, R., Bian, W., Zhu, X., & Gai, K. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11601 · Cited by 121
Li, D., Tasci, S., Ghosh, S., Zhu, J., Zhang, J., & Heck, L. (2019) · Proceedings of the 4th ACM/IEEE Symposium on Edge Computing · doi:10.1145/3318216.3363317 · Cited by 94
Object Detection with Transformers: A Review
Shehzadi, T., Hashmi, K. A., Liwicki, M., Stricker, D., & Afzal, M. Z. (2025) · Sensors · doi:10.3390/s25196025 · Cited by 93 · cited 1×
“DETRDistill [198] introduces a method to enhance transformer-based object detectors (DETRs) [199,200] through knowledge distillation, as shown in Figure 27.”
Self-citationTraining Object Detectors With Noisy Data
Chadwick, S., & Newman, P. (2019) · 2019 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2019.8814137 · Cited by 46
Research works printed but not linked: 196
- Hinton, G.E., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv.
- Sauer, A., Asaadi, S., and Küch, F. (2022, January 27). Knowledge distillation meets few-shot learning: An approach for few-shot intent classification within and across domains. Proceedings of the 4th Workshop on NLP for Conversational AI, Dublin, Ireland. · DOI
- Alkhulaifi, 2021, Knowledge distillation in deep learning and its applications, PeerJ Comput. Sci., 7, e474, 10.7717/peerj-cs.474 · DOI · Paper page
- Kothandaraman, D., and Nambiar, A. (2021, January 5–9). Domain adaptive knowledge distillation for driving scene semantic segmentation. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), Waikola, HI, USA. · DOI
- Ni, Y., Zhang, X., and Li, Y. (2023). Knowledge Distillation Facilitates the Lightweight and Efficient Plant Diseases Detection Model. Agriculture, 13. · DOI
- Ruan, 2022, An Incremental Learning of YOLOv3 Without Catastrophic Forgetting for Smart City Applications, IEEE Consum. Electron. Mag., 10, 56
- Pham, C., Hoang, T., and Do, T.T. (2023, January 7). Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA. · DOI
- Zhang, H., Chen, D., and Wang, C. (2023, January 10–14). Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning. Proceedings of the 2023 IEEE International Conference on Multimedia and Expo (ICME), Brisbane, Australia. · DOI
- Zhang, H., Chen, D., and Wang, C. (2022, January 23–27). Confidence-Aware Multi-Teacher Knowledge Distillation. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore. · DOI
- Yuan, F., Shou, L., Pei, J., Lin, W., Gong, M., Fu, Y., and Jiang, D. (2021, January 2–9). Reinforced Multi-Teacher Selection for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada. · DOI
- Larochelle, 2020, Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space, Advances in Neural Information Processing Systems, Volume 33, 12345
- Liu, 2020, Adaptive multi-teacher multi-level knowledge distillation, Neurocomputing, 415, 106, 10.1016/j.neucom.2020.07.048 · DOI · Paper page
- Li, L., Dong, P., Wei, Z., and Yang, Y. (2023, January 1–6). Automated Knowledge Distillation via Monte Carlo Tree Search. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., and Wang, J. (2019, January 15–20). Structured Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Liu, 2023, Structured Knowledge Distillation for Dense Prediction, IEEE Trans. Pattern Anal. Mach. Intell., 45, 7035, 10.1109/TPAMI.2020.3001940 · DOI · Paper page
- Yang, C., Zhou, H., An, Z., Jiang, X., Xu, Y., and Zhang, Q. (2022, January 19–24). Cross-Image Relational Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI · Paper page
- Yang, Z., Li, Z., Shao, M., Shi, D., Yuan, Z., and Yuan, C. (2022, January 23–27). Masked Generative Distillation. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel. · DOI · Paper page
- Shu, C., Liu, Y., Gao, J., Yan, Z., and Shen, C. (2021, January 11–17). Channel-Wise Knowledge Distillation for Dense Prediction. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada. · DOI · Paper page
- Chen, J., Zhu, D., Qian, G., Ghanem, B., Yan, Z., Zhu, C., Xiao, F., Culatana, S.C., and Elhoseiny, M. (2023, January 1–6). Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Liu, L., Wang, Z., Phan, M.H., Zhang, B., Ge, J., and Liu, Y. (2024, January 7–11). BPKD: Boundary Privileged Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii. · DOI
- Wang, Y., Zhou, W., Jiang, T., Bai, X., and Xu, Y. (2020, January 23–28). Intra-class Feature Variation Distillation for Semantic Segmentation. Proceedings of the Computer Vision—ECCV Cham, Switzerland. · DOI · Paper page
- Lin, S., Xie, H., Wang, B., Yu, K., Chang, X., Liang, X., and Wang, G. (2022, January 19–24). Knowledge Distillation via the Target-Aware Transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI
- Berrada, T., Couprie, C., Alahari, K., and Verbeek, J. (2024, January 7–11). Guided Distillation for Semi-Supervised Instance Segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii. · DOI
- Xu, Y., Yang, Y., and Zhang, L. (2023, January 1–6). Multi-Task Learning with Knowledge Distillation for Dense Prediction. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Yang, Z., Li, R., Ling, E., Zhang, C., Wang, Y., Huang, D., Ma, K.T., Hur, M., and Lin, G. (2023, January 1–6). Label-Guided Knowledge Distillation for Continual Semantic Segmentation on 2D Images and 3D Point Clouds. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Ji, D., Wang, H., Tao, M., Huang, J., Hua, X.S., and Lu, H. (2022, January 19–24). Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI
- Phan, M.H., Ta, T.A., Phung, S.L., Tran-Thanh, L., and Bouzerdoum, A. (2022, January 19–24). Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI
- Oh, 2023, Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation, Advances in Neural Information Processing Systems, Volume 36, 51359
- Koyejo, 2022, Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation, Advances in Neural Information Processing Systems, Volume 35, 10380
- Lin, J., Yuan, Y., Shao, T., and Zhou, K. (2020, January 13–19). Towards High-Fidelity 3D Face Reconstruction From In-the-Wild Images Using Graph Convolutional Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. · DOI
- Jackson, A.S., Bulat, A., Argyriou, V., and Tzimiropoulos, G. (2017, January 22–29). Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. · DOI · Paper page
- Zheng, H., Cao, D., Xu, J., Ai, R., Gu, W., Yang, Y., and Liang, Y. (2024). Distilling Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection. arXiv.
- Wang, 2023, IFKD: Implicit field knowledge distillation for single view reconstruction, Math. Biosci. Eng., 20, 13864, 10.3934/mbe.2023617 · DOI
- Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. Towards Efficient 3D Object Detection with Knowledge Distillation. Advances in Neural Information Processing Systems, Curran Associates, Inc.. 28 November–9 December 2022.
- Zhang, L., Dong, R., Tai, H.S., and Ma, K. (2023, January 19–24). PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada. · DOI
- Landeghem, J.V., Maity, S., Banerjee, A., Blaschko, M.B., Moens, M.F., Llad’os, J., and Biswas, S. (2024). DistilDoc: Knowledge Distillation for Visually-Rich Document Applications. Document Analysis and Recognition—ICDAR 2024: 18th International Conference, Athens, Greece, 30 August–4 September 2024, Springer.
- Banerjee, A., Biswas, S., Llad’ós, J., and Pal, U. (2024). GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation. Document Analysis and Recognition—ICDAR 2024: 18th International Conference, Athens, Greece, 30 August–4 September 2024, Springer. · DOI
- Chen, X., He, B., Hui, K., Sun, L., and Sun, Y. (2020). Simplified TinyBERT: Knowledge Distillation for Document Retrieval. arXiv. · DOI
- Fink, G.A., Jain, R., Kise, K., and Zanibbi, R. (2023). Multi-teacher Knowledge Distillation for End-to-End Text Image Machine Translation. Proceedings of the Document Analysis and Recognition—ICDAR 2023, Springer.
- Elhanashi, A., Dini, P., Saponara, S., and Zheng, Q. (2023). Integration of Deep Learning into the IoT: A Survey of Techniques and Challenges for Real-World Applications. Electronics, 12. · DOI
- Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
- Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy. · DOI · Paper page
- Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. Lecture Notes in Computer Science, Springer International Publishing. · DOI · Paper page
- He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2018). Mask R-CNN. arXiv, Available online: http://arxiv.org/abs/1703.06870. · arXiv
- Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv, Available online: http://arxiv.org/abs/2010.11929. · arXiv
- Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.-M. (2020). End-to-End Object Detection with Transformers. Computer Vision—ECCV 2020, Springer. · doi:10.1007/978-3-030-58542-6
- Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea. · DOI · Paper page
- Shehzadi, T., Hashmi, K.A., Sarode, S., Stricker, D., and Afzal, M.Z. (2025, January 12–19). STEP-DETR: Advancing DETR-based Semi-Supervised Object Detection with Super Teacher and Pseudo-Label Guided Text Queries. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA.
- Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv. · DOI · Paper page
- Law, H., and Deng, J. (2018, January 8–14). CornerNet: Detecting Objects as Paired Keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany. · DOI · Paper page
- Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
- Li, Y., Chen, Y., Wang, N., and Zhang, Z. (November, January 27). Scale-Aware Trident Networks for Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
- Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2021). Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv.
- Vincent, O.R., and Folorunso, O. (2009, January 12–15). A Descriptive Algorithm for Sobel Image Edge Detection. Proceedings of the InSITE 2009: Informing Science + IT Education Conference, Macon, GA, USA. · DOI · Paper page
- Canny, 1986, A Computational Approach To Edge Detection, IEEE Trans. Pattern Anal. Mach. Intell., PAMI-8, 679, 10.1109/TPAMI.1986.4767851 · DOI · Paper page
- Harris, C.G., and Stephens, M.J. (1988, January 1–3). A Combined Corner and Edge Detector. Proceedings of the Alvey Vision Conference, London, UK. · DOI · Paper page
- Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
- Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile. · DOI · Paper page
- Campos, 2021, Optimized convolutional neural network architectures for efficient on-device vision-based object detection, Neural Comput. Appl., 34, 10469
- Du, Z., Zhang, R., Chang, M.F., Zhang, X., Liu, S., Chen, T., and Chen, Y. (2021, January 6–14). Distilling Object Detectors with Feature Richness. Proceedings of the Neural Information Processing Systems, Red Hook, NY, USA.
- Wang, 2020, Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks, IEEE Trans. Pattern Anal. Mach. Intell., 44, 3048, 10.1109/TPAMI.2021.3055564 · DOI · Paper page
- Chen, G., Choi, W., Yu, X., Han, T.X., and Chandraker, M. (2017, January 4–9). Learning Efficient Object Detection Models with Knowledge Distillation. Proceedings of the Neural Information Processing Systems, Long Beach, CA, USA.
- Zhang, F., Zhu, X., and Ye, M. (2019, January 20). Fast Human Pose Estimation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Meng, Z., Li, J., Zhao, Y., and Gong, Y. (2019, January 12–17). Conditional Teacher-student Learning. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK. · DOI
- Kim, S., and Kim, H.E. (2017, January 24–26). Transferring Knowledge to Smaller Network with Class-Distance Loss. Proceedings of the International Conference on Learning Representations, Toulon, France.
- Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2015, January 7–9). FitNets: Hints for Thin Deep Nets. Proceedings of the ICLR, San Diego, CA, USA.
- Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21–26). Like What You Like: Knowledge Distill via Neuron Selectivity Transfer. Proceedings of the CVPR, Honolulu, HI, USA.
- Zagoruyko, S., and Komodakis, N. (2017, January 24–26). Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer. Proceedings of the ICLR, Toulon, France.
- Yim, J., Joo, D., Bae, J., and Kim, J. (2018, January 3–8). Paraphrasing Complex Network: Network Compression via Factor Transfer. Proceedings of the NIPS, Montreal, QC, Canada.
- Heo, B., Lee, M., Yun, S., and Choi, J.Y. (February, January 27). Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons. Proceedings of the AAAI, Honolulu, HI, USA.
- Liu, J., Wen, D., Gao, H., Tao, W., Chen, T.W., Osa, K., and Kato, M. (2019, January 16–20). Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA. · DOI
- Shen, 2019, Amalgamating Knowledge towards Comprehensive Classification, Proc. AAAI Conf. Artif. Intell., 33, 3068
- Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.M. (2020). Feature Normalized Knowledge Distillation for Image Classification. Proceedings of the Computer Vision—ECCV 2020, Springer. · doi:10.1007/978-3-030-58542-6
- Guan, Y., Zhao, P., Wang, B., Zhang, Y., Yao, C., Bian, K., and Tang, J. (2020). Differentiable Feature Aggregation Search for Knowledge Distillation. arXiv. · DOI
- Yang, J., Martínez, B., Bulat, A., and Tzimiropoulos, G. (2020). Knowledge distillation via adaptive instance normalization. arXiv.
- Wang, X., Fu, T., Liao, S., Wang, S., Lei, Z., and Mei, T. (2020, January 23–28). Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition. Proceedings of the European Conference on Computer Vision, Glasgow, UK. · DOI
- Passban, P., Wu, Y., Rezagholizadeh, M., and Liu, Q. (2020, January 7–12). ALP-KD: Attention-Based Layer Projection for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
- Chen, D., Mei, J.P., Zhang, Y., Wang, C., Wang, Z., Feng, Y., and Chen, C. (2020, January 7–12). Cross-Layer Distillation with Semantic Calibration. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
- Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21–26). A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. · DOI · Paper page
- You, S., Xu, C., Xu, C., and Tao, D. (2017, January 13–17). Learning from Multiple Teacher Networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada. · DOI · Paper page
- Zhang, C., and Peng, Y. (2018, January 13–19). Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden. · DOI · Paper page
- Chen, Y., Wang, N., and Zhang, Z. (2017). DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer. arXiv. · DOI · Paper page
- Lee, S., and Song, B.C. (2019, January 9–12). Graph-based Knowledge Distillation by Multi-head Attention Network. Proceedings of the British Machine Vision Conference, Cardiff, UK.
- Park, W., Kim, D., Lu, Y., and Cho, M. (2019, January 29). Relational Knowledge Distillation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Liu, Y., Cao, J., Li, B., Yuan, C., Hu, W., Li, Y., and feng Duan, Y. (2019, January 29). Knowledge Distillation via Instance Relationship Graph. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Tung, F., and Mori, G. (November, January 27). Similarity-Preserving Knowledge Distillation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea. · DOI · Paper page
- Peng, B., Jin, X., Liu, J., Zhou, S., Wu, Y., Liu, Y., Li, D., and Zhang, Z. (November, January 27). Correlation Congruence for Knowledge Distillation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea. · DOI · Paper page
- Yu, L., Yazici, V.O., Liu, X., van de Weijer, J., Cheng, Y., and Ramisa, A. (2019, January 15–20). Learning Metrics From Teachers: Compact Networks for Image Embedding. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI
- Passalis, 2020, Probabilistic Knowledge Transfer for Lightweight Deep Representation Learning, IEEE Trans. Neural Netw. Learn. Syst., 32, 2030, 10.1109/TNNLS.2020.2995884 · DOI · Paper page
- Passalis, N., Tzelepi, M., and Tefas, A. (2020, January 13–19). Heterogeneous Knowledge Distillation Using Information Flow Modeling. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. · DOI
- Chen, 2018, Learning Student Networks via Feature Embedding, IEEE Trans. Neural Netw. Learn. Syst., 32, 25, 10.1109/TNNLS.2020.2970494 · DOI
- Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. (2019). TinyBERT: Distilling BERT for Natural Language Understanding. arXiv. · DOI · Paper page
- Tang, R., Lu, Y., Liu, L., Mou, L., Vechtomova, O., and Lin, J.J. (2019). Distilling Task-Specific Knowledge from BERT into Simple Neural Networks. arXiv.
- Li, 2018, Learning without Forgetting, IEEE Trans. Pattern Anal. Mach. Intell., 40, 2935, 10.1109/TPAMI.2017.2773081 · DOI · Paper page
- Peng, Z., Li, Z., Zhang, J., Li, Y., Qi, G.J., and Tang, J. (November, January 27). Few-Shot Image Recognition with Knowledge Transfer. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea. · DOI · Paper page
- Bagherinezhad, H., Horton, M., Rastegari, M., and Farhadi, A. (2018). Label Refinery: Improving ImageNet Classification through Label Progression. arXiv.
- Chen, W.C., Chang, C.C., Lu, C.Y., and Lee, C.R. (2018, January 2–6). Knowledge Distillation with Feature Maps for Image Classification. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.
- Wang, 2019, DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation, IEEE Trans. Vis. Comput. Graph., 25, 2168, 10.1109/TVCG.2019.2903943 · DOI
- Mukherjee, P., Das, A., Bhunia, A.K., and Roy, P.P. (2018, January 7–10). Cogni-Net: Cognitive Feature Learning Through Deep Visual Perception. Proceedings of the 2019 IEEE International Conference on Image Processing, Athens, Greece. · DOI
- Zhu, M., Han, K., Zhang, C., Lin, J., and Wang, Y. (2019, January 12–17). Low-resolution Visual Recognition via Deep Feature Distillation. Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK. · DOI
- Rijk, P.D., Schneider, L., Cordts, M., and Gavrila, D. (December, January 28). Structural Knowledge Distillation for Object Detection. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA.
- Cao, W., Zhang, Y., Gao, J., Cheng, A., Cheng, K., and Cheng, J. (December, January 28). PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA.
- Zhao, B., Cui, Q., Song, R., Qiu, Y., and Liang, J. (2022, January 19–24). Decoupled Knowledge Distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI · Paper page
- Jin, Y., Wang, J., and Lin, D. (2023, January 19–24). Multi-Level Logit Distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada. · DOI
- Ranzato, 2021, Instance-Conditional Knowledge Distillation for Object Detection, Advances in Neural Information Processing Systems, Volume 34, 16468
- Oh, 2023, KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs, Advances in Neural Information Processing Systems, Volume 36, 69490
- Wang, L., Li, X., Liao, Y., Jiang, Z., Wu, J., Wang, F., Qian, C., and Liu, S. (2022, January 23–27). HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel. · DOI
- Dai, X., Jiang, Z., Wu, Z., Bao, Y., Wang, Z., Liu, S., and Zhou, E. (2021, January 20–25). General Instance Distillation for Object Detection. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI · Paper page
- Guo, J., Han, K., Wang, Y., Wu, H., Chen, X., Xu, C., and Xu, C. (2021, January 20–25). Distilling Object Detectors via Decoupled Features. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI · Paper page
- Wang, T., Yuan, L., Zhang, X., and Feng, J. (2019, January 15–20). Distilling Object Detectors with Fine-Grained Feature Imitation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Yang, Z., Li, Z., Jiang, X., Gong, Y., Yuan, Z., Zhao, D., and Yuan, C. (2022, January 19–24). Focal and Global Knowledge Distillation for Detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI · Paper page
- Zhang, L., and Ma, K. (2021, January 3–7). Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors. Proceedings of the International Conference on Learning Representations, Virtual Event.
- Wu, D., Chen, P., Yu, X., Li, G., Han, Z., and Jiao, J. (2023, January 1–6). Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Bilen, H., and Vedaldi, A. (2016, January 27–30). Weakly Supervised Deep Detection Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA. · DOI · Paper page
- Liu, C., Wang, K., Lu, H., Cao, Z., and Zhang, Z. (2022). Robust Object Detection with Inaccurate Bounding Boxes. arXiv, Available online: http://arxiv.org/abs/2207.09697. · DOI
- Chen, P., Yu, X., Han, X., Hassan, N., Wang, K., Li, J., Zhao, J., Shi, H., Han, Z., and Ye, Q. (2022, January 23–27). Point-to-Box Network for Accurate Object Detection via Single Point Supervision. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel. · DOI
- Li, J., Xiong, C., Socher, R., and Hoi, S.C.H. (2020). Towards Noise-resistant Object Detection with Noisy Annotations. arXiv.
- Mao, J., Yu, Q., Yamakata, Y., and Aizawa, K. (2021). Noisy Annotation Refinement for Object Detection. arXiv. · DOI
- Xu, 2021, Training Robust Object Detectors From Noisy Category Labels and Imprecise Bounding Boxes, IEEE Trans. Image Process., 30, 5782, 10.1109/TIP.2021.3085208 · DOI
- Ma, Z., Luo, G., Gao, J., Li, L., Chen, Y., Wang, S., Zhang, C., and Hu, W. (2022, January 19–24). Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.
- Zareian, A., Rosa, K.D., Hu, D.H., and Chang, S.F. (2021, January 20–25). Open-Vocabulary Object Detection Using Captions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI · Paper page
- Bansal, A., Sikka, K., Sharma, G., Chellappa, R., and Divakaran, A. (2018, January 8–14). Zero-Shot Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany. · DOI · Paper page
- Gu, X., Lin, T.Y., Kuo, W., and Cui, Y. (2022, January 25–29). Open-vocabulary Object Detection via Vision and Language Knowledge Distillation. Proceedings of the International Conference on Learning Representations, Virtual Event.
- Li, Z., Yao, L., Zhang, X., Wang, X., Kanhere, S.S., and Zhang, H. (27–1, January 27). Zero-Shot Object Detection with Textual Descriptions. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
- Zheng, Y., Huang, R., Han, C., Huang, X., and Cui, L. (December, January 30). Background Learnable Cascade for Zero-Shot Object Detection. Proceedings of the Asian Conference on Computer Vision (ACCV), Kyoto, Japan.
- Zheng, Y., Wu, J., Qin, Y., Zhang, F., and Cui, L. (2021, January 19–25). Zero-Shot Instance Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI
- Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning Transferable Visual Models From Natural Language Supervision. arXiv, Available online: http://arxiv.org/abs/2103.00020. · arXiv
- Zheng, 2023, Localization Distillation for Object Detection, IEEE Trans. Pattern Anal. Mach. Intell., 45, 10070, 10.1109/TPAMI.2023.3248583 · DOI
- Cho, J.H., and Hariharan, B. (November, January 27). On the Efficacy of Knowledge Distillation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.
- Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., and Ghasemzadeh, H. (February, January 27). Improved Knowledge Distillation via Teacher Assistant. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
- Yang, C., Xie, L., Su, C., and Yuille, A.L. (2019, January 16–20). Snapshot Distillation: Teacher-Student Optimization in One Generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. · DOI · Paper page
- Zhang, Y., Xiang, T., Hospedales, T.M., and Lu, H. (2017, January 21–26). Deep Mutual Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI · Paper page
- Pham, C., Nguyen, V.A., Le, T., Phung, D., Carneiro, G., and Do, T.T. (2024, January 4–8). Frequency Attention for Knowledge Distillation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA. · DOI
- Ji, M., Heo, B., and Park, S. (2021). Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching. arXiv. · DOI
- Shin, S., Lee, J., Lee, J., Yu, Y., and Lee, K. (2022, January 23–27). Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel. · DOI
- Heideman, 1985, Gauss and the history of the fast Fourier transform, Arch. Hist. Exact Sci., 34, 265, 10.1007/BF00348431 · DOI · Paper page
- Li, Q., Jin, S., and Yan, J. (2017, January 21–26). Mimicking Very Efficient Network for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. · DOI · Paper page
- Huang, J., and Guo, Z. (2023, January 1–10). Pixel-Wise Contrastive Distillation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Lee, S.H., Kim, D.H., and Song, B.C. (2018, January 8–14). Self-supervised Knowledge Distillation Using Singular Value Decomposition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany. · DOI
- Hadsell, R., Chopra, S., and LeCun, Y. (2006, January 17–22). Dimensionality Reduction by Learning an Invariant Mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA.
- He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13–19). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. · DOI · Paper page
- Koyejo, 2022, Knowledge Distillation from a Stronger Teacher, Advances in Neural Information Processing Systems, Volume 35, 33716
- Zhou, H., Song, L., Chen, J., Zhou, Y., Wang, G., Yuan, J., and Zhang, Q. (2021). Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective. arXiv.
- Cao, Y., Xu, J., Lin, S., Wei, F., and Hu, H. (November, January 27). GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea. · DOI · Paper page
- Lao, S., Song, G., Liu, B., Liu, Y., and Yang, Y. (2023, January 1–10). UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Lu, X., Li, Q., Li, B., and Yan, J. (2020). MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection. arXiv. · DOI
- Yao, L., Pi, R., Xu, H., Zhang, W., Li, Z., and Zhang, T. (2021, January 11–17). G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada. · DOI
- Oh, 2023, Knowledge Diffusion for Distillation, Advances in Neural Information Processing Systems, Volume 36, 65299
- Ho, J., Jain, A., and Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. arXiv.
- Song, J., Meng, C., and Ermon, S. (2021, January 3–7). Denoising Diffusion Implicit Models. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
- He, X., Zhao, K., and Chu, X. (2019). AutoML: A Survey of the State-of-the-Art. arXiv.
- Godlewski, 2022, Monte Carlo Tree Search: A review of recent modifications and applications, Artif. Intell. Rev., 56, 2497
- Larochelle, 2020, Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection, Advances in Neural Information Processing Systems, Volume 33, 21002
- Qiu, H., Li, H., Wu, Q., and Shi, H. (2020, January 13–19). Offset Bin Classification Network for Accurate Object Detection. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. · DOI
- Son, W., Na, J., Choi, J., and Hwang, W. (2021, January 11–17). Densely Guided Knowledge Distillation Using Multiple Teacher Assistants. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada. · DOI
- Yang, L., Zhou, X., Li, X., Qiao, L., Li, Z., Yang, Z., Wang, G., and Li, X. (2023, January 1–10). Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Guo, Q., Mu, Y., Chen, J., Wang, T., Yu, Y., and Luo, P. (2022, January 18–24). Scale-Equivalent Distillation for Semi-Supervised Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI
- Liu, Y.C., Ma, C.Y., He, Z., Kuo, C.W., Chen, K., Zhang, P., Wu, B., Kira, Z., and Vajda, P. (2021). Unbiased Teacher for Semi-Supervised Object Detection. arXiv, Available online: http://arxiv.org/abs/2102.09480. · arXiv
- Sohn, K., Zhang, Z., Li, C.L., Zhang, H., Lee, C.Y., and Pfister, T. (2020). A Simple Semi-Supervised Learning Framework for Object Detection. arXiv.
- Yang, Q., Wei, X., Wang, B., Hua, X.S., and Zhang, L. (2021, January 20–25). Interactive Self-Training with Mean Teachers for Semi-Supervised Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI
- Zhou, Q., Yu, C., Wang, Z., Qian, Q., and Li, H. (2021, January 20–25). Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. · DOI · Paper page
- Zhu, Y., Zhou, Q., Liu, N., Xu, Z., Ou, Z., Mou, X., and Tang, J. (2023, January 19–25). ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada. · DOI
- Cai, Z., and Vasconcelos, N. (2018, January 18–22). Cascade R-CNN: Delving Into High Quality Object Detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI · Paper page
- He, R., Sun, S., Yang, J., Bai, S., and Qi, X. (2022, January 18–24). Knowledge Distillation As Efficient Pre-Training: Faster Convergence, Higher Data-Efficiency, and Better Transferability. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI
- Joulin, A., Van Der Maaten, L., Jabri, A., and Vasilache, N. (2016, January 8–16). Learning Visual Features from Large Weakly Supervised Data. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands. · DOI · Paper page
- Kuznetsova, 2018, The Open Images Dataset V4, Int. J. Comput. Vis., 128, 1956, 10.1007/s11263-020-01316-z · DOI · Paper page
- Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22–29). Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy. · DOI · Paper page
- Klema, 1980, The singular value decomposition: Its computation and some applications, IEEE Trans. Autom. Control, 25, 164, 10.1109/TAC.1980.1102314 · DOI · Paper page
- Tomani, C., Cremers, D., and Buettner, F. (2022, January 23–27). Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel. · DOI
- Liu, Z., Hu, X., and Nevatia, R. (2024, January 1–5). Efficient Feature Distillation for Zero-Shot Annotation Object Detection. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA. · DOI
- Kang, M., Zhang, J., Zhang, J., Wang, X., Chen, Y., Ma, Z., and Huang, X. (2023, January 1–7). Alleviating Catastrophic Forgetting of Incremental Object Detection via Within-Class and Between-Class Knowledge Distillation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Menezes, 2023, Continual Object Detection: A Review of Definitions, Strategies, and Challenges, Neural Netw., 161, 476, 10.1016/j.neunet.2023.01.041 · DOI
- Wang, L., Liu, Y., Du, P., Ding, Z., Liao, Y., Qi, Q., Chen, B., and Liu, S. (2023, January 19–24). Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada. · DOI
- Chang, J., Wang, S., Xu, H.M., Chen, Z., Yang, C., and Zhao, F. (2023, January 1–7). DETRDistill: A Universal Knowledge Distillation Framework for DETR-families. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Shehzadi, T., Hashmi, K.A., Stricker, D., and Afzal, M.Z. (2024, January 17–22). Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vienna, Austria. · DOI
- Liu, Z., Liu, Q., Liu, T., Wang, Y., and Wen, W. (2018, January 18–22). Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA. · DOI · Paper page
- Li, L., Miao, J., Shi, D., Tan, W., Ren, Y., Yang, Y., and Pu, S. (2023, January 1–7). Distilling DETR with Visual-Linguistic Knowledge for Open-Vocabulary Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France. · DOI
- Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q.V., Sung, Y.H., Li, Z., and Duerig, T. (2021, January 18–24). Scaling Up Visual and Vision-Language Representation Learning with Noisy Text Supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), Virtual. Proceedings of Machine Learning Research.
- Shehzadi, T., Stricker, D., and Afzal, M.Z. (2024). Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer. arXiv.
- Yousaf, 2021, Protein active site prediction for early drug discovery and designing, Int. Rev. Appl. Sci. Eng., 13, 98
- Shehzadi, T., Majid, A., Hameed, M., Farooq, A., and Yousaf, A. (2020, January 12–14). Intelligent predictor using cancer-related biologically information extraction from cancer transcriptomes. Proceedings of the 2020 International Symposium on Recent Advances in Electrical Engineering & Computer Sciences (RAEE & CS), Islamabad, Pakistan. · DOI
- Shehzadi, T., Hashmi, K.A., Pagani, A., Liwicki, M., Stricker, D., and Afzal, M.Z. (2022). Mask-Aware Semi-Supervised Object Detection in Floor Plans. Appl. Sci., 12. · DOI
- Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 15–17). FD-SSD: Semi-supervised Detection of Bone Fenestration and Dehiscence in Intraoral Images. Proceedings of the Medical Image Understanding and Analysis (MIUA), Leeds, UK. · DOI
- Xie, Q., Hovy, E.H., Luong, M.T., and Le, Q.V. (2020, January 13–19). Self-Training with Noisy Student Improves ImageNet Classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA. · DOI · Paper page
- Zhu, Y., and Wang, Y. (2021, January 10–17). Student Customized Knowledge Distillation: Bridging the Gap Between Student and Teacher. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada. · DOI
- Beyer, L., Zhai, X., Royer, A., Markeeva, L., Anil, R., and Kolesnikov, A. (2022, January 18–24). Knowledge distillation: A good teacher is patient and consistent. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA. · DOI · Paper page
- Li, Z., Li, X., Yang, L., Zhao, B., Song, R., Luo, L., Li, J., and Yang, J. (2023, January 7–14). Curriculum Temperature for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.
- Qiu, Z., Ma, X., Yang, K., Liu, C., Hou, J., Yi, S., and Ouyang, W. (2023, January 1–5). Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
- Park, J., and No, A. (2022, January 23–27). Prune Your Model Before Distill It. Proceedings of the Computer Vision—ECCV 2022, Tel Aviv, Israel. · DOI
- Wang, X., Li, Y., Chen, Z., and Zhao, H. (2025, January 16–21). SemiTabDETR: End-to-End Semi-supervised Table Detection with Transformer-Based Enhanced Query Approach. Proceedings of the Document Analysis and Recognition—ICDAR, Wuhan, China.
- Shehzadi, T., Stricker, D., and Afzal, M.Z. (September, January 30). A Hybrid Approach for Document Layout Analysis in Document Images. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), Athens, Greece.
- Shehzadi, T., Hashmi, K.A., Stricker, D., Liwicki, M., and Afzal, M.Z. (2023, January 21–26). Towards End-to-End Semi-Supervised Table Detection with Deformable Transformer. Proceedings of the Document Analysis and Recognition—ICDAR 2023, San José, CA, USA. · DOI
- Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 19–23). DocSemi: Efficient Document Layout Analysis with Guided Queries. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA.
- Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 19–25). Efficient Additive Attention for Transformer-based Semi-supervised Document Layout Analysis. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA.
- Lin, T.Y., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland. · DOI · Paper page
Metadata
- Authors
- Shehzadi, T., Noor, R., Ifza, I., Liwicki, M., Stricker, D., & Afzal, M. Z.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2026-01-02
- Volume
- 26
- Issue
- 1
- Pages
- 292
- ISSN
- 1424-8220
- Cited by
- 9
- References
- 24
- Language
- en
- Funders
- European project AIRISE (101092312)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.