Residual error based knowledge distillation

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works

ImageNet classification with deep convolutional neural networks

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works

ImageNet: A large-scale hierarchical image database

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works

Densely Connected Convolutional Networks

Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works

Learning Multiple Layers of Features from Tiny Images

Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works

Distilling the Knowledge in a Neural Network

Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983 · see citing works

Towards Evaluating the Robustness of Neural Networks

Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870 · see citing works

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., & Loy, C. C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_5 · Cited by 5,085 · see citing works

A survey of deep neural network architectures and their applications

Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239 · see citing works

Untitled work

(1989) · Cited by 3,214 · see citing works

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,123 · see citing works

FitNets: Hints for Thin Deep Nets

Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859 · see citing works

Deep learning for visual understanding: A review

Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504 · see citing works

Model compression

Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · see citing works

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

Zagoruyko, S., & Komodakis, N. (2016) · arXiv · doi:10.48550/arxiv.1612.03928 · Cited by 2,027 · see citing works

Deep Learning Face Representation from Predicting 10,000 Classes

Sun, Y. J., Wang, X., & Tang, X. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.244 · Cited by 1,995 · see citing works

Deep Mutual Learning

Zhang, Y., Xiang, T., Hospedales, T. M., & Lu, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00454 · Cited by 1,933 · see citing works

Advances in Neural Information Processing Systems 19 (NIPS 2006)

Simonotto, E., Whalley, H. C., Lawrie, S. M., Murray, L. M., McGonigle, D. J., & Storkey, A. (2006) · Cited by 1,651 · see citing works

A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning

Yim, J., Joo, D., Bae, J., & Kim, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.754 · Cited by 1,635 · see citing works

Do Deep Nets Really Need to be Deep?Preprint

Ba, J. L., & Caruana, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6184 · Cited by 1,592 · see citing works

Similarity-Preserving Knowledge Distillation

Tung, F., & Mori, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00145 · Cited by 1,179 · see citing works

Born Again Neural NetworksPreprint

Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04770 · Cited by 798 · see citing works

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons

Heo, B., Lee, M., Yun, S., & Choi, J. Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013779 · Cited by 550 · see citing works

Cascade Residual Learning: A Two-Stage Convolutional Neural Network for Stereo Matching

Pang, J., Sun, W., Ren, J., Yang, C., & Yan, Q. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.108 · Cited by 499 · see citing works

Paraphrasing Complex Network: Network Compression via Factor Transfer

Kim, J., Park, S., & Kwak, N. (2018) · arXiv · doi:10.48550/arxiv.1802.04977 · Cited by 411 · see citing works

Knowledge Distillation by On-the-Fly Native EnsemblePreprint

Xu, L., Zhu, X., & Gong, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04606 · Cited by 363 · see citing works

Efficient and accurate approximations of nonlinear convolutional networks

Zhang, X., Zou, J., Xiang, M., He, K., & Sun, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298809 · Cited by 267 · see citing works

Fixed-point feedforward deep neural network design using weights +1, 0, and −1

Hwang, K., & Sung, W. (2014) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips.2014.6986082 · Cited by 258 · see citing works

Multimodal Residual Learning for Visual QAPreprint

Kim, J.-H., Lee, S., Kwak, D., Heo, M.-O., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01455 · Cited by 250 · see citing works

Self-supervised Knowledge Distillation Using Singular Value Decomposition

Lee, S., Kim, D. H., & Song, B. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_21 · Cited by 162 · see citing works

International Conference on Computer Vision（ICCV）2019

佐藤, 具. (2020) · Cited by 129 · see citing works

N2N Learning: Network to Network Compression via Policy Gradient Reinforcement LearningPreprint

Ashok, A., Rhinehart, N., Beainy, F., & Kitani, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06030 · Cited by 122 · see citing works

Improved Knowledge Distillation via Teacher Assistant

Mirzadeh, S. I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., & Ghasemzadeh, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5963 · Cited by 101 · see citing works

Progressive Blockwise Knowledge Distillation for Neural Network Acceleration

Wang, H., Zhao, H., Li, X., & Tan, X. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/384 · Cited by 63 · see citing works

Adversarial Network Compression

Belagiannis, V., Farshad, A., & Galasso, F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_37 · Cited by 62 · see citing works

Self-Referenced Deep Learning

Xu, L., Zhu, X., & Gong, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20890-5_19 · Cited by 19 · see citing works

Knowledge Distillation: A Survey

Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931 · see citing works

Multi-Residual Networks: Improving the Speed and Accuracy of Residual NetworksPreprint

Abdi, M., & Nahavandi, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05672 · Cited by 54 · see citing works

Summary of Studies on Structure of "X+Tou"

Yu-jing, F. (2008) · Journal of Lianyungang Teachers College

Using accumulation to optimize deep residual neural nets

(2018) · doi:10.48550/arxiv.1803.05778

Residual Networks of Residual Networks: Multilevel Residual Networks

Zhang, K., Sun, M., Han, T. X., Yuan, X., Guo, L., & Liu, T. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2654543 · Cited by 398 · see citing works

Multi-Residual Networks.Preprint

Abdi, M., & Nahavandi, S. (2016) · arXiv (Cornell University) · Cited by 15 · see citing works