Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Open accessAbstract
We propose a new method for creating computationally efficient and compact convolutional neural networks (CNNs) using a novel sparse connection structure that resembles a tree root. This allows a significant reduction in computational cost and number of parameters compared to state-of-the-art deep CNNs, without compromising accuracy, by exploiting the sparsity of inter-layer filter dependencies. We validate our approach by using it to train more efficient variants of state-of-the-art CNN architectures, evaluated on the CIFAR10 and ILSVRC datasets. Our results show similar or higher accuracy than the baseline architectures with much less computation, as measured by CPU and GPU timings. For example, for ResNet 50, our model has 40% fewer parameters, 45% fewer floating point operations, and is 31% (12%) faster on a CPU (GPU). For the deeper ResNet 200 our model has 48% fewer parameters and 27% fewer floating point operations, while maintaining state-of-the-art accuracy. For GoogLeNet, our model has 7% fewer parameters and is 21% (16%) faster on a CPU (GPU).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent advances in light-weight neural network architectures are mostly based on the metric of FLOPs and do not consider the proposed properties.
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [12]
The Introduction (p. 1) identifies this as work on reducing CNN computational cost and model size, so it supports citing the paper as an example of a lightweight architecture. But the Conclusion (p. 8) evaluates efficiency using CPU and GPU timings as well as parameter counts, and GPU Implementation (p. 7) notes that runtime gains differed from FLOPs-based predictions. The copy therefore does not support characterizing this work as mostly based on FLOPs; the unspecified “properties above” cannot be assessed from the citing sentence. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,266 · see citing works
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · see citing works
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · see citing works
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · see citing works
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
One weird trick for parallelizing convolutional neural networksPreprint
Krizhevsky, A. (2014) · arXiv.org · doi:10.48550/arxiv.1404.5997 · Cited by 1,435 · see citing works
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, R., Zaremba, W., Bruna, J., LeCun, Y., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1404.0736 · Cited by 1,414 · see citing works
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · see citing works
Speeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.88 · Cited by 1,164 · see citing works
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974 · see citing works
(2014) · arXiv (Cornell University) · Cited by 942 · see citing works
Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1302.4389 · Cited by 935 · see citing works
Predicting Parameters in Deep Learning
Denil, M., Shakibi, B., Dinh, L., Ranzato, M., & de Freitas, N. (2013) · arXiv · doi:10.48550/arxiv.1306.0543 · Cited by 853 · see citing works
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile ApplicationsPreprint
Kim, Y., Park, E., Yoo, S., Choi, T., Lu, Y., & Shin, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06530 · Cited by 736 · see citing works
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · Cited by 707 · see citing works
Speeding up Convolutional Neural Networks with Low Rank ExpansionsPreprint
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3866 · Cited by 576 · see citing works
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-DecompositionPreprint
Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I. V., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6553 · Cited by 511 · see citing works
Predicting Parameters in Deep Learning
Shakibi, B. (2014) · University of British Columbia · doi:10.14288/1.0165555 · Cited by 462 · see citing works
Deformable part models are convolutional neural networks
Girshick, R., Iandola, F., Darrell, T., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298641 · Cited by 423 · see citing works
Fast Training of Convolutional Networks through FFTs
Mathieu, M., Henaff, M., & LeCun, Y. (2014) · International Conference on Learning Representations · Cited by 343 · see citing works
Spectral Representations for Convolutional Neural NetworksPreprint
Rippel, O., Snoek, J., & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03767 · Cited by 276 · see citing works
Reducing Overfitting in Deep Networks by Decorrelating RepresentationsPreprint
Cogswell, M., Ahmed, F., Girshick, R., Zitnick, L., & Batra, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06068 · Cited by 247 · see citing works
Simplifying ConvNets for Fast Learning
Mamalet, F., & García, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33266-1_8 · Cited by 152 · see citing works
Do Deep Nets Really Need to be DeepPreprint
Ba, L. J., & Caruana, R. (2013) · arXiv (Cornell University) · Cited by 140 · see citing works
Training CNNs with Low-Rank Filters for Efficient Image ClassificationPreprint
Ioannou, Y., Robertson, D., Shotton, J., Cipolla, R., & Criminisi, A. (2015) · arXiv (Cornell University) · doi:10.13140/rg.2.1.3727.8163 · Cited by 99 · see citing works
Self-citationSironi, A., Tekin, B., Rigamonti, R., Lepetit, V., & Fua, P. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2343229 · Cited by 96 · see citing works
A GEMM interface and implementation on NVIDIA GPUs for multiple small matrices
Jhurani, C., & Mullowney, P. J. (2014) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2014.09.003 · Cited by 32 · see citing works
Flagged first, then by citations
7 of the 46 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [8][8] M. Minsky and S. Papert, Perceptrons . MIT press, 1988 (cit. on p. 1). -
Cited by
Sorted by citations
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., & Hu, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01155 · Cited by 9,641 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · see citing works
“Group convolution is at the core of modern network architectures [33, 35, 12, 34, 31, 26].”
BackgroundA Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · see citing works
“…efficient NN model architectures : One line of work has focused on optimizing the NN model architecture in terms of its micro-architecture [100, 110, 125, 163, 164, 205, 245, 272] (e.g., kernel types such as depth-wise convolution or low-rank factorization) as well as its macro-architecture…”
Two-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471 · see citing works
Li, J., Xie, H., Li, J., Wang, Z., & Zhang, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00639 · Cited by 357 · see citing works
Interleaved Group Convolutions
Zhang, T., Qi, G.-J., Xiao, B., & Wang, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.469 · Cited by 323 · see citing works
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
Liu, Y., Fan, B., Meng, G., Lu, J., Xiang, S., & Pan, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00534 · Cited by 306 · see citing works
Han, C., Wu, C., Guo, H., Hu, M., & Chen, H. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3264802 · Cited by 268 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · arXiv · Cited by 7,344 · see citing works
Temporally Distributed Networks for Fast Video Semantic Segmentation
Hu, P., Caba, F., Wang, O., Lin, Z., Sclaroff, S., & Perazzi, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00884 · Cited by 219 · see citing works
DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks
Zhong, J., Chen, J., & Mian, A. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3151138 · Cited by 222 · see citing works
Ms RED: A novel multi-scale residual encoding and decoding network for skin lesion segmentation
Dai, D., Dong, C., Xu, S., Yan, Q., Li, Z., Zhang, C., & Luo, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102293 · Cited by 204 · see citing works
Energy-Efficient Neural Network Accelerator Based on Outlier-Aware Low-Precision Computation
Park, E., Kim, D., & Yoo, S. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00063 · Cited by 203 · see citing works
SBNet: Sparse Blocks Network for Fast Inference
Ren, M., Pokrovsky, A., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00908 · Cited by 198 · see citing works
NASA Neural Articulated Shape Approximation
Deng, B., Lewis, J. P., Jeruzalski, T., Pons‐Moll, G., Hinton, G. E., Norouzi, M. H., & Tagliasacchi, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_36 · Cited by 196 · see citing works
Holistic CNN Compression via Low-Rank Decomposition with Knowledge Transfer
Lin, S., Ji, R., Chen, C., Tao, D., & Luo, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2873305 · Cited by 165 · see citing works
Interleaved Structured Sparse Convolutional Neural Networks
Xie, G., Wang, J., Zhang, T., Lai, J., Hong, R., & Qi, G.-J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00922 · Cited by 142 · see citing works
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer
Idelbayev, Y., & Carreira-Perpiñán, M. Á. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00807 · Cited by 132 · see citing works
LMSD-YOLO: A Lightweight YOLO Algorithm for Multi-Scale SAR Ship Detection
Guo, Y., Chen, S., Zhan, R., Wang, W., & Zhang, J. (2022) · Remote Sensing · doi:10.3390/rs14194801 · Cited by 126 · see citing works
Efficient Inductive Vision Transformer for Oriented Object Detection in Remote Sensing Imagery
Zhang, C., Su, J., Ju, Y., Lam, K., & Wang, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3292418 · Cited by 120 · see citing works
The Power of Sparsity in Convolutional Neural NetworksPreprint
Changpinyo, S., Sandler, M., & Zhmoginov, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06257 · Cited by 116 · see citing works
Wang, Y., Zhang, C., Xie, Z., Guo, C., Liu, Y., & Leng, J. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00088 · Cited by 113 · see citing works
Liu, D., Li, J., & Yuan, Q. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3049875 · Cited by 108 · see citing works
Multi-View Separable Pyramid Network for AD Prediction at MCI Stage by 18F-FDG Brain PET Imaging
Pan, X., Phan, T.-L., Adel, M., Fossati, C., Gaidon, T., Wojak, J., & Guedj, É. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3022591 · Cited by 110 · see citing works
UEDG:Uncertainty-Edge Dual Guided Camouflage Object Detection
Lyu, Y., Zhang, H., Li, Y., Liu, H., Yang, Y. F., & Yuan, D. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3295095 · Cited by 105 · see citing works
Robust Facial Landmark Detection via a Fully-Convolutional Local-Global Context Network
Merget, D., Rock, M., & Rigoll, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00088 · Cited by 96 · see citing works
Restoring Extremely Dark Images in Real Time
Lamba, M., & Mitra, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00349 · Cited by 96 · see citing works
Chen, W., Xie, D., Zhang, Y., & Pu, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00741 · Cited by 95 · see citing works
Sparsely Connected CNN for Efficient Automatic Modulation Recognition
Tunze, G. B., Huynh‐The, T., Lee, J., & Kim, D. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3042638 · Cited by 93 · see citing works
Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
Wang, X., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00926 · Cited by 84 · see citing works
Applications and Techniques for Fast Machine Learning in Science
Deiana, A. M., Tran, N. V., Agar, J., Blott, M., Guglielmo, G. D., Duarte, J. M., Harris, P. C., Hauck, S., Liu, M., Neubauer, M. S., et al. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.787421 · Cited by 81 · see citing works
RF-UAVNet: High-Performance Convolutional Network for RF-Based Drone Surveillance Systems
Huynh‐The, T., Pham, Q., Nguyen, T.-V., da Costa, D. B., & Kim, D. (2022) · IEEE Access · doi:10.1109/access.2022.3172787 · Cited by 76 · see citing works
Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels
Bragman, F., Tanno, R., Sebastien Ourselin, A., Alexander, D. C., & Cardoso, M. J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00147 · Cited by 74 · see citing works
Extreme Network Compression via Filter Group Approximation
Peng, B., Tan, W., Li, Z., Zhang, S., Xie, D., & Pu, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_19 · Cited by 68 · see citing works
SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization
Cao, S., Ma, L., Xiao, W., Zhang, C., Liu, Y., Zhang, L., Nie, L., & Yang, Z. Q. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01147 · Cited by 73 · see citing works
Lightweight image super-resolution based on deep learning: State-of-the-art and future directions
Gendy, G., He, G., & Sabor, N. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.024 · Cited by 73 · see citing works
A Deep Learning Method for Early Detection of Diabetic Foot Using Decision Fusion and Thermal Images
Munadi, K., Saddami, K., Oktiana, M., Roslidar, R., Muchtar, K., Melinda, M., Muharar, R., Syukri, M., Abidin, T. F., & Arnia, F. (2022) · Applied Sciences · doi:10.3390/app12157524 · Cited by 64 · see citing works
Comparative Analysis of Recent Architecture of Convolutional Neural Network
Saleem, M. A., Senan, N., Wahid, F., Aamir, M., Samad, A., & Khan, M. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/7313612 · Cited by 62 · see citing works
An Efficient Convolutional Neural Network for Paddy Leaf Disease and Pest Classification
Senan, N., Aamir, M., Ibrahim, R., Nikita Konstantina, S., & Muda, W. H. N. W. (2020) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2020.0110716 · Cited by 61 · see citing works
Multiple Attention Siamese Network for High-Resolution Image Change Detection
Huang, J., Shen, Q., Wang, M., & Yang, M. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3127580 · Cited by 61 · see citing works
GET: Group Event Transformer for Event-Based Vision
Peng, Y., Zhang, Y., Xiong, Z., Sun, X., & Wu, F. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00555 · Cited by 67 · see citing works
Scalable Multimodal Convolutional Networks for Brain Tumour Segmentation
Fidon, L., Li, W., Garcia-Peraza-Herrera, L. C., Ekanayake, J., Kitchen, N., Ourselin, S., & Vercauteren, T. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66179-7_33 · Cited by 60 · see citing works
Accelerating Convolutional Neural Networks via Activation Map Compression
Georgiadis, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00725 · Cited by 58 · see citing works
RFDOA-Net: An Efficient ConvNet for RF-Based DOA Estimation in UAV Surveillance Systems
Akter, R., Doan, V., Huynh‐The, T., & Kim, D. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3114058 · Cited by 58 · see citing works
Tang, G., Yi, C., Liu, L., Xingguo, Y., Du, X., Zhou, Q., & Lin, J. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106395 · Cited by 57 · see citing works
Depth-Wise Separable Convolution Attention Module for Garbage Image Classification
Liu, F., Xu, H. F., Qi, M., Liu, D., Wang, J., & Kong, J. (2022) · Sustainability · doi:10.3390/su14053099 · Cited by 56 · see citing works
Traffic sign detection in unconstrained environment using improved YOLOv4
Saxena, S., Dey, S., Shah, M., & Gupta, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121836 · Cited by 56 · see citing works
A lightweight hybrid vision transformer network for radar-based human activity recognition
Huan, S., Wang, Z., Wang, X., Wu, L., Yang, X., Huang, H., & Dai, G. (2023) · Scientific Reports · doi:10.1038/s41598-023-45149-5 · Cited by 52 · see citing works
Super Sparse Convolutional Neural Networks
Lu, Y., Lu, G., Zhang, B., Xu, Y., & Li, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014440 · Cited by 52 · see citing works
CNN and Transformer Fusion for Remote Sensing Image Semantic Segmentation
Chen, X., Li, D.-F., Liu, M.-Y., & Jia, J. (2023) · Remote Sensing · doi:10.3390/rs15184455 · Cited by 45 · see citing works
Jiang, G., Wei, J., Xu, Y., He, Z., Zeng, H., Wu, J., Qin, G., Chen, W., & Lu, Y. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3071544 · Cited by 50 · see citing works
Lu, Y., Yu, Q., Gao, J., Li, Y., Zou, J.-C., & Qiao, H. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.117 · Cited by 48 · see citing works
Deep-Sea Debris Identification Using Deep Convolutional Neural Networks
Xue, B., Huang, B., Chen, G., Li, H., & Wei, W. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3107853 · Cited by 47 · see citing works
Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks
Kundu, S., Nazemi, M., Pedram, M., Chugg, K. M., & Beerel, P. A. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2972520 · Cited by 41 · see citing works
DCFNet: Deep Neural Network with Decomposed Convolutional FiltersPreprint
Qiu, Q., Cheng, X., Calderbank, R., & Sapiro, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04145 · Cited by 41 · see citing works
Cui, B., Liang, L., Ji, B., Zhang, L., Zhao, L., Zhang, K., Shi, F., & Créput, J.-C. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3424232 · Cited by 39 · see citing works
Seismic Impedance Inversion Based on Residual Attention Network
Wu, B., Xie, Q., & Wu, B. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3193563 · Cited by 38 · see citing works
Small neural nets are beautiful
Iandola, F., & Keutzer, K. (2017) · International Conference on Hardware/Software Codesign and System Synthesis · doi:10.1145/3125502.3125606 · Cited by 36 · see citing works
Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks
Schuler, J. P. S., Romaní, S., Abdel‐Nasser, M., Rashwan, H. A., & Puig, D. (2022) · MENDEL · doi:10.13164/mendel.2022.1.023 · Cited by 36 · see citing works
Lai, X., Zhang, K., Zheng, Q. R., Li, Z., Ding, G., & Ding, K. (2023) · Measurement · doi:10.1016/j.measurement.2023.112833 · Cited by 36 · see citing works
Faster and Better: A Lightweight Transformer Network for Remote Sensing Scene Classification
Huang, X., Liu, F., Cui, Y., Chen, P., Li, L., & Li, P. (2023) · Remote Sensing · doi:10.3390/rs15143645 · Cited by 35 · see citing works
Hyperspectral-cube-based mobile face recognition: A comprehensive review
Zhang, X., & Zhao, H. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.04.003 · Cited by 36 · see citing works
Navarro, P., Miller, L., Rosique, F., Fernández-Isla, C., & Gila-Navarro, A. (2021) · Electronics · doi:10.3390/electronics10111266 · Cited by 35 · see citing works
GLMDriveNet: Global–local Multimodal Fusion Driving Behavior Classification Network
Liu, W., Gong, Y., Zhang, G., Lu, J., Zhou, Y., & Liao, J. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107575 · Cited by 36 · see citing works
Realtime Face-Detection and Emotion Recognition Using MTCNN and miniShuffleNet V2
Ghofrani, A., Toroghi, R. M., & Ghanbari, S. (2019) · International Conference on Knowledge-Based Engineering and Innovation · doi:10.1109/kbei.2019.8734924 · Cited by 34 · see citing works
Color-Aware Two-Branch DCNN for Efficient Plant Disease Classification
Schuler, J. P. S., Romaní, S., Abdel‐Nasser, M., Rashwan, H. A., & Puig, D. (2022) · MENDEL · doi:10.13164/mendel.2022.1.055 · Cited by 33 · see citing works
Li, C., Liu, X., & Shutao Li, A. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3326545 · Cited by 32 · see citing works
Li, J., Xie, H., Yu, L., Gao, X., & Zhang, Y. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3117003 · Cited by 33 · see citing works
An Improved Helmet Detection Algorithm Based on YOLO V4
Yang, B., & Wang, J. (2022) · International Journal of Foundations of Computer Science · doi:10.1142/s0129054122420205 · Cited by 32 · see citing works
SRGC-Nets: Sparse Repeated Group Convolutional Neural Networks
Lu, Y., Lu, G., Lin, R., Li, J., & Zhang, D. D. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2933665 · Cited by 31 · see citing works
Fu, R., Xu, H. G., Wang, Z., Shen, L., Cao, M., Liu, T., & Novák, D. (2020) · Sensors · doi:10.3390/s20072021 · Cited by 33 · see citing works
Self-grouping convolutional neural networks
Guo, Q., Wu, X., Kittler, J., & Feng, Z. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.09.015 · Cited by 30 · see citing works
Ad 2 Attack: Adaptive Adversarial Attack on Real-Time UAV Tracking
Fu, C., Li, S., Yuan, X., Ye, J., Cao, Z., & Ding, F. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812056 · Cited by 30 · see citing works
BlockSwap: Fisher-guided Block Substitution for Network Compression on a BudgetPreprint
Turner, J., Crowley, E. J., O'Boyle, M., Storkey, A., & Gray, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04113 · Cited by 38 · see citing works
HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs
Singh, P. P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00497 · Cited by 37 · see citing works
Building Efficient Deep Neural Networks With Unitary Group Convolutions
Zhao, R., Hu, Y., Dotzel, J., De, C., & Zhang, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01156 · Cited by 27 · see citing works
Ship Detection in Multispectral Satellite Images Under Complex Environment
Xie, X., Li, B. H., & Wei, X. (2020) · Remote Sensing · doi:10.3390/rs12050792 · Cited by 28 · see citing works
Pruning Depthwise Separable Convolutions for MobileNet Compression
Tu, C.-H., Lee, J., Chan, Y.-M., & Chen, C. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207259 · Cited by 29 · see citing works
HDIQA: A Hyper Debiasing Framework for Full Reference Image Quality Assessment
Zhou, M., Wang, H., Wei, X., Feng, Y., Luo, J. T., Pu, H., Zhao, J., Wang, L., Chu, Z., Wang, X., et al. (2024) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2024.3353573 · Cited by 27 · see citing works
Xie, Z., Chen, J., Shi, Z., Liu, S., & He, S. (2024) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2024.111824 · Cited by 26 · see citing works
HetConv: Beyond Homogeneous Convolution Kernels for Deep CNNs
Singh, P. P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01264-3 · Cited by 25 · see citing works
A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection
Jiang, M., Shimasaki, K., Hu, S., Senoo, T., & Ishii, I. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3048653 · Cited by 26 · see citing works
Tuong, V. D., Noh, W., & Cho, S. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3254651 · Cited by 26 · see citing works
Tang, G., Yi, C., Liu, L., Du, X., Zhou, Q., Hu, Y., Zhou, P., & Lin, J. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107407 · Cited by 26 · see citing works
On Correlation of Features Extracted by Deep Neural Networks
Ayinde, B. O., Inanc, T., & Żurada, J. M. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852296 · Cited by 26 · see citing works
Residual-Guided Multiscale Fusion Network for Bit-Depth Enhancement
Liu, J., Wen, X. L., Nie, W., Su, Y., Jing, P., & Yang, X. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3098707 · Cited by 25 · see citing works
Log-DenseNet: How to Sparsify a DenseNetPreprint
Hu, H., Dey, D., Del Giorno, A., Hebert, M., & Bagnell, J. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00002 · Cited by 25 · see citing works
Li, S., Li, Q. X., Li, R., Wu, W., Zhao, J., Qiang, Y., & Tian, Y. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.103543 · Cited by 24 · see citing works
A deep interpretable representation learning method for speech emotion recognition
Jing, E., Liu, Y., Chai, Y., Sun, J., Samtani, S., Jiang, Y., & Qian, Y. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103501 · Cited by 24 · see citing works
MSGC-YOLO: An Improved Lightweight Traffic Sign Detection Model under Snow Conditions
Chen, B., & Fan, X. (2024) · Mathematics · doi:10.3390/math12101539 · Cited by 24 · see citing works
Xiao, P., Qin, Z., Chen, D., Zhang, N., Ding, Y., Deng, F., Qin, Z., & Pang, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3235651 · Cited by 23 · see citing works
Tang, G., Yi, C., Liu, L., Zhan, X., Zhou, Q., & Lin, J. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110870 · Cited by 21 · see citing works
A Closer Look at Structured Pruning for Neural Network CompressionPreprint
Crowley, E. J., Turner, J., Storkey, A., & O’boyle, M. F. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04622 · Cited by 21 · see citing works
Guo, J., Yang, J., Yue, H., & Li, K. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3067513 · Cited by 20 · see citing works
Zhao, C., Guo, D., Shao, C., Zhao, K., Sun, M., & Shuai, H. (2024) · IEEE Access · doi:10.1109/access.2024.3382245 · Cited by 20 · see citing works
BED-YOLO: An Enhanced YOLOv8 for High-Precision Real-Time Bearing Defect Detection
Han, T., Dong, Q., Wang, X., & Sun, L. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3472791 · Cited by 20 · see citing works
- Authors
- Ioannou, Y., Robertson, D., Cipolla, R., & Criminisi, A.
- Affiliations
- University of Cambridge (GB)
- Microsoft (United States) (US)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 5977-5986
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2400429454
- Cited by
- 321
- References
- 46
- Citations by year
- 2026: 8 · 2025: 34 · 2024: 42 · 2023: 56 · 2022: 42 · 2021: 44 · 2020: 32 · 2019: 39 · 2018: 15 · 2017: 9
- FWCI
- 7.60
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- sparse connections, compact convolutional neural networks, computational efficiency, parameter reduction, ResNet, GoogLeNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
