Convolutional neural networks at constrained time cost
Open accessThe DOI matches its Crossref record; 26 of the 30 references it cites were found in a registry or an index.
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The performance of neural network models is closely related to the setting of hyperparameters [42].”
From Prediction of Heat and Cold Loads of Factory Mushroom Houses Based on EWT Decomposition (2023) · cites this paper as [42]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 9×
“No other data augmentation, such as scale jittering [10, 22, 9], is used.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 14×
“The actual running time of our model is 20% faster than AlexNet.”
BackgroundMethodResultInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 7×
“Through the investigations, we obtain a model that achieves 11.8% top-5 error (10-view test) on ImageNet [3] and only takes 3 to 4 days training on a single GPU.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Self-citationBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Background
Research works printed but not linked: 1
- lin, 2013, Network in network, arXiv 1312 4400
Cited by
Items 501–600 of 1,001
Liu, Z., & Bărbulescu, A. (2024) · Water · doi:10.3390/w16020289 · Cited by 34
Deep Non-Cooperative Spectrum Sensing Over Rayleigh Fading Channel
Su, Z., Teh, K. C., Razul, S. G., & Kot, A. C. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3138593 · Cited by 35
“According to [17], for CNN layers, the computation complexity for one signal sample is O( ∑D l = 1 Cl−1 · Cl ·Kl ·Ml), where D is the total number of convolutional and deconvolutional layers.”
BackgroundLi, W., & Gao, J. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1561 · Cited by 38
Kolbusz, J., Różycki, P., & Wilamowski, B. M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59063-9_9 · Cited by 35
Approximate FPGA-Based LSTMs Under Computation Time Constraints
Rizakis, M., Venieris, S. I., Kouris, A., & Bouganis, C.-S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-78890-6_1 · Cited by 37
Road safety performance function analysis with visual feature importance of deep neural nets
Pan, G., Fu, L., Chen, Q., Yu, M., & Muresan, M. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003108 · Cited by 31
“Since its proposal, it has been successfully applied to solve complex problems in a variety of fields, including but not limited to pattern recognition, game theory, computer vision, medical treatment, transportation logistics and financial [13]–[18].”
BackgroundHybrid SVM-CNN Classification Technique for Human–Vehicle Targets in an Automotive LFMCW Radar
Wu, Q., Gao, T. F., Lai, Z., & Li, D. (2020) · Sensors · doi:10.3390/s20123504 · Cited by 33
“The overall computational complexity of the CNN approach is about O(N ∑l=1 M2 l K 2 l Cl−1Cl) [44], where d is the network depth, Ml represents the side length of the feature map in the lth convolutional layer, Kl denotes the side length of the convolution kernel in the l-th convolutional layer, Cl is the number of co…”
BackgroundCui, R., Chen, Z., Wu, J., Tan, Y., & Yu, G. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3024563 · Cited by 31
“In general, the performance of a deeper neural network will be improved, but it will also lead to more computation and slower convergence speed [42], so the depth of the network needs to be set within a reasonable range.”
BackgroundRen, Z. X., Kong, Q., Han, J., Plumbley, M. D., & Schuller, B. W. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3037534 · Cited by 33
“However, more convolutional layers lead to more parameters, so that the training procedure tends to be sub-optimised [35].”
BackgroundUnifying Nonlocal Blocks for Neural Networks
Zhu, L., She, Q., Li, D., Lu, Y., Kang, X., Hu, J., & Wang, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01207 · Cited by 39
“Firstly, repeating convolutional operations comes with higher computational and memory costs as well as the risk of over-fitting [10].”
BackgroundBalanced Neural Architecture Search and Its Application in Specific Emitter Identification
Du, M.-Y., He, X., Cai, X., & Bi, D.-P. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3107633 · Cited by 28
“More precisely, for CNNs, we mainly take the number of parameters regarding convolutional layers and fully connected layers into consideration [44], which can be computed by where l is the index of the convolutional layer, d is the depth or the number of convolutional layers, n l is the number of…”
MethodAutomatic Fault Detection for Deep Learning Programs Using Graph Transformations
Nikanjam, A., Braiek, H. B., Morovati, M. M., & Khomh, F. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3470006 · Cited by 39
A Self-Tuning Cyber-Attacks’ Location Identification Approach for Critical Infrastructures
Alabassi, A., Jahromi, A. N., Karimipour, H., Dehghantanha, A., Siano, P., & Leung, H. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3133361 · Cited by 31
“The computational complexity of training and testing the used algorithms is shown in Table III [38], [47].”
MethodTSE DeepLab: An efficient visual transformer for medical image segmentation
Yang, J., Tu, J., Zhang, X., Yu, S., & Xianyou, Z. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.104376 · Cited by 33
Handwritten Chinese character recognition with spatial transformer and deep residual networks
Zhong, Z., Zhang, X.-Y., Yin, F., & Liu, C. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900166 · Cited by 31
“However, when the network becomes deeper, it will encounter the degradation problem as reported in [22].”
BackgroundAutomatic fetal brain extraction from 2D in utero fetal MRI slices using deep neural network
Li, J., Luo, Y., Shi, L., Zhang, X., Li, M., Zhang, B., & Wang, D. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.10.032 · Cited by 33
Bhandari, S., Vu, T. X., & Chatzinotas, S. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3365883 · Cited by 37
“…The computational complexity to train the CNN-CSI model with l c layers is given by O p E max D ř l c l “ 1 n f,l ´ 1 s 2 f,l n f,l 2 U q [41], where n f,l is the number of filters in the l -th layer, s f,l is the spatial size of the filters in the l -th layer, D is the total number of…”
MethodComparison of Three Different CNN Architectures for Age Classification
Demirci, M. F., Celik, V., & Demirci, M. F. (2017) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2017.61 · Cited by 30
“A. 6 Layer CNN The structure of the 6 layer CNN is shown in Fig.”
MethodA Reinforcement Learning Based Smart Cache Strategy for Cache-Aided Ultra-Dense Network
Li, W., Wang, J., Zhang, G., Li, L., Dang, Z., & Li, S. (2019) · IEEE Access · doi:10.1109/access.2019.2905589 · Cited by 32
A Saak Transform Approach to Efficient, Scalable and Robust Handwritten Digits Recognition
Chen, Y., Xu, Z., Cai, S., Lang, Y., & Kuo, C. ‐. J. (2018) · Picture Coding Symposium · doi:10.1109/pcs.2018.8456277 · Cited by 34
Cotrim, W. D. S., Félix, L. B., Minim, V. P. R., Campos, R. C., & Minim, L. A. (2021) · Chemical Engineering Science · doi:10.1016/j.ces.2021.116679 · Cited by 30
Deep learning application for sensing available spectrum for cognitive radio: An ECRNN approach
Goyal, S. B., Bedi, P., Kumar, J., & Varadarajan, V. (2021) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-021-01169-4 · Cited by 30
“While training any network, the complexity of CNN for processing one data sample is evaluated to be as in Eqn (12) [29]: where P = Number of convolution layer with N k , p − 1 to be the number of input channels.”
MethodA novel method for transformer fault diagnosis based on refined deep residual shrinkage network
Hu, H., Ma, X., & Shang, Y. (2021) · IET Electric Power Applications · doi:10.1049/elp2.12147 · Cited by 33
Han, H., Xu, Y., Jin, Z., Li, W., Chen, X., Fang, G., & Xu, Y. (2021) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3125337 · Cited by 30
An Optimized Convolution Neural Network Architecture for Paddy Disease Classification
Saleem, M. A., Aamir, M., Ibrahim, R., Senan, N., & Alyas, T. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.022215 · Cited by 33
“Study in [9] improved the accuracy by designing architecture while adding up to sixteen layers.”
TransCloudSeg: Ground-Based Cloud Image Segmentation With Transformer
Liu, S., Zhang, J., Zhang, Z., Cao, X., & Durrani, T. S. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3194316 · Cited by 33
Zilelioglu, H., Khodabandelou, G., Chibani, A., & Amirat, Y. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3267243 · Cited by 32
“Following [57], the time complexity of d consecutive convolutional layers with n flat kernels of size s can be expressed as where m l is the spatial size of the output feature map.”
BackgroundLi, Y., Xu, Y., Li, G., Gong, Y., Liu, X., Wang, H., & Li, W. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3307950 · Cited by 35
Automatic myocardial segmentation by using a deep learning network in cardiac MRI
Curiale, A. H., Colavecchia, F. D., Kaluza, P., Isoardi, R., & Mato, G. (2017) · Latin American Computer Conference (CLEI) · doi:10.1109/clei.2017.8226420 · Cited by 36
“Unexpect-edly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported [23], [24].”
BackgroundHuman Action Prediction with 3D-CNN
Alfaifi, R., & Artoli, A. M. (2020) · SN Computer Science · doi:10.1007/s42979-020-00293-x · Cited by 30
Traffic Analytics With Low-Frame-Rate Videos
Luo, Z., Jodoin, P., Su, S., Li, S., & Larochelle, H. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2632439 · Cited by 29
“and “Vehicle” for the surface occupied by vehicles that is identical to the one used in [16] and [45].”
BackgroundMethodBio-Inspired Representation Learning for Visual Attention Prediction
Yuan, Y., Ning, H., & Lu, X. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2931735 · Cited by 29
“According to [60], the time complexity of each convolutional and pooling layer can be computed as , where M denotes the size of the output feature map, In addition, according to [22], a summary of these methods is provided in Table VI.”
MethodA Smartphone Lightweight Method for Human Activity Recognition Based on Information Theory
Bragança, H. L. D. S., Colonna, J. G., Lima, W. S., & Souto, E. (2020) · Sensors · doi:10.3390/s20071856 · Cited by 32
“The DNNs have a high computational cost when compared with traditional approaches, which makes them unfit for real-time applications implemented on mobile devices with low computational power (CPU and low-performance GPU) [35].”
BackgroundDeep Learning Based Signal Detection for OFDM VLC Systems
Amran, N. A., Soltani, M. D., Yaghoobi, M., & Safari, M. (2020) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops49005.2020.9145364 · Cited by 31
“The time complexity for all convolutional layers [25] for the CNN model is O( ∑d l=1 nl−1 · s(2)l · nl ·m(2)l ) where l and d are the index and the number of convolutional layers, respectively.”
BackgroundTarget Detection Network for SAR Images Based on Semi-Supervised Learning and Attention Mechanism
Wei, D., Du, Y., Du, L., & Li, L. (2021) · Remote Sensing · doi:10.3390/rs13142686 · Cited by 32
“By analyzing the calculation principle of CNN [26], the computational complexity of CNN is:”
BackgroundMethodAutomatic glottis segmentation for laryngeal endoscopic images based on U-Net
Ding, H., Cen, Q., Si, X., Pan, Z., & Chen, X. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103116 · Cited by 35
Zhou, K., Oh, S., Witold Pedrycz, C., & Qiu, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105580 · Cited by 31
Sankari, V. M. R., Umapathy, S., Chandrasekaran, A., & Baskaran, P. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104883 · Cited by 30
Arshad, S., Shahzad, M., Riaz, Q., & Fraz, M. M. (2019) · IEEE Access · doi:10.1109/access.2019.2918862 · Cited by 32
“It is easier to approximate identity mapping with residual functions as compared to approximate with multiple nonlinear stacked layers [31], [55], [56].”
BackgroundConvolutional-Neural-Network-Based Detection Algorithm for Uplink Multiuser Massive MIMO Systems
Li, L., Hou, H., & Meng, W. (2020) · IEEE Access · doi:10.1109/access.2020.2985083 · Cited by 30
“…layer, the computational complexity is O( p 2 q 2 st ), where p 2 is the scale of each feature map output from one convolutional filter, q 2 is the scale of each convolutional filter, s is the number of input channels for each filter, and t is the number of convolutional filters in the layer [33].”
BackgroundWang, Y., Wu, J., Hua, X., Liu, C. H., Li, G., Zhao, J., Yuan, Y., & Wang, G. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00140 · Cited by 38
“We compute the time complexity of forward process according to [43].”
MethodCNN-based object detection solutions for embedded heterogeneous multicore SoCs
Wang, C., Wang, Y., Han, Y., Song, L., Quan, Z., Li, J., & Li, X. (2017) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2017.7858304 · Cited by 29
“The design of two networks are following the guidelines of [8].”
BackgroundTransfer learning across human activities using a cascade neural network architecture
Du, X., Farrahi, K., & Niranjan, M. (2019) · International Symposium on Wearable Computers · doi:10.1145/3341163.3347730 · Cited by 33
“Although the above approaches show state of the art generalization performance, the computational complexity and memory requirements of CNNs is noted to be generally higher than the corresponding feedforward MLPs used [5, 13].”
BackgroundA Shallow Convolutional Neural Network for Apple Classification
Li, J., Xie, S., Chen, Z., Liu, H., Kang, J., Fan, Z., & Li, W. (2020) · IEEE Access · doi:10.1109/access.2020.3002882 · Cited by 31
“For CNN, the algorithm complexity contains two separate parts: time complexity, which is represented by floating-point operations (FLOPs), and space complexity, which is defined as the number of parameters to be trained [32].”
BackgroundChen, K., Zhang, S., Zhu, L., Chen, S., & Zhao, H. (2021) · Sensors · doi:10.3390/s21020449 · Cited by 31
“Hence, classic networks [37,38] on the challenging ImageNet dataset [39] all adapt deep models.”
BackgroundRUL Estimation Enhancement using Hybrid Deep Learning Methods
Remadna, I., Terrissa, L. S., Ayad, S., & Noureddine Zerhouni, A. (2021) · International Journal of Prognostics and Health Management · doi:10.36001/ijphm.2021.v12i1.2378 · Cited by 29
“The complexity of the pooling and the Fully Connected layers (FC) takes 5-10 % of the overall computational time (He & Sun, 2015).”
BackgroundFully Convolutional Neural Network-Based CSI Limited Feedback for FDD Massive MIMO Systems
Fan, G., Sun, J., Gui, G., Gacanin, H., Adebisi, B., & Ohtsuki, T. O. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3119945 · Cited by 29
Self-citationA novel unsupervised deep learning method for the generalization of urban form
Cai, J., & Chen, Y. (2022) · Geo-spatial Information Science · doi:10.1080/10095020.2022.2068384 · Cited by 30
“The use of residual blocks can also improve the network’s performance by deepening the network without the negative effects such as vanishing/exploding gradient and the degradation problem (He et al. 2016; He and Sun 2015).”
BackgroundAn Improved Magnetic Tracking Approach Based on ResNet-LM Fusion Algorithm
Guo, P., Dai, H., Yang, Q., Huang, Q., & Yao, H. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3216597 · Cited by 30
Alfano, P. D., Pastore, V. P., Rosasco, L., & Odone, F. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104894 · Cited by 36
“In [27] the authors focus on the impact of a CNN complexity on the training time, evaluating the accuracy with constrained time cost, considering the influence of different factors including depth, number of filters, and filter size.”
BackgroundWang, H., Zhou, Z., Zhang, L., & Yan, R. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3338664 · Cited by 38
“The details of ResNet can be found in [32].”
BackgroundMethodInfluentialRadar-Aided Beam Selection in MIMO Communication Systems: A Federated Transfer Learning Approach
Zhou, Q., Gong, Y., & Nallanathan, A. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3373496 · Cited by 27
Recognition and Classification of Mixed Defect Pattern Wafer Map Based on Multi-Path DCNN
Hou, X., Yi, M., Chen, S., Liu, M., & Zhu, Z. (2024) · IEEE Transactions on Semiconductor Manufacturing · doi:10.1109/tsm.2024.3418520 · Cited by 27
“In addition, a similar description is available in the paper by He and Sun [32] in 2015.”
MethodZhang, S., Zhang, S., Yuan, W., & Quek, T. Q. S. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3486673 · Cited by 35
“Furthermore, according to [45], the complexity of a CNN module can be written as O{ l i ∈ L C a l i b l i c 2 l i } , where the set of CNN layers is represented by L C ≜ { l 1 , . . . , l M c } , the number of CNN layers is indicated by M c , the length of the output feature at layer l i is a l i ,…”
Dai, W., Fan, R., & Nan, J. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3550617 · Cited by 27
“To fully verify the performance of SCN in modeling accuracy and efficiency, this article compares four deep neural network models: LSTM [25], AlexNet [26], ResNet [27], and 1D-CNN [28]. in terms of accuracy, but its modeling time is also longer than SCN.”
Keserwani, P., Ali, T., & Roy, P. P. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-019-00938-1 · Cited by 28
“[43, 44].”
Semi‐supervised convolutional generative adversarial network for hyperspectral image classification
Xue, Z. (2019) · IET Image Processing · doi:10.1049/iet-ipr.2019.0869 · Cited by 30
“However, scholars have found that as the network depth increases, the deep neural networks often suffer serious degradation problem, which leads to the accuracy gets saturated and then degrades rapidly [32, 33].”
BackgroundFast brain tumour segmentation using optimized U-Net and adaptive thresholding
Isunuri, B. V., & Kakarla, J. (2020) · Automatika · doi:10.1080/00051144.2020.1760590 · Cited by 28
“Common demerit of these methods is high computational cost [14,15].”
BackgroundA Road Health Monitoring System Using Sensors in Optimal Deep Neural Network
Mishra, R., Gupta, H. P., & Dutta, T. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3005998 · Cited by 38
EdgeNet: Semantic Scene Completion from a Single RGB- D Image
Dourado, A., de Campos, T. E., Kim, H., & Hilton, A. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413252 · Cited by 33
“We address the degradation problem of deeper networks [18], by replacing simple convolutional blocks of U-Net by ResNet modules [15].”
MethodPolSAR Image Classification Based on Robust Low-Rank Feature Extraction and Markov Random Field
Bi, H., Yao, J. T., Wei, Z., Hong, D., & Chanussot, J. (2020) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2020.3034700 · Cited by 27
SafeDrive: A New Model for Driving Risk Analysis Based on Crash Avoidance
Wang, Y., Xu, W. X., Zhang, W., & Zhao, J. L. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3033276 · Cited by 29
Ridgelet-Nets With Speckle Reduction Regularization for SAR Image Scene Classification
Qian, X., Liu, F., Jiao, L., Zhang, X., Guo, Y., Liu, X., & Cui, Y. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3051057 · Cited by 27
LSV-ANet: Deep Learning on Local Structure Visualization for Feature Matching
Chen, J., Chen, S., Chen, X., Yang, Y., Xing, L., Fan, X., & Rao, Y. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3062498 · Cited by 27
“Fully connected layers and max-pooling often take 5–10% computational time and this time complexity applies to both training and testing [59].”
BackgroundHuang, M., Zhang, X., Lan, X., Wang, H., & Tang, Y. D. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3073558 · Cited by 30
Interferometric orbital angular momentum mode detection in turbulence with deep learning
Cox, M. A., Çelik, T., Genga, Y. O., & Drozdov, A. V. (2021) · Applied Optics · doi:10.1364/ao.444954 · Cited by 27
Spatial–Spectral Split Attention Residual Network for Hyperspectral Image Classification
Shu, Z.-Q., Liu, Z.-G., Zhou, J., Tang, S., Yu, Z., & Wu, X.-J. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3225928 · Cited by 24
“Moreover, the training of the deep network is very computationally intensive and leads to network degradation [26],”
BackgroundChen, M., Yi, M., Huang, M., Huang, G., Ren, Y., & Liu, A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119743 · Cited by 27
Deeply Reinforcing Android GUI Testing with Deep Reinforcement Learning
Lan, Y., Lu, Y., Li, Z., Pan, M., Yang, W., Zhang, T., & Li, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623344 · Cited by 49
“Although deeper networks typically perform better [31], we prioritize a more compact but efficient network since deeper networks take longer to operate and are harder to converge, which would negatively reduce our testing efficiency within such a limited time.”
BackgroundResidual squeeze CNDS deep learning CNN model for very large scale places image recognition
Verma, A., Qassim, H., & Feinzimer, D. (2017) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon.2017.8248975 · Cited by 32
Li, J., Sun, A., & Xing, Z. (2018) · Information Sciences · doi:10.1016/j.ins.2018.03.014 · Cited by 31
Human action recognition using Lie Group features and convolutional neural networks
Cai, L., Liu, C., Yuan, R., & Ding, H. (2020) · Nonlinear Dynamics · doi:10.1007/s11071-020-05468-y · Cited by 26
“The time complexity and the space complexity of the CNNs model can be described by (7) and (8), respectively [38]”
Background3D-RVP: A method for 3D object reconstruction from a single depth view using voxel and point
Zhao, M., Xiong, G., Zhou, M., Shen, Z., & Wang, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.097 · Cited by 26
Power Control for Body Area Networks: Accurate Channel Prediction by Lightweight Deep Learning
Yang, Y., Smith, D. B., Rajasegaran, J., & Seneviratne, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3024820 · Cited by 27
“D convolutional layers is O ( k · T · n l · s l ) [34], [35], k denotes the kernel size, and T is the number of time steps. n l and s l are the size of output and the number of channels at layer l .”
BackgroundCloud Identification and Properties Retrieval of the Fengyun-4A Satellite Using a ResUnet Model
Zhao, Z., Zhang, F., Wu, Q., Li, Z., Tong, X., Li, J., & Han, W. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3252023 · Cited by 26
“However, it was found that the accuracy of the training dataset would drop significantly with the deepening of the network, and problems like gradient disappearance, gradient explosion, and gradient degradation would also occur in actual experiments [46], [47].”
BackgroundCascade2vec: Learning Dynamic Cascade Representation by Recurrent Graph Neural Networks
Huang, Z., Wang, Z., & Zhang, R. (2019) · IEEE Access · doi:10.1109/access.2019.2942853 · Cited by 27
“Therefore, we also analyze the outputs in each layer and estimate the memory requirements following the method introduced in [48], [49].”
MethodCNN based localisation of forged region in object‐based forgery for HD videos
Kohli, A., Gupta, A. K., & Singhal, D. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2019.0397 · Cited by 28
“[ 29] and Zhao et al .”
Learning a deep network with cross-hierarchy aggregation for crowd counting
Guo, Q., Zeng, X., Hu, S., Phoummixay, S., & Ye, Y. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106691 · Cited by 27
An Autonomous Emotion Recognition Strategy Employing Deep Learning for Self-Learning
Singh, K. U., Kumar, A., Kumar, G., Singh, T., Kumar, S., & Yadav, S. P. (2023) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs59847.2023.10389867 · Cited by 26
“Because of the selection of hyperparameters, the number of filters included inside each convolutional layer is maintained at a level of consistency that is reasonably stable, which protects the information transmission across the network [21].”
BackgroundHetConv: Beyond Homogeneous Convolution Kernels for Deep CNNs
Singh, P., Verma, V., Rai, P., & Namboodiri, V. P. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01264-3 · Cited by 26
“This raises interest in designing efficient models (He and Sun 2015).”
BackgroundGeneric Deep Learning-Based Linear Detectors for MIMO Systems Over Correlated Noise Environments
He, K., Wang, Z., Huang, W., Deng, D., Xia, J., & Fan, L. (2020) · IEEE Access · doi:10.1109/access.2020.2973000 · Cited by 26
“For the DCNN, the computational complexity is given by [40]:”
MethodSelf-citationA Novel Machine Learning Aided Antenna Selection Scheme for MIMO Internet of Things
An, W., Zhang, P., Xu, J., Luo, H., Huang, L., & Zhong, S. (2020) · Sensors · doi:10.3390/s20082250 · Cited by 26
“According to [32], the total training time complexity of all convolutional layers may be presented as”
BackgroundMishra, R., Gupta, A., Gupta, H. P., & Dutta, T. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3015546 · Cited by 39
Chen, Z., Cheng, G., Xu, Z., Guo, S., Zhou, Y., & Zhao, Y. (2021) · Digital Communications and Networks · doi:10.1016/j.dcan.2021.09.009 · Cited by 29
HybNet: A Hybrid Deep Learning-Matched Filter Approach for IoT Signal Detection
Dakic, K., Homssi, B. A., Lech, M., & Al‐Hourani, A. (2023) · IEEE Transactions on Machine Learning in Communications and Networking · doi:10.1109/tmlcn.2023.3270131 · Cited by 30
“The theoretical time complexity for all the networks can be expressed as O L =1 K l − 1 F l W l K l [40], where l is the index of convolutional layers and M is the number of layers.”
BackgroundWang, Y., Liu, C. H., Piao, C., Yuan, Y., Han, R., Wang, G., & Tang, J. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00039 · Cited by 29
CovMnet–Deep Learning Model for classifying Coronavirus (COVID-19)
Jawahar, M., Anbarasi, L. J., Ravi, V., Prassanna, J., Jasmine, S. G., Manikandan, R., Sekaran, R., & Suthendran, K. (2022) · Health and Technology · doi:10.1007/s12553-022-00688-1 · Cited by 25
Li, X., Su, J., Yue, Z., & Duan, F. (2022) · Sensors · doi:10.3390/s22207707 · Cited by 30
Chang, B. R., Tsai, H. F., & Hsieh, C.-W. (2023) · Electronics · doi:10.3390/electronics12102161 · Cited by 30
Exploiting Web Images for Video Highlight Detection With Triplet Deep Ranking
Kim, H., Mei, T., Byun, H., & Yao, T. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2806224 · Cited by 29
“ory (LSTM) cells is calculated by [50] and [51], respectively.”
MethodExploiting deep convolutional neural networks for a neural-based learning classifier system
Kim, J.-Y., & Cho, S.-B. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.05.137 · Cited by 25
An Evaluation of Parametric Activation Functions for Deep Learning
Godfrey, L. B. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8913972 · Cited by 38
“Unintuitively, networks without access to the identity function tend to perform worse as they become deeper [20], [21], [22].”
BackgroundSaraswat, S., Gupta, A., Gupta, H. P., & Dutta, T. (2019) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2019.2961066 · Cited by 25
“Next, the time complexity of CNN is given as O( ∑d i=1 pi−1 ·F2 i · fi ·o(2)i ) [23], where d is the number of convolution layers, p is number of input channels, F is spatial size of filter, f is number of filters, and o is size of output feature map, at ith layer.”
BackgroundSun, R., Sun, X., Chen, F., Pan, H., & Song, Q. (2020) · International Journal of Remote Sensing · doi:10.1080/01431161.2020.1727584 · Cited by 25
“In the reference of Kaiming and Sun (2015), the consecutive 2 × 2 convolutions were e ff ectively used in a relatively shallow CNN architecture.”
MethodAn Autoencoder-Based Deep Learning Approach for Load Identification in Structural Dynamics
Rosafalco, L., Manzoni, A., Mariani, S., & Corigliano, A. (2021) · Sensors · doi:10.3390/s21124207 · Cited by 26
“Skip connections are employed to address the degradation problem usually suffered by deep NNs [38].”
MethodUncertainty-Aware Fast Curb Detection Using Convolutional Networks in Point Clouds
Jung, Y., Jeon, M., Kim, C., Seo, S., & Kim, S.-W. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561358 · Cited by 25
“In addition, most of the operations in the detection network were based on simple convolutional layers [37].”
BackgroundSDN-Based Application-Aware Segment Routing for Large-Scale Network
Tong, V., Souihi, S., Tran, H. A., & Mellouk, A. (2021) · IEEE Systems Journal · doi:10.1109/jsyst.2021.3123809 · Cited by 25
“Regarding time complexity of this module, it is based on CNN, and the total time complexity of all convolutional layers [27] is estimated by the following:”
MethodGraham, C. A., Shamkhalichenar, H., Browning, V. E., Byrd, V. J., Liu, Y., Gutierrez‐Wing, M. T., Novelo, N. D., Choi, J., & Tiersch, T. R. (2022) · Aquaculture · doi:10.1016/j.aquaculture.2022.738039 · Cited by 27
Hybrid U‐Net‐based deep learning model for volume segmentation of lung nodules in CT images
Wang, Y., Zhou, C., Chan, H., Hadjiiski, L. M., Chughtai, A. R., & Kazerooni, E. A. (2022) · Medical Physics · doi:10.1002/mp.15810 · Cited by 27
“36,39 In general, each layer will produce a lossy-compression-like effect after passing through the convolution kernel.”
BackgroundDing, X., Ni, T., Zou, Y., & Zhang, G. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3211845 · Cited by 24
“In the sprit of [28] and [29], the computational-complexity formulas of the C-CNN-LSTM, of the conventional CD-DetectNets, of the DetectNet, of the CNN, and of the ED can be respectively given by Table II, wherein l is the index of the neural-network layer. c l , k l and m l…”
Method
Metadata
- Authors
- He, K., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 5353-5360
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1932847118
- Cited by
- 1,558
- References
- 30
- Citations by year
- 2026: 32 · 2025: 91 · 2024: 143 · 2023: 174 · 2022: 216 · 2021: 252 · 2020: 227 · 2019: 209 · 2018: 110 · 2017: 61 · 2016: 32 · 2015: 11
- FWCI
- 29.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, time complexity, inference speed, network architecture design, speed-accuracy trade-off, ImageNet classification, AlexNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.