Convolutional neural networks at constrained time cost
Open accessThe DOI matches its Crossref record; 26 of the 30 references it cites were found in a registry or an index.
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The performance of neural network models is closely related to the setting of hyperparameters [42].”
From Prediction of Heat and Cold Loads of Factory Mushroom Houses Based on EWT Decomposition (2023) · cites this paper as [42]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 9×
“No other data augmentation, such as scale jittering [10, 22, 9], is used.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 14×
“The actual running time of our model is 20% faster than AlexNet.”
BackgroundMethodResultInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 7×
“Through the investigations, we obtain a model that achieves 11.8% top-5 error (10-view test) on ImageNet [3] and only takes 3 to 4 days training on a single GPU.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Self-citationBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Background
Research works printed but not linked: 1
- lin, 2013, Network in network, arXiv 1312 4400
Cited by
Items 801–900 of 1,001
Rasch, M. J., Gokmen, T., Rigotti, M., & Haensch, W. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00753 · Cited by 14
“…o × n s with an input matrix of size n s × n p is not proportional to the overall amount of compute ( ∝ n o n s n p , as for conventional hardware; He and Sun, 2015), but instead only scales linearly with the number of columns of the input matrix n p and is invariant with respect to the size of…”
BackgroundAn Automatic Design of Factors in a Human-Pose Estimation System Using Neural Networks
Chan, K.-C., Koh, C., & Lee, C. S. G. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2557218 · Cited by 13
Going Deeper into OSNR Estimation with CNN
Shen, F., Zhou, J., Huang, Z., & Li, L. (2021) · Photonics · doi:10.3390/photonics8090402 · Cited by 13
“In [47], He investigated the relationship between the accuracy of CNNs and the complexity.”
BackgroundLocomotion Mode Recognition Using Sensory Data With Noisy Labels: A Deep Learning Approach
Mishra, R., Gupta, A., & Gupta, H. P. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3135878 · Cited by 16
“per time step of the input sequence [36], where l is the number of convolution layers, c is the number of input channels, u is number of filters, s is the size of a filter, and v is the size of output feature vector, at i th convolution layer.”
BackgroundWu, W., Yang, F., Yang, G., Wang, X., Si, P., Zhang, Y., & Yu, F. R. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3190883 · Cited by 16
“1) Centralized Training Process: Similar to [30], the time complexity for one forward propagation or one backward propagation on the Q-evaluation network or the Q-target network is O (cid:19)(cid:2) (cid:20) where V is the number of fully connected layers, L v is the number of neurons in the v -th…”
BackgroundFeature Distillation Siamese Networks for Object Tracking
Huang, H., Liu, G., Zhang, Y., & Xiong, R. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109912 · Cited by 14
Deep learning based text detection using resnet for feature extraction
Huang, L.-K., Tseng, H., Hsieh, C.-C., & Yang, C.-S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15449-z · Cited by 13
“However, the experiments show that the degradation problem has been observed in more profound neural network training [10].”
BackgroundStress Monitoring in Free-Living Environments
Sah, R. K., Cleveland, M. J., & Ghasemzadeh, H. G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3315755 · Cited by 15
“Furthermore, the training complexity of a machine learning model depends linearly on the number of parameters of the model or has a polynomial relationship with the size of the dataset [24], [25], [26].”
BackgroundLi, W., Chen, Q., Gu, G., & Sui, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122038 · Cited by 15
Heart Murmur Severity Stages Classification Using Multikernel Residual CNN
Das, S., & Dandapat, S. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3373226 · Cited by 14
“RL creates a deeper structure while ensuring the detection performance remains uncompromised [40], [41].”
BackgroundEnd-to-End Direct Phase Retrieval From a Single-Frame Interferogram Based on Deep Learning
Zhang, T., Lu, M.-F., Hu, Y. G., Hao, Q., Wu, J.-M., Zhang, N., Yang, S., He, W., Zhang, F., & Tao, R. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3418112 · Cited by 25
“In terms of neural network training, [36] has shown that residual learning can effectively cope with the degradation problem [47], making the training of a large model much easier.”
BackgroundLi, Y., Xu, Y., Li, W., Li, G., Feng, Z., Liu, S., Du, J., & Li, X. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3552297 · Cited by 14
“The FLOPs (Floating Point Operations Per Second) of the CNN for one time step can be roughly estimated as [45]: where input channels and output channels refer to the distinct components of the initial data, like the red, green, and blue channels in an RGB image.”
Residual Recurrent Neural Network with Sparse Training for Offline Arabic Handwriting Recognition
Yan, R., Peng, L., Bin, G., Wang, S., & Cheng, Y. (2017) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.171 · Cited by 13
Generation of a diagnosis model for hybrid-electric vehicles using machine learning
Meckel, S., Schuessler, T., Jaisawal, P. K., Yang, J.-U., & Obermaisser, R. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103071 · Cited by 15
A hybrid neural network for predicting Days on Market a measure of liquidity in real estate industry
Kamara, A. F., Chen, E., Liu, Q., & Pan, Z. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106417 · Cited by 15
Padmasini, N., & Rengasamy, U. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2020.0612 · Cited by 13
“However, the accuracy also gets saturated and degrades rapidly [20, 21].”
BackgroundAdversarial interference mitigation for automotive radar
Jiang, C., Chen, T., & Yang, B. (2021) · IEEE Radar Conference (RadarConf) · doi:10.1109/radarconf2147009.2021.9455216 · Cited by 13
“Specifically, it is in the order of O ( (cid:80) i H i W i C i k i l i C i +1 ) [26], where H i , W i and C i denote the height, width and channel number of the i -th layer, respectively. k i and l i are the height and width of the convolution kernel in this layer.”
BackgroundUltrasound Medical Images Classification Based on Deep Learning Algorithms: A Review
Kareem, F. Q., & Abdulazeez, A. M. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4621288 · Cited by 17
Deep Learning of Transferable MIMO Channel Modes for 6G V2X Communications
Cazzella, L., Tagliaferri, D., Mizmizi, M., Badini, D., Mazzucco, C., Matteucci, M., & Spagnolini, U. (2022) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2022.3169950 · Cited by 13
Towards Feature-free TSP Solver Selection: A Deep Learning Approach
Zhao, K., Liu, S., Yu, J. X., & Rong, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533538 · Cited by 30
Image Classification and Recognition of Medicinal Plants Based on Convolutional Neural Network
Liu, Y., Li, Y., Zhao, Y., & Na, X. (2021) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct52962.2021.9658028 · Cited by 13
“The local receptive field allows neurons in each plane of each layer to extract the local features of a specific area in the image [8].”
BackgroundA Survey: Image Classification Models Based on Convolutional Neural Networks
Peng, L., Qiang, B., & Wu, J. (2022) · International Conference on Computer Research and Development · doi:10.1109/iccrd54409.2022.9730565 · Cited by 14
“When the complexity of a model increases, the optimization of stochastic gradient descent becomes difficult, which will result in gradient vanishing, gradient explosion and degradation problem [7].”
BackgroundENG-Face: cross-domain heterogeneous face synthesis with enhanced asymmetric CycleGAN
Zhang, Y., Yu, L., Sun, B., & He, J. Y. D. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03302-z · Cited by 13
“Given the specific computational cost, the trade-offs among the layer depth, number of channels, kernel sizes could be investigated when designing the architectures of networks [39].”
BackgroundBoosting RUL Prediction Using a Hybrid Deep CNN-BLSTM Architecture
Remadna, I., Terrissa, L. S., Sayah, M., Ayad, S., & Noureddine Zerhouni, A. (2022) · Automatic Control and Computer Sciences · doi:10.3103/s014641162204006x · Cited by 13
“The complexity of the pooling and the FC layers takes 5–10% of the overall computational time [21].”
BackgroundPatel, M. J., Kothari, A. M., & Koringa, H. P. (2022) · RADIOELECTRONIC AND COMPUTER SYSTEMS · doi:10.32620/reks.2022.3.12 · Cited by 16
“However, given the concern of cost-effectiveness in training time and commercial cost, trade-offs between layer depth, the number of channels, kernel sizes, and other network attributes must still be considered when designing network architectures [31] for experimental research and practical application.”
BackgroundSingha, A., & Bhowmik, M. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14098-y · Cited by 19
“In case of TNBC dataset, the network models listed are DeconvNet [22], Ensemble [18], two-stage learning U-Net (DLA) [19], two-stage stacked U-Net (SU-Net) [12], and gradient boosting network U-Net (GB U-Net) [23].”
MethodDT-CGRA: Dual-track coarse-grained reconfigurable architecture for stream applications
Fan, X., Li, H., Cao, W., & Wang, L. (2016) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2016.7577309 · Cited by 16
“For example, the convolutional layers and the fully connected layers account for nearly 90% of the computation time according to [14] in the state-of-the-art convolutional neural network (CNN) [13].”
BackgroundResidual Encoder and Convolutional Decoder Neural Network for Glioma Segmentation
Pawar, K., Chen, Z., Shah, N. J., & Egan, G. F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75238-9_23 · Cited by 14
Product Surface Defect Detection Based on Deep Learning
Lien, P. C., & Zhao, Q. (2018) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/datacom/cyberscitec.2018.00051 · Cited by 13
“On the other hand, CNN is the best known “deep learner” for image classification [10].”
BackgroundMethodSaguil, D., & Azim, A. (2020) · IEEE Access · doi:10.1109/access.2020.2981411 · Cited by 15
“However, convolutional layers have high complexity; their complexity can be seen in Equation 2 [17], where n l is the number of filters, s l is the size of the filter, and m l is the size of the output.”
BackgroundMultisource Data Fusion Diagnosis Method of Rolling Bearings Based on Improved Multiscale CNN
Jin, Y., Chen, C., & Zhao, S. (2021) · Journal of Sensors · doi:10.1155/2021/2251530 · Cited by 14
“For a convolutional neural network, its time complexity can be expressed as where M is the size of the output feature map, K is the size of the convolution kernel, C in is the number of input channels of the convolution layer, and C out is the number of output channels of the convolution layer [48].”
BackgroundAdaptive hierarchical hyper-gradient descent
Jie, R., Gao, J., Vasnev, A. L., & Tran, M. (2022) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-022-01625-4 · Cited by 19
“For convolutional neural networks, we have learned that the total time complexity of all convolutional layers is [22]:”
BackgroundAn interpretable semi-supervised framework for patch-based classification of breast cancer
Shawi, R. E., Kilanava, K., & Sakr, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-20268-7 · Cited by 14
“Then, updating filter weights of l − th convolution layer for one input costs is O ( f l s 2 l n l m 2 l ) 73 .”
Ultrasound Medical Images Classification Based on Deep Learning Algorithms: A Review
admin, A., & Abdulazeez, A. M. (2021) · Fusion Practice and Applications · doi:10.54216/fpa.030102 · Cited by 16
Zhang, Z., Xing, F., Su, H., Shi, X., & Yang, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.07281 · Cited by 20
“is raised by a common question: why it is difficult to train very deep networks? For examples, suppose we can train a 30 layer network, when the depth increases to 30 + 10. The error rate will rise up [71,196]. However, intuitively, if we setting the extra 10 layers as identity mapping, i.e., passing the same output of…”
BackgroundYe, M., Pan, C., Xu, Y., & Li, C. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3391309 · Cited by 20
“According to [30] and [47], the required FLOPs of the proposed Pix2pix-cGAN become (cid:2) ) , where M s , 1 and M s , 2 denote width and height of feature map output in the s -the layer, respectively.”
BackgroundSmart Jamming for Secrecy: Deep Reinforcement Learning Enabled Secure Visible Light Communication
Liu, S., Liu, X., Du, X., & Guizani, M. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3458046 · Cited by 12
Bi, C., Liu, D., Zhu, L., Lu, C., Li, S., & Tang, Y. (2024) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2024.110376 · Cited by 16
Heritage Image Classification by Convolution Neural Networks
Vu, M.-T., Beurton‐Aimar, M., & Le, V.-L. (2018) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr.2018.8337517 · Cited by 12
“input = output and so skipping the learning progress on some layers [16].”
BackgroundArtificial intelligence enhances the performance of chaotic baseband wireless communication
Ren, H., Yin, H., Zhao, H., Bai, C., & Grebogi, C. (2021) · IET Communications · doi:10.1049/cmu2.12162 · Cited by 15
“The total time required of all convolutional layers [35] is the order of”
BackgroundModelling the Refractive Index Structure Parameter: A ResNet Approach
Lamprecht, C., Bekhrad, P., Ivanov, H., & Leitgeb, E. (2020) · International Conference on Broadband Communications for Next Generation Networks and Multimedia Applications (CoBCom) · doi:10.1109/cobcom49975.2020.9174186 · Cited by 12
“decreases with an increasing network depth [13] [14].”
BackgroundDSCP: Depthwise Separable Convolution‐Based Passive Indoor Localization Using CSI Fingerprint
Han, C., Xun, W., Sun, L., Lin, Z., & Guo, J. W. (2021) · Wireless Communications and Mobile Computing · doi:10.1155/2021/8821129 · Cited by 12
CNN-Bi-LSTM Based Household Energy Consumption Prediction
Gaur, K., & Singh, S. K. (2021) · doi:10.1109/icspc51351.2021.9451797 · Cited by 12
An application of CNN to classify barchan dunes into asymmetry classes
van der Merwe, B., Pillay, N., & Coetzee, S. (2022) · Aeolian Research · doi:10.1016/j.aeolia.2022.100801 · Cited by 12
A novel denoising method for low SNR NMR logging echo signal based on deep learning
Liu, Y., Cai, J., Jiang, Z., Zhang, P., & Cheng, J. (2022) · Measurement Science and Technology · doi:10.1088/1361-6501/ac97fc · Cited by 12
“Three factors should be taken into account when we design a network: depth, width, convolutional kernel size [26].”
Cone Geometry-Based Simultaneous 3D Position and Orientation Estimation for Visible Light Systems
Lu, Y., Gao, S., Qiu, H., Chen, Y., & Tu, G. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3212321 · Cited by 14
“For the FP-SPO system, the time complexity of training step calculated from [47] is O ( N fi N t N ALL N p ) , where N fi is the iteration number for training and N t is the number of training samples.”
MethodJi, Y., & Chen, L. (2023) · IEEE Transactions on Semiconductor Manufacturing · doi:10.1109/tsm.2023.3264255 · Cited by 12
Multi-source Domain Adaptation for Unsupervised Road Defect Segmentation
Yu, J., Oh, H., Fichera, S., Paoletti, P., & Luo, S. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161099 · Cited by 14
“The computational complexity of our method, which is estimated based on the model complexity study of convolutional neural networks [37], is O(ns ∑d i=1WiHiDiwihidi), where n s denotes the number of source domains.”
MethodTransformer-based rapid human pose estimation network
Wang, D., Xie, W., Cai, Y., Li, X., & Liu, X. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.09.001 · Cited by 14
Du, J., Yang, T., Xue, C., Chai, J., Zhao, Y. K., & Shi, S. (2020) · Optics Communications · doi:10.1016/j.optcom.2020.126007 · Cited by 12
Deep Learning for Resource-Limited Devices
Merzoug, M. A., Mostefaoui, A., Kechout, M. H., & Tamraoui, S. (2020) · ACM Symposium on QoS and Security for Wireless and Mobile Networks (Q2SWinet) · doi:10.1145/3416013.3426445 · Cited by 14
“) yields larger activation maps and higher classification accuracy [17].”
BackgroundChinese Decorative Color Based on Improved AlexNet in Interior Decoration Design
Shan, W. L., Jin, R. M., & Ding, X. Y. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/2358905 · Cited by 12
“is paper attempts to explore the relationship between the interior ecological landscape and interior decoration design by analyzing the relevant theories and design techniques and with the help of the improved AlexNet convolution neural network [24, 25].”
MethodELM-based convolutional neural networks making move prediction in Go
Zhao, X., Ma, Z., Li, B., Zhang, Z., & Liu, H. (2018) · Soft Computing · doi:10.1007/s00500-018-3158-1 · Cited by 13
“It requires multiple iterations to adjusting weights of each layer to optimal and takes much time (He and Sun 2015).”
BackgroundReal-time convolutional networks for sonar image classification in low-power embedded systemsPreprint
Valdenegro-Toro, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02153 · Cited by 14
“A large literature exists about model compression [4] [5] [6] [7], where a machine learning model is compressed in order to reduce the number of parameters, typically by slightly reducing model performance.”
BackgroundWhen Work Matters: Transforming Classical Network Structures to Graph CNNPreprint
Zhao, W., Xu, C., Cui, Z., Zhang, T., Jiang, J., Zhang, Z., & Yang, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02653 · Cited by 19
“And it addresses the degradation problem [26], [27] of accuracy that arises with the network depth increasing by introducing a deep residual learning framework.”
BackgroundCNN-based Channel Estimation using NOMA for mmWave Massive MIMO System
Anu, T. S., & Raveendran, T. (2023) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp53291.2023.10207968 · Cited by 12
“The computational complexity of CNN processing, according to [14], is:”
BackgroundNumber of FLOPs of Training DNNs for Learning Precoding
Cong, P., & Yang, C. (2023) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2023-spring57618.2023.10200945 · Cited by 13
“In [13], the time complexity of training was analyzed, but only the FLOPs in a single mini-batch were considered.”
BackgroundChen, Y., Qiu, S., Wang, Z., Zhao, H., & Cao, X. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3376017 · Cited by 11
“D CNN layer is provided in [41], and we define the complexity of the 1-”
BackgroundFusion of satellite and street view data for urban traffic accident hotspot identification
Guo, W., Cheng, X., & Jin, S. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103853 · Cited by 28
Dogru, D., Özdemir, G. D., Özdemir, M. A., Ercan, U. K., Topaloğlu, N., & Güren, O. (2024) · BMC Medical Imaging · doi:10.1186/s12880-024-01332-2 · Cited by 13
Wu, H., Wang, P., & Narayana, C. V. A. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3577910 · Cited by 17
“BP for CNN: From [55], the complexity of the BP process for convolutional layers is roughly twice that of the FP process.”
Road structure classification through artificial neural network for automotive radar systems
Sim, H., Lee, S., Lee, B., & Kim, S. (2019) · IET Radar Sonar & Navigation · doi:10.1049/iet-rsn.2018.5610 · Cited by 11
“In addition, s l is the spatial size of the filter, n l is the number of filters in the lth layer, and m l is the spatial size of the output feature map [28].”
BackgroundTag Prediction at Flickr: A View from the Darkroom
Boakye, K., Farfade, S., Izadinia, H., Kalantidis, Y., & Garrigues, P. J. (2017) · Thematic Workshops of ACM Multimedia · doi:10.1145/3126686.3126745 · Cited by 12
“We draw inspiration from the layer replacement strategy used by He and Sun in [10].”
BackgroundMethodWide-Band Butterfly Network: Stable and Efficient Inversion Via Multi-Frequency Neural Networks
Li, M., Demanet, L., & Zepeda-Núñez, L. (2022) · Multiscale Modeling and Simulation · doi:10.1137/20m1383276 · Cited by 18
Echo state network based symbol detection in chaotic baseband wireless communication
Yin, H., Bai, C., & Ren, H. (2022) · Digital Communications and Networks · doi:10.1016/j.dcan.2022.06.006 · Cited by 12
Unsupervised Deep Learning Approach for Near Optimal Power Allocation in CRAN
Labana, M., & Hamouda, W. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3082776 · Cited by 14
Li, W., Wang, J., Li, L., Peng, Q., Wei, H., Chen, X., & Li, S. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3098978 · Cited by 13
Multi-scale multi-stream deep network for car logo recognition
Surwase, S. V., & Pawar, M. M. (2021) · Evolutionary Intelligence · doi:10.1007/s12065-021-00671-1 · Cited by 11
“Consequently,increase in network depth degrades training accuracy [10, 39].”
BackgroundAutomatic modulation classification of noise‐like radar intrapulse signals using cascade classifier
Meng, X., Chao-xuan, S., Dong, J., Fu, X., & Lang, P. (2021) · ETRI Journal · doi:10.4218/etrij.2020-0338 · Cited by 11
Zhang, C., Bai, Y., Yang, C., Cheng, R., Tan, X., Zhang, W., & Zhang, G. (2022) · Biochemical and Biophysical Research Communications · doi:10.1016/j.bbrc.2022.06.004 · Cited by 13
Spectrum Sensing Using CNN With Attention on Switch of Channel States
Cong, Z., Jin, M., Guo, Q., Zhou, Z., & Tian, Y. (2022) · IEEE Communications Letters · doi:10.1109/lcomm.2022.3193302 · Cited by 11
“(cid:12) The complexity of a CNN network is given by O ( =1 n i − 1 s i n i m i ) where I is the number of Conv layers, n i − 1 is the number of input channels in the i th layer, s i is the size of the convolution kernel, and m i is the size of the output from the i th layer [18], [22].”
BackgroundDeLightLCD: A Deep and Lightweight Network for Loop Closure Detection in LiDAR SLAM
Xiang, H., Zhu, X., Shi, W., Fan, W., Chen, P., & Bao, S. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3206506 · Cited by 11
“Solid evidence is yielded in [31] that increasing the depth is a prior condition for improving the accuracy.”
BackgroundAdversarial data hiding with only one pixel
Li, M., Wang, X., Cui, Q., & Zhang, J. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103222 · Cited by 13
ElGhondakly, R., Moussa, S. M., & Badr, N. L. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110430 · Cited by 11
Topic to Image: A Rumor Detection Method Inspired by Image Forgery Recognition Technology
Pang, Y., Li, X., Wei, S., Li, Q., & Xiao, Y. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3302307 · Cited by 12
“According to the time complexity formula for CNNs [43], Algorithm 2 has a time complexity of where D denotes the number of convolutional layers, F l is the edge length of the output feature graph of each convolutional kernel, K l is the edge length of each convolutional kernel, C ( l − 1 ) denotes…”
BackgroundAydemir, M. S., & Bilgin, G. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.02.092 · Cited by 11
Image Classification Based on Convolutional Denoising Sparse Autoencoder
Chen, S., Liu, H., Zeng, X., Qian, S., Yu, J., & Guo, W. (2017) · Mathematical Problems in Engineering · doi:10.1155/2017/5218247 · Cited by 12
“Most recently, He and Sun [66] propose the theoretical complexity of all convolutional layers.”
BackgroundTime-efficient offloading for machine learning tasks between embedded systems and fog nodes
Saguil, D., & Azim, A. (2019) · IEEE International Symposium on Real-Time Distributed Computing (ISORC) · doi:10.1109/isorc.2019.00022 · Cited by 13
“The resulting complexity, as calculated by Microsoft Research in [9], shows that these layers have a time complexity which has an exponentially larger growth than the dense layers in normal neural networks.”
MethodSiamese Dilated Inception Hashing With Intra-Group Correlation Enhancement for Image Retrieval
Lu, X., Chen, Y., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2935118 · Cited by 12
“Following [47], the computational cost of each convolutional layer can be given as O(M2 ·K2 · Cin ·Cout), where M denotes the size of the feature map in the previous layer and K denotes the size of each convolution kernel in this layer.”
MethodSmart connected parking lots based on secured multimedia IoT devices
Merzoug, M. A., Mostefaoui, A., Gianini, G., & Damiani, E. (2021) · Computing · doi:10.1007/s00607-021-00921-1 · Cited by 12
“– Delayed downsampling the delayed reduction (e.g., pooling layers, etc.) gives larger activation maps, which in turn lead to higher classification accuracy [18].”
BackgroundThe Complex Convolutional Neural Network for Adaptive Spatio-Temporal Broadband Beamforming
Wu, X., Xue, C., Zhang, S., Zhu, H., Han, Y., & Sheng, W. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3366971 · Cited by 10
Wang, J., Pan, Y., Wang, Y., & Guo, L. (2024) · Applied Thermal Engineering · doi:10.1016/j.applthermaleng.2024.124226 · Cited by 10
Sherstyukov, R. O., Moges, S. T., Kozlovsky, A. E., & Ulich, T. (2024) · Earth and Space Science · doi:10.1029/2023ea003446 · Cited by 10
“The increasing of neural network depth cause accuracy saturates and then degrades rapidly (He & Sun, 2015).”
Zhang, E., Yao, M., Li, Y., Wang, Q., Song, X., Chen, Y., Liu, K., Zhao, W., Xing, X., Zhou, Y., et al. (2024) · BMC Medical Imaging · doi:10.1186/s12880-024-01489-w · Cited by 10
Zou, L., Vo, Q. H., Kim, K. T., Le, H. Q., Thwal, C. M., Zhang, C., & Hong, C. S. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3517621 · Cited by 14
“As shown in [69], the overall time complexity of all the convolutional layers is given by summarizing the TC of each convolutional layer.”
HFFTrack: Transformer tracking via hybrid frequency features
Ma, S., Wan, Z., Zhang, L., Hu, B., Zhang, J., & Zhao, X. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107269 · Cited by 11
Learning deep transmission network for efficient image dehazing
Ling, Z., Fan, G., Gong, J., & Guo, S. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5687-0 · Cited by 10
“A recent study [15] suggests that CNNs could benefit from increasing the depth of network moderately.”
BackgroundFacing Erosion Identification in Railway Lines Using Pixel-Wise Deep-Based Approaches
Nogueira, K., Machado, G. L. S., Gama, P. H. T., da Silva, C. C. V., Balaniuk, R., & dos Santos, J. A. (2020) · Remote Sensing · doi:10.3390/rs12040739 · Cited by 10
Lin, H., Quan, P., Liang, Z., Lou, Y., Wei, D., & Di, S. (2022) · Sensors · doi:10.3390/s22093439 · Cited by 10
“The total computational complexity of all convolutional layers can be expressed as follows [52]: O n ∙ s ∙ n ∙ m (13) where d is the number of convolutional layers and l is the index of a convolutional layer.”
BackgroundA nonlinear solver based on residual network for seepage equation
Li, D., Lv, S., Zha, W., Shen, L., & Xing, Y. T. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106850 · Cited by 10
Multivariate CNN Model for Human Locomotion Activity Recognition with a Wearable Exoskeleton Robot
Son, C.-S., & Kang, W. (2023) · Bioengineering · doi:10.3390/bioengineering10091082 · Cited by 11
“We limited convolutional blocks to three to avoid degradation from potential gradient vanishing and exploding as network depth increases [51–53].”
Zhou, X., Hu, K., Guan, Z., Yu, C., Wang, S., Fan, M., Sun, Y., Cao, Y., Wang, Y., & Miao, G. (2023) · Ecological Indicators · doi:10.1016/j.ecolind.2023.110908 · Cited by 10
Sohaib, M., Hasan, M. J., Chen, J., & Zheng, Z. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad296c · Cited by 10
“However, due to the known gradient vanishing problem, training a deeper network is challenging [36].”
A Deep Learning Approach for Website Fingerprinting Attack
Xiao-min, H., Wang, J., He, Y., & Shi, Y. (2018) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/compcomm.2018.8780755 · Cited by 10
“However, in some practical classification tasks, the use of these methods to optimize deep neural networks does not improve the accuracy of classification, but instead causes the problem of reduced accuracy [10], which is not due to overfitting.”
BackgroundTrajectory Data Driven V2V/V2I Mode Switching and Bandwidth Allocation for Vehicle Networks
Zhang, Z., Li, X., Liu, D., Luo, T., & Zhang, Y. (2020) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2020.2970054 · Cited by 10
“According to [13], the complexity of a DNN model depends on its width, depth, and number of parameters.”
BackgroundKey, C., & Notaroš, B. M. (2020) · IEEE Antennas and Wireless Propagation Letters · doi:10.1109/lawp.2020.2973937 · Cited by 10
“case where the fully connected network has a similar number of neurons in a hidden layer as the number of inputs [19].”
BackgroundAn Experimental Study of the Impact of Pre-Training on the Pruning of a Convolutional Neural Network
Hubens, N., Mancaş, M., Décombas, M., Preda, M., Zaharia, T. B., Gosselin, B. H., & Dutoit, T. (2020) · Applications Intelligent Systems · doi:10.1145/3378184.3378224 · Cited by 10
“This computation cost can be reduced by downsampling the feature maps earlier in the network [11] through various pooling mechanisms or by replacing the convolution operations by factorized convolutions [4, 14].”
DISL: Deep Isomorphic Substructure Learning for network representations
Cui, S., Li, T., Chen, S., Shyu, M., Li, Q., & Zhang, H. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.105086 · Cited by 10
Scene-Aware Error Modeling of LiDAR/Visual Odometry for Fusion-Based Vehicle Localization
Ju, X., Xu, D., & Zhao, H. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3058054 · Cited by 11
Zhang, W., Huang, Y., Tong, J., Bao, M., & Li, X. (2021) · Sensors · doi:10.3390/s21082767 · Cited by 10
“According to [34] and network structures involved in the article, the computational complexity of the CFCN and CNN-DOA are O(M 3 4 I 2F) and O(M×MF×M/2× 64× 64 + 2× 64× F× 512 + 2× 512× I), respectively.”
Background
Metadata
- Authors
- He, K., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 5353-5360
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1932847118
- Cited by
- 1,558
- References
- 30
- Citations by year
- 2026: 32 · 2025: 91 · 2024: 143 · 2023: 174 · 2022: 216 · 2021: 252 · 2020: 227 · 2019: 209 · 2018: 110 · 2017: 61 · 2016: 32 · 2015: 11
- FWCI
- 29.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, time complexity, inference speed, network architecture design, speed-accuracy trade-off, ImageNet classification, AlexNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.