Convolutional neural networks at constrained time cost
Open accessThe DOI matches its Crossref record; 26 of the 30 references it cites were found in a registry or an index.
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The performance of neural network models is closely related to the setting of hyperparameters [42].”
From Prediction of Heat and Cold Loads of Factory Mushroom Houses Based on EWT Decomposition (2023) · cites this paper as [42]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 9×
“No other data augmentation, such as scale jittering [10, 22, 9], is used.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 14×
“The actual running time of our model is 20% faster than AlexNet.”
BackgroundMethodResultInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 7×
“Through the investigations, we obtain a model that achieves 11.8% top-5 error (10-view test) on ImageNet [3] and only takes 3 to 4 days training on a single GPU.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Self-citationBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Background
Research works printed but not linked: 1
- lin, 2013, Network in network, arXiv 1312 4400
Cited by
Items 401–500 of 1,001
Huangfu, B., Cui, C., Zhang, C., & Xu, L. (2022) · IEEE Journal of Emerging and Selected Topics in Power Electronics · doi:10.1109/jestpe.2022.3189078 · Cited by 46
“Referring to [16] and [28], the time complexity of the controller is equal to the time complexity of the actor network, which can be shown as , where M 1 , M 2 are the number of the units in the first and second hidden layer.”
BackgroundZhang, C., Perezhogin, P. A., Gültekin, C., Adcroft, A., Fernandez‐Granda, C., & Zanna, L. (2023) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2023ms003697 · Cited by 57
“The total time complexity (He & Sun, 2014) of one time step inference is where l is the index of a convolutional layer, d is the depth (number of convolutional layers), w l is the number of output channels (also known as “width”) for the l -th layer, w l −1 is the number of input channels of the l…”
BackgroundResource Allocation of Federated Learning for the Metaverse With Mobile Augmented Reality
Zhou, X., Liu, C., & Zhao, J. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3326884 · Cited by 51
“From [48], in the first layer, m 1 can be computed by In the above equation, padding and stride are set in a CNN [48], and s n × s n is the resolution of the input image on MAR device n .”
Joint User Association and Power Control for Cell-Free Massive MIMO
Hao, C., Vu, T. T., Ngo, H. Q., Dao, M. N., Dang, X., Wang, C., & Matthaiou, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3351633 · Cited by 50
“For a single convolutional layer, the complexity is O f 2 map k 2 siz c in c out [49].”
BackgroundFault diagnosis of industrial process using attention mechanism with 3DCNN-LSTM
Chen, Y., Zhang, R., & Gao, F. (2024) · Chemical Engineering Science · doi:10.1016/j.ces.2024.120059 · Cited by 45
Deep learning in material recovery: Development of method to create training database
Vrancken, C., Longhurst, P., & Wagland, S. T. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.01.077 · Cited by 44
Semantic Image Networks for Human Action Recognition
Khowaja, S. A., & Lee, S.-L. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01248-3 · Cited by 44
“2009; Rodriguez et al. 2008; Shechtman and Irani 2005). Ji et al. (2013) proposed the 3D CNNs which capture the spatial as well as temporal information from the videos. The spatial information is captured using 2D, and the motion information from multiple frames is accounted for the third dimension to learn the featur…”
BackgroundInfluentialMulti-Classification Algorithm for Human Motion Recognition Based on IR-UWB Radar
Qi, R. M., Li, X., Zhang, Y., & Li, Y. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3000498 · Cited by 41
“Referring to [32], the time complexity and space complexity of Convolutional where l is the index of a convolutional layer, and d is the number of convolutional layers. n l is the number of filters in the l -th layer.”
BackgroundIntermittent-Aware Neural Architecture Search
Mendis, H. R., Kang, C.-K., & Hsiu, P.-C. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3476995 · Cited by 52
“In each iteration, the NAS controller generates a child network, parameterized by the kernel size (K ) and number of weight kernels (M) per layer, both of which crucially affect model accuracy and inference latency [23].”
MethodCNN based framework for detection of epileptic seizures
Sameer, M., & Gupta, B. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12702-9 · Cited by 45
“In addition to that, all deep learning networks are very time-consuming in the training process, and usually, a GPU is required [15].”
BackgroundGaze Estimation via the Joint Modeling of Multiple Cues
Chen, W., Xu, H. F., Zhu, C., Liu, X., Lu, Y., Zheng, C., & Kong, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3071621 · Cited by 46
A neural network-based PDE solving algorithm with high precision
Jiang, Z., Jiang, J., Yao, Q., & Yang, G. (2023) · Scientific Reports · doi:10.1038/s41598-023-31236-0 · Cited by 56
CNN-Based LOS/NLOS Identification in 3-D Massive MIMO Systems
Zeng, T., Chang, Y., Zhang, Q., Hu, M., & Li, J. (2018) · IEEE Communications Letters · doi:10.1109/lcomm.2018.2872522 · Cited by 41
“Inspired by [13], denote CL number as Lc, and time complexity of proposed CLs is O(LcC2 L(2)exN(2) T ), since the size of ifmaps and ofmaps for a certain CL is nearlinear with LexNT.”
A view-based 3D CAD model reuse framework enabling product lifecycle reuse
Zhang, C., & Zhou, G. (2018) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2018.09.001 · Cited by 42
Pham, H., Khoudour, L., Crouzil, A., Zegers, P., & Velastín, S. A. (2019) · IET Computer Vision · doi:10.1049/iet-cvi.2018.5014 · Cited by 44
“In addition, another problem called “the degradation phenomenon [56]” also impedes the convergence of deeper networks.”
BackgroundMultipath-DenseNet: A Supervised ensemble architecture of densely connected convolutional networks
Lodhi, B. A., & Kang, J. (2019) · Information Sciences · doi:10.1016/j.ins.2019.01.012 · Cited by 43
“However, these networks suffer from convergence complexity, overfitting, and vanishing gradient issues [17,18] .”
BackgroundLi, Y., Wang, X., Liu, D., Guo, Q., Liu, X., Zhang, J., & Xu, Y. (2019) · Applied Sciences · doi:10.3390/app9071361 · Cited by 39
GHTnet: Tri-Branch deep learning network for real-time electricity price forecasting
Yang, H., & Schell, K. R. (2021) · Energy · doi:10.1016/j.energy.2021.122052 · Cited by 47
Deepthi, K., Jereesh, A. S., & Liu, Y. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107945 · Cited by 43
Sensing-Enabled Predictive Beamforming Design for RIS-Assisted V2I Systems: A Deep Learning Approach
Xia, F., Fei, Z., Huang, J., Wang, X., Wang, R., Yuan, W., & Ng, D. W. K. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3327362 · Cited by 47
“For the CAP-Net, the complexity of the CNN module is O τ [51], where N c and subscript n denote the number of convolutional layers and the index of convolutional layers, respectively. τ denotes the number of historical time slots. c i and c n denote the number of input channels and output channels,…”
BackgroundA Deep Neural Network Approach for Classification of GNSS Interference and Jamming
Mehr, I. E., & Dovis, F. (2024) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2024.3462662 · Cited by 50
Predicting repayment of borrows in peer‐to‐peer social lending with deep dense convolutional network
Kim, J., & Cho, S. (2019) · Expert Systems · doi:10.1111/exsy.12403 · Cited by 43
“However, DenseNet includes many structures based on the combination of hyperparameters, which affect the process of capturing the features of the borrower (He & Sun, 2015).”
BackgroundTowards Repayment Prediction in Peer-to-Peer Social Lending Using Deep Learning
Kim, J.-Y., & Cho, S.-B. (2019) · Mathematics · doi:10.3390/math7111041 · Cited by 40
“Hyperparameters affect the process of extracting features, learning time and performance [33].”
BackgroundLu, X., Liao, W., Huang, W., Xu, Y., & Chen, X. (2020) · Journal of Vibration and Control · doi:10.1177/1077546320933756 · Cited by 39
“This phenomenon is similar to the conclusions of He and Sun (2015) and Chollet (2018).”
BackgroundResultInfluentialEnhanced Lightweight Multiscale Convolutional Neural Network for Rolling Bearing Fault Diagnosis
Shi, Y., Deng, A., Deng, M., Zhu, J., Liu, Y., & Cheng, Q. (2020) · IEEE Access · doi:10.1109/access.2020.3041735 · Cited by 40
A deep neural network based method for magnetic anomaly detection
Wang, Y., Han, Q., Zhao, G., Li, M., Zhan, D., & Li, Q. (2021) · IET Science Measurement & Technology · doi:10.1049/smt2.12084 · Cited by 40
Learning to Schedule Joint Radar-Communication With Deep Multi-Agent Reinforcement Learning
Lee, J., Niyato, D. T., Guan, Y. L., & Kim, D. I. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3124810 · Cited by 42
“For each convolutional layer j , the time complexity scales with O ( n j − 1 s 2 j n j m 2 j ) , where n j is the number of filters in the j th layer, s j is the size of each filter, and m j is the spatial size of the output [43].”
BackgroundDeep residual networks for plankton classification
Li, X., & Cui, Z. (2016) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans.2016.7761223 · Cited by 55
“According to [7, 8], adding more layers to a suitable deep model leads to higher training error.”
BackgroundCuriale, A. H., Colavecchia, F. D., & Mato, G. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.002 · Cited by 43
“Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as it was reported by [15].”
BackgroundPham, H., Salmane, H., Khoudour, L., Crouzil, A., Zegers, P., & Velastín, S. A. (2019) · Sensors · doi:10.3390/s19081932 · Cited by 45
“In particular, D-CNNs with architectures such as DenseNet [54] can improve accuracy in 105 the image recognition task since this kind of network is able to prevent overfitting and degradation 106 phenomena [61] by maximizing information flow and facilitating features reuse as each layer in its 107 architecture has dir…”
BackgroundZhou, G., Yang, X., Zhang, C., Li, Z., & Xiao, Z. (2019) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2019.04.007 · Cited by 39
Wide-residual-inception networks for real-time object detection
Lee, Y., Kim, H., Park, E., Cui, X., & Kim, H. (2017) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2017.7995808 · Cited by 41
“He et al. [29] experimentally claimed that a network has three factors that are most important when constructing a network architecture: depth, width, and filter size.”
BackgroundOptimizing the Learning Performance in Mobile Augmented Reality Systems With CNN
He, Y., Ren, J., Yu, G., & Cai, Y. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2992329 · Cited by 42
“According to [27], the time complexity of the l-th convolutional layer can be expressed as”
BackgroundImproving the Performance of VGG Through Different Granularity Feature Combinations
Zhou, Y., Chang, H., Lu, Y., Lu, X., & Zhou, R. (2020) · IEEE Access · doi:10.1109/access.2020.3031908 · Cited by 41
“Several works have shown that significant information of the input images is lost with depth in the successful ImageNet classification CNNs [10], and the deeper structure does not always guarantee a better result, as validated by the experiments of He and Sun [11].”
MethodYan, L., Feng, J., Hang, T., & Zhu, Y. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107228 · Cited by 38
Pintelas, E. G., Liaskos, M., Livieris, I. E., Kotsiantis, S. B., & Pintelas, P. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06141-0 · Cited by 40
Han, J., Lee, G. H., Park, S., Lee, J., & Choi, J. K. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3124673 · Cited by 38
Diwakar, M., Singh, P., Karetla, G. R., Narooka, P., Yadav, A., Maurya, R., Gupta, R., Arias‐Gonzáles, J. L., Singh, M. P., Shetty, D. K., et al. (2022) · Electronics · doi:10.3390/electronics11203375 · Cited by 38
RetractedGraph Neural Networks for Joint Communication and Sensing Optimization in Vehicular Networks
Li, X., Chen, M., Liu, Y., Zhang, Z., Liu, D., & Mao, S. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3322761 · Cited by 43
“According to [44], the complexity of training a neural network depends on its width, depth, and number of parameters.”
BackgroundChen, G., Xiang, D., Liu, T., Xu, F., & Li, W. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112180 · Cited by 41
Das, D. K., & Dutta, P. K. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.11.001 · Cited by 42
Vijayan, V., & Sherly, E. (2019) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-169909 · Cited by 38
HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs
Singh, P. P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00497 · Cited by 185
“This raises interest in designing efficient models [7].”
BackgroundPan, G., Fu, L., Yu, R., & Muresan, M. (2019) · International Conference on Transportation Information and Safety · doi:10.1109/ictis.2019.8883540 · Cited by 39
“Since then, substantial research has been done to advance the start of the art in the field, and several useful pre-trained models like ResNet, Google’s Xception and Microsoft’s Inception have been developed [18-21].”
BackgroundInfluentialVirnodkar, S. S., Pachghare, V., Patil, V. C., & Jha, S. K. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.09.005 · Cited by 38
A Globally Regularized Joint Neural Architecture for Music Classification
Ashraf, M., Geng, G., Wang, X., Ahmad, F., & Abid, F. (2020) · IEEE Access · doi:10.1109/access.2020.3043142 · Cited by 42
Polarization imaging based bruise detection of nectarine by using ResNet-18 and ghost bottleneck
Yu, Y., Wang, L., Huang, M., Zhu, Q., & Wang, R. (2022) · Postharvest Biology and Technology · doi:10.1016/j.postharvbio.2022.111916 · Cited by 37
Wang, J., Nishikawa, R. M., & Yang, Y. (2017) · Journal of medical imaging · doi:10.1117/1.jmi.4.2.024501 · Cited by 42
Parallel memetic algorithm for training recurrent neural networks for the energy efficiency problem
Ruiz, L. G. B., Capel, M. I., & Pegalajar, M. C. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.12.028 · Cited by 37
Tongguang, Y., Yu, X., Li, G., Dou, J., & Baojia, D. (2019) · Measurement Science and Technology · doi:10.1088/1361-6501/ab5342 · Cited by 36
“Considering the characteristics of complexity and the intelligence of the aero-engine hydraulic pipeline, as well as its ‘ big data ’ feature, it is difficult for the traditional data method to meet the requirements of actual diagnosis [22].”
MethodPark, S., Im, J., Han, D., & Rhee, J. (2020) · Remote Sensing · doi:10.3390/rs12213499 · Cited by 44
“, support vector regression or RF) approaches [55].”
BackgroundStride and Translation Invariance in CNNs
Mouton, C., Myburgh, J. C., & Davel, M. H. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-66151-9_17 · Cited by 42
“Furthermore, through spatial reduction, the size of the input layer to the fully connected layers is greatly reduced, implying there are less learn-able parameters which further reduces training time and memory consumption [9].”
BackgroundInfluentialPseudo-colored rate map representation for speech emotion recognition
Özer, İ. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102502 · Cited by 37
Hyperspectral-cube-based mobile face recognition: A comprehensive review
Zhang, X., & Zhao, H. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.04.003 · Cited by 41
Han, H., Wang, X., Gu, F., Li, W., Cai, Y., Xu, Y., & Xu, Y. (2021) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3081509 · Cited by 36
Image scrambling adversarial autoencoder based on the asymmetric encryption
Bao, Z., Xue, R., & Jin, Y. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11043-3 · Cited by 41
“Then the time complexity of these convolution of our model is O [16].”
BackgroundDeep learning method for predicting the strengths of microcracked brittle materials
Xu, B., Sang, Y., Li, M., Zhao, H., & Feng, X. (2022) · Engineering Fracture Mechanics · doi:10.1016/j.engfracmech.2022.108600 · Cited by 39
Multi-instance attention network for few-shot learning
Qin, Z., Wang, H., Mawuli, C. B., Han, W., Zhang, R., Yang, Q., & Shao, J. (2022) · Information Sciences · doi:10.1016/j.ins.2022.07.013 · Cited by 40
Used Car Price Prediction Based on the Iterative Framework of XGBoost+LightGBM
Cui, B., Ye, Z., Zhao, H., Renqing, Z., Meng, L., & Yang, Y. (2022) · Electronics · doi:10.3390/electronics11182932 · Cited by 36
Over-the-Air Design of GAN Training for mmWave MIMO Channel Estimation
Doshi, A. S., Gupta, M., & Andrews, J. G. (2022) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2022.3222479 · Cited by 45
RF Transmitter Fingerprinting Exploiting Spatio-Temporal Properties in Raw Signal Data
Roy, D., Mukherjee, T., Chatterjee, M., & Pasiliao, E. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00023 · Cited by 40
“We also limited the number of recurrent layers and fully connected layers for each model for faster training [39], since no significant increase in the validation accuracy was observed after increasing the number of layers.”
BackgroundMulti-User MISO With Stacked Intelligent Metasurfaces: A DRL-Based Sum-Rate Optimization Approach
Liu, H., An, J., Alexandropoulos, G. C., Ng, D. W. K., Yuen, C., & Gan, L. (2025) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2025.3558008 · Cited by 58
“As detailed in [57], the complexity considering the output feature map size can be expressed as c l − 1 × s 2 c × c l × s f , where c l − 1 denotes the number of channels in the previous layer, s c is the convolutional kernel size, c l refers to the output channel number, and s f indicates the…”
Multi-focus Image Fusion using Fully Convolutional Two-stream Network for Visual Sensors
Xu, K., Qin, Z., Wang, G., Zhang, H., Huang, K., & Ye, S. (2018) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2018.05.019 · Cited by 35
“For number of layers, we gain that CNN could achieve good performance by increasing the depth of network moderately [31].”
BackgroundAdversarial Learning for Personalized Tag Recommendation
Quintanilla, E., Rawat, Y. S., Sakryukin, A., Shah, M., & Kankanhalli, M. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2992941 · Cited by 37
“and [70] with a modified loss function for multi-label classification.”
MethodCapsNet regularization and its conjugation with ResNet for signature identification
Jampour, M., Abbaasi, S., & Javidi, M. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107851 · Cited by 35
Benchmarking Modern Edge Devices for AI Applications
Kang, P., & Jo, J. (2021) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2020edp7160 · Cited by 36
“ResNet 7 Additional CNN benchmarks for evaluating GPU systems addresses the “degradation problem” [28] in deep neural networks by using residual blocks and identity blocks.”
BackgroundZhang, X., Fang, X., Pan, M., Yuan, L., Zhang, Y., Yuan, M., Lv, S., & Yu, H. (2021) · Sensors · doi:10.3390/s21217205 · Cited by 44
“With the increase of the network depth, higher training errors will occur in the training process, and the accuracy will reach saturation and drop rapidly [34].”
BackgroundDesign Guidelines for Mammogram-Based Computer-Aided Systems Using Deep Learning Techniques
Azour, F., & Boukerche, A. (2022) · IEEE Access · doi:10.1109/access.2022.3151830 · Cited by 38
“The reference [147] conducted a study under constrained time to solve the issue of time complexity and consequently understand the effect of CNN factors such as depth, width, number of filters, filter size, etc.”
BackgroundDeep Transfer Learning for Automatic Prediction of Hemorrhagic Stroke on CT Images
Rao, B. N., Mohanty, S. S., Sen, K. K., Acharya, U. R., Cheong, K. H., & Sabut, S. (2022) · Computational and Mathematical Methods in Medicine · doi:10.1155/2022/3560507 · Cited by 40
Muhammad, L. J., Haruna, A. A., Usman, S. S., & Mohammed, M. B. (2022) · Health and Technology · doi:10.1007/s12553-022-00711-5 · Cited by 36
“The sharing of weight in the convolutional layer together with appropriate chosen pooling schemes, endows the CNN with some invariance properties [7, 22–24].”
A Survey on Deep Learning in COVID-19 Diagnosis
Han, X., Hu, Z., Wang, S., & Zhang, Y. (2022) · Journal of Imaging · doi:10.3390/jimaging9010001 · Cited by 38
Huynh‐The, T., Nguyen, T.-V., Pham, Q., da Costa, D. B., Kwon, G.-H., & Kim, D. (2022) · IEEE Systems Journal · doi:10.1109/jsyst.2022.3207377 · Cited by 35
Sensitivity of deep learning applied to spatial image steganalysis
Tabares-Soto, R., Arteaga-Arteaga, H. B., Mora-Rubio, A., Bravo-Ortíz, M. A., Arias-Garzón, D., Alzate-Grisales, J. A., Orozco-Arias, S., Isaza, G. A., & Ramos-Pollán, R. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.616 · Cited by 34
DRL-Based Pricing-Driven for Task Offloading and Dynamic Resource in Vehicle Edge Computing
Wu, S., Yang, L., Li, J., Guo, H., Ahmad, I., da Costa, D. B., Jiang, H., & Niyato, D. T. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3569817 · Cited by 40
Residual neural network-based fully convolutional network for microstructure segmentation
Jang, J., Van, D., Jang, H., Baik, D. H., Yoo, S. D., Park, J., Mhin, S., Mazumder, J., & Lee, S. H. (2019) · Science and Technology of Welding & Joining · doi:10.1080/13621718.2019.1687635 · Cited by 35
“More layers require more parameters, which are directly related to computational cost [30].”
BackgroundA Deep Learning Method to Accelerate the Disaster Response Process
Antoniou, V., & Potsiou, C. (2020) · Remote Sensing · doi:10.3390/rs12030544 · Cited by 43
“Furthermore, as [18] explained, modern ML/DL models, while trying to achieve more accurate results, increase the computation cost needed to be trained.”
BackgroundAudio classification using braided convolutional neural networks
Sinha, H., Awasthi, V., & Ajmera, P. K. (2020) · IET Signal Processing · doi:10.1049/iet-spr.2019.0381 · Cited by 38
SP-GAN: Self-Growing and Pruning Generative Adversarial Networks
Song, X., Chen, Y., Feng, Z., Hu, G., Yu, D., & Wu, X. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3005574 · Cited by 37
Big Data for 5G Intelligent Network Slicing Management
Chergui, H., & Verikoukis, C. V. (2020) · IEEE Network · doi:10.1109/mnet.011.1900437 · Cited by 34
Deep ensemble neural-like P systems for segmentation of central serous chorioretinopathy lesion
Xue, J., Wang, Z., Kong, D., Wang, Y., Liu, X., Fan, W., Yuan, S., Niu, S., & Li, D. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.08.016 · Cited by 37
Multi-Occupancy Fall Detection Using Non-Invasive Thermal Vision Sensor
Zhong, C., Ng, W. W. Y., Zhang, S., Nugent, C., Shewell, C., & Medina-Quero, J. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3032728 · Cited by 37
“As to the pooling layers and fully connected layers of CNN, they often take 5% to10% computational time [37].”
BackgroundComputer Vision Aided Codebook Design for MIMO Communications Systems
Chen, J., Gao, F., Tao, X., Liu, G., Pan, C., & Alkhateeb, A. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3217664 · Cited by 36
“According to [36], the time complexity of CNN network is (cid:7)(cid:25) (cid:9)”
BackgroundImage super-resolution with densely connected convolutional networks
Kuang, P., Ma, T., Chen, Z., & Li, F. (2019) · Applied Intelligence · doi:10.1007/s10489-018-1234-y · Cited by 33
“The truth is that deeper structure used in super-resolution task is not able to play a key role, just like it does in image classification field [42].”
BackgroundProgress of the Computer-Generated Holography Based on Deep Learning
Zhang, Y., Zhang, M., Liu, K., He, Z., & Cao, L. (2022) · Applied Sciences · doi:10.3390/app12178568 · Cited by 37
Zhou, Z., Yang, X., Ji, H., & Zhu, Z. (2023) · ICES Journal of Marine Science · doi:10.1093/icesjms/fsad041 · Cited by 37
A Rotating Machinery Fault Diagnosis Method Based on Multi-Sensor Fusion and ECA-CNN
Wang, H., Zhu, H., & Li, H. (2023) · IEEE Access · doi:10.1109/access.2023.3320065 · Cited by 34
A comprehensive review of convolutional neural networks: foundations, enhancements and applications
Reza, M. H., Sibly, M. N. B., Rabbani, M. G., Sadi, S. H., Ahamed, M. F., Shafi, F. B., Sarmun, R., & Chowdhury, M. E. H. (2026) · Neural Computing and Applications · doi:10.1007/s00521-025-11827-w · Cited by 46
“In order to accomplish this, the input image is padded with zeros (or occasionally other values) so that the convolution operation can cover all area of the input, including the edges, without lowering the output size [57].”
CNN Convolutional layer optimisation based on quantum evolutionary algorithm
Lu, T.-C. (2020) · Connection Science · doi:10.1080/09540091.2020.1841111 · Cited by 33
Pathak, D., & Raju, U. S. N. (2021) · Optik · doi:10.1016/j.ijleo.2021.167754 · Cited by 36
Multi-view stereoscopic attention network for 3D tumor classification in automated breast ultrasound
Ding, W., Zhang, H., Zhuang, S., Zhuang, Z., & Gao, Z. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120969 · Cited by 32
CNN-Based Automatic Modulation Classification Under Phase Imperfections
Oikonomou, T. K., Evgenidis, N. G., Nixarlidis, D. G., Tyrovolas, D., Tegos, S. A., Diamantoulakis, P. D., Sarigiannidis, P. G., & Karagiannidis, G. K. (2024) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2024.3379198 · Cited by 34
“Moreover, as shown in [5], more complex CNN architectures typically increase their complexity due to the addition of extra layers and filters, thus it is important to keep the CNN structure as simple as possible to achieve real-time decisions for the AMC task.”
BackgroundMethodInfluentialFast Branch Convolutional Neural Network for Traffic Sign Recognition
Hu, W., Zhuo, Q., Zhang, C., & Li, J. (2017) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2017.2709780 · Cited by 33
“[25] explores the performance and time consumed of different layers in a CNN model, then they propose to design a model at constrained time cost by a series of subjective layer replacement.”
MethodAn Efficient Residual Learning Neural Network for Hyperspectral Image Superresolution
Liu, W., & Lee, J. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2901752 · Cited by 33
Chou, F.-I., Tsai, Y.-K., Chen, Y., Tsai, J., & Kuo, C.-C. (2019) · IEEE Access · doi:10.1109/access.2019.2918563 · Cited by 33
Primary user emulation and jamming attack detection in cognitive radio via sparse coding
Furqan, H. M., Aygül, M. A., Nazzal, M., & Arslan, H. (2020) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-020-01736-y · Cited by 32
“Moreover, the computational complexity of training per sample is roughly double as compared to the complexity of testing per sample [33].”
BackgroundMa, Y., Ning, W., Wang, B., & Liang, X. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3065426 · Cited by 32
Balanced Convolutional Neural Networks for Pneumoconiosis Detection
Hao, C., Jin, N., Qiu, C., Ba, K., Wang, X., Zhang, H., Zhao, Q., & Huang, B. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18179091 · Cited by 33
“It differs from overfitting [28], in which case training loss decreases while test loss increases.”
BackgroundAn Improved Residual Network for Pork Freshness Detection Using Near-Infrared Spectroscopy
Zou, L., Liu, W., Lei, M., & Yu, X. (2021) · Entropy · doi:10.3390/e23101293 · Cited by 32
“Although increasing the depth of CNN can extract more abstract features and therefore improve the performance, it may suffer from degradation [30].”
BackgroundEvolutionary neural networks for deep learning: a review
Ma, Y., & Xie, Y. (2022) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-022-01578-8 · Cited by 39
“The deep learning methods [24] can be divided into supervised learning, semi-supervised learning, and unsupervised learning, where supervision refers to whether the data An important feature of network performance improvement is the increase of model complexity and training parameters.”
BackgroundA hybrid deep learning CNN-ELM approach for parking space detection in Smart Cities
Kaur, R., Roul, R. K., & Batra, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08426-y · Cited by 32
“[73] revealed that the training stage of CNN, including fine-tuning of convolution layer parameters, consumes the maximum amount of time which is represented using Eq.”
Background
Metadata
- Authors
- He, K., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 5353-5360
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1932847118
- Cited by
- 1,558
- References
- 30
- Citations by year
- 2026: 32 · 2025: 91 · 2024: 143 · 2023: 174 · 2022: 216 · 2021: 252 · 2020: 227 · 2019: 209 · 2018: 110 · 2017: 61 · 2016: 32 · 2015: 11
- FWCI
- 29.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, time complexity, inference speed, network architecture design, speed-accuracy trade-off, ImageNet classification, AlexNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.