Convolutional neural networks at constrained time cost
Open accessThe DOI matches its Crossref record; 26 of the 30 references it cites were found in a registry or an index.
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The performance of neural network models is closely related to the setting of hyperparameters [42].”
From Prediction of Heat and Cold Loads of Factory Mushroom Houses Based on EWT Decomposition (2023) · cites this paper as [42]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 9×
“No other data augmentation, such as scale jittering [10, 22, 9], is used.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 14×
“The actual running time of our model is 20% faster than AlexNet.”
BackgroundMethodResultInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 7×
“Through the investigations, we obtain a model that achieves 11.8% top-5 error (10-view test) on ImageNet [3] and only takes 3 to 4 days training on a single GPU.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Self-citationBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Background
Research works printed but not linked: 1
- lin, 2013, Network in network, arXiv 1312 4400
Cited by
Items 201–300 of 1,001
An Accurate and Robust Approach of Device-Free Localization With Convolutional Autoencoder
Zhao, L., Huang, H., Li, X., Ding, S., Zhao, H., & Han, Z. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2907580 · Cited by 91
“According to [59], for a CNN based neural network, the total time complexity of all convolutional layers is given as follows:”
BackgroundExploring the role of deep neural networks for post-disaster decision support
Chaudhuri, N., & Bose, I. (2020) · Decision Support Systems · doi:10.1016/j.dss.2019.113234 · Cited by 80
Deep learning enabled intelligent fault diagnosis: Overview and applications
Duan, L., Xie, M., Wang, J., & Bai, T. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-17938 · Cited by 98
“In real-world industrial applications, engineers and developers are frequently confronted with constrained time budgets [36].”
Ke, H., Chen, D., Shah, T., Liu, X., Zhang, X., Zhang, L., & Li, X. (2018) · Software Practice and Experience · doi:10.1002/spe.2668 · Cited by 97
Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control
Venturini, F., Mason, F., Pase, F., Chiariotti, F., Testolin, A., Zanella, A., & Zorzi, M. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3063170 · Cited by 98
“We can then compute the complexity of one convolutional layer, as done in [29], when neither batch normalization nor pooling layers are present.”
MethodGöçeri, E. (2019) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2019.8936083 · Cited by 95
“Therefore, training of very deep CNNs is hard due to vanishing gradients in the long forward feed and backward propagate process [7,8].”
BackgroundSurface Defect Detection Using Image Pyramid
Xiao, L., Wu, B., & Hu, Y. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.2977366 · Cited by 88
“However, stack more layers to a suitable neural network model led to higher training error [34].”
BackgroundCheng, Y., Xu, Y., Zhong, H., & Liu, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3000771 · Cited by 84
“The computational complexity of TCN and LSTM can be, respectively, defined as O t ( n ) [8], [41], [42] and O l ( n ) [8], [43].”
BackgroundDeep Dictionary Learning: A PARametric NETwork Approach
Mahdizadehaghdam, S., Panahi, A., Krim, H., & Dai, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2914376 · Cited by 89
“Contrary to conventional wisdom, several experimental studies [17], [18] have reported that deeper neural networks are more difficult to train, and adding more layers, eventually leads to decreased performance.”
ResultPasupa, K., & Ayutthaya, T. S. N. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101615 · Cited by 91
Effects of Hypertension, Diabetes, and Smoking on Age and Sex Prediction from Retinal Fundus Images
Kim, Y. D., Noh, K. J., Byun, S. J., Lee, S., Kim, T., Sunwoo, L., Lee, K. J., Kang, S., Park, K. H., & Park, S. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-61519-9 · Cited by 96
A Systematic Survey of Deep Learning-Based Single-Image Super-Resolution
Li, J., Pei, Z., Li, W., Gao, G., Wang, L., Wang, Y., & Zeng, T. (2024) · ACM Computing Surveys · doi:10.1145/3659100 · Cited by 110
“To solve these problems, some interesting methods have been proposed.”
MethodBangla Speech Emotion Recognition and Cross-Lingual Study Using Deep CNN and BLSTM Networks
Sultana, S., Iqbal, M. Z., Selim, M. R., Rashid, M. M., & Rahman, M. S. (2021) · IEEE Access · doi:10.1109/access.2021.3136251 · Cited by 95
“Given, d is the total number of convolutional layers, the computational complexity of multiple convolutional layers is expressed as [40]: where, for each convolutional layer l, m l is the spatial size of the output feature map, k l is the size of the kernel, n l is the number of input channels, o l…”
BackgroundDeep convolutional neural networks for computer-aided breast cancer diagnostic: a survey
Oza, P. R., Sharma, P., Patel, S., & Kumar, P. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06804-y · Cited by 86
“This was stated in [43, 44] and practically demonstrated in [45].”
BackgroundDeep Model Compression and Architecture Optimization for Embedded Systems: A Survey
Berthelier, A., Château, T., Duffner, S., García, C., & Blanc, C. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-020-01596-1 · Cited by 98
“Furthermore, the designed models have the same computational cost or memory consumption than the original ResNet but with a better accuracy.”
InfluentialUAV Anti-Jamming Video Transmissions With QoE Guarantee: A Reinforcement Learning-Based Approach
Xiao, L., Ding, Y., Huang, J., Liu, S., Tang, Y., & Dai, H. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3087787 · Cited by 84
“According to [63], we have According to (24) and [64], we have”
BackgroundA Physically Constrained Variational Autoencoder for Geochemical Pattern Recognition
Xiong, Y., Zuo, R., Luo, Z., & Wang, X. (2021) · Mathematical Geosciences · doi:10.1007/s11004-021-09979-1 · Cited by 89
Exploiting deep residual networks for human action recognition from skeletal data
Pham, H., Khoudour, L., Crouzil, A., Zegers, P., & Velastín, S. A. (2018) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.03.003 · Cited by 92
A residual convolutional neural network for polarimetric SAR image super-resolution
Shen, H., Lin, L., Li, J., Yuan, Q., & Zhao, L. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.01.006 · Cited by 84
“The very deep convolutional network (VDSR) model (Kim et al., 2016) used residual learning to solve the model degradation problem caused by the increase in the number of layers (He and Sun, 2015; He et al., 2016).”
Yang, S., Yang, B., Kang, Z., & Deng, L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.015 · Cited by 96
Ensemble Convolutional Neural Networks for Mode Inference in Smartphone Travel Survey
Yazdizadeh, A., Patterson, Z., & Farooq, B. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2918923 · Cited by 80
“The trade-offs between the depth (number of layers) and width (number of kernels) of neural networks have been investigated by some researchers [17].”
BackgroundKe, H., Chen, D., Shi, B., Zhang, J., Liu, X., Zhang, X., & Li, X. (2019) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2019.2962673 · Cited by 81
“The total time complexity of all convolutional layers can be computed as [34]:”
BackgroundRETRACTED ARTICLE: Military object detection in defense using multi-level capsule networks
Janakiramaiah, B., Kalyani, G., Arava, K., Prasad, L. V. N., & Krishna, M. H. (2021) · Soft Computing · doi:10.1007/s00500-021-05912-0 · Cited by 85
“From one viewpoint, it has been called attention to that deep network structures assumes the main part in improving performance of the network He and Sun (2014).”
BackgroundRetractedVisual Cause Analytics for Traffic Congestion
Pi, M., Yeon, H., Son, H., & Jang, Y. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2940580 · Cited by 90
“When classifying data with simple feature information such as TD patterns, a deep network can have less classification accuracy than a relatively simple network [91].”
MethodLu, F., Tong, Q., Feng, Z., & Wan, Q. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3217541 · Cited by 78
“We use Big-O notation [38] to describe the asymptotic upper bound of the order of magnitude of the function.”
MethodScene classification with improved AlexNet model
Xiao, L., Qin, Y., & Deng, S. (2017) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske.2017.8258820 · Cited by 78
“Thus AlexNet model makes a breakthrough progress in the classification of large-scale images and wins ILSVRC-2012 [10].”
BackgroundMethodInfluentialSingh, P., & Shankar, A. (2021) · Journal of Real-Time Image Processing · doi:10.1007/s11554-020-01060-0 · Cited by 82
“Therefore, the TC of all convolutional layers is [39]:”
BackgroundFocusCovid: automated COVID-19 detection using deep learning with chest X-ray images
Agrawal, T., & Choudhary, P. (2021) · Evolving Systems · doi:10.1007/s12530-021-09385-2 · Cited by 83
“With adding more layers, accuracy might increase but as reported in He and Sun (2015); Srivastava et al. (2015) with increasing depth, accuracy gets saturated and then degrades rapidly.”
MethodShahid, S. M., Ko, S.-H., & Kwon, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116233 · Cited by 80
Ding, A., Qin, Y., Wang, B., Jia, L., & Cheng, X. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3231325 · Cited by 83
Liu, S., Fan, H., Niu, X., Ng, H.-C., Chu, Y., & Luk, W. W. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242900 · Cited by 86
“For this reason, we propose an optimized U-Net CNN architecture based on the original one through three general model compression methods: 1) channel prunning [8]; 2) depth (number of layers) reduction [7] and 3) resolution multiplier [6, 9].”
MethodRI-GAN: An End-To-End Network for Single Image Haze Removal
Dudhane, A. A., Aulakh, H. S., & Murala, S. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00253 · Cited by 81
“Here, we make use of identity mapping [14] shown by the red line in Figure 2 (c) to add these learned features with the input features.”
BackgroundMethodXu, J., Gui, C., & Han, Q. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12563 · Cited by 78
“…Bottou, Bengio, & Haffner, 1998), ZFNet (Zeiler & Fergus, 2014), VGG (Simonyan & Zisser-man, 2014), and ResNet50 (He, Zhang, Ren, & Sun, 2016a) solving the problem that the training error increases with the deepening of the network (He & Sun, 2015) were adopted to train base rust grade classifiers.”
MethodSoftware-Defined Vehicular Networks With Trust Management: A Deep Reinforcement Learning Approach
Zhang, D., Yu, F. R., Yang, R., & Zhu, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3025684 · Cited by 70
Deep Residual Inception Encoder–Decoder Network for Medical Imaging Synthesis
Gao, F., Wu, T., Chu, X., Yoon, H., Xu, Y., & Patel, B. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2912659 · Cited by 81
“However, the additional layers may lead to issues of gradient vanishing or degradation, which has long been criticized from very deep networks [24], [29].”
BackgroundLu, X., Xiao, L., Niu, G., Ji, X., & Wang, Q. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3149396 · Cited by 74
Lin, H., Chen, H., Yin, C., Zhang, Q., Li, Z., Shi, Y. H., & Men, H. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3174251 · Cited by 74
Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving
Sharma, S., Ball, J. E., Tang, B., Carruth, D. W., Doude, M., & Islam, M. A. (2019) · Sensors · doi:10.3390/s19112577 · Cited by 81
“For any CNN, the total computational complexity of the convolutional layer can be expressed as follows [39]:”
BackgroundRetracted: Categorical Image Classification Based On Representational Deep Network (RESNET)
Mahajan, A. D., & Chaudhary, S. (2019) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca.2019.8822133 · Cited by 85
“A promptly act about preparation multi-layer perceptron’s (MLPs) may be on include a straight layer joined starting with those organize information of the yield [6, 16].”
BackgroundRetractedWu, W. Y., Yang, T., Li, R., Chen, C., Liu, T., Kai, Z. H. O. U., Sun, C., Li, C., Zhu, X., & Guo, W. (2020) · Journal of Integrative Agriculture · doi:10.1016/s2095-3119(19)62803-0 · Cited by 83
Xiao, D., Qin, C., Yu, H., Huang, Y., Liu, C., & Zhang, J. (2021) · Measurement · doi:10.1016/j.measurement.2021.109186 · Cited by 79
Dense Skip Attention Based Deep Learning for Day-Ahead Electricity Price Forecasting
Li, Y., Ding, Y., Liu, Y., Yang, T., Wang, P., Wang, J., & Yao, W. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3217579 · Cited by 77
“With more parameters, the model becomes more complex, and thus the training of the model is more prone to overfitting and may reduce the DAEPF performance [53], [54].”
Fog computing and convolutional neural network enabled prognosis for machining process optimization
Liang, Y., Li, W., Lu, X. B., & Wang, S. (2019) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2019.05.003 · Cited by 74
“The time complexity of the convolutional layer on the fog layer is calculated below [53]: where 𝑇𝑋 2 is the time complexity of the convolutional layer on the fog layer; 𝑙 is no. of a convolutional layer in the CNN; 𝑑 is the total number of convolutional layers; 𝑛 𝑙 is the number of filters in…”
BackgroundDeep Learning Prior Model for Unsupervised Seismic Data Random Noise Attenuation
Qiu, C., Wu, B., Liu, N., Zhu, X., & Ren, H. (2021) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2021.3053760 · Cited by 82
“Due to the computational cost which will increase sharply as the network goes deeper and wider [31], this generative network architecture is used for seismic data denoising without proceeding to deeper depth.”
BackgroundWang, Z., Hong, H., Ye, K., Zhang, G.-E., Jiang, M., & Tan, K. C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3113158 · Cited by 73
“It takes O( ∑d l=1 nl · nl−1 · N2n2) to execute the GAN once [60], where d is the depth of the neural networks, and nl is the number of neurons in the lth layer.”
BackgroundImproving the Efficiency of Multistep Short-Term Electricity Load Forecasting via R-CNN with ML-LSTM
Alsharekh, M. F., Habib, S., Dewi, D. A., Albattah, W., Islam, M., & Albahli, S. (2022) · Sensors · doi:10.3390/s22186913 · Cited by 72
“The implementation was performed in Python 3.5 using the Keras framework.”
MethodInfluentialDeep limits of residual neural networks
Thorpe, M., & van Gennip, Y. (2022) · Research in the Mathematical Sciences · doi:10.1007/s40687-022-00370-y · Cited by 97
“rom the previous layer “skipping layer i” and being transmitted to the next layer without being transformed. The reason for introducing these shortcut connections is to tackle the degradation problem [31,32]: It has been observed that increasing the depth ofanetwork (i.e. itsnumber oflayers) canleadtoanincrease inthee…”
Deep Unsupervised Learning for Joint Antenna Selection and Hybrid Beamforming
Liu, Z., Yang, Y., Gao, F., Zhou, T., & Ma, H. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3143122 · Cited by 83
“The complexity of offline training mainly comes from the gradient computation using backpropagation, which has the same order of complexity as the forward propagation according to [52].”
CNN-Fusion: An effective and lightweight phishing detection method based on multi-variant ConvNet
Hussain, M., Chi, C., Xu, R., & Afzal, M. (2023) · Information Sciences · doi:10.1016/j.ins.2023.02.039 · Cited by 79
Li, L. (2019) · Remote Sensing · doi:10.3390/rs11111378 · Cited by 70
“Residual (shortcut) connections in neural networks have been demonstrated to address vanishing/exploding gradients [38] and accuracy degradation in CNNs [39,40].”
BackgroundTang, S., Wang, J., & Tang, C. (2020) · Rock Mechanics and Rock Engineering · doi:10.1007/s00603-020-02259-0 · Cited by 72
“When a deep networks is able to start converging, a degradation problem might be exposed: as the network depth increases, the accuracy becomes saturated, which is not caused by overfitting (He and Sun 2015; Srivastava et al. 2015).”
BackgroundCervical Cancer Classification Using Combined Machine Learning and Deep Learning Approach
Alquran, H., Mustafa, W. A., Abu-Qasmieh, I. F., Yacob, Y. M., Alsalatie, M., Al-Issa, Y., & Alqudah, A. M. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.025692 · Cited by 72
Almutairi, H., Hassan, G. M., & Datta, A. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102906 · Cited by 68
Global sensitivity analysis for studying hot-mix asphalt dynamic modulus parameters
Owais, M., & Moussa, G. (2024) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2023.134775 · Cited by 76
A MIMO Detector With Deep Learning in the Presence of Correlated Interference
Xia, J., He, K., Xu, W., Zhang, S., Fan, L., & Karagiannidis, G. K. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2972806 · Cited by 72
“For the DCCN setting in this paper, the computational complexity at each iteration is equal to O( ∑L l=1 (Fl−1RlNFl)) [23].”
MethodSelf-citationConditional Random Field and Deep Feature Learning for Hyperspectral Image Classification
Alam, F. I., Zhou, J. W., Liew, A. W., Jia, X., Chanussot, J., & Gao, Y. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2867679 · Cited by 73
Distributed Deep Convolutional Neural Networks for the Internet-of-Things
Disabato, S., Roveri, M., & Alippi, C. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3062227 · Cited by 70
“However, such solutions are also characterized by a high computational load and memory occupation [6], [7], [8] and, for this reason, their use is mostly restricted to highperformance computing platforms [9], [10], [11].”
BackgroundTrends in deep convolutional neural Networks architectures: a review
Elhassouny, A., & Smarandache, F. (2019) · International Conference of Computer Science and Renewable Energies (ICCSRE) · doi:10.1109/iccsre.2019.8807741 · Cited by 78
“…intuition of [3] is that large activation maps (due to delayed downsampling) can lead to higher classification accuracy confirmed by the research of [32] The main role of squeeze layer is to limit the number of input channels to the 3 x 3 filters by fitting s 1 x 1 to be less than ( e 1 x 1 + e 3 x 3…”
BackgroundBansal, G., Chamola, V., Narang, P., Kumar, S., & Raman, S. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2019.1164 · Cited by 65
“A general expression for the overall time complexity of all convolutional layers in a CNN was presented by He and Sun [52] and demonstrated by Sinha and Ajmera [53].”
MethodFaster RCNN‐based detection of cervical spinal cord injury and disc degeneration
Ma, S., Huang, Y., Che, X., & Gu, R. (2020) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.13001 · Cited by 69
“In addition, the numerical experiments showed that Faster R‐CNN combined with novel CNN models has a better recognition performance compared with the performance of several traditional detection methods.(32) Moreover Sa et al.”
ResultA Voxel Graph CNN for Object Classification with Event Cameras
Deng, Y., Chen, H., Liu, H., & Li, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00124 · Cited by 110
“We follow the calculation method described in [3,20,22, 32] to compute FLOPs for these methods.”
MethodEnhanced convolutional neural network for plankton identification and enumeration
Cheng, K., Cheng, X., Wang, Y., Bi, H., & Benfield, M. C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219570 · Cited by 72
An Indoor and Outdoor Navigation System for Visually Impaired People
Croce, D., Giarré, L., Pascucci, F., Tinnirello, I., Galioto, G. E., Garlisi, D., & Valvo, A. L. (2019) · IEEE Access · doi:10.1109/access.2019.2955046 · Cited by 75
“Both models consist of different convolutional, max-pooling and fully connected layers, with a dominant contribution on the complexity given by the convolutional layer (the other layers usually contribute for no more that 5-10% [33]).”
BackgroundA Comparative Study of Deep Learning Models for Diagnosing Glaucoma From Fundus Images
Alghamdi, M., & Abdel-Mottaleb, M. S. (2021) · IEEE Access · doi:10.1109/access.2021.3056641 · Cited by 71
Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances
Moser, B. B., Raue, F., Frolov, S., Palacio, S., Hees, J., & Dengel, A. R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243794 · Cited by 86
“The primary reason behind adding skip connections is twofold: To avoid vanishing gradients and mitigate the accuracy saturation problem [99].”
BackgroundResultPan-sharpening via deep metric learning
Xing, Y., Wang, M., Yang, S., & Jiao, L. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.01.016 · Cited by 72
Convolution with even-sized kernels and symmetric paddingPreprint
Wu, S., Wang, G., Tang, P. S., Chen, F., & Shi, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08385 · Cited by 94
“In [10], 2×2 kernels are tested with relatively shallow (about 10 layers) models, where the FM sizes between two convolution layers are not preserved strictly.”
MethodWang, J., Luo, C., Huang, H., Zhao, H., & Wang, S. (2017) · Remote Sensing · doi:10.3390/rs9030225 · Cited by 67
“Furthermore, MSRA-Net is designed deeper by replacing the 5 × 5 filters with two series 3 × 3 filters [34].”
MethodData-Driven Approaches for Diagnosis of Incipient Faults in DC Motors
Munikoti, S., Das, L., Natarajan, B., & Srinivasan, B. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2895132 · Cited by 66
“On the other hand, the NN models (CNN and CNN-LSTM) used in this paper have a complexity of O ( N 2 ) [28].”
MethodIdriss, L. K., & Owais, M. (2023) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2023.134498 · Cited by 73
Offline SLA-Constrained Deep Learning for 5G Networks Reliable and Dynamic End-to-End Slicing
Chergui, H., & Verikoukis, C. V. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2959186 · Cited by 65
“In this regard, we notice that this CNN strategy is of high complexity [8].”
BackgroundA Radio Anomaly Detection Algorithm Based on Modified Generative Adversarial Network
Zhou, X., Xiong, J., Zhang, X., Liu, X., & Wei, J. (2021) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3074135 · Cited by 67
SSconv: Explicit Spectral-to-Spatial Convolution for Pansharpening
Wang, Y., Deng, L.-J., Zhang, T.-J., & Wu, X. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475600 · Cited by 69
“, 2×2 and 4×4, are used to downsample the images, whose effectiveness has be verified in [12, 22, 35].”
BackgroundWiAU: An Accurate Device-Free Authentication System with ResNet
Lin, C., Hu, J., Sun, Y., Ma, F., Wang, L., & Wu, G. (2018) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2018.8397108 · Cited by 72
“However, simply piling up convolution layers suffers from problems of slow convergence, easy over-fitting, information missing, and so on [22].”
BackgroundAn AMP-Based Network With Deep Residual Learning for mmWave Beamspace Channel Estimation
Wei, Y. M., Zhao, M.-M., Zhao, M., Lei, M., & Yu, Q. (2019) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2019.2916786 · Cited by 65
“Note that for a single convolutional layer, its computational complexity can be expressed as O(P2K (2)CinCout) [17], where P and K are the spatial lengths of the filter and the output feature map, Cin and Cout are the numbers of input and output channels.”
BackgroundDetecting cerebral microbleeds with transfer learning
Hong, J., Cheng, H., Zhang, Y., & Liu, J. (2019) · Machine Vision and Applications · doi:10.1007/s00138-019-01029-5 · Cited by 67
“However, the performance will be saturated and degraded rapidly when the networks go deeper [49, 50].”
BackgroundA deep learning framework for text-independent writer identification
Javidi, M., & Jampour, M. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103912 · Cited by 63
DeepACSON automated segmentation of white matter in 3D electron microscopy
Abdollahzadeh, A., Belevich, I. N., Jokitalo, E., Sierra, A., & Tohka, J. (2021) · Communications Biology · doi:10.1038/s42003-021-01699-w · Cited by 77
“For the general analysis of the time complexity of FCNs, we refer to 23 .”
BackgroundGenerative adversarial network to detect unseen Internet of Things malware
Moti, Z., Hashemi, S., Karimipour, H., Dehghantanha, A., Jahromi, A. N., Abdi, L., & Alavi, F. (2021) · Ad Hoc Networks · doi:10.1016/j.adhoc.2021.102591 · Cited by 64
Building Footprint Generation Using Improved Generative Adversarial Networks
Shi, Y., Li, Q., & Zhu, X. X. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2878486 · Cited by 64
“Similar phenomenas have been reported in [15].”
RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey
Gao, M., Jiang, J., Zou, G., John, V., & Liu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2907071 · Cited by 60
“[152] H. Liu, F. Li, X. Xu, and F. Sun, ‘‘Multi-modal local receptive field extreme learning machine for object recognition,’’ Neurocomputing, vol. 277, pp. 4–11, Feb. 2018.”
BackgroundInfluentialTowards Empathetic Human-Robot Interactions
Fung, P., Bertero, D., Wan, Y., Dey, A., Chan, R. H. Y., Bin Siddique, F., Yang, Y., Wu, C.-S., & Lin, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-75487-1_14 · Cited by 53
“time contribution is due to the computations inside the network [14], which with one”
BackgroundKausar, T., Wang, M., Idrees, M., & Lu, Y. (2019) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2019.09.003 · Cited by 68
Music emotion recognition based on segment-level two-stage learning
He, N., & Ferguson, S. (2022) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-022-00230-z · Cited by 75
“This is powerful for learning music representations, but it introduces more training parameters and increases the complexity of computing which requiresmore computing resources and aggravates the burden of operating environment, even more time cost [16].”
BackgroundChristopoulou, M. I., Barmpounakis, S., Koumaras, H. G., & Kaloxylos, A. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2022.100569 · Cited by 77
Khan, M. A., Park, H., & Chae, J. (2023) · Electronics · doi:10.3390/electronics12081802 · Cited by 76
Deep Learning-based Estimation for Multitarget Radar Detection
Delamou, M., Bazzi, A., Chafii, M., & Amhoud, E. M. (2023) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2023-spring57618.2023.10200157 · Cited by 62
GAN-Based Key Secret-Sharing Scheme in Blockchain
Zheng, W., Wang, K., & Wang, F.-Y. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2963138 · Cited by 58
“According to the computational complexity method [67], the total time complexity of all convolutional layers is where the number of filters (also known as “width”) in the l th layer is denoted as n l .”
MethodTwo-Dimensional Convolutional Neural Network-Based Signal Detection for OTFS Systems
Enku, Y. K., Bai, B., Wan, F., Guyo, C. U., Tiba, I. N., Zhang, C., & Li, S. (2021) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2021.3106039 · Cited by 67
E2DR: A Deep Learning Ensemble-Based Driver Distraction Detection with Recommendations Model
Aljasim, M., & Kashef, R. F. (2022) · Sensors · doi:10.3390/s22051858 · Cited by 64
“The same filter size is applied several times, allowing the network to represent complex features [47].”
BackgroundInfluentialXu, T.-B., Yang, P., Zhang, X.-Y., & Liu, C. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.10.029 · Cited by 66
Xiao, L., Jiang, D., Chen, Y., Su, W., & Tang, Y. (2019) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2019.2910938 · Cited by 61
“We have the following result for Algorithm 2 according to [21] and [22].”
MethodA Multiscale Neural Network Based on Hierarchical Matrices
Fan, Y., Lin, L., Ying, L., & Zepeda-Núñez, L. (2019) · Multiscale Modeling and Simulation · doi:10.1137/18m1203602 · Cited by 84
“In addition, there is a fine balance between the number of parameters, the architecture, and the degree of approximation [27, 28, 48].”
BackgroundRegion-Based CNN Method with Deformable Modules for Visually Classifying Concrete Cracks
Deng, L., Chu, H.-H., Shi, P., Wang, W., & Kong, X. (2020) · Applied Sciences · doi:10.3390/app10072528 · Cited by 64
Qu, R., Xu, G., Ding, C., Jia, W., & Sun, M. (2020) · IEEE Access · doi:10.1109/access.2020.2991845 · Cited by 59
“Although deep CNNs have achieved remarkable success in medical image analysis in recent years, their relative low identification accuracy and expensive computational costs limit the application of CNNs in clinical practice [49].”
BackgroundMoussa, G., & Owais, M. (2021) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2021.123589 · Cited by 64
Series Arc Fault Detection Using Regular Signals and Time-Series Reconstruction
Jiang, R., & Zheng, Y. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3165260 · Cited by 59
Predictive Precoder Design for OTFS-Enabled URLLC: A Deep Learning Approach
Liu, C., Li, S., Yuan, W., Liu, X., & Ng, D. W. K. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3280984 · Cited by 73
Al-Ja’Afreh, M. A. A., Mokryani, G., & Amjad, B. (2023) · Energy Reports · doi:10.1016/j.egyr.2023.08.003 · Cited by 61
Multi-view Convolutional Neural Network for lung nodule false positive reduction
El-Regaily, S. A., Salem, M. A. ‐., Abdel-Aziz, M. H., & Roushdy, M. I. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.113017 · Cited by 61
Metadata
- Authors
- He, K., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 5353-5360
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1932847118
- Cited by
- 1,558
- References
- 30
- Citations by year
- 2026: 32 · 2025: 91 · 2024: 143 · 2023: 174 · 2022: 216 · 2021: 252 · 2020: 227 · 2019: 209 · 2018: 110 · 2017: 61 · 2016: 32 · 2015: 11
- FWCI
- 29.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, time complexity, inference speed, network architecture design, speed-accuracy trade-off, ImageNet classification, AlexNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.