Convolutional neural networks at constrained time cost
Open accessThe DOI matches its Crossref record; 26 of the 30 references it cites were found in a registry or an index.
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The performance of neural network models is closely related to the setting of hyperparameters [42].”
From Prediction of Heat and Cold Loads of Factory Mushroom Houses Based on EWT Decomposition (2023) · cites this paper as [42]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 9×
“No other data augmentation, such as scale jittering [10, 22, 9], is used.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 14×
“The actual running time of our model is 20% faster than AlexNet.”
BackgroundMethodResultInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 7×
“Through the investigations, we obtain a model that achieves 11.8% top-5 error (10-view test) on ImageNet [3] and only takes 3 to 4 days training on a single GPU.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Self-citationBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Background
Research works printed but not linked: 1
- lin, 2013, Network in network, arXiv 1312 4400
Cited by
Items 601–700 of 1,001
A benchmarking protocol for SAR colorization: From regression to deep learning approaches
Shen, K., Vivone, G., Yang, X., Lolli, S., & Schmitt, M. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.10.058 · Cited by 24
Monocular depth estimation with guidance of surface normal map
Han, Y., Zhang, S., Zhang, Y., & Zhang, L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.08.074 · Cited by 26
Detection of double JPEG compression using modified DenseNet model
Zeng, X., Feng, G., & Zhang, X. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6737-3 · Cited by 24
“Among them, the residual deep learning framework introduced in [7] was designed to address the degradation problem: as the number of the layers increases, instead of steadily improving or remaining stable, accuracy degrades rapidly and training error rises with the depth of the network increasing [6, 9, 13, 20, 21].”
BackgroundAccelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies
Zeng, L., & Tian, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2873762 · Cited by 24
“Many previous works attempted to remove redundancy in CNNs [14], [16], [18]–[21], [23]–[34].”
BackgroundCorbat, L., Nauval, M., Henriet, J., & Lapayre, J. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113200 · Cited by 25
Khotimah, W. N., Bennamoun, M., Boussaïd, F., Sohel, F. A., & Edwards, D. (2020) · Remote Sensing · doi:10.3390/rs12193137 · Cited by 26
“In some cases, when the networks are deeper, their accuracy degrades because of the vanishing/exploding gradients problem [60].”
BackgroundAccurate Deep CNN-Based Waveform Recognition for Intelligent Radar Systems
Huynh‐The, T., Hua, C.-H., Doan, V., Pham, Q., & Kim, D. (2021) · IEEE Communications Letters · doi:10.1109/lcomm.2021.3095278 · Cited by 27
Automated Optimal Architecture of Deep Convolutional Neural Networks for Image Recognition
Albelwi, S. A., & Mahmood, A. (2016) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2016.0018 · Cited by 24
“Simonyan and Zisserman [11] further improved the accuracy by increasing the depth up to 19 layers with small (3×3) kernel sizes for all convolutional layers.”
BackgroundA lightweight convolutional neural network (CNN) model for diatom classification: DiatomNet
Gündüz, H., & Günal, S. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1970 · Cited by 25
“However, simply increasing depth does not always lead to improved accuracy ( He & Jian, 2015 ).”
BackgroundCao, M., Dai, Z., Chen, J., Yin, H., Zhang, X., Wu, J., Thanh, H. V., & Soltanian, M. R. (2024) · Water Research · doi:10.1016/j.watres.2024.122706 · Cited by 23
Deep Learning-Based Decoding of Constrained Sequence Codes
Cao, C., Li, D., & Fair, I. J. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933954 · Cited by 24
“Following the study in [40], [41], we now consider the time and space complexities of the proposed networks at the test phase.”
BackgroundDV-DCNN: Dual-view deep convolutional neural network for matching detected masses in mammograms
Alghamdi, M., & Abdel-Mottaleb, M. S. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106152 · Cited by 25
AI Model Compression for Edge Devices Using Optimization Techniques
Kulkarni, U., Meena, S. M., Gurlahosur, S. V., Benagi, P., Kashyap, A. K., Ansari, A., & Karnam, V. E. G. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-68291-0_17 · Cited by 30
Li, W., Qin, Y., Feng, Z., Han, H., Chen, J., & Xu, Y. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2022.3165633 · Cited by 23
Deep-Learning Based Signal Detection for MIMO-OTFS Systems
Enku, Y. K., Bai, B., Li, S., Liu, M., & Tiba, I. N. (2022) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops53468.2022.9814608 · Cited by 32
Zhang, D., Shi, W., St‐Hilaire, M., & Yang, R. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3183927 · Cited by 26
“In the first case, according to the security analysis of [31], due to the nature of the blockchain, the data written to the blockchain cannot be tampered with.”
BackgroundSSCLNet: A Self-Supervised Contrastive Loss-Based Pre-Trained Network for Brain MRI Classification
Mishra, A., Jha, R., & Bhattacharjee, V. (2023) · IEEE Access · doi:10.1109/access.2023.3237542 · Cited by 23
“But the degradation problem comes up when the deep networks start converging, and accuracy gets saturated and degrades rapidly [20].”
BackgroundShin, M., Peng, Z., Kim, H.-J., Yoo, S., & Yoon, K. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107591 · Cited by 27
Res-BiANet: A Hybrid Deep Learning Model for Arrhythmia Detection Based on PPG Signal
Wu, Y., Tang, Q., Zhan, W., Li, S., & Chen, Z. (2024) · Electronics · doi:10.3390/electronics13030665 · Cited by 25
Supervised Deep Sparse Coding Networks
Sun, X., Nasrabadi, N. M., & Tran, T. D. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451701 · Cited by 35
“to combine the best of both strategies. For instance, the imperative for sparse codes to adequately reconstruct an input signal [4] ameliorates information degeneracy issues within deep architectures [12, 15]. Furthermore, multilayer sparse coding networks demand less training data as compared to deep neural networks.…”
BackgroundXiang, Q., Yang, Y., Zhang, Q., & Yao, Y. (2019) · IEEE photonics journal · doi:10.1109/jphot.2019.2929913 · Cited by 23
“The overall computational complexity in CNN is directly related to the structure of neural network and the size of image, which can be expressed as O ( ∑d i=1 n i − 1s 2 i n i m 2 i ) [26].”
BackgroundFace Inpainting via Nested Generative Adversarial Networks
Li, Z., Zhu, H., Cao, L., Jiao, L., Zhong, Y., & Ma, A. (2019) · IEEE Access · doi:10.1109/access.2019.2949614 · Cited by 25
“The result illustrates that our method is outperform than those in [38] and [39].”
BackgroundMethodResultRace estimation with deep networks
Ahmed, M. A., Choudhury, R. D., & Kashyap, K. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.11.029 · Cited by 28
“For R-Net model, the total (testing) time complexity of the 6 convolution layers, according to (12), is: Also, time complexity of the train phase is three times greater than of the test phase (He and Sun, 2015) as it includes both forward and backward propagation.”
Efficient Prediction of Link Outage in Mobile Optical Wireless Communications
Wu, Z.-Y., Ismail, M., Serpedin, E., & Wang, J. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3029029 · Cited by 26
“Meanwhile, the convolutional neural network (CNN) is proved to be the most powerful candidate in such classification thus far [33], and it requires fewer parameters than other classification approaches [34] due to its shared weights and translation invariance performed by the convolutions.”
BackgroundMethodXia, H., Tang, J., Qiao, J., Zhang, J., & Yu, W. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06809-7 · Cited by 27
“Although CNNs are typically improved by increasing its depth and width [155, 156], unavoidable problems, such as overfitting, hyperparameter increase, and deceleration of training speed, also occur.”
BackgroundFlowCon: Elastic Flow Configuration for Containerized Deep Learning Applications
Zheng, W., Tynes, M., Gorelick, H., Mao, Y., Cheng, L., & Hou, Y. (2019) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3337821.3337868 · Cited by 36
“For example, in the work [22], the authors evaluate a series of models under constrained time cost, and propose models that are fast for practical applications yet are more accurate than existing fast models.”
BackgroundECG Signal Analysis Using 2-D Image Classification with Convolutional Neural Network
Wasimuddin, M., Elleithy, K. M., Abuzneid, A., Faezipour, M., & Abuzaghleh, O. (2019) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci49370.2019.00180 · Cited by 22
“In Equation 2, d is the number of convolutional layers, n l denotes the filter’s width in a given l -th layer, and s l and m l are the sizes of the filter and output feature map [15].”
BackgroundAbnormal event detection using adversarial predictive coding for motion and appearance
Yu, J., Kim, J.-G., Gwak, J., Lee, B., & Jeon, M. (2021) · Information Sciences · doi:10.1016/j.ins.2021.11.001 · Cited by 24
A Data Augmentation Method for Skeleton-Based Action Recognition with Relative Features
Chen, J., Yang, W., Liu, C., & Yao, L. (2021) · Applied Sciences · doi:10.3390/app112311481 · Cited by 27
“This means our model consumes less memory and has higher efficiency than DenseNet121 [32,33].”
ResultSpiking VGG7: Deep Convolutional Spiking Neural Network with Direct Training for Object Recognition
Xiang, S., Jiang, S., Liu, X., Zhang, T., & Yu, L. (2022) · Electronics · doi:10.3390/electronics11132097 · Cited by 22
Wang, W., He, Q., Zhang, Z., & Feng, Z. (2022) · Ultrasonics · doi:10.1016/j.ultras.2022.106823 · Cited by 22
Wang, L., Song, F., Feng, Z., Li, G., Jiao, Y., Wang, H., & Xu, Y. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2022.3204898 · Cited by 23
“Motivated by [12], we measure the complexity of the proposed algorithm by floating-point operations (FLOPs).”
MethodCyber-Attack Detection: Modeling and Roof-PV Generation System Defending
Qiu, W., Sun, K., Li, K., Li, Y., Duan, J., & Zhu, K. (2022) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2022.3213629 · Cited by 23
“The total time complexity of all convolutional layers is O(DK2 C2 N 2), where D is the number of convolutional layers,K andC are the sizes of the data dimension for convolutional kernel and channels of the convolutional layer, respectively [46].”
BackgroundA Multichannel Convolutional Decoding Network for Graph Classification
Guang, M., Yan, C., Xu, Y., Wang, J., & Jiang, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266243 · Cited by 22
“D convolution operator is the most time-consuming part of the decoder, and its time complexity is O ( k 2 p 2 q l − 1 q l ) [58].”
BackgroundLow-Overhead Clustered Federated Learning for Personalized Stress Monitoring
Jiang, S., Firouzi, F., & Chakrabarty, K. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3299736 · Cited by 28
“Details for computation of T NN ( · ) for a given neural network architecture are explained in [51].”
BackgroundKanakis, M., Maurer, S., Spallanzani, M., Chhatkuli, A., & Gool, L. V. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00651 · Cited by 37
“These solutions can be partitioned in topological optimizations, aiming at increasing accuracy-per-operation or accuracy-per-parameter [31], [32], software optimizations such as tensor decomposition and parameter pruning [33], [34], [35], and hardware-aware optimizations [11].”
BackgroundDeep Learning for Historical Cadastral Maps Digitization: Overview, Challenges and Potential
Ignjatić, J., Nikolic, B. Z., Rikalović, A., & Culibrk, D. R. (2018) · Computer Science Research Notes · doi:10.24132/csrn.2018.2803.6 · Cited by 24
“That model is created by selecting different kernels or tuning the parameters by end-to-end optimization with the minimum human inference. where the parameters of DNN model are trained jointly [He15], [Sze17].”
MethodDeep Learning Driven Venue Recommender for Event-Based Social Networks
Pramanik, S., Haldar, R., Kumar, A., Pathak, S., & Mitra, B. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2915523 · Cited by 22
Whole volume brain extraction for multi-centre, multi-disease FLAIR MRI datasets
Khademi, A., Reiche, B., DiGregorio, J., Arezza, G., & Moody, A. R. (2019) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2019.08.022 · Cited by 23
“In [23], a convolutional neural network was used, and this model is generally regarded as being very complex [45].”
MethodA Model for Generating Workplace Procedures Using a CNN-SVM Architecture
Patalas‐Maliszewska, J., & Halikowski, D. (2019) · Symmetry · doi:10.3390/sym11091151 · Cited by 24
“The total time complexity of all convolutional layers (1) is [55]: O n ∙ s ∙ n ∙m (1)”
BackgroundAygül, M. A., Nazzal, M., Sağlam, M. İ., da Costa, D. B., Ateş, H. F., & Arslan, H. (2020) · Sensors · doi:10.3390/s21010135 · Cited by 22
“Besides, the computational complexity of training per sample is approximately two times more as compared to the complexity of testing per sample [39].”
BackgroundXiang, Q., Yang, Y., Zhang, Q., & Yao, Y. (2021) · Optics Express · doi:10.1364/oe.415138 · Cited by 22
56-Gbit/s PAM-4 Optical Signal Transmission Over 100-km SMF Enabled by TCNN Regression Model
Zhang, J., Yan, L., Jiang, L., Yi, A., Pan, Y., Pan, W., & Luo, B. (2021) · IEEE photonics journal · doi:10.1109/jphot.2021.3092003 · Cited by 22
Zhang, S., Mao, Y., Clerckx, B., & Quek, T. Q. S. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3489219 · Cited by 24
“Furthermore, according to [67], the complexity of a CNN layer can be written as O{ 2 Y Zι 2 ζ 2 } , where ζ 2 is the filter size.”
Pyramidal RoR for image classification
Zhang, K., Guo, L., Gao, C., & Zhao, Z. (2017) · Cluster Computing · doi:10.1007/s10586-017-1443-x · Cited by 24
Wei, H., Shen, H., Li, J., Zhao, R., & Chen, Z. (2020) · Optics & Laser Technology · doi:10.1016/j.optlastec.2020.106773 · Cited by 24
A Multi-Scale Residual Attention Network for Retinal Vessel Segmentation
Jiang, Y., Yao, H., Wu, C., & Liu, W. (2020) · Symmetry · doi:10.3390/sym13010024 · Cited by 21
“The time complexity calculation formula of a convolutional neural network [37] is as follows:”
MethodArab, H., Ghaffari, I., Evina, R. M., Tatu, S. O., & Dufour, S. (2022) · IEEE Access · doi:10.1109/access.2022.3147980 · Cited by 23
Reliable resource allocation with RF fingerprinting authentication in secure IoT networks
Wu, W., Hu, S., Lin, D., & Wu, G. (2022) · Science China Information Sciences · doi:10.1007/s11432-021-3284-y · Cited by 21
ResNet-based bio-acoustics presence detection technology of Hainan gibbon calls
Ruan, W., Wu, K., Chen, Q., & Zhang, C. (2022) · Applied Acoustics · doi:10.1016/j.apacoust.2022.108939 · Cited by 21
Tseng, F., Yeh, K., Kao, F.-Y., & Chen, C. (2022) · ISA Transactions · doi:10.1016/j.isatra.2022.07.030 · Cited by 21
A Method for UWB Localization Based on CNN-SVM and Hybrid Locating Algorithm
Gao, Z., Jiao, Y., Yang, W., Li, X., & Wang, Y. (2023) · Information · doi:10.3390/info14010046 · Cited by 21
Toward Multi-User Authentication Using WiFi Signals
Kong, H., Lü, L., Yu, J., Chen, Y. J., Xu, X., & Lyu, F. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3237686 · Cited by 25
Zhang, L., Miao, Z., Xia, Y., Zhou, F., & Yuan, X. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3322484 · Cited by 21
CViT: A Convolution Vision Transformer for Video Abnormal Behavior Detection and Localization
Roka, S., & Diwakar, M. (2023) · SN Computer Science · doi:10.1007/s42979-023-02294-y · Cited by 23
“The theoretical time complexity of the network can be measured by following the procedure of [38–40]; however, the actual running time may vary depending upon implementations and hardware.”
MethodBreast lesion classification from mammograms using deep neural network and test-time augmentation
Oza, P. R., Sharma, P., & Patel, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09165-w · Cited by 27
“‘‘ This was stated in [45, 46] and shown practically in [11].”
Hybrid Neural Network Models for Detecting Fake News Articles
Khalil, A., Jarrah, M., & Aldwairi, M. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00055-x · Cited by 21
“The pooling layer was eliminated to preserve the spatial dimension of features, where the model extracts more features without increasing the computational complexity [34].”
A convolutional neural networks based transportation mode identification algorithm
Yanyun, G., Zhao, F., Chen, S., & Luo, H. (2017) · International Conference on Indoor Positioning and Indoor Navigation · doi:10.1109/ipin.2017.8115930 · Cited by 22
Big data in nanoscale connectomics, and the greed for training labels
Motta, A., Sievers, M., Staffler, B., & Helmstaedter, M. N. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.03.012 · Cited by 26
Scalable Learning With a Structural Recurrent Neural Network for Short-Term Traffic Prediction
Kim, Y., Wang, P., & Mihaylova, L. S. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2933823 · Cited by 24
Deep Multi-Scale Recurrent Network for Synthetic Aperture Radar Images Despeckling
Zhou, Y., Shi, J., Yang, X., Wang, C., Kumar, D., Wei, S., & Zhang, X. (2019) · Remote Sensing · doi:10.3390/rs11212462 · Cited by 21
“Increasing the number of convolutional layers can enhance the feature extraction ability [35,36].”
BackgroundFarhangi, M. M., Petrick, N., Sahiner, B., Frigui, H., Amini, A. A., & Pezeshk, A. X. (2020) · Medical Physics · doi:10.1002/mp.14076 · Cited by 23
Data-Driven Link Assignment With QoS Guarantee in Mobile RF-Optical HetNet of Things
Wu, Z.-Y., Ismail, M., Serpedin, E., & Wang, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2975851 · Cited by 26
“…to compress the time cost in this stage; otherwise, the actual equivalent timing delay error will be greatly enlarged, which is why it is not recommended to involve more complicated methods such as convolutional neural networks (CNNs) [36] in the channel state abstraction and other procedures.”
BackgroundMicrostructural classification of unirradiated LiAlO2 pellets by deep learning methods
Pazdernik, K., LaHaye, N. L., Artman, C. M., & Zhu, Y. (2020) · Computational Materials Science · doi:10.1016/j.commatsci.2020.109728 · Cited by 22
OCT Image Recognition of Cardiovascular Vulnerable Plaque Based on CNN
Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3007599 · Cited by 21
Water Level Estimation in Sewer Pipes Using Deep Convolutional Neural Networks
Haurum, J. B., Bahnsen, C., Pedersen, M., & Moeslund, T. B. (2020) · Water · doi:10.3390/w12123412 · Cited by 23
“It has even been shown that for some architectures it can harm the performance if the depth is overly increased due to the degradation problem [23].”
BackgroundLi, H., Yue, X., Wang, Z., Wang, W., Tomiyama, H., & Meng, L. (2021) · Measurement Sensors · doi:10.1016/j.measen.2021.100080 · Cited by 25
Fast ConvNets Using Group-wise Brain DamagePreprint
Lebedev, V., & Lempitsky, V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02515 · Cited by 45
“It is interesting to see that the optimization process decided to shrink the receptive fields towards the center compared to the full version (which is consistent with the findings in [43, 19]).”
BackgroundResultWang, D., Hu, Q., & Yang, C. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105654 · Cited by 24
Momen‐Tayefeh, M., Momen-Tayefeh, M., Ghahramani, S. A. G., & Hemmatyar, A. M. A. (2024) · Signal Processing · doi:10.1016/j.sigpro.2024.109710 · Cited by 22
HEVC Intra Frame Coding Based on Convolutional Neural Network
Yeh, C., Zhang, Z.-T., Chen, M., & Lin, C. (2018) · IEEE Access · doi:10.1109/access.2018.2867342 · Cited by 20
“As known in [8], we can derive the computational complexity of each convolutional layer, as shown in Table 2.”
MethodA Deep Support Vector Data Description Method for Anomaly Detection in Helicopters
Liu, C., & Gryllias, K. (2021) · PHM Society European Conference · doi:10.36001/phme.2021.v6i1.2957 · Cited by 23
“The computational time complexity of the proposed CNN-based deep SVDD model is analyzed based on the time cost of the convolutional layers described as follows (He & Sun, 2015): where l represents the convolutional layer index.”
MethodRoy, D., Mukherjee, T., Riden, A., Paquet, J., Pasiliao, E., & Blasch, E. (2022) · IEEE Access · doi:10.1109/access.2022.3169420 · Cited by 20
“In [29], the authors noted that the time complexity for training all the convolutional layers is: , where ζ is the number of convolutional layers, τ is the index of a convolutional layer, η τ − 1 is the number of input channels for the τ th layer, ν τ is the spatial size of the filters at the τ th…”
BackgroundMethodZhou, K., Oh, S., Qiu, J., Witold Pedrycz, C., & Seo, K. (2022) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2022.3186181 · Cited by 22
“Meanwhiles, the computational complexity of EDLT-MCO (the number of swap is constant, so the computational complexity of MCO is O(1)) and CNN is O((m-k)2m-k) and O( ∑D l=1 M 2 l K 2 l Cl−1Cl), respectively [18], [38].”
BackgroundRapid classification of local seismic events using machine learning
Jia, L., Chen, H., & Xing, K. (2022) · Journal of Seismology · doi:10.1007/s10950-022-10109-5 · Cited by 21
“He and Jian (2015) conducted experiments using 5 × 5 and 3 × 3 convolution kernels and noted that changing a two-layer 3 × 3 convolution kernel into a fourlayer 2 × 2 convolution kernel did not increase the number of parameters but did improve the accuracy.”
MethodArtificial Intelligence in Manufacturing Equipment, Automation, and Robots
Chryssolouris, G., Alexopoulos, K., & Arkouli, Z. (2023) · Studies in systems, decision and control · doi:10.1007/978-3-031-21828-6_3 · Cited by 31
Semi-universal geo-crack detection by machine learning
Shi, Y., Ballesio, M., Johansen, K., Trentman, D., Huang, Y., McCabe, M. F., Bruhn, R. L., & Schuster, G. T. (2023) · Frontiers in Earth Science · doi:10.3389/feart.2023.1073211 · Cited by 24
“• Residual function connection: In deep learning models, the convergence rate and accuracy can become degraded with an increase in the number of layers (He and Sun, 2015).”
BackgroundDhalla, S., Mittal, A., & Gupta, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09157-w · Cited by 20
“However, as the network tends to increase its depth, accuracy starts to saturate and training loss tends to increase [50, 51].”
Hyperparameter Optimization and Importance Ranking in Deep Learning–Based Crack Segmentation
Canchila, C., Zhou, S., & Song, W. (2023) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-5512 · Cited by 20
Federated clustered multi-domain learning for health monitoring
Jiang, S., Li, Y., Firouzi, F., & Chakrabarty, K. (2024) · Scientific Reports · doi:10.1038/s41598-024-51344-9 · Cited by 30
“Techniques for detecting outliers to improve the privacy of the FL will be investigated in future work.”
BackgroundMoving Target Indication Using Deep Convolutional Neural Network
Liu, Z., Ho, K. C., Xu, X., & Yang, J. (2018) · IEEE Access · doi:10.1109/access.2018.2877018 · Cited by 20
“The order of the computation complexity is the total numbers of the multiplication operations in all convolution layers [41].”
BackgroundDeep Learning-Based Optimal RIS Interaction Exploiting Previously Sampled Channel Correlations
Aygül, M. A., Nazzal, M., & Arslan, H. (2021) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc49053.2021.9417591 · Cited by 20
“computational complexity of testing is roughly half of the per-sample training computational complexity since the testing stage does not require back-propagation [20].”
BackgroundWang, Y., Zheng, K., Tian, D., Duan, X., & Zhou, J. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900637 · Cited by 20
“For convolutional layers, the total time complexity of all convolutional layers is (He KM and Sun, 2015) where n is the index of a convolutional layer, C n is the number of filters (also known as “width”) in the n th convolutional layer, C n − 1 is also known as the number of input channels of the n…”
BackgroundMei, S., Mei, Y., Cui, J. F., Dong, S., & Zhao, J. (2022) · Computers & Industrial Engineering · doi:10.1016/j.cie.2022.108129 · Cited by 24
Residual Squeeze VGG16Preprint
Qassim, H., Feinzimer, D., & Verma, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.03004 · Cited by 31
“He & Sun [23] observed higher classification accuracy after testing downsampling implemented into four unique CNN architectures.”
BackgroundWinter Road Surface Condition Recognition Using A Pretrained Deep Convolutional NetworkPreprint
Pan, G., Fu, L., Yu, R., & Muresan, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06858 · Cited by 40
Chen, P., Tseng, B., Yang, Z.-H., Yu, C., Lin, K.-T., Chen, J., & Liu, P. (2024) · iScience · doi:10.1016/j.isci.2024.111318 · Cited by 19
“22,23 Recognizing this, the ResNet model emerged in 2015, providing a robust solution for various image processing tasks, from segmentation to recognition.”
Mitigating gradient conflicts via expert squads in multi-task learning
Chen, J., & Er, M. J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128832 · Cited by 27
Lu, Q., Lin, T. R., & Zhu, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3108625 · Cited by 22
“However, different from the conventional algorithms, the matrix multiplication of the HBF-Net and the CE-HBF-Net can be efficiently accelerated by parallel computation via graphics processing unit [40].”
BackgroundMethodStructure-Enhanced DRL for Optimal Transmission Scheduling
Chen, J., Liu, W., Quevedo, D. E., Khosravirad, S. R., Li, Y., & Vucetic, B. S. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3277861 · Cited by 19
“The computation complexity of an FCNN is O L − 1 l =1 u l u l +1 , where u l is the number of nodes in layer l and L is the total number of layers of the NN [39].”
BackgroundBreast Cancer Diagnosis Using Lightweight Deep Convolution Neural Network Model
Kausar, T., Lu, Y. H., & Kausar, A. (2023) · IEEE Access · doi:10.1109/access.2023.3326478 · Cited by 25
“From the BreakHis dataset, about 30% of data (598 images) is selected in the testing set, while the remaining 70% (1397 images) is selected in the training set as per the work in [12], [33], and [34].”
BackgroundMethodPandey, A. K., Nayak, S. C., & Kim, S. (2024) · Bioresource Technology · doi:10.1016/j.biortech.2024.130496 · Cited by 19
Image Classification and Object Detection Algorithm Based on Convolutional Neural Network
Group of Network Computation, D. O. M. A. C., & Leonard, J. K. (2019) · Science Insights · doi:10.15354/si.19.re117 · Cited by 23
“in the network based on the appropriate network depth will bring the problem of network degradation with increased training errors (49).”
BackgroundImproved External Speaker-Robust Keyword Spotting for Hearing Assistive Devices
López‐Espejo, I., Tan, Z., & Jensen, J. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2984089 · Cited by 20
“Nevertheless, training very deep neural networks is not an easy task and, as an example of this, the extension of a well-trained model with additional layers might even lead to higher training error [24].”
BackgroundReconstructing patchy reionization with deep learning
Guzman, E., & Meyers, J. (2021) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.104.043529 · Cited by 23
Snake Detection and Classification using Deep Learning
Yang, Z., & Sinnott, R. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.148 · Cited by 20
“However, as the layers of network increase, the complexity of the training process grows, while the final accuracy may not always be improved due to the vanishing gradient [14].”
BackgroundA Shallow 1-D Convolution Neural Network for Fetal State Assessment Based on Cardiotocogram
Fasihi, M., Nadimi-Shahraki, M. H., & Jannesari, A. (2021) · SN Computer Science · doi:10.1007/s42979-021-00694-6 · Cited by 21
“CNN architecture can increment computational complexity that can lead to overfitting [30].”
BackgroundAn Efficient Face Recognition Method Using CNN
Fredj, H. B., Sghaier, S., & Souani, C. (2021) · International Conference of Women in Data Science at Taif University (WiDSTaif) · doi:10.1109/widstaif52235.2021.9430209 · Cited by 19
“The second problem resides in the field of memory and computational requirements of CNNs [11,12].”
BackgroundUnfolded Hybrid Beamforming With GAN Compressed Ultra-Low Feedback Overhead
Balevi, E., & Andrews, J. G. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3092350 · Cited by 21
Sumit, S. S., Rambli, D. R. A., Mirjalili, S., Ejaz, M. M., & Miah, M. S. U. (2022) · Applied Sciences · doi:10.3390/app12189331 · Cited by 21
Metadata
- Authors
- He, K., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 5353-5360
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1932847118
- Cited by
- 1,558
- References
- 30
- Citations by year
- 2026: 32 · 2025: 91 · 2024: 143 · 2023: 174 · 2022: 216 · 2021: 252 · 2020: 227 · 2019: 209 · 2018: 110 · 2017: 61 · 2016: 32 · 2015: 11
- FWCI
- 29.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, time complexity, inference speed, network architecture design, speed-accuracy trade-off, ImageNet classification, AlexNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.