A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Open accessThe DOI matches its Crossref record; 220 of the 272 references it cites were found in a registry or an index; 48 of the 149 sources its claims cite were checked for support.
Abstract
A convolutional neural network (CNN) is one of the most significant networks in the deep learning field. Since CNN made impressive achievements in many areas, including but not limited to computer vision and natural language processing, it attracted much attention from both industry and academia in the past few years. The existing reviews mainly focus on CNN's applications in different scenarios without considering CNN from a general perspective, and some novel ideas proposed recently are not covered. In this review, we aim to provide some novel ideas and prospects in this fast-growing field. Besides, not only 2-D convolution but also 1-D and multidimensional ones are involved. First, this review introduces the history of CNN. Second, we provide an overview of various convolutions. Third, some classic and advanced CNN models are introduced; especially those key points making them reach state-of-the-art results. Fourth, through experimental analysis, we draw some conclusions and provide several rules of thumb for functions and hyperparameter selection. Fifth, the applications of 1-D, 2-D, and multidimensional convolution are covered. Finally, some open issues and promising directions for CNN are discussed as guidelines for future work.
Overview
CoChat Summary
TL;DR
This survey reviews CNN fundamentals, classic and advanced architectures, activation/loss/optimizer choices, applications across 1D/2D/multi-dimensional convolutions, and future directions. It adds comparative experiments on activation functions and optimizers, reporting that ELU often gives the best accuracy among activation functions while ReLU/Leaky ReLU are stable, and that RMSprop achieved the highest CIFAR-10 validation accuracy among tested optimizers.
Key findings
- The survey synthesizes CNN history and models from LeNet-5 through AlexNet, VGG, GoogLeNet/Inception, ResNet, DCGAN, MobileNets, ShuffleNets, and GhostNet, emphasizing innovations such as ReLU/dropout/LRN, small kernels, residual connections, depthwise separable convolutions, channel shuffle, NAS/NetAdapt, SE, and h-swish.
- Activation-function experiments: LeNet-5 on MNIST reached 99.20% with ELU/PReLU, 99.18% with ReLU, and 98.56% with linear; on CIFAR-10 ELU reached 65.70%, ReLU 64.40%, and linear 62.56%; on CIFAR-100 ELU reached 35.10%, Leaky ReLU 33.81%, and linear 31.24%. Pretrained VGG-16 on MNIST reached 99.55% with ReLU, 99.48% with Leaky ReLU, and 99.45% with PReLU; on CIFAR-100 Leaky ReLU reached 48.22%, PReLU 47.46%, and ReLU 44.77%, while sigmoid/tanh/ELU sometimes failed to converge.
- Optimizer experiments with pretrained VGG-16 on CIFAR-10: RMSprop reached 87.32%, Momentum 86.37%, Nadam 86.26%, Adamax 86.18%, Adadelta 86.06%, MBGD 85.70%, Adagrad 84.68%, Nesterov 84.32%, AMSgrad 83.25%, and Adam 83.09%. Almost all tested optimizers converged, MBGD was slowest, and Nesterov/Adagrad/RMSprop/Adamax/Nadam oscillated; learning rate strongly affected convergence.
- Applications covered include 1D CNN for time-series prediction and signal identification, 2D CNN for image classification, object detection, segmentation, and face recognition, and 3D CNN for human action recognition and object recognition/detection; future directions include model compression, security/adversarial attacks, NAS, and Capsule Neural Networks.
Methods
- Design: a narrative survey/review plus experimental analysis, covering CNN building blocks, classic models, activation functions, loss functions, optimizers, applications, and prospects.
- Data/sample: activation functions were tested with LeNet-5 and VGG-16 on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100; optimizer comparison tested ten optimizers on CIFAR-10 using a pretrained VGG-16 without its last three ImageNet layers.
- Main analysis: comparative tables/figures and rules of thumb; activation experiments used cross-entropy loss, Adam, batch sizes 256/512, and specified epochs, while optimizer experiments used cross-entropy, ReLU, batch size 512, and 100 epochs.
Limitations
- Stated: “Due to the limitation of paper length,” the model section is an overview of representative models and emphasizes innovations “but not all details”; multi-dimensional CNN discussion uses 3D as an example because higher dimensions are uncommon and hard for humans to understand.
- Evident: The experimental comparisons are narrow—activation functions only on LeNet-5/VGG-16 and four datasets, and optimizers only on CIFAR-10 with VGG-16; Table I reports no data augmentation, and the text does not report repeated runs or statistical significance tests.
- Evident: The survey’s application and prospect coverage is selective rather than exhaustive, as it presents representative examples and promising directions rather than a systematic review of all CNN work.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 37 Not valid · 67 Not checked · 41 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Afterward, deeper network structures emerged, such as GoogLeNet [33] and VGGNets [32], which significantly improve the accuracy in classification tasks.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > B. Applications of two-dimensional CNN 1) Image Classification
What the sources say
- Partly supportedcites [33] sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes VGG-nets as a very deep, effective architecture and says the Inception family achieves compelling accuracy. It does not establish that GoogLeNet and VGGNets significantly improved classification accuracy over earlier networks; that comparative part of the claim is unsupported by these passages. (LLM, conf=0.82)
- Disputedcites [32] sourceExperimental — accuracy not yet measured
The citation is attached to “GoogLeNet,” but the cited paper is CondenseNet, not the GoogLeNet paper. Its Introduction describes experiments on CondenseNets; the general discussion of CNN classification accuracy does not make this source the named example. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Hence, ternary quantization and multivariate quantization are harnessed to strike a proper balance between model size and accuracy.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > A. Model Compression
What the sources say
- Could not verifycites [163] sourceExperimental — accuracy not yet measured
The Training section (p. 4) discusses reducing GPU-memory overhead by using large input tiles, not reducing model size through quantization. Searches for ternary and multivariate quantization, reduced-precision weights, and a model-size–accuracy trade-off found no passage addressing the claim in our copy. (LLM, conf=0.76)
- Disputedcites [164] sourceExperimental — accuracy not yet measured
The Performance Analysis (p. 6) says ENet's small parameter count alleviates the need for model compression; compression techniques could be applied under stricter memory constraints. It does not describe harnessing ternary or multivariate quantization to balance model size and accuracy. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [165] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Different from the traditional feature extraction methods [17], [18], [19] , CNN does not need to extract features manually.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > II. BRIEF OVERVIEW OF CNN
What the sources say
- Could not verifycites [17] sourceExperimental — accuracy not yet measured
Our copy of Cohen & Welling (2016) never contrasts CNNs with traditional/hand-crafted feature extraction: the Introduction (p. 1) frames the background as learning invariant vs. equivariant representations and the need for large labelled datasets, and Sections 2-3 (pp. 2-8) define feature maps, capsules and equivariant filter banks, with no statement about manual feature extraction being unnecessary. Searches for 'traditional/manual feature extraction', 'hand-engineered descriptors (SIFT, HOG)', 'features learned automatically' and 'end-to-end hierarchical features' returned only unrelated passages and reference-list entries (pp. 10-11), so nothing in our copy supports or contradicts the attributed point. The paper is not truncated; the claim simply lies outside its scope. (LLM, conf=0.68)
- Partly supportedcites [18] sourceExperimental — accuracy not yet measured
Li et al. (2018) do draw the handcrafted-vs-learned contrast, but only in a narrow domain: 'Neural Networks on Molecular Graph' (p. 3) says chemical-property prediction 'was usually handled by handcrafted features and feature embedding (Mayr et al. 2016; Weiss, Torralba, and Fergus 2009)', whereas Duvenaud/Wallach/Wu-built networks construct models 'on raw molecules for learning representations'. The paper nowhere states the general proposition attributed here — that CNN as such 'does not need to extract features manually'; its Introduction (p. 1) discusses CNNs only for their success on ML problems and their tensor-input requirement, and the representation-learning point is credited to other works rather than asserted as a property of CNNs. So the manual-feature-extraction contrast is supported for molecular/graph data, while the blanket CNN claim and its attribution as this paper's statement are not. (LLM, conf=0.70)
- Partly supportedcites [19] sourceExperimental — accuracy not yet measured
The Introduction (p. 1) says the graph convolutional model learns node representations, and Appendix A.1 (pp. 11–12) shows it extracting node features even with featureless input. These passages support the point for this GCN, but the paper does not make the broader comparison with traditional feature-extraction methods or establish it for CNNs generally. (LLM, conf=0.78)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“It successfully leverages ReLU activation function, dropout, and local response normalization (LRN) for the first time on CNN.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > B. AlexNet
What the sources say
- Could not verifycites [11] sourceExperimental — accuracy not yet measured
Section 3.3 (p. 5) mentions ReLU only as an example of a CNN activation function. Searches for AlexNet, dropout, and local response normalization found no passage saying AlexNet used these features or was the first CNN to do so. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The LRN layer was removed since the author of VGGNets found the effect of LRN in deep CNNs was not obvious.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > C. VGGNets
What the sources say
- Could not verifycites [32] sourceExperimental — accuracy not yet measured
The Introduction (page not provided) mentions VGG only as an ImageNet accuracy comparison. The passages retrieved for LRN, local response normalization, and VGG normalization do not address whether VGGNets removed LRN or why. (LLM, conf=0.73)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
44 more claims · 7 with their cited sources checked
Create a free account to see all 154 claims and to verify the 52 references still open.
- Probably not a substantial claim. 72
- It may state two separate claims as one. 63
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Deep learning techniques [78], particularly those based on convolutional neural networks (CNNs) [79], are widely used due to their performance in image recognition tasks.”
From A Review of Passenger Counting in Public Transport Concepts with Solution Proposal Based on Image Processing and Machine Learning (2024) · cites this paper as [79]
The cited source says what the claim attributes to it.
In the citing paper
“Since AlexNet [123] was proposed in 2012, researchers have invented a variety of deeper, wider, and lighter CNN models [124].”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > 4) Face Recognition
“FaceNet [85] proposed by Google in 2015 utilizes 22-layer CNN and 200 million pictures, including eight million people, to train a model.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > References
“85. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, no. Jul, pp. 2121-2159, 2011.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > I. INTRODUCTION
“In 2006, Hinton et al. [11] proposed the following points:”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > References
“11. A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Advances in neural information processing systems, vol. 25, no. 2, 2012.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > 3) Inception v3
“And factorizing 5 × 5 and 3 × 3 convolution kernels into two one-dimensional ones (one by seven and seven by one, one by three and three by one, respectively).”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > 2) Inception v2
“One n x n convolutional layer can be replaced by one 1 x n and one n x 1 convolutional layer shown in Fig. 8 (b).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > 2) Experimental Evaluation
“• From the perspective of accuracy, ELU possesses the best accuracy, but only a little better than ReLU, Leaky ReLU, and PReLU.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > 3) Rules of Thumb for Selection
“VGG-16 (pretrained) MNIST -Cross entropy Adam 512 30 Linear 9.82% 598.14 Sigmoid 11.35% 600.27Tanh 11.35% 596.32 ReLU 99.55% 606.83 Leaky ReLU 99.48% 608.91 PReLU 99.45% 607.27ELU 11.35% 614.81”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > I. INTRODUCTION
“In 1943, McCulloch and Pitts [1] proposed the first mathematical model of neurons-the MP model.”
§ A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects > References
“1. W. S. Mcculloch, and W. H. Pitts, "A logical Calculus of Ideas Immanent in Nervous Activity," The Bulletin of Mathematical Biophysics, vol. 5, pp. 115-133, 1942.”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“Then, Simonyan and Zisserman [39] emphasize the importance of depth, but these primitive CNNs are not more than ten layers.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 7×
“AlexNet [11], the best model in ILSVRC-2012, uses ReLU as the activation function of the CNN-based model, which mitigates gradient vanishing problem when the network is deep and verifies that the use of ReLU surpasses sigmoid in deep networks.”
BackgroundMethodInfluentialU-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“Another frequently used optimizer is adaptive moment estimation [98].”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 9×
“Then, Girshick [136], summarizing the shortcomings of R-CNN [135] and drawing lessons from the SPP-Net [120], proposed fast R-CNN, which introduced the ROI pooling layer, making the network faster.”
BackgroundMethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 6×
“Although it did not achieve satisfactory results on complicated large-scale data sets, such as CIFAR-100 or ImageNet, we can see that it is potential.”
MethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 11×
“LeNet-5 [10] is regarded as the first application used in hand-written digits classification.”
BackgroundMethodInfluentialYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 7×
“Afterward, YOLO v2 [130] proposed a new classification model, darknet-19, which includes 19 convolutional layers and five max-pooling layers.”
MethodInfluentialDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · cited 2×
“Chen et al. [121] proposed a double path network (DPN) for image classification by analyzing the similarities and differences between ResNet [44] and DenseNet [122].”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 2×
“Afterward, deeper network structures emerged, such as GoogLeNet [40] and VGGNets [39], which significantly improve the accuracy in classification tasks.”
BackgroundFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · cited 1×
“[139] proposed the concept of fully convolutional networks and applied CNN structures to image semantic segmentation for the first time.”
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · cited 1×
“In addition, SENet [51] and MobileNet v3 [48] need to transform the output value to (0, 1) for attention mechanism, in which sigmoid is an appropriate way to implement.”
BackgroundHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · cited 9×
“Then, Girshick [136], summarizing the shortcomings of R-CNN [135] and drawing lessons from the SPP-Net [120], proposed fast R-CNN, which introduced the ROI pooling layer, making the network faster.”
MethodInfluentialRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 1×
“[4] proposed a multilayer feedforward network trained by the error backpropagation algorithm—backpropagation network (BP network), which addressed some problems that single-layer perceptron could not solve.”
Background(2016) · Cited by 31,590
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 7×
“They proposed panoramic FPN [138], which combines the FPN network with mask-RCNN to generate a branch of semantic segmentation.”
MethodInfluentialFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“Lots of classic CNN models use it to train their networks in original articles, such as AlexNet [11], VGG [39], inception v2 [41], ResNet [44] and DenseNet [85].”
MethodSemi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 1×
“Besides, ENet [141], PSPNet [142], and so on [143], [144] were proposed to handle specific problems.”
BackgroundChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128 · cited 1×
“The approaches of object detection can be divided into one-stage approaches, such as you only look once (YOLO) [129]–[131], single shot multibox detector (SSD) [132], and CornerNet [133], [134], and two-stage approaches, such as R-CNN [135], fast R-CNN [136], and faster R-CNN [137].”
MethodXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · cited 1×
“Afterward, YOLO v2 [130] proposed a new classification model, darknet-19, which includes 19 convolutional layers and five max-pooling layers.”
MethodDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 6×
“FaceNet [71] uses Adagrad as the optimizer in training.”
MethodInfluentialConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 1×
“Adadelta are leveraged in many tasks [87], [88], [89].”
BackgroundA Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., Jindal, A., & Singh, A. K. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,040
(2010) · International Conference on Machine Learning · Cited by 13,995
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680 · cited 1×
“The architecture of CNN is inspired by visual perception [20].”
MethodAggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · cited 5×
“To minimize the design burden, the smaller convolutional blocks used in ResNeXt have the same topology.”
BackgroundInfluentialThe perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280 · cited 2×
“Rosenblatt [2], [3] proposed a single-layer perceptron model by adding learning ability to the MP model.”
BackgroundUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“[7] also constructed a CNN for handwritten zip code recognition and first used the term “convolution,” which is the original version of LeNet.”
MethodExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2×
“[182] reckoned that the main factor for the vulnerability of neural networks is the linear characteristics, such as ReLU and Maxout.”
BackgroundShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · cited 1×
“In addition, SENet [51] and MobileNet v3 [48] need to transform the output value to (0, 1) for attention mechanism, in which sigmoid is an appropriate way to implement.”
BackgroundThe graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582 · cited 1×
“[34] extended the existing neural network method to process the graph domain data.”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556
(2010) · Cited by 8,261
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00482 · Cited by 7,781
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 1×
“Furthermore, in order for convolution kernels to perceive larger areas, dilated convolution [23] was proposed.”
BackgroundDeformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 2×
“[24] used a parallel convolution layer to learn offsets and added it to the convolution kernel’s original position to realize scale transformation, such as translation and rotation, and then”
BackgroundMethodShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 1×
“Some commonly used algorithms and toolkits include bayesian optimization [111], Nevergrad [112], and ZOOpt [113].”
MethodDeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · cited 3×
“DeepFace [150] and DeepID [68] achieved excellent results on the LFW [81] data set, surpassing humans for the first time in the unconstrained scenarios.”
BackgroundFashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108 · cited 1×
“2) Experimental Evaluation: To compare aforementioned activation functions, two classic CNN models, LeNet-5 [10] and VGG-16 [39], are tested on four benchmark data sets, including the Mixed National Institute of Standards and Technology (MNIST) database [10], fashion-MNIST [63], CIFAR-10 [64], and CIFAR-100 [64].”
MethodFukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikainen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,584 · cited 1×
“Different from the traditional feature extraction methods [17]–[19], CNN does not need to extract features manually.”
MethodParkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479 · cited 1×
“Besides, VGGFace [72] also deploys triplet loss to train the model.”
MethodDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336 · cited 1×
“Contrastive loss [66] enlarges the distance between different categories and minimizes the distance within the same categories.”
BackgroundADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · cited 1×
“The Adadelta algorithm [93], an extension of the Adagrad, is designed to reduce its monotonically decreasing learning rate.”
MethodGhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222 · cited 2×
“[51] proposed GhostNet to reduce computational cost effectively.”
BackgroundVoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813 · cited 2×
“Zhou and Tuzel [161] proposed a single end-to-end VoxelNet for point-cloud-based 3-”
BackgroundHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 1×
“[166] proposed deep pruning for CNN to prune networks by learning essential connections.”
BackgroundInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · cited 1×
“MobileNet [46], inception v3 [42], and inception v4 [43] achieved their best models using RMSprop.”
Method3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298801 · Cited by 4,550 · cited 1×
“[158] proposed a generative 3D CNN of shape named 3-D ShapeNet, which can be applied to object detection of RGBD images.”
MethodGhostNet: More Features From Cheap OperationsPreprint
Han, K., Wang, Y.-H., Tian, Q., Guo, J.-Y., Xu, C.-J., & Xu, C. (2019) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1911.11907 · Cited by 4,540
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · cited 1×
“Exponential linear units (ELU) function [60] [see Fig.”
BackgroundA general regression neural network
Specht, D. F. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.97934 · Cited by 4,492 · cited 1×
“Chaotic neural networks [8] and A general regression neural network [9].”
BackgroundXNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · cited 1×
“[168] proposed binary-weight networks and XNOR-networks.”
Dynamic Routing Between Capsules
Sabour, S., Frosst, N., & Hinton, G. E. (2017) · arXiv · doi:10.48550/arxiv.1710.09829 · Cited by 4,167
A Discriminative Feature Learning Approach for Deep Face Recognition
Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_31 · Cited by 4,077 · cited 2×
“Another one is the center loss [77], which is an improvement based upon cross entropy.”
BackgroundMethodSelf-citationLearning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · cited 1×
“It was first used in SiameseNet [67] and later was deployed in DeepID2 [68], DeepID2+ [69], and DeepID3 [70].”
MethodSpectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983 · cited 1×
“[35] found that CNN can use the local translation invariance of the signal class in its domain, which is very useful in image and audio recognition tasks.”
MethodNeural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974 · cited 1×
“[186] proposed a machine learning search algorithm that uses reinforcement learning to maximize the target network and implements an auto-built network on the CIFAR-10 data set.”
MethodCornerNet: Detecting Objects as Paired Keypoints
Law, H., & Deng, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_45 · Cited by 3,852
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503 · cited 1×
“Rawat and Wang [14] reviewed CNN for image recognition.”
BackgroundVoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458 · cited 2×
“Maturana and Scherer [159] proposed VoxNet, which is a 3D CNN architecture that contains two 3-”
BackgroundHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · cited 1×
“[188] designed the mobile neural architecture search to solve the CNN inferring delay problem.”
BackgroundCosFace: Large Margin Cosine Loss for Deep Face Recognition
Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., & Liu, W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00552 · Cited by 3,034
Self-citationSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Deformable ConvNets V2: More Deformable, Better Results
Zhu, X., Hu, H., Lin, S., & Dai, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00953 · Cited by 2,943 · cited 1×
“[25] found that although deformable convolution can adapt to geometric transformation and expand the receptive field by moving the convolution kernel unit, the training is affected by many features that are not related to the available domain.”
BackgroundConvergence Results for Neural Networks via Electrodynamics
Panigrahy, R., Rahimi, A., Sachdeva, S., & Zhang, Q. (2018) · Leibniz international proceedings in informatics · doi:10.4230/lipics.itcs.2018.22 · Cited by 2,761
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
In Defense of the Triplet Loss for Person Re-Identification
Hermans, A., Beyer, L., & Leibe, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07737 · Cited by 2,687 · cited 1×
“CNNs with triplet loss or its variants can be used in identification problems, such as face identification [71]–[73], person re-identification [74], [75], and vehicle re-identification [76].”
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643 · cited 1×
“[5] proposed a time-delay neural network for speech recognition, which can be viewed as a 1-D CNN.”
BackgroundYOLACT: Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00925 · Cited by 2,536 · cited 1×
“[146] based on RetinaNet [147] harnessed ResNet-101 and FPN to fuse multiscale features.”
MethodHawkins, D. M. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0342472 · Cited by 2,522 · cited 1×
“After convolution, feature maps consist of a large number of features that is prone to causing overfitting problem [21].”
BackgroundSearching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335 · cited 5×
“In addition to sigmoid, tanh, and ReLU-likes, Google proposed another activation function called Swish [61] [see Fig.”
BackgroundInfluentialOn the momentum term in gradient descent learning algorithms
Qian, N. (1999) · Neural Networks · doi:10.1016/s0893-6080(98)00116-6 · Cited by 2,312 · cited 1×
“proposed the Momentum algorithm [80].”
SphereFace: Deep Hypersphere Embedding for Face Recognition
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.713 · Cited by 2,280
Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
Ghiasi, G., Lin, T.-Y., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00720 · Cited by 2,099 · cited 2×
“[189] proposed NAS-FPN by applying NAS to feature pyramid structure search of object detection.”
BackgroundMethodA new model for learning in graph domains
Gori, M., Monfardini, G., & Scarselli, F. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1555942 · Cited by 2,075
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021 · cited 1×
“Adadelta are leveraged in many tasks [94]–[96].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021 · cited 1×
“Akhtar and Mian [184] listed three directions of defense against adversarial attacks, which are, respectively, improved on training examples, modified trained networks, and additional networks.”
BackgroundDeep Learning Face Representation from Predicting 10,000 Classes
Sun, Y., Wang, X., & Tang, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.244 · Cited by 1,997
Deep Learning Face Representation by Joint Identification-Verification
Sun, Y., Wang, X., & Tang, X. (2014) · arXiv · doi:10.48550/arxiv.1406.4773 · Cited by 1,906 · cited 4×
“It was first used in SiameseNet [67] and later was deployed in DeepID2 [68], DeepID2+ [69], and DeepID3 [70].”
BackgroundMethodInfluentialENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826 · cited 2×
“Besides, ENet [141], PSPNet [142], and so on [143], [144] were proposed to handle specific problems.”
BackgroundOn the Convergence of Adam and Beyond
Reddi, S. J., Kale, S., & Kumar, S. (2019) · arXiv · doi:10.48550/arxiv.1904.09237 · Cited by 1,771
PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS
ROSENBLATT, F. (1961) · doi:10.21236/ad0256582 · Cited by 1,691
Kirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00963 · Cited by 1,659
Incorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,595
(2018) · International Conference on Machine Learning · Cited by 1,465
Abdeljaber, O., Avci, O., Kiranyaz, S., Gabbouj, M., & Inman, D. J. (2017) · Journal of Sound and Vibration · doi:10.1016/j.jsv.2016.10.043 · Cited by 1,278 · cited 1×
“[108] proposed a direct damage identification method based on 1D CNN that can apply to the original environmental vibration signals.”
MethodPerson Re-identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function
Cheng, D., Gong, Y., Zhou, S., Wang, J., & Zheng, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.149 · Cited by 1,247 · cited 1×
“CNNs with triplet loss or its variants can be used in identification problems, such as face identification [71]–[73], person re-identification [74], [75], and vehicle re-identification [76].”
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · cited 1×
“Besides, multivariate quantization is leveraged to represent weights by several values [171], [172].”
BackgroundAihara, K., Takabe, T., & Toyoda, M. (1990) · Physics Letters A · doi:10.1016/0375-9601(90)90136-c · Cited by 1,175
Speeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.88 · Cited by 1,164 · cited 1×
“[164] proposed a structure-independent method using cross channel redundancy or filter redundancy to reconstruct a low-rank filter.”
MethodConvolutional neural network: a review of models, methodologies and applications to object detection
Dhillon, A., & Verma, G. K. (2020) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00203-0 · Cited by 1,119 · cited 1×
“Dhillon and Verma [13] discussed the architectures of some classic networks, but there are many new-generation networks, after their work, that have been proposed, such as MobileNet v3, inception v4, and ShuffleNet series, which deserve researchers’ attention.”
BackgroundConvolutional Neural Networks for Sentence ClassificationPreprint
Kim, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5882 · Cited by 1,112
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
Liu, K., Dolan-Gavitt, B., & Garg, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00470-5_13 · Cited by 1,055 · cited 1×
“[183] proposed a method of fine-tuning and pruning that can effectively defend against the threat of backdoor injection.”
MethodNeocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition
Fukushima, K., & Miyake, S. (1982) · Lecture Notes in Biomathematics · doi:10.1007/978-3-642-46466-9_18 · Cited by 991
Deeply learned face representations are sparse, selective, and robust
Sun, Y., Wang, X., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298907 · Cited by 939
CondenseNet: An Efficient DenseNet Using Learned Group Convolutions
Huang, G., Liu, S., Maaten, L. V. D., & Weinberger, K. Q. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00291 · Cited by 904
Medical image classification with convolutional neural network
Li, Q., Cai, W., Wang, X., Zhou, Y., Feng, D. D., & Chen, M. (2014) · 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV) · doi:10.1109/icarcv.2014.7064414 · Cited by 880 · cited 2×
“[123] designed a custom CNN with shallow convolution layers to the classification of interstitial lung disease.”
MethodZhu, C., Han, S., Mao, H., & Dally, W. J. (2016) · arXiv · doi:10.48550/arxiv.1612.01064 · Cited by 865
A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2)
Nesterov, Y. E. (1983) · Medical Entomology and Zoology · Cited by 865
(2016) · Cited by 844
DeepID3: Face Recognition with Very Deep Neural Networks
Sun, Y. J., Liang, D., Wang, X., & Tang, X. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.00873 · Cited by 842 · cited 2×
“It was first used in SiameseNet [67] and later was deployed in DeepID2 [68], DeepID2+ [69], and DeepID3 [70].”
BackgroundMethodAdaptive Graph Convolutional Neural Networks
Li, R., Wang, S., Zhu, F., & Huang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11691 · Cited by 842 · cited 1×
“[37] constructs a residual graph using a learnable distance function with two-node features as input.”
MethodVideo Frame Interpolation via Adaptive Separable Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.37 · Cited by 725 · cited 1×
“The AdaMax algorithm [98] is a variant of Adam that makes the boundary range of the learning rate simpler, and it has been used to train CNN models [101], [102].”
MethodSelf-citationAnalyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12470 · Cited by 703 · cited 1×
“More precisely, researchers [171],[172],[173],[174] have proposed some methods to deceive CNN, resulting in a”
BackgroundDeep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Song, S., & Xiao, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.94 · Cited by 681 · cited 2×
“Song and Xiao [160] proposed a 3-”
BackgroundNeural Network for Graphs: A Contextual Constructive Approach
Micheli, A. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2010350 · Cited by 639 · cited 2×
“To record each layer’s features, NN4G [55] uses residual structure and skip connections.”
BackgroundMethodA review on deep convolutional neural networks
Aloysius, N., & Geetha, M. (2017) · 2017 International Conference on Communication and Signal Processing (ICCSP) · doi:10.1109/iccsp.2017.8286426 · Cited by 636 · cited 2×
“Before our work, there exist several researchers who have reviewed CNN. Aloysius and Geetha [12] paid attention to frameworks of deep learning chronologically.”
BackgroundDenseNet: Implementing Efficient ConvNet Descriptor Pyramids
Iandola, F., Moskewicz, M. W., Karayev, S., Girshick, R., Darrell, T. J., & Keutzer, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.1869 · Cited by 629 · cited 3×
“Chen et al. [121] proposed a double path network (DPN) for image classification by analyzing the similarities and differences between ResNet [44] and DenseNet [122].”
Scale Invariant Feature Transform
Lindeberg, T. (2012) · Scholarpedia · doi:10.4249/scholarpedia.10491 · Cited by 621 · cited 1×
“Different from the traditional feature extraction methods [17]–[19], CNN does not need to extract features manually.”
MethodDiverse Region-Based CNN for Hyperspectral Image Classification
Zhang, M., Li, W., & Du, Q. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2809606 · Cited by 604 · cited 1×
“In addition, CNN can be deployed in medical image classification [123], [124], traffic scenes related classification [125], [126], and so on [127], [128].”
MethodGeneral $E(2)$-Equivariant Steerable CNNs
Weiler, M., & Cesa, G. (2019) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1911.08251 · Cited by 600
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572
Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA
Guo, K., Sui, L., Qiu, J., Yu, J., Wang, J., Yao, S., Han, S., Wang, Y., & Yang, H. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2705069 · Cited by 562
Hand gesture recognition with 3D convolutional neural networks
Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301342 · Cited by 543 · cited 1×
“NAG has been used to train CNN-based models in many tasks [89]–[91].”
MethodMish: A Self Regularized Non-Monotonic Activation Function
Misra, D. (2019) · arXiv · doi:10.48550/arxiv.1908.08681 · Cited by 541
Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment
Gao, H., Cheng, B., Wang, J., Li, K., Zhao, J., & Li, D. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2822828 · Cited by 524 · cited 1×
“A lot of impressive applications of CNN are emerging in modern society, from simple cat–dog classifiers [190] to sophisticated autonomous vehicles [189], [191], [192].”
BackgroundEfficient Neural Architecture Search via Parameter Sharing
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03268 · Cited by 517
Towards Accurate Binary Convolutional Neural Network
Lin, X., Zhao, C., & Pan, W. (2017) · arXiv · doi:10.48550/arxiv.1711.11294 · Cited by 453 · cited 1×
“[169] use a linear combination of multiple binary weight bases to approximate full-precision weights and multiple binary activations to reduce information loss, which suppresses the prediction accuracy degradation caused by previous binary CNN.”
MethodSpeeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3866 · Cited by 447
Mish: A Self Regularized Non-Monotonic Activation Function
Misra, D. (2020) · Proceedings of the British Machine Vision Conference 2020 · doi:10.5244/c.34.191 · Cited by 432
Why do deep convolutional networks generalize so poorly to small image transformations?
Azulay, A., & Weiss, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.12177 · Cited by 428
Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification
Zhuang, C., & Ma, Q. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186116 · Cited by 423 · cited 1×
“Zhuang and Ma [38] implemented a dual-GCN (DGCN) by parallelizing two graph convolution layers.”
MethodAccelerating Deep Convolutional Neural Networks Using Specialized Hardware
Ovtcharov, K., Ruwase, O., Kim, J.-Y., Fowers, J., Strauß, K., & Chung, E. S. (2015) · Cited by 412
Large-Margin Softmax Loss for Convolutional Neural Networks
Liu, W., Wen, Y., Yu, Z., & Yang, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02295 · Cited by 408
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06870 · Cited by 406
Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., & Feng, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01629 · Cited by 397
Fake news detection on social media using geometric deep learning
Monti, F., Frasca, F., Eynard, D., Mannion, D., & Bronstein, M. M. (2019) · arXiv · doi:10.48550/arxiv.1902.06673 · Cited by 392
CNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
Jiang, Y., Chen, L., Zhang, H., & Xiao, X. (2019) · PLOS ONE · doi:10.1371/journal.pone.0214587 · Cited by 365 · cited 2×
“[124] proposed a method based on SE-ResNet modules to classify breast cancer tissues.”
MethodPhoneme Recognition Using Time-Delay Neural Networks
WAIBEL, A., HANAZAWA, T., HINTON, G., SHIKANO, K., & LANG, K. J. (1990) · Readings in Speech Recognition · doi:10.1016/b978-0-08-051584-7.50037-1 · Cited by 302
A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks
Gokhale, V., Jin, J., Dundar, A., Martini, B., & Culurciello, E. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.106 · Cited by 300
Action Recognition using Visual Attention
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 298
Zhang, Q., Zhou, D., & Zeng, X. (2017) · IEEE Access · doi:10.1109/access.2017.2707460 · Cited by 282 · cited 1×
“[117] proposed a multiresolution 1-D CNN structure to identify arrhythmia and other diseases based on ECG data.”
BackgroundProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 267
On-Line Learning in Neural Networks
MTW, & Saad, D. (2000) · Journal of the American Statistical Association · doi:10.2307/2669811 · Cited by 260
A patch-based convolutional neural network for remote sensing image classification
Sharma, A., Liu, X., Yang, X., & Shi, D. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.07.017 · Cited by 260 · cited 1×
“In addition, CNN can be deployed in medical image classification [123], [124], traffic scenes related classification [125], [126], and so on [127], [128].”
MethodAttention-Based 3D-CNNs for Large-Vocabulary Sign Language Recognition
Huang, J., Zhou, W., Li, H., & Li, W. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2870740 · Cited by 232 · cited 1×
“[156] designed a 3D CNN structure to carry out sign language recognition that the 3-D convolutional layers automatically extract distinct spatial and temporal features from the video stream, which are input to the FC layer for classification.”
Method3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
Weiler, M., Geiger, M., Welling, M., Boomsma, W., & Cohen, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02547 · Cited by 231 · cited 1×
“[31] proposed the 3-D steerable CNN that is equivalent to rigid body motion, representing data in 3-D Euclidean space using scalar, vector, and tensor fields and mapping in these representations using equal-variant convolution.”
MethodEnergy-Efficient CNN Implementation on a Deeply Pipelined FPGA Cluster
Zhang, C., Wu, D., Sun, J., Sun, G., Luo, G., & Cong, J. (2016) · Proceedings of the 2016 International Symposium on Low Power Electronics and Design · doi:10.1145/2934583.2934644 · Cited by 227
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Mao, H., Han, S., Pool, J., Li, W., Liu, X., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1705.08922 · Cited by 219 · cited 1×
“[167] deemed that coarse pruning reaches a better compression ratio than fine-grained pruning.”
BackgroundOne dimensional convolutional neural network architectures for wind prediction
Harbola, S., & Coors, V. (2019) · Energy Conversion and Management · doi:10.1016/j.enconman.2019.05.007 · Cited by 219 · cited 2×
“Harbola and Coors [115] proposed 1-D Single CNN for predicting dominant wind speed and direction.”
BackgroundA feature agnostic approach for glaucoma detection in OCT volumes
Maetschke, S., Antony, B., Ishikawa, H., Wollstein, G., Schuman, J., & Garnavi, R. (2019) · PLOS ONE · doi:10.1371/journal.pone.0219126 · Cited by 213 · cited 1×
“Nadam is used in many tasks [97], [98], [99].”
MethodStructured Transforms for Small-Footprint Deep LearningPreprint
Sindhwani, V., Sainath, T. N., & Kumar, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.01722 · Cited by 208
Very deep multilingual convolutional neural networks for LVCSR
Sercu, T., Puhrsch, C., Kingsbury, B., & LeCun, Y. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472620 · Cited by 205
CNN Design for Real-Time Traffic Sign Recognition
Shustanov, A., & Yakimov, P. (2017) · Procedia Engineering · doi:10.1016/j.proeng.2017.09.594 · Cited by 195 · cited 1×
“There are many applications of CNN in daily life, including security identification system [173], [174], medical image identification [163], [175], [176], traffic sign recognition [177], and license plate recognition [178].”
BackgroundCornerNet-Lite: Efficient Keypoint Based Object Detection
Law, H., Teng, Y., Russakovsky, O., & Deng, J. (2019) · arXiv · doi:10.48550/arxiv.1904.08900 · Cited by 172
Predicting sex from brain rhythms with deep learning
van Putten, M. J. A. M., Olbrich, S., & Arns, M. (2018) · Scientific Reports · doi:10.1038/s41598-018-21495-7 · Cited by 169 · cited 2×
“The practice has proved that the Adam algorithm works well on many problems and is applicable to many different CNN structures [99]–[101].”
BackgroundMethodAn End-To-End Network for Panoptic Segmentation
Liu, H., Peng, C., Yu, C., Wang, J., Liu, X., Yu, G., & Jiang, W. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00633 · Cited by 168 · cited 1×
“[149] proposed OANet that also introduced the FPN based on the mask-RCNN.”
MethodCornerNet-Lite: Efficient Keypoint based Object Detection
Law, H., Teng, Y., Russakovsky, O., & Deng, J. (2020) · Proceedings of the British Machine Vision Conference 2020 · doi:10.5244/c.34.5 · Cited by 165
Kosiorek, A. R., Sabour, S., Teh, Y. W., & Hinton, G. E. (2019) · arXiv · doi:10.48550/arxiv.1906.06818 · Cited by 163
Simplifying ConvNets for Fast Learning
Mamalet, F., & García, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33266-1_8 · Cited by 154
Vehicle Re-identification: an Efficient Baseline Using Triplet Embedding
Kuma, R., Weill, E., Aghdasi, F., & Sriram, P. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852059 · Cited by 151 · cited 1×
“CNNs with triplet loss or its variants can be used in identification problems, such as face identification [71]–[73], person re-identification [74], [75], and vehicle re-identification [76].”
Building DNN acoustic models for large vocabulary speech recognition
Maas, A. L., Qi, P., Xie, Z., Hannun, A., Lengerich, C. T., Jurafsky, D. S., & Ng, A. Y. (2016) · Computer Speech & Language · doi:10.1016/j.csl.2016.06.007 · Cited by 147 · cited 1×
“Nesterov Accelerated Gradient has been used to train CNN-based models in many tasks [82], [83], [84].”
MethodF-CNN: An FPGA-based framework for training Convolutional Neural Networks
Zhao, W., Fu, H., Luk, W. W., Yu, T., Wang, S., Feng, B., Ma, Y., & Yang, G. (2016) · IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP) · doi:10.1109/asap.2016.7760779 · Cited by 147 · cited 1×
“Through experiments, some researchers [194]–[197], [200]–[202] found that FPGA has a great advantage over CPU and GPU in power efficiency.”
BackgroundTowards the Limit of Network Quantization
Choi, Y., El-Khamy, M., & Lee, J. (2016) · arXiv · doi:10.48550/arxiv.1612.01543 · Cited by 143
CNN-based anti-spoofing two-tier multi-factor authentication system
Sajjad, M., Khan, S., Hussain, T., Muhammad, K., Sangaiah, A. K., Castiglione, A., Esposito, C., & Baik, S. W. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.015 · Cited by 140 · cited 1×
“There are many applications of CNN in daily life, including security identification system [173], [174], medical image identification [163], [175], [176], traffic sign recognition [177], and license plate recognition [178].”
BackgroundFault Detection and Severity Identification of Ball Bearings by Online Condition Monitoring
Abdeljaber, O., Sassi, S., Avcı, O., Serkan Kiranyaz, B., Ibrahim, A., & Gabbouj, M. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2018.2886789 · Cited by 131 · cited 1×
“[119] designed a compact 1D CNN used in fault detection and severity identification of ball bearings.”
MethodLarge Margin Softmax Loss for Speaker Verification
Liu, Y., He, L. J., & Liu, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2357 · Cited by 131 · cited 1×
“Large-margin softmax loss was used in face recognition [81], emotion recognition [82], speaker verification [83], and so on.”
MethodHolistic recognition of low quality license plates by CNN using track annotated data
Špaňhel, J., Sochor, J., Juránek, R., Herout, A., Marsik, L., & Zemčík, P. (2017) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078501 · Cited by 130 · cited 1×
“There are many applications of CNN in daily life, including security identification system [173], [174], medical image identification [163], [175], [176], traffic sign recognition [177], and license plate recognition [178].”
Background3D convolutional neural network for automatic detection of lung nodules in chest CT
Hamidian, S., Sahiner, B., Petrick, N., & Pezeshk, A. X. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2255795 · Cited by 127
Deep 3D convolution neural network for CT brain hemorrhage classification
Jnawali, K., Arbabshirani, M. R., Rao, N. A. H. K., & Patel, A. A. (2018) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2293725 · Cited by 124 · cited 2×
“A lot of practitioners [163] are dedicated to these jobs.”
BackgroundBackdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Liao, C., Zhong, H., Squicciarini, A., Zhu, S., & Miller, D. (2018) · arXiv · doi:10.48550/arxiv.1808.10307 · Cited by 110 · cited 1×
“[180] introduced that generated perturbation masks are injected into the training samples as a backdoor to deceive the model.”
BackgroundKe, H., Chen, D., Li, X., Tang, Y., Shah, T., & Ranjan, R. K. (2018) · IEEE Access · doi:10.1109/access.2018.2810882 · Cited by 101 · cited 1×
“There are many applications of CNN in daily life, including security identification system [173], [174], medical image identification [163], [175], [176], traffic sign recognition [177], and license plate recognition [178].”
BackgroundUrtnasan, E., Kim, H.-G., Park, J.-U., Kang, D., & Lee, K.-J. (2019) · Journal of Korean Medical Science · doi:10.3346/jkms.2019.34.e64 · Cited by 89 · cited 1×
“[114] proposed a method based on 1D CNN to predict atrial fibrillation using short-term ECG data automatically.”
MethodA fully convolutional deep auditory model for musical chord recognition
Korzeniowski, F., & Widmer, G. (2016) · IEEE International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2016.7738895 · Cited by 76
Group emotion recognition with individual facial emotion CNNs and global image based CNNs
Tan, L., Zhang, K., Wang, K., Zeng, X., Peng, X., & Qiao, Y. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143008 · Cited by 73 · cited 1×
“Large-margin softmax loss was used in face recognition [81], emotion recognition [82], speaker verification [83], and so on.”
MethodNguyen, D. Q., & Verspoor, K. (2018) · Proceedings of the BioNLP 2018 workshop · doi:10.18653/v1/w18-2314 · Cited by 67
A Review of Image Recognition with Deep Convolutional Neural Network
Liu, Q., Zhang, N., Yang, W., Wang, S., Cui, Z., Chen, X., & Chen, L. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63309-1_7 · Cited by 64 · cited 1×
“[15] discussed CNN for image recognition.”
Self-citationCoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs
Huang, Q.-J., Wang, D., Dong, Z., Gao, Y.-Z., Cai, Y., Li, T., Wu, B.-C., Keutzer, K., & Wawrzynek, J. (2021) · Symposium on Field Programmable Gate Arrays · doi:10.1145/3431920.3439295 · Cited by 63 · cited 1×
“[26] combined depthwise convolution with deformable convolution.”
Convolutional neural network based image segmentation: a review
Rehman, S., Ajmal, H., Farooq, U., et al. (2018) · doi:10.1117/12.2304711 · Cited by 62 · cited 1×
“[16] discussed CNN for image segmentation.”
Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation
Pastor, F., Gandarias, J. M., García-Cerezo, A. J., & Gómez-de-Gabriel, J. M. (2019) · Sensors · doi:10.3390/s19245356 · Cited by 49 · cited 1×
“[162] designed TactNet3D, harnessing tactile features to recognize objects.”
BackgroundZOOpt: a toolbox for derivative-free optimization
Liu, Y.-R., Hu, Y.-Q., Qian, H., Qian, C., & Yang, Y. (2022) · Science China Information Sciences · doi:10.1007/s11432-021-3416-y · Cited by 41 · cited 1×
“Some commonly used algorithms and toolkits include bayesian optimization [111], Nevergrad [112], and ZOOpt [113].”
MethodAggressive Deep Driving: Model Predictive Control with a CNN Cost ModelPreprint
Drews, P., Williams, G., Goldfain, B., Theodorou, E. A., & Rehg, J. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05303 · Cited by 39 · cited 1×
“A lot of impressive applications of CNN are emerging in modern society, from simple cat–dog classifiers [190] to sophisticated autonomous vehicles [189], [191], [192].”
BackgroundDifferentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks
Zhang, Z., Li, J., Shao, W., Peng, Z., Zhang, R., Wang, X., & Luo, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00364 · Cited by 38 · cited 1×
“[29] proposed Groupable ConvNet to combine network architecture search (NAS) with group convolution, which can automatically learn the number of groups using an end-to-end approach.”
BackgroundDeep person re-identification with improved embedding and efficient training
Jin, H., Wang, X., Liao, S., & Li, S. Z. (2017) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2017.8272706 · Cited by 37 · cited 1×
“with CNN in face recognition [77], image retrieval [78], person re-identification [79], speaker recognition [80], and so on.”
Method
Research works printed but not linked: 7
- itqan, 2016, User identification system based on finger-vein patterns using convolutional neural network, ARPN J Eng Appl Sci, 11, 3316
- cao, 2016, Action recognition with joints-pooled 3D deep convolutional descriptors, Proc IJCAI, 1, 3
- krizhevsky, 2010, Convolutional deep belief networks on CIFAR-10, Unpublished manuscript, 40, 1
- hagan, 2002, Neural Network Design, 9
- zhang, 1988, Shift-invariant pattern recognition neural network and its optical architecture, Proc Annu Conf Jpn Soc Appl Phys, 734
- jajodia, 2019, Image classification–cat and dog images, Image, 6, 570
- rapin, 2018, Nevergrad—A Gradient-Free Optimization Platform Version 0 2
Cited by
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Evaluation metrics and statistical tests for machine learning
Rainio, O., Teuho, J., & Klén, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-56706-x · Cited by 1,338
A review of convolutional neural networks in computer vision
Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10721-6 · Cited by 1,148
Taye, M. M. (2023) · Computers · doi:10.3390/computers12050091 · Cited by 1,057
“12 , 91 The application of neural network-based DL technology is widespread across many industries and areas of study, including healthcare, sentiment analysis, NLP, the id of images, BI, cybersecurity, and many more [45,52,53].”
BackgroundMethodConvolutional Neural Networks: A Survey
Krichen, M. (2023) · Computers · doi:10.3390/computers12080151 · Cited by 623
“Convolutional neural networks (CNNs) [14–16] are a popular type of ML model that have been widely utilized for various tasks, including image recognition [17–20], speech recognition [21–25], and NLP [26–30].”
Metadata
- Authors
- Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J.
- Affiliations
- Hohai University (CN)
- Griffith University (AU)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2022-12
- Volume
- 33
- Issue
- 12
- Pages
- 6999-7019
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3168997536
- Cited by
- 5,273
- References
- 272
- Citations by year
- 2026: 853 · 2025: 1745 · 2024: 1500 · 2023: 851 · 2022: 244
- FWCI
- 40.90
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Griffith Research Online (submittedVersion)
- arXiv (Cornell University)
- Funders
- Natural Science Foundation of Jiangsu Province (BK20191298)
- Fundamental Research Funds for the Central Universities (B200202175)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- convolutional neural networks, multidimensional convolution, convolutional neural network architecture, hyperparameter optimization, computer vision, natural language processing, open problems
- MeSH terms
- Natural Language Processing, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.