Convolutional networks and applications in vision
Open accessThe DOI matches its Crossref record; 52 of the 56 references it cites were found in a registry or an index.
Abstract
Intelligent tasks, such as visual perception, auditory perception, and language understanding require the construction of good internal representations of the world (or "features")? which must be invariant to irrelevant variations of the input while, preserving relevant information. A major question for Machine Learning is how to learn such good features automatically. Convolutional Networks (ConvNets) are a biologically-inspired trainable architecture that can learn invariant features. Each stage in a ConvNets is composed of a filter bank, some nonlinearities, and feature pooling layers. With multiple stages, a ConvNet can learn multi-level hierarchies of features. While ConvNets have been successfully deployed in many commercial applications from OCR to video surveillance, they require large amounts of labeled training samples. We describe new unsupervised learning algorithms, and new non-linear stages that allow ConvNets to be trained with very few labeled samples. Applications to visual object recognition and vision navigation for off-road mobile robots are described.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3×
“Most common feature extraction systems like SIFT [40], HoG [41] are composed of filterbanks (oriented edge detectors at multiple scales) followed by non-linearities (winner take all) and pooling (histogramming).”
MethodHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 2×
“Most common feature extraction systems like SIFT [40], HoG [41] are composed of filterbanks (oriented edge detectors at multiple scales) followed by non-linearities (winner take all) and pooling (histogramming).”
MethodReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“The innovation in [5], [1] was to simplify the architecture and to use the back-propagation algorithm to train the entire system in a supervised fashion.”
MethodSelf-citationBeyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · cited 4×
“A Pyramid Match Kernel (PMK) SVM [39] classifer can also be seen as another layer of feature extraction since it performs a K-means based feature extraction followed by local histogramming.”
MethodInfluentialGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“Recent research in deep learning [30], [31], [32] has shown that unsupervised learning can be used to train each stage one after the other using only unlabeled data, reducing the requirement for labeled samples significantly.”
Background(1989) · neural information processing systems · Cited by 3,736
Self-citationSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. · doi:10.1109/icdar.2003.1227801 · Cited by 2,741
Research works printed but not linked: 4
- lecun, 0, Handwritten digit recognition with a backpropagation network, NIPS'89
- frome, 0, Large-scale privacy protection in street-level imagery, ICCV'09
- collobert, 2008, Torch, Workshop on Machine Learning Open Source Software NIPS
- lecun, 2002, The Lush Reference Manual
Cited by
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
“Traditionally, the neighborhoods summarized by adjacent pooling units do not overlap (e.g., [17, 11, 4]).”
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317
“However, these same architectures have been shown to be unstable to small, well sought, perturbations of the images.”
BackgroundOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“a form of structured sparsity is orthogonal to the sparsity that occurs from network pruning as described in Section VII-B2. Accordingly, a convolutional neural network (CNN) is a popular form of DNN [35]. 1) Convolutional Neural Networks (CNNs): CNNs are composed of multiple convolutional layers (CONV), as shown in F…”
BackgroundGeometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
“and computer vision [5], [6], [7], [8], [9], [10], [11] (the reader is referred to [12], [13] for many additional examples of successful applications of deep learning).”
BackgroundSelf-citationDeep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
“These included image and object classification and recognition (Chellapilla, Puri, & Simard, 2006; Ranzato et al., 2007; Weston et al., 2008; Jarrett et al., 2009; Lee et al., 2009; LeCun et al., 2010; Scherer et al., 2010; Boureau et al., 2010; Masci, Meier, Ciresan, & Schmidhuber, 2011), face detection (Nasse, Thura…”
BackgroundMethodInfluentialEyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
“The CNN algorithm is constructed by stacking multiple computation layers for feature extraction and classifi-cation [30].”
MethodA survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
“In equation (7), ( ) i f x is a transformed signal, whereas i x is original input.”
BackgroundDeep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art
Zhang, L., Zhang, L., & Du, B. (2016) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2540798 · Cited by 2,208
“Finally, objects are assembled into scenes [107], [108].”
BackgroundCan AI help in screening viral and COVID-19 pneumonia?
Chowdhury, M. E. H., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M. A., Mahbub, Z. B., Islam, K. R., Khan, M. S., Iqbal, A., Emadi, N. A., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3010287 · Cited by 1,913
“com/cnn/understand‐resnet‐alexnet‐vgg‐inception/ [49] Y.”
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883
“asserts that one way we could achieve this would be to utilize and integrate both unsupervised and supervised learning [24].”
BackgroundA review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
Invariant Scattering Convolution Networks
Bruna, J., & Mallat, S. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.230 · Cited by 1,633
“A scattering is implemented by a deep convolution network [20] having a very specific architecture.”
BackgroundDeep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377
“Deep learning models perform extremely well with correlated data, which contributed to substantial improvements in computer vision [45], image processing, video processing, face recognition [80], speech recognition [33], text-to-speech systems [62] and natural language processing [2, 14, 88].”
BackgroundAutomated Identification of Diabetic Retinopathy Using Deep Learning
Gargeya, R., & Leng, T. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.02.008 · Cited by 1,362
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases
Janowczyk, A., & Madabhushi, A. (2016) · Journal of Pathology Informatics · doi:10.4103/2153-3539.186902 · Cited by 1,348
“In past approaches, [34] a sigmoid function was typically used, but more recent implementations [35,36] have shown that a rectified linear (ReLu) activation has more favorable properties.”
MethodTarget Classification Using the Deep Convolutional Networks for SAR Images
Chen, S., Wang, H., Xu, F., & Jin, Y.-Q. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2551720 · Cited by 1,347
“In ConvNets, the hidden units of convolution and pooling layers are organized as a set of 2-D arrays, which are called feature maps [21], as shown in Fig.”
MethodA review of unsupervised feature learning and deep learning for time-series modeling
Längkvist, M., Karlsson, L., & Loutfi, A. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.01.008 · Cited by 1,243
“For music recognition, a commonly used set of features are MFCCs, chroma, constant-Q spectrograms (CQT) [109], local contrast normalization (LCN) [75], or Compressive Sampling (CS) [18].”
MethodMachine learning on big data: Opportunities and challenges
Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.026 · Cited by 1,228
Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001177 · Cited by 1,182
“A convolutional neural network (CNN) is constructed by stacking multiple computation layers as a directed acyclic graph [36].”
MethodMajorana zero modes and topological quantum computation
Sarma, S. D., Freedman, M., & Nayak, C. (2015) · npj Quantum Information · doi:10.1038/npjqi.2015.1 · Cited by 1,180
Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images
Rahman, T., Khandakar, A., Qiblawey, Y., Tahir, A., Kiranyaz, S., Abul Kashem, S. B., Islam, M. T., Al Maadeed, S., Zughaier, S. M., Khan, M. S., et al. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104319 · Cited by 1,147
“degradation problem [26] and learn from residuals instead of features [30].”
Mallat, S. (2012) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.21413 · Cited by 1,038
Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery
Yang, X., Wang, Y., Byrne, R., Schneider, G., & Yang, S. (2019) · Chemical Reviews · doi:10.1021/acs.chemrev.8b00728 · Cited by 1,022
Gland segmentation in colon histology images: The glas challenge contest
Sirinukunwattana, K., Pluim, J. P. W., Chen, H., Qi, X., Heng, P.-A., Guo, Y. B., Wang, L. Y., Matuszewski, B. J., Bruni, E., Sanchez, U., et al. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.008 · Cited by 979
Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life
Sateesh Babu, G., Zhao, P., & Li, X.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-32025-0_14 · Cited by 962
“In this paper we treat RUL estimation problem as multivariate time series regression and solve it by adapting one particular deep learning model, namely Convolutional Neural Network (CNN) adapted from deep learning model for image classification [1,12,13], which is the first attempt to leverage deep learning to estima…”
MethodBreast cancer histopathological image classification using Convolutional Neural Networks
Spanhol, F. A., Oliveira, L. S., Petitjean, C., & Heutte, L. (2016) · 2016 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727519 · Cited by 942
ShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2750389 · Cited by 933
“Yet, for now, the figure should give the reader a sufficient broad idea of the mapping to follow the development of the architecture in the next section.”
CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope
Bhatt, D., Patel, C., Talsania, H., Patel, J., Vaghela, R., Pandya, S., Modi, K., & Ghayvat, H. (2021) · Electronics · doi:10.3390/electronics10202470 · Cited by 890
Dynamic Key-Value Memory Networks for Knowledge Tracing
Zhang, J., Shi, X., King, I., & Yeung, D.-Y. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052580 · Cited by 862
“yesian models by a large margin regarding prediction accuracy. This is the rst attempt to integrate deep learning models [14, 26], which has achieved great success in other areas like computer vision [15, 13] and natural language processing [3, 17], into knowledge tracing. 2.2 Memory-Augmented Neural Networks Inspired…”
BackgroundInfluentialGraph neural networks for materials science and chemistry
Reiser, P., Neubert, M., Eberhard, A., Torresi, L., Zhou, C., Shao, C., Metni, H., van Hoesel, C., Schopmans, H., Sommer, T., et al. (2022) · Communications Materials · doi:10.1038/s43246-022-00315-6 · Cited by 860
Sequential Deep Learning for Human Action Recognition
Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., & Baskurt, A. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25446-8_4 · Cited by 782
“Schematically, LeCun’s ConvNet architecture [14,15] is a succession of layers alternating 2D-convolutions (to capture salient information) and sub-samplings (to reduce dimension), both with trainable weights.”
BackgroundMethodDeep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2019) · Medical Physics · doi:10.1002/mp.13264 · Cited by 740
“The most successful and popular DL architecture in imaging is the convolutional neural network (CNN).(30) Nearby pixels in an image are correlated with one another both in areas that exhibit local smoothness and areas consisting of structures (e.”
BackgroundA Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs
Wu, Z., Huang, Y., Wang, L., Wang, X., & Tan, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2545669 · Cited by 729
“Particularly, the deep convolutional neural networks (CNNs) [13] were used to tackle with various tasks, updating the record scores one after another.”
MethodFault diagnosis for rotating machinery using multiple sensors and convolutional neural networks
Xia, M., Li, T., Xu, L., Liu, L., & de Silva, C. W. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2728371 · Cited by 723
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
“Pooling is usually considered as an intrinsic layer of the representation [70].”
BackgroundAnden, J., & Mallat, S. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2326991 · Cited by 666
“The computational structure is similar to a convolutional deep neural network [12]–[19], but involves no learning.”
BackgroundA survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658
“CNNs have been widely studied for image recognition and processing [60] and are considered the state-ofthe-art for many computer vision tasks.”
BackgroundTime Series Classification Using Multi-Channels Deep Convolutional Neural Networks
Zheng, Y., Liu, Q., Chen, E., Ge, Y., & Zhao, J. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-08010-9_33 · Cited by 652
“Then the MC-DCNN model combines the learnt features of each channel and feeds them into a Multilayer Perceptron (MLP) to perform classification finally.”
BackgroundMethodInfluentialReliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization
Rahman, T., Khandakar, A., Kadir, M. A., Islam, K. R., Islam, K. F., Mazhar, R., Hamid, T., Islam, M. T., Kashem, S., Mahbub, Z. B., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3031384 · Cited by 651
“while ResNet learns residuals instead of features [56].”
BackgroundMax-pooling convolutional neural networks for vision-based hand gesture recognition
Nagi, J., Ducatelle, F., Di Caro, G. A., Ciresan, D., Meier, U., Giusti, A., Nagi, F., Schmidhuber, J., & Gambardella, L. M. (2011) · 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA) · doi:10.1109/icsipa.2011.6144164 · Cited by 644
“They were used for detection and recognition of objects including faces [21], hands, logos, text [25], with record accuracy and real-time performance.”
BackgroundMethodRahman, T., Chowdhury, M. E. H., Khandakar, A., Islam, K. R., Islam, K. F., Mahbub, Z. B., Kadir, M. A., & Kashem, S. (2020) · Applied Sciences · doi:10.3390/app10093233 · Cited by 638
“Figure 4: Structure of ResNet18 [48].”
MethodInfluentialDeep learning for plant identification using vein morphological patterns
Grinblat, G. L., Uzal, L. C., Larese, M. G., & Granitto, P. M. (2016) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2016.07.003 · Cited by 628
“For a more detailed description of this kind of models we refer the reader to [24] and the references therein.”
BackgroundWaveletKernelNet: An Interpretable Deep Neural Network for Industrial Intelligent Diagnosis
Li, T., Zhao, Z., Sun, C., Cheng, L., Chen, X., Yan, R., & Gao, R. X. (2022) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3048950 · Cited by 618
“[13] is used, where its first layer is replaced by CWConv.”
BackgroundMethodDeep-learning inversion: A next-generation seismic velocity model building method
Yang, F., & Ma, J. (2019) · GEOPHYSICS · doi:10.1190/geo2018-0249.1 · Cited by 612
Avendi, M. R., Kheradvar, A., & Jafarkhani, H. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.01.005 · Cited by 593
“Convolu-tional networks were chosen for localization and extracting an ROI because they are invariant to spacial translation and changes in scale and pixels’ intensity (LeCun et al., 2010; Sermanet et al., 2014).”
MethodInfluentialRobust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
You, Q., Luo, J., Jin, H., & Yang, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9179 · Cited by 548
Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.40 · Cited by 534
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
Sajjadi, M. S. M., Javanmardi, M., & Taşdizen, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04586 · Cited by 532
“Stochastic gradient descent, dropout [8], randomized data transformation and augmentation [9] and many other training techniques that are essential for fast convergence and effective generalization of the learning functions introduce some non-deterministic behavior to the learning system.”
BackgroundCruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N. N. C., Tomaszewski, J., González, F. A., & Madabhushi, A. (2017) · Scientific Reports · doi:10.1038/srep46450 · Cited by 532
“The ConvNet classifier 41,42 , was trained using a set of image tiles extracted from invasive (positive examples) and non-invasive (negative examples) tissue regions, annotated on whole slide digitized images by expert pathologists.”
MethodXing, F., & Yang, L. (2016) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2016.2515127 · Cited by 519
“automatically learn multilevel hierarchies of features that are invariant to irrelevant variations of samples while preserving relevant information [150], [151], and has achieved great success on image classification and scene labeling [152], [153].”
BackgroundBreast cancer detection using deep convolutional neural networks and support vector machines
Ragab, D. A., Sharkas, M., Marshall, S., & Ren, J. (2019) · PeerJ · doi:10.7717/peerj.6201 · Cited by 513
“A convolutional neural network (CNN) consists of multiple trainable stages stacked on top of each other, followed by a supervised classifier and sets of arrays named feature maps ( LeCun, Kavukcuoglu & Farabet, 2010 ).”
MethodProtein interface prediction using graph convolutional networks
Fout, A., Ben‐Hur, A., Anderson, C., Chitsaz, H., & Zhou, W. (2017) · Digital Collections of Colorado (Colorado State University) · doi:10.25675/3.021826 · Cited by 513
Understanding error propagation in deep learning neural network (DNN) accelerators and applications
Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W. (2017) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3126908.3126964 · Cited by 492
“A deep learning neural network (DNN) is a directed acyclic graph consisting of multiple computation layers [34].”
BackgroundLearning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification
Yang, J., Zhao, Y.-Q., & Chan, J. C.-W. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2698503 · Cited by 484
“i th feature map in the lth layer is [32]”
BackgroundDeep Convolutional Neural Networks for breast cancer screening
Chougrad, H., Zouaki, H., & Alheyane, O. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.01.011 · Cited by 470
Deep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Deep Neural Networks Motivated by Partial Differential Equations
Ruthotto, L., & Haber, E. (2020) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-019-00903-1 · Cited by 460
“alled weights, from training data. CNNs are widely used for solving large-scale learning tasks involving data that represent a discretization of a continuous function, e.g., voice, images, and videos [25, 26, 28]. By design, CNNs exploit the local relation between image information, lead to translation invariance, and…”
MethodArtificial intelligence as the next step towards precision pathology
Acs, B., Rantalainen, M., & Hartman, J. (2020) · Journal of Internal Medicine · doi:10.1111/joim.13030 · Cited by 450
“Several types of deep neural networks exist, whilst convolutional neural network (CNN) [79] is the most commonly used in pathology image analysis [80].”
BackgroundLi, Q., Dietrich, F., Bollt, E. M., & Kevrekidis, I. G. (2017) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4993854 · Cited by 444
Forecasting monthly gas field production based on the CNN-LSTM model
Zha, W., Liu, Y., Wan, Y., Luo, R., Li, D., Yang, S., & Xu, Y. (2022) · Energy · doi:10.1016/j.energy.2022.124889 · Cited by 440
Tang, Z., Chen, Z., Bao, Y., & Li, H. (2019) · Structural Control and Health Monitoring · doi:10.1002/stc.2296 · Cited by 436
Scene Classification via a Gradient Boosting Random Convolutional Network Framework
Zhang, F., Du, B., & Zhang, L. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2488681 · Cited by 424
“In an image, pixels are assembled into edgelets, edgelets are assembled into motifs, motifs are assembled into parts, parts are assembled into objects, and finally, objects are assembled into scenes [10].”
BackgroundMixed Pooling for Convolutional Neural Networks
Yu, D., Wang, H., Chen, P., & Wei, Z. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11740-9_34 · Cited by 409
“In general, CNNs are representatives of the multi-stage Hubel-Wiesel architecture [13], which extract local features at a high resolution and successively combine these into more complex features at lower resolutions.”
BackgroundRotation, Scaling and Deformation Invariant Scattering for Texture Discrimination
Sifre, L., & Mallat, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.163 · Cited by 408
A Review of Convolutional Neural Network Applied to Fruit Image Processing
Naranjo-Torres, J., Mora, M., Hernández-García, R., Barrientos, R. J., Fredes, C., & Valenzuela, A. (2020) · Applied Sciences · doi:10.3390/app10103443 · Cited by 406
“Each stage aim to represent features as sets of arrays called feature maps (see Figure 2) [12,19,42,43].”
BackgroundInfluentialA Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
Wiatowski, T., & Bolcskei, H. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2776228 · Cited by 399
Large language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395
Chest pathology detection using deep learning with non-medical training
Bar, Y., Diamant, I., Wolf, L., Lieberman, S., Konen, E., & Greenspan, H. (2015) · 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2015.7163871 · Cited by 392
“Deep learning refers to machine learning models such as Convolutional Neural Networks (CNNs) that represent mid-level and high-level abstractions obtained from raw data (e.g. images) [7].”
BackgroundA review of convolutional neural network architectures and their optimizations
Cong, S., & Zhou, Y. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10213-5 · Cited by 381
“Similarly, there are also some well-known studies which discussed different CNNs algorithms and their applications (LeCun et al. 2010; Srinivas et al. 2016; Alzubaidi et al. 2021).”
BackgroundClass-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
“Hence, ideal CIL algorithms should work for any type of backbones, e.g. , multi-layer perceptron (MLP) [30], convolutional neural network (CNN) [31], and Vision Transformer (ViT) [19].”
BackgroundText feature extraction based on deep learning: a review
Liang, H., Sun, X., Sun, Y., & Gao, Y. (2017) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-017-0993-1 · Cited by 373
“Now, in the field of image recognition, CNN has become a highly efficient method of identification [90].”
MethodA review on modern defect detection models using DCNNs – Deep convolutional neural networks
Tulbure, A.-A., Tulbure, A.-A., & Dulf, E.-H. (2022) · Journal of Advanced Research · doi:10.1016/j.jare.2021.03.015 · Cited by 372
“For a further summary please refer to [3,1], where applications such as: Face recognition, pedestrian detection, activity recognition, human pose estimation and others are described.”
BackgroundCross-Modal Retrieval With CNN Visual Features: A New Baseline
Wei, Y., Zhao, Y., Lu, C., Wei, S., Liu, L., Zhu, Z., & Yan, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2519449 · Cited by 370
“opment of convolutional neural network (CNN) [25], [27].”
BackgroundSemi-Supervised Sequence Modeling with Cross-View Training
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1217 · Cited by 366
“Early approaches to deep semi-supervised learning pre-train neural models on unlabeled data, which has been successful for applications in computer vision (Jarrett et al., 2009; LeCun et al., 2010) and NLP.”
BackgroundDeep Learning-Based Parkinson’s Disease Classification Using Vocal Feature Sets
Gunduz, H. (2019) · IEEE Access · doi:10.1109/access.2019.2936564 · Cited by 360
“Pooling type indicates whether the pooling process will be applied to each filter map (outputs of the filtering process) or globally to the feature maps that are the results of the multiple filters [31].”
MethodFingerprint Liveness Detection Using Convolutional Neural Networks
Nogueira, R. F., de Alencar Lotufo, R., & Campos Machado, R. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2520880 · Cited by 359
“filter weights can be learned, filters with random weights can perform well and they have the advantage that they do not need to be learned [28]–[30].”
BackgroundLong, E., Lin, H., Liu, Z., Wu, X., Wang, L., Jiang, J., An, Y., Lin, Z., Li, X., Chen, J., et al. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-016-0024 · Cited by 358
Waheed, A., Goyal, M., Gupta, D., Khanna, A., Hassanien, A. E., & Pandey, H. M. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105456 · Cited by 356
“Convolutional neural networks (CNNs) are one such class [10,11,12,13], but established CNN architectures give a high-performance rate with many parameters or high convergence time.”
BackgroundOne-step and Two-step Classification for Abusive Language Detection on Twitter
Park, J. H., & Fung, P. (2017) · Proceedings of the First Workshop on Abusive Language Online · doi:10.18653/v1/w17-3006 · Cited by 353
An Automatic Learning-Based Framework for Robust Nucleus Segmentation
Xing, F., Xie, Y., & Yang, L. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2481436 · Cited by 351
“Instead of relying on hand-crafted features which need to be specifically designed, CNN can automatically learn multi-level hierarchies of features which are invariant to irrelevant variations of samples while preserving relevant information [58].”
BackgroundBuilding a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
You, Q., Luo, J., Jin, H., & Yang, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9987 · Cited by 348
“…and Hin-ton 2012)), to solve other computer vision problems, such as scene parsing (Grangier, Bottou, and Collobert 2009), feature learning (LeCun, Kavukcuoglu, and Farabet 2010), visual recognition (Kavukcuoglu et al. 2010) and image clas-sification (Krizhevsky, Sutskever, and Hinton 2012).”
BackgroundSubject-Independent Brain–Computer Interfaces Based on Deep Convolutional Neural Networks
Kwon, O.-Y., Lee, M.-H., Guan, C., & Lee, S.-W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2946869 · Cited by 347
“The whole CNN framework is trained to reduce the loss function by stochastic gradient descent [40]–[43].”
BackgroundMethodQualitatively characterizing neural network optimization problems
Goodfellow, I. J., Vinyals, O., & Saxe, A. M. (2014) · arXiv · doi:10.48550/arxiv.1412.6544 · Cited by 341
“convolutional networks (LeCun et al., 2010), unsupervised models, recurrent models of sequences, and analytically tractable factored linear models.”
BackgroundMethodSeismic fault detection with convolutional neural network
Xiong, W., Ji, X., Ma, Y., Wang, Y., AlBinHassan, N. M., Ali, M. N., & Luo, Y. (2018) · GEOPHYSICS · doi:10.1190/geo2017-0666.1 · Cited by 333
Deep Learning Advances in Computer Vision with 3D Data: A Survey
Ioannidou, A., Chatzilari, E., Nikolopoulos, S., & Kompatsiaris, I. (2018) · ACM Computing Surveys · doi:10.1145/3042064 · Cited by 327
Lung Cancer Detection and Classification with 3D Convolutional Neural Network (3D-CNN)
Alakwaa, W., Nassef, M., & Badr, A. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080853 · Cited by 326
“This activation function has been widely used in a number of domains [24, 16] and is believed to be particularly helpful in classification tasks as the sparsity it induces in the outputs helps create separation between classes during learning [17, 3].”
BackgroundMethodFeature Learning for Image Classification Via Multiobjective Genetic Programming
Shao, L., Liu, L., & Li, X. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2293418 · Cited by 324
“In addition, basic arithmetic functions are chosen based on the desire to allow the MOGP process computing features from images more naturally.”
BackgroundAn Introduction to Neural Information Retrieval
Mitra, B., & Craswell, N. (2018) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000061 · Cited by 317
A survey of deep learning methods and software tools for image classification and object detection
Druzhkov, P. N., & Kustikova, V. D. (2016) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661816010065 · Cited by 311
DeepFool: a simple and accurate method to fool deep neural networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Pascal Frossard, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04599 · Cited by 100
“However, these same architectures have been shown to be unstable to small, well sought, perturbations of the images.”
BackgroundConvolutional Neural Network Architectures for Signals Supported on Graphs
Gama, F., Marques, A. G., Leus, G., & Ribeiro, A. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2887403 · Cited by 308
A deep feature based framework for breast masses classification
Jiao, Z., Gao, X., Wang, Y., & Li, J. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.02.060 · Cited by 304
The Impact of Imbalanced Training Data for Convolutional Neural Networks
Masko, D., & Hensman, P. (2015) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 302
Jagannath, A., Jagannath, J., & Kumar, P. S. P. V. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.109455 · Cited by 300
Macular OCT Classification Using a Multi-Scale Convolutional Neural Network Ensemble
Rasti, R., Rabbani, H., Mehridehnavi, A., & Hajizadeh, F. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2780115 · Cited by 298
Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge
Zafeiriou, S., Kollias, D., Nicolaou, M. A., Papaioannou, A., Zhao, G., & Kotsia, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.248 · Cited by 297
“Deep Convolutional Neural Networks (CNNs) [20] [21] include convolutional layers with feature maps composed of neurons with local receptive fields, shared weights and pooling layers, which are able to automatically extract non-linear features.”
BackgroundQin, B., & Li, D. (2020) · Sensors · doi:10.3390/s20185236 · Cited by 296
Robertson, S., Azizpour, H., Smith, K., & Hartman, J. (2018) · Translational Research · doi:10.1016/j.trsl.2017.10.010 · Cited by 295
Mukhamediev, R. I., Popova, Y., Kuchin, Y., Zaitseva, E., Kalimoldayev, A., Symagulov, A., Levashenko, V., Abdoldina, F., Gopejenko, V., Yakunin, K., et al. (2022) · Mathematics · doi:10.3390/math10152552 · Cited by 292
Metadata
- Authors
- LeCun, Y., Kavukcuoglu, K., & Farabet, C.
- Affiliations
- Courant Institute of Mathematical Sciences (US)
- ORCID
- LeCun, Yann
- Venue
- Proceedings of 2010 IEEE International Symposium on Circuits and Systems
- Publisher
- IEEE
- Published
- 2010-05
- Pages
- 253-256
- ISSN
- 0271-4302 (ISSN-L)
- OpenAlex
- W2015861736
- Semantic Scholar
- c43025c429b1fbf6f1379f61801a1b40834d62e7
- Cited by
- 2,213
- References
- 56
- Citations by year
- 2026: 30 · 2025: 112 · 2024: 168 · 2023: 201 · 2022: 223 · 2021: 242 · 2020: 255 · 2019: 240 · 2018: 206 · 2017: 183 · 2016: 142 · 2015: 94 · 2014: 59 · 2013: 21 · 2012: 21
- FWCI
- 15.63
- Type
- conference paper
- Language
- en
- Open access
- green
- hal.science
- Licence
- other-oa
- Locations
- IEEE International Symposium on Circuits and Systems proceedings (publishedVersion)
- HAL (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, unsupervised learning, representation learning, hierarchical feature representation, filter banks, feature pooling, nonlinear activation functions, visual object recognition, visual navigation, mobile robots, few labeled samples
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.