A novel connectivity of deep convolutional neural networks
Abstract
Residual network(ResNet) is an effective instance and a significant extension of deep convolutional neural network. ResNet utilizes skip-connection between input layers and output layers to solve the vanishing gradient problem. Due to the powerfulness of skip-connection, the gradient can flow directly through the identity function from later layers to the earlier layers. However, skip-connection makes ResNet overemphasizes the effect of the original input feature. In this paper, we further research the learning behavior of deep residual networks(ResNet) and proposed a novel connectivity pattern to replace skip-connection, which builds upon the success of interpreting the ResNet as an exponential ensemble of relatively shallow networks. The proposed method is implemented by making more input layers link with the output layer. The proposed method is called spare residual network(SRN). It generates model that is wider rather than deeper. The experimental results show that the proposed SRN can achieve impressive results on CIFAR-10, and outperform most of the existing architecture.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Jump connections in the inverted residual structure with step size 1 solve the network degradation problem caused by deepening the number of network layers.
“As shown in the inverted residual structure with a step size of 1 in Figure 6a, this structure uses jump connections to connect and sum the inputs and outputs through one residual edge, which can solve the network degradation problem caused by deepening the number of network layers [37].”
From Digital Image Copyright Protection and Management Approach—Based on Artificial Intelligence and Blockchain Technology (2025) · cites this paper as [37]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 7,048 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 5,037 · see citing works
Wide Residual NetworksPreprint
Zagoruyko, S., & Komodakis, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07146 · Cited by 1,850 · see citing works
Deeply-Supervised NetsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 1,661 · see citing works
Training Very Deep NetworksPreprint
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,453 · see citing works
A Multiobjective Sparse Feature Learning Model for Deep Neural Networks
Gong, M., Liu, J., Li, H., Cai, Q., & Su, L. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2469673 · Cited by 198 · see citing works
Learning Deep and Wide: A Spectral Method for Learning Deep Networks
Shao, L., Wu, D., & Li, X. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2308519 · Cited by 154 · see citing works
Deep and Shallow Architecture of Multilayer Neural Networks
Chang, C.-H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2387439 · Cited by 78 · see citing works
Residual Networks are Exponential Ensembles of Relatively Shallow Networks.Preprint
Veit, A., Wilber, M. J., & Belongie, S. (2016) · arXiv (Cornell University) · Cited by 76 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Xu, J., Zhang, J., & Wang, J. (2025) · Journal of Theoretical and Applied Electronic Commerce Research · doi:10.3390/jtaer20020076 · Cited by 9 · see citing works
“2025 , 20 , 76 contrast, in the inverted residual structure with a step size of 2 in Figure 6b, the feature graphs of the inputs and outputs are not of the same size, and then the jump connection operation cannot be used. of network layers [37].”
- Authors
- Shen, Z., & Liu, Y.
- Affiliations
- Chongqing University (CN)
- ORCID
- Shen, Zhixi
- Venue
- 2017 Chinese Automation Congress (CAC)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 7779-7783
- OpenAlex
- W2781802848
- Cited by
- 5
- References
- 19
- Citations by year
- 2026: 1 · 2025: 4
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Chinese Automation Congress (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- ResNet, skip connections, network connectivity, gradient vanishing, network architecture design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
