The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks
Open accessAbstract
Benefitting from large-scale training datasets and the complex training network, Convolutional Neural Networks (CNNs) are widely applied in various fields with high accuracy. However, the training process of CNNs is very time-consuming, where large amounts of training samples and iterative operations are required to obtain high-quality weight parameters. In this paper, we focus on the time-consuming training process of large-scale CNNs and propose a Bi-layered Parallel Training (BPT-CNN) architecture in distributed computing environments. BPT-CNN consists of two main components: (a) an outer-layer parallel training for multiple CNN subnetworks on separate data subsets, and (b) an inner-layer parallel training for each subnetwork. In the outer-layer parallelism, we address critical issues of distributed and parallel computing, including data communication, synchronization, and workload balance. A heterogeneous-aware Incremental Data Partitioning and Allocation (IDPA) strategy is proposed, where large-scale training datasets are partitioned and allocated to the computing nodes in batches according to their computing power. To minimize the synchronization waiting during the global weight update process, an Asynchronous Global Weight Update (AGWU) strategy is proposed. In the inner-layer parallelism, we further accelerate the training process for each CNN subnetwork on each computer, where computation steps of convolutional layer and the local weight training are parallelized based on task-parallelism. We introduce task decomposition and scheduling strategies with the objectives of thread-level load balancing and minimum waiting time for critical paths. Extensive experimental results indicate that the proposed BPT-CNN effectively improves the training performance of CNNs while maintaining the accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211 · cited 1×
“However, the training process of CNN is very time-consuming, in which large amounts of training samples and iterative operations are required to obtain high-quality weight parameters [6], [7].”
BackgroundA dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343
Cited by
Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing
Chen, J., Li, K., Deng, Q., Li, K., & Yu, P. S. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2909473 · Cited by 253
Self-citationChen, J., Li, K., & Yu, P. S. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3105682 · Cited by 116
Self-citationExploring Adversarial Attacks in Federated Learning for Medical Imaging
Darzi, E., Dubost, F., Sijtsema, N. M., & van Ooijen, P. M. A. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3423457 · Cited by 45
“This is particularly important in considering the feasibility of attack methods in large-scale training of federated networks, and performing adversarial attacks on them efficiently [27], [28].”
Cheng, H., Shi, Y., Wu, L., Guo, Y., & Xiong, N. (2021) · Information Sciences · doi:10.1016/j.ins.2020.12.041 · Cited by 29
Metadata
- Authors
- Chen, J., Li, K., Bilal, K., Zhou, X., Li, K., & Yu, P. S.
- Venue
- IEEE Transactions on Parallel and Distributed Systems
- Published
- 2019-05-01
- Volume
- 30
- Issue
- 5
- Pages
- 965-976
- ISSN
- 1045-9219 (ISSN-L)
- 1558-2183
- 2161-9883
- OpenAlex
- W2895891814
- Cited by
- 185
- References
- 5
- FWCI
- 26.61
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks. https://app.cochat.ai/papers/10.1109/tpds.2018.2877359
- The paper
- Chen, J., Li, K., Bilal, K., Zhou, X., Li, K., & Yu, P. S. (2019). A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks. IEEE Transactions on Parallel and Distributed Systems. https://doi.org/10.1109/tpds.2018.2877359
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.