HierTrain: Fast Hierarchical Edge AI Learning With Hybrid Parallelism in Mobile-Edge-Cloud Computing
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Nowadays, deep neural networks (DNNs) are the core enablers for many emerging edge AI applications. Conventional approaches for training DNNs are generally implemented at central servers or cloud centers for centralized learning, which is typically time-consuming and resource-demanding due to the transmission of a large number of data samples from the edge device to the remote cloud. To overcome these disadvantages, we consider accelerating the learning process of DNNs on the Mobile-Edge-Cloud Computing (MECC) paradigm. In this paper, we propose HierTrain, a hierarchical edge AI learning framework, which efficiently deploys the DNN training task over the hierarchical MECC architecture. We develop a novel hybrid parallelism method, which is the key to HierTrain, to adaptively assign the DNN model layers and the data samples across the three levels of the edge device, edge server and cloud center. We then formulate the problem of scheduling the DNN training tasks at both layer-granularity and sample-granularity. Solving this optimization problem enables us to achieve the minimum training time. We further implement a hardware prototype consisting of an edge device, an edge server and a cloud server, and conduct extensive experiments on it. Experimental results demonstrate that HierTrain can achieve up to $6.9\times $ speedups compared to the cloud-based hierarchical training approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · cited 1×
“Therefore, when training the DNN with the back-propagation rule in the stochastic gradient descent (SGD) algorithm [18], the workers need to communicate to exchange the intermediate results.”
BackgroundThe case for vm-based cloudlets in mobile computing
Satyanarayanan, M., Bahl, P., Caceres, R., & Davies, N. (2009) · IEEE Pervasive Computing · doi:10.1109/mprv.2009.82 · Cited by 3,660
Deep Neural Networks for YouTube Recommendations
Covington, P., Adams, J., & Sargin, E. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959190 · Cited by 3,539
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · cited 3×
“These data samples are often generated by mobile and IoT devices at the network edge that have limited communication and computation capabilities, such as mobile phones, smart watches, smart robots, etc.”
BackgroundInfluentialSelf-citation
Cited by
Singh, R., & Gill, S. S. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.02.004 · Cited by 572
Distributed Artificial Intelligence Empowered by End-Edge-Cloud Computing: A Survey
Duan, S., Wang, D., Ren, J., Lyu, F., Zhang, Y., Wu, H., & Shen, X. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3218527 · Cited by 408
Yu, S., Chen, X., Zhou, Z., Gong, X., & Wu, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3026589 · Cited by 353
Self-citationArtificial Intelligence in the IoT Era: A Review of Edge AI Hardware and Software
Sipola, T., Alatalo, J., Kokkonen, T., & Rantonen, M. (2022) · 2022 31st Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct54823.2022.9770931 · Cited by 100
Liu, B., Luo, Z., Chen, H., & Li, C. (2022) · IEEE Access · doi:10.1109/access.2022.3176106 · Cited by 36
“Liu et al. [122] proposed a framework, HierTrain, and a new hybrid parallel approach.”
Background
Metadata
- Authors
- Liu, D., Chen, X., Zhou, Z., & Ling, Q.
- Venue
- IEEE Open Journal of the Communications Society
- Published
- 2020
- Volume
- 1
- Pages
- 634-645
- ISSN
- 2644-125X (ISSN-L)
- OpenAlex
- W3024862772
- Cited by
- 73
- References
- 20
- FWCI
- 4.71
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.