CoChat

Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge

Hu, Chuang, Bao, Wei, Wang, Dan, Liu, Fengming · 2019 · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications

The DOI matches its Crossref record; 22 of the 23 references it cites were found in a registry or an index.

Abstract

Recent advances in deep neural networks (DNNs) have substantially improved the accuracy and speed of a variety of intelligent applications. Nevertheless, one obstacle is that DNN inference imposes heavy computation burden to end devices, but offloading inference tasks to the cloud causes transmission of a large volume of data. Motivated by the fact that the data size of some intermediate DNN layers is significantly smaller than that of raw input data, we design the DNN surgery, which allows partitioned DNN processed at both the edge and cloud while limiting the data transmission. The challenge is twofold: (1) Network dynamics substantially influence the performance of DNN partition, and (2) State-of-the-art DNNs are characterized by a directed acyclic graph (DAG) rather than a chain so that partition is greatly complicated. In order to solve the issues, we design a Dynamic Adaptive DNN Surgery (DADS) scheme, which optimally partitions the DNN under different network condition. Under the lightly loaded condition, DNN Surgery Light (DSL) is developed, which minimizes the overall delay to process one frame. The minimization problem is equivalent to a min-cut problem so that a globally optimal solution is derived. In the heavily loaded condition, DNN Surgery Heavy (DSH) is developed, with the objective to maximize throughput. However, the problem is NP-hard so that DSH resorts an approximation method to achieve an approximation ratio of 3. Real-world prototype based on self-driving car video dataset is implemented, showing that compared with executing entire the DNN on the edge and cloud, DADS can improve latency up to 6.45 and 8.08 times respectively, and improve throughput up to 8.31 and 14.01 times respectively.

Overview

Integrity signals

Registry

Crossref · 10.1109/infocom.2019.8737614

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Cited by

See all 493

Metadata

Authors
Hu, C., Bao, W., Wang, D., & Liu, F.
Affiliations
  • Hong Kong Polytechnic University (HK)
  • The University of Sydney (AU)
Venue
IEEE INFOCOM 2019 - IEEE Conference on Computer Communications
Publisher
IEEE
Published
2019-04
Pages
1423-1431
OpenAlex
W2920031528
Cited by
416
References
23
Citations by year
2026: 29 · 2025: 61 · 2024: 65 · 2023: 92 · 2022: 62 · 2021: 72 · 2020: 26 · 2019: 9
FWCI
36.52
Type
conference paper
Language
en
Open access
closed
Locations
  • IEEE Conference on Computer Communications (INFOCOM) (publishedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Hardware and Architecture › Cloud Computing and Remote Desktop Technologies
Keywords
DNN partitioning, cloud-edge collaboration, inference acceleration, dynamic network conditions, directed acyclic graphs, minimum cut, approximation algorithms, throughput maximization, latency reduction, data transmission reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.