The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Abstract
The computation for today's intelligent personal assistants such as Apple Siri, Google Now, and Microsoft Cortana, is performed in the cloud. This cloud-only approach requires significant amounts of data to be sent to the cloud over the wireless network and puts significant computational pressure on the datacenter. However, as the computational resources in mobile devices become more powerful and energy efficient, questions arise as to whether this cloud-only processing is desirable moving forward, and what are the implications of pushing some or all of this compute to the mobile devices on the edge. In this paper, we examine the status quo approach of cloud-only processing and investigate computation partitioning strategies that effectively leverage both the cycles in the cloud and on the mobile device to achieve low latency, low energy consumption, and high datacenter throughput for this class of intelligent applications. Our study uses 8 intelligent applications spanning computer vision, speech, and natural language domains, all employing state-of-the-art Deep Neural Networks (DNNs) as the core machine learning technique. We find that given the characteristics of DNN algorithms, a fine-grained, layer-level computation partitioning strategy based on the data and computation variations of each layer within a DNN has significant latency and energy advantages over the status quo approach. Using this insight, we design Neurosurgeon, a lightweight scheduler to automatically partition DNN computation between mobile devices and datacenters at the granularity of neural network layers. Neurosurgeon does not require per-application profiling. It adapts to various DNN architectures, hardware platforms, wireless networks, and server load levels, intelligently partitioning computation for best latency or best mobile energy. We evaluate Neurosurgeon on a state-of-the-art mobile development platform and show that it improves end-to-end latency by 3.1X on average and up to 40.7X, reduces mobile energy consumption by 59.5% on average and up to 94.7%, and improves datacenter throughput by 1.5X on average and up to 6.7X.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
MAUI: making smartphones last longer with code offload
Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., & Bahl, P. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814441 · Cited by 2,281
(2011) · doi:10.1145/1966445.1966473 · Cited by 1,870
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
A close examination of performance and power characteristics of 4G LTE networks
Huang, J., Qian, F., Gerber, A., Mao, Z. M., Sen, S., & Spatscheck, O. (2012) · Proceedings of the 10th international conference on Mobile systems, applications, and services · doi:10.1145/2307636.2307658 · Cited by 995
Bubble-Up: increasing utilization in modern warehouse scale computers via sensible co-locations
Mars, J., Tang, L., Hundt, R., Skadron, K., & Soffa, M. L. (2011) · Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/2155620.2155650 · Cited by 607
Self-citationAccelerating Deep Convolutional Neural Networks Using Specialized Hardware
Ovtcharov, K., Ruwase, O., Kim, J.-Y., Fowers, J., Strauß, K., & Chung, E. S. (2015) · Cited by 412
Odessa: enabling interactive perception applications on mobile devices
Ra, M.-R., Sheth, A., Mummert, L., Pillai, P., Wetherall, D., & Govindan, R. (2011) · Proceedings of the 9th international conference on Mobile systems, applications, and services · doi:10.1145/1999995.2000000 · Cited by 361
COMET: code offload by migrating execution transparently
Gordon, M. S., Jamshidi, D. A., Mahlke, S. A., Mao, Z. M., & Xu, C. (2012) · doi:10.5555/2387880.2387890 · Cited by 357
Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers
Yang, H., Breslow, A., Mars, J., & Tang, L. (2013) · Proceedings of the 40th Annual International Symposium on Computer Architecture · doi:10.1145/2485922.2485974 · Cited by 345
Self-citationPuDianNao: A Polyvalent Machine Learning Accelerator
Liu, D., Chen, T., Liu, S., Zhou, J., Zhou, S., Teman, O., Feng, X., Zhou, X., & Chen, Y. (2015) · Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/2694344.2694358 · Cited by 240
AUGEM: automatically generate high performance dense linear algebra kernels on x86 CPUs
Wang, Q., Zhang, X., Zhang, Y., & Yi, Q. (2013) · Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis · doi:10.1145/2503210.2503219 · Cited by 199
Hauswald, J., Laurenzano, M. A., Zhang, Y., Li, C., Rovinski, A., Khurana, A., Dreslinski, R. G., Mudge, T., Petrucci, V., Tang, L., et al. (2015) · Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/2694344.2694347 · Cited by 193
Self-citationWhare-map: heterogeneity in "homogeneous" warehouse-scale computers
Mars, J., & Tang, L. (2013) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2485922.2485975 · Cited by 169
Self-citationZhang, Y., Laurenzano, M. A., Mars, J., & Tang, L. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.53 · Cited by 163
Self-citationDjiNN and Tonic: DNN as a service and its implications for future warehouse scale computers
Hauswald, J., Kang, Y., Laurenzano, M. A., Chen, Q., Li, C., Mudge, T., Dreslinski, R. G., Mars, J., & Tang, L. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2749472 · Cited by 161
Self-citationHalpern, M., Zhu, Y., & Reddi, V. J. (2016) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2016.7446054 · Cited by 154
Towards energy proportionality for large-scale latency-critical workloads
Lo, D., Cheng, L., Govindaraju, R., Barroso, L. A., & Kozyrakis, C. (2014) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2014.6853237 · Cited by 153
Adrenaline: Pinpointing and reining in tail queries with quick voltage boosting
Hsu, C.-H., Zhang, Y., Laurenzano, M. A., Meisner, D., Wenisch, T. F., Mars, J., Tang, L., & Dreslinski, R. (2015) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2015.7056039 · Cited by 145
Self-citationBigHouse: A simulation infrastructure for data center systems
Meisner, D., Wu, J., & Wenisch, T. F. (2012) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2012.6189204 · Cited by 143
Accurate and compact large vocabulary speech recognition on mobile devices
Lei, X., Senior, A., Gruenstein, A., & Sorensen, J. S. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-189 · Cited by 120
Mobile Network Performance from User Devices: A Longitudinal, Multidimensional Analysis
Nikravesh, A., Choffnes, D., Katz-Bassett, E., Mao, Z. M., & Welsh, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-04918-2_2 · Cited by 106
Octopus-Man: QoS-driven task management for heterogeneous multicores in warehouse-scale computers
Petrucci, V., Laurenzano, M. A., Doherty, J., Zhang, Y., Mossé, D., Mars, J., & Tang, L. (2015) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2015.7056037 · Cited by 96
Self-citationZhang, Y., Meisner, D., Mars, J., & Tang, L. (2016) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2016.47 · Cited by 74
Self-citationA hybrid approach to offloading mobile image classification
Hauswald, J., Manville, T., Zheng, Q., Dreslinski, R., Chakrabarti, C., & Mudge, T. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6855235 · Cited by 67
Self-citationProtean Code: Achieving Near-Free Online Code Transformations for Warehouse Scale Computers
Laurenzano, M. A., Zhang, Y., Tang, L., & Mars, J. (2014) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2014.21 · Cited by 57
Self-citationSkach, M., Arora, M., Hsu, C.-H., Li, Q., Tullsen, D. M., Tang, L., & Mars, J. (2015) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2749469.2749474 · Cited by 50
Self-citationContinuous shape shifting: Enabling loop co-optimization via near-free dynamic code rewriting
Jain, A., Laurenzano, M. A., Tang, L., & Mars, J. (2016) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783726 · Cited by 17
Self-citation(2015) · Cited by 14
PowerChop: identifying and managing non-critical units in hybrid processor architectures
Laurenzano, M. A., Zhang, Y., Jiang, C., Tang, L., & Mars, J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001152 · Cited by 11
Self-citation
Cited by
Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing
Li, E., Zeng, L., Zhou, Z., & Chen, X. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946140 · Cited by 929
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services
Xu, M., Du, H., Niyato, D., Kang, J., Xiong, Z., Mao, S., Han, Z., Jamalipour, A., Kim, D. I., Shen, X., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3353265 · Cited by 449
A Survey of Recent Advances in Edge-Computing-Powered Artificial Intelligence of Things
Chang, Z., Liu, S., Xiong, X., Cai, Z., & Tu, G. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3088875 · Cited by 418
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436
Metadata
- Authors
- Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L.
- Venue
- ACM SIGARCH Computer Architecture News
- Published
- 2017-04-04
- Volume
- 45
- Issue
- 1
- Pages
- 615-629
- ISSN
- 0163-5964 (ISSN-L)
- OpenAlex
- W4236099117
- Cited by
- 806
- References
- 34
- FWCI
- 10.82
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.