The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
AI Benchmark: All About Deep Learning on Smartphones in 2019
Open accessAbstract
The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs. The current 4th generation of mobile NPUs is already approaching the results of CUDA-compatible Nvidia graphics cards presented not long ago, which together with the increased capabilities of mobile deep learning frameworks makes it possible to run complex and deep AI models on mobile devices. In this paper, we evaluate the performance and compare the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference. We also discuss the recent changes in the Android ML pipeline and provide an overview of the deployment of deep learning models on mobile devices. All numerical results provided in this paper can be found and are regularly updated on the official project website.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“performance of desktop CPUs and GPUs. Section6analyzes the results. Finally, Section7concludes the paper. 2.Hardware Acceleration Though many deep learning algorithms were presented back in the 1990s [40,41,22], the lack of appropriate (and affordable) hardware to train such models prevented them from being extensivel…”
BackgroundRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870 · cited 2×
“Model: VGG-19 (VDSR) [35]”
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 1×
“Additionally, many tasks deal with time series processing, e.g. , human activity recognition [39, 26], gesture recognition [60], sleep monitoring [69], adaptive power management [50, 47], music tracking [86] and classi-fication [73].”
BackgroundDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · cited 1×
“…computer vision problems like image classi-fication [38, 82, 23], image enhancement [27, 28, 32, 30], image super-resolution [17, 42, 83], bokeh simulation [85], object tracking [87, 25], optical character recognition [56], face detection and recognition [44, 70], augmented reality [3, 16], etc.”
BackgroundICNet for Real-Time Semantic Segmentation on High-Resolution Images
Zhao, H., Qi, X., Shen, X., Shi, J., & Jia, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_25 · Cited by 1,709
A convolutional neural network cascade for face detection
Li, H., Lin, Z., Shen, X., Brandt, J., & Hua, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299170 · Cited by 1,390
(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229
Large-scale deep unsupervised learning using graphics processors
Raina, R., Madhavan, A., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553486 · Cited by 742 · cited 1×
“…presented back in the 1990s [40, 41, 22], the lack of appropriate (and affordable) hardware to train such models prevented them from being extensively used by the research community till 2009, when it became possible to effectively accelerate their training with general-purpose consumer GPUs [65].”
BackgroundAn Adaptive Recurrent Neural Network for Remaining Useful Life Prediction of Lithium-ion Batteries
Liu, J., Saxena, A., Goebel, K., Saha, B., & Wang, W. (2010) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2010.v2i1.1896 · Cited by 274
Research works printed but not linked: 46
- 10.1109/ICASSP.2014.6854949 · DOI
- 10.1145/2766462.2767830 · DOI · Paper page
- iulian, 2017, A deep reinforcement learning chatbot
- richard, 0, Recursive deep models for semantic compositionality over a sentiment treebank, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 1631
- 10.1145/332040.332044 · DOI · Paper page
- simonyan, 2014, Very Deep Convolutional Networks for Large-scale Image Recognition
- song, 0, 7.1 an 11.5 tops/w 1024-mac butterfly structure dual-core sparsity-aware neural processing unit in 8nm flagship mobile soc, 2019 IEEE International Solid-State Circuits Conference-(ISSCC), 130
- 10.1109/ICTAI.2017.00140 · DOI
- ignatov, 0, Pirm challenge on perceptual image enhancement on smartphones: Report, European Conf on Comp Vision Workshop
- ignatov, 0, Ai benchmark: Running deep neural networks on android smartphones, Proceedings of the European Conference on Computer Vision (ECCV), 0
- ignatov, 0, Ntire 2019 challenge on image enhancement: Methods and results, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 0
- krishnamoorthi, 2018, Quantizing deep convolutional networks for efficient inference A whitepaper
- 10.1109/ICDAR.2005.65 · doi:10.1109/icdar.2005.65
- 10.1109/ICCV.2015.203 · DOI · Paper page
- ignatov, 2017, Wespe Weakly supervised photo enhancer for digital cameras
- 10.1109/ICCV.2017.355 · DOI · Paper page
- 10.1109/CVPRW.2018.00112 · DOI · Paper page
- sathyanarayana, 2016, Sleep quality prediction from wearable data using deep learning, JMIR mhealth and uhealth, 4
- abadi, 2016, Tensorflow: a system for large-scale machine learning, OSDI, 16, 265
- 10.1109/ICDSC.2007.4357512 · DOI
- hu, 2014, Convolutional neural network architectures for matching natural language sentences, Advances in neural information processing systems, 2042
- 10.1016/j.asoc.2017.09.027 · DOI · Paper page
- mannor, 2006, Machine learning for adaptive power management, Autonomic computing, 10, 299
- matsunaga, 2005, Universal design activities for mobile phone: Raku raku phone, FUJITSU Sci Tech J, 41, 78
- 10.1109/CVPR.2017.660 · DOI · Paper page
- van den oord, 2016, Pixel recurrent neural networks
- niezen, 0, Gesture recognition as ubiquitous input for mobile phones, International Workshop on Devices that Alter Perception (DAP 2008) in conjunction with Ubicomp, 17
- netzer, 0, Reading digits in natural images with unsupervised feature learning, NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011, 5
- jawad, 0, Max-pooling convolutional neural networks for vision-based hand gesture recognition, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), 342
- 10.1109/ICCV.2019.00141 · DOI · Paper page
- 10.1145/1851322.1851328 · DOI
- detone, 2016, Deep image homography estimation
- szegedy, 2017, Inception-v4, inception-resnet and the impact of residual connections on learning, AAAI, 4, 12
- 10.1109/IOT.2010.5678457 · doi:10.1109/iot.2010.5678457
- emmett, 2013, Voice activated virtual assistant, US Patent App
- 10.1109/CVPRW.2018.00130 · DOI · Paper page
- sutskever, 2014, Sequence to sequence learning with neural networks, Advances in neural information processing systems, 3104
- 10.1109/ICPR.2002.1047827 · doi:10.1109/icpr.2002.1047827
- alhaija, 0, Augmented reality meets deep learning for car instance segmentation in urban scenes, British Machine Vision Conference, 1, 2
- neal, 2018, Synthetic depth-of-field with a single-camera mobile phone, ACM Transactions on Graphics (TOG), 37, 64
- chetlur, 2014, cuDNN Efficient Primitives for Deep Learning
- 10.1145/1145287.1145312 · doi:10.1145/1145287.1145312
- louizos, 2018, Relaxed quantization for discretized neural networks
- yonghui, 2016, Google's neural machine translation system Bridging the gap between human and machine translation
- 10.1016/j.pmcj.2009.07.007 · DOI · Paper page
- lee, 2019, On-device neural net inference with mobile gpus
Cited by
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
Meta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
“tions involving deployment of learning algorithms on mobile devices such as smartphones [6], for example to achieve personalisation.”
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Reddi, V. J., Cheng, C., Kanter, D., Mattson, P., Schmuelling, G., Wu, C.-J., Anderson, B., Breughe, M., Charlebois, M., Chou, W., et al. (2020) · 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca45697.2020.00045 · Cited by 505
Self-citationEfficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436
“Benchmarking quantized DL models to accelerate on smartphones is presented in [14].”
Method
Metadata
- Authors
- Ignatov, A., Timofte, R., Kulik, A., Yang, S., Wang, K., Baum, F., Wu, M., Xu, L., & Van Gool, L.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 3617-3635
- OpenAlex
- W2981147706
- Cited by
- 247
- References
- 27
- FWCI
- 26.18
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.