pCAMP: Performance Comparison of Machine Learning Packages on the Edges
Open accessAbstract
Machine learning has changed the computing paradigm. Products today are built with machine intelligence as a central attribute, and consumers are beginning to expect near-human interaction with the appliances they use. However, much of the deep learning revolution has been limited to the cloud. Recently, several machine learning packages based on edge devices have been announced which aim to offload the computing to the edges. However, little research has been done to evaluate these packages on the edges, making it difficult for end users to select an appropriate pair of software and hardware. In this paper, we make a performance comparison of several state-of-the-art machine learning packages on the edges, including TensorFlow, Caffe2, MXNet, PyTorch, and TensorFlow Lite. We focus on evaluating the latency, memory footprint, and energy of these tools with two popular types of neural networks on different edge devices. This evaluation not only provides a reference to select appropriate combinations of hardware and software packages for end users but also points out possible future directions to optimize packages for developers.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
No single DL library or hardware platform can entirely outperform others, and loading the DL model may take more time than executing it.
“Experimental results show that no single DL library or hardware platform can entirely outperform others, and loading the DL model may take more time than that of executing it.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [186]
Cites
Still being checked.
Cited by
Sorted by citations
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · see citing works
“, TensorFlow Lite, Caffe2, CoreML, and MXNet); however, empirical measurements [115] show that there is no single winner that can outperform other frameworks in all metrics.”
BackgroundConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“In [186], for the first time, the performance of DL libraries is evaluated by executing DL inference on”
MethodAll one needs to know about fog computing and related edge computing paradigms: A complete survey
Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J., & Jue, J. P. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.02.009 · Cited by 1,368 · see citing works
A Survey on Edge Computing Systems and Tools
Liu, F., Tang, G., Li, Y., Cai, Z., Zhang, X., & Zhou, T. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2920341 · Cited by 340 · see citing works
Self-citationA Survey on optimized implementation of deep learning models on the NVIDIA Jetson platform
Mittal, S. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.01.011 · Cited by 295 · see citing works
Bandwidth-Efficient Live Video Analytics for Drones Via Edge Computing
Wang, J., Feng, Z., Chen, Z., George, S., Bala, M., Pillai, P., Yang, S.-W., & Satyanarayanan, M. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00019 · Cited by 204 · see citing works
A First Look at Deep Learning Apps on Smartphones
Xu, M., Liu, J., Liu, Y., Lin, F. X., Liu, Y., & Liu, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313591 · Cited by 185 · see citing works
Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review
Rosendo, D., Costan, A., Valduriez, P., & Antoniu, G. (2022) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2022.04.004 · Cited by 144 · see citing works
OpenEI: An Open Framework for Edge Intelligence
Zhang, X., Wang, Y., Lu, S., Liu, L., xu, L., & Shi, W. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00182 · Cited by 139 · see citing works
“made a comprehensive performance comparison of several state-of-the-art deep learning frameworks on the edges and evaluated the latency, memory footprint, and energy of these frameworks with two popular deep learning models on different edge devices [48].”
BackgroundSelf-citationCharacterizing the deployment of deep neural networks on commercial edge devices
Hadidi, R., Cao, J., Xie, Y., Asgari, B., Krishna, T., & Kim, H. (2019) · 2019 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc47752.2019.9041955 · Cited by 118 · see citing works
Green AI: do deep learning frameworks have different costs?
Georgiou, S., Kechagia, M., Sharma, T., Sarro, F., & Zou, Y. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510221 · Cited by 102 · see citing works
Edge Intelligence: Concepts, Architectures, Applications, and Future Directions
Mendez, J., Bierzynski, K., Cuéllar, M. P., & Morales, D. P. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3486674 · Cited by 108 · see citing works
A Survey on Edge Performance Benchmarking
Varghese, B., Wang, N., Bermbach, D., Hong, C.-H., de Lara, E., Shi, W., & Stewart, C. (2021) · ACM Computing Surveys · doi:10.1145/3444692 · Cited by 100 · see citing works
Self-citationAlmeida, M., Laskaridis, S., Ilias Leontiadis, A., Venieris, S. I., & Lane, N. D. (2019) · doi:10.1145/3325413.3329793 · Cited by 74 · see citing works
Learning to See: Convolutional Neural Networks for the Analysis of Social Science Data
Torres, M., & Cantú, F. (2021) · Political Analysis · doi:10.1017/pan.2021.9 · Cited by 69 · see citing works
AI on the Edge: Characterizing AI-based IoT Applications Using Specialized Edge Architectures
Liang, Q., Shenoy, P. J., & Irwin, D. (2020) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc50251.2020.00023 · Cited by 75 · see citing works
Demystifying TensorRT: Characterizing Neural Network Inference Engine on Nvidia Edge Devices
Shafi, O., Rai, C., Sen, R., & Ananthanarayanan, G. (2021) · 2021 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc53511.2021.00030 · Cited by 51 · see citing works
Energy-Efficient Machine Learning on the Edges
Kumar, M., Zhang, X.-Z., Liu, L., Wang, Y.-F., & Shi, W.-S. (2020) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw50202.2020.00153 · Cited by 34 · see citing works
Self-citationHussain, N., Farooque, A. A., Schumann, A. W., Abbas, F., Acharya, B., McKenzie‐Gopsill, A., Barrett, R., Afzaal, H., Zaman, Q. U., & Cheema, M. J. M. (2021) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2021.106040 · Cited by 37 · see citing works
Efficient Environmental Context Prediction for Lower Limb Prostheses
Zhong, B., da Silva, R. L., Tran, M., Huang, H., & Lobaton, E. J. (2021) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3084036 · Cited by 31 · see citing works
Collaborative cloud-edge computation for personalized driving behavior modeling
Zhang, X., Qiao, M., Liu, L., Xu, Y., & Shi, W. (2019) · Proceedings of the 4th ACM/IEEE Symposium on Edge Computing · doi:10.1145/3318216.3363310 · Cited by 30 · see citing works
Self-citationHao, J., Subedi, P., Ramaswamy, L. M., & Kim, I. K. (2022) · ACM Transactions on Internet Technology · doi:10.1145/3546192 · Cited by 29 · see citing works
Preliminary Study of Trusted Execution Environments on Heterogeneous Edge Platforms
Ning, Z., Liao, J., Zhang, F., & Shi, W. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00057 · Cited by 28 · see citing works
Self-citationEdgeFaaSBench: Benchmarking Edge Devices Using Serverless Computing
Rajput, K. R., Kulkarni, C. D., Cho, B., Wang, W., & Kim, I. K. (2022) · IEEE International Conference on Edge Computing and Communications (EDGE) · doi:10.1109/edge55608.2022.00024 · Cited by 29 · see citing works
Subedi, P., Hao, J., Kim, I. K., & Ramaswamy, L. M. (2021) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud53861.2021.00016 · Cited by 25 · see citing works
Enhancing Energy-Awareness in Deep Learning through Fine-Grained Energy Measurement
Rajput, S., Widmayer, T., Shang, Z., Kechagia, M., Sarro, F., & Sharma, T. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3680470 · Cited by 28 · see citing works
Reviewing inference performance of state-of-the-art deep learning frameworks
Ülker, B., Stuijk, S., Corporaal, H., & Wijnhoven, R. G. J. (2020) · International Workshop on Software and Compilers for Embedded Systems (SCOPES) · doi:10.1145/3378678.3391882 · Cited by 22 · see citing works
An Erasure-Coded Storage System for Edge Computing
Liang, L., He, H., Zhao, J., Liu, C., Luo, Q., & Chu, X. (2020) · IEEE Access · doi:10.1109/access.2020.2995973 · Cited by 20 · see citing works
Hu, Z., Tarakji, A. B., Raheja, V., Phillips, C., Wang, T., & Mohomed, I. (2019) · doi:10.1145/3325413.3329787 · Cited by 19 · see citing works
Exploiting LoRa, edge, and fog computing for traffic monitoring in smart cities
Gia, T. N., Queralta, J. P., & Westerlund, T. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-818880-4.00017-x · Cited by 18 · see citing works
Rachmanto, R. D., Sukma, Z., Nabhaan, A. N. L., Setyanto, A., Jiang, T., & Kim, I. K. (2024) · IEEE International Conference on Edge Computing and Communications (EDGE) · doi:10.1109/edge62653.2024.00023 · Cited by 16 · see citing works
Filling Two Needs With One Deed: Combo Pricing Plans for Computing-Intensive Multimedia Applications
Zang, S., Bao, W., Yeoh, P. L., Vucetic, B. S., & Li, Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2916451 · Cited by 16 · see citing works
Artificial intelligence in edge devices
Singh, A., Saini, K., Nagar, V., Aseri, V., Sankhla, M. S., Pandit, P. P., & Chopade, R. L. (2022) · Advances in computers · doi:10.1016/bs.adcom.2022.02.013 · Cited by 16 · see citing works
AI on the Edge: Rethinking AI-based IoT Applications Using Specialized Edge ArchitecturesPreprint
Liang, Q., Shenoy, P. J., & Irwin, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12488 · Cited by 15 · see citing works
Machine Learning at the Edge: Efficient Utilization of Limited CPU/GPU Resources by Multiplexing
Dhakal, A., Kulkarni, S. G., & Ramakrishnan, K. K. (2020) · IEEE International Conference on Network Protocols (ICNP) · doi:10.1109/icnp49622.2020.9259361 · Cited by 14 · see citing works
GPU-based embedded edge server configuration and offloading for a neural network service
Kim, J., Ullah, S., & Kim, D. (2021) · The Journal of Supercomputing · doi:10.1007/s11227-021-03623-9 · Cited by 13 · see citing works
A Comparative Analysis for Optimizing Machine Learning Model Deployment in IoT Devices
Islam, M. M., Sayeedi, M. F. A., Deepti, J. F., Osmani, A. M. I. M., Rahman, T., & Islam, M. M. (2024) · Applied Sciences · doi:10.3390/app14135459 · Cited by 11 · see citing works
ECML: Improving Efficiency of Machine Learning in Edge Clouds
Dhakal, A., Kulkarni, S. G., & Ramakrishnan, K. K. (2020) · IEEE International Conference on Cloud Networking · doi:10.1109/cloudnet51028.2020.9335804 · Cited by 10 · see citing works
Velasco-Montero, D., Femandez-Bemi, J., Carmona‐Galán, R., & Rodriguez-Vazquez, A. (2019) · International Conference on Systems, Signals and Image Processing · doi:10.1109/iwssip.2019.8787329 · Cited by 9 · see citing works
Characterizing Resource Heterogeneity in Edge Devices for Deep Learning Inferences
Hao, J., Subedi, P., Kim, I. K., & Ramaswamy, L. M. (2020) · doi:10.1145/3452411.3464446 · Cited by 8 · see citing works
Durán, F., Martínez, M., Lago, P., & Martínez‐Fernández, S. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112574 · Cited by 7 · see citing works
Accelerating Native Inference Model Performance in Edge Devices using TensorRT
Gopikrishna, P. B., Rubell Marion Lincy, G., Rishekeeshan, A., & Deekshitha (2024) · IEEE Recent Advances in Intelligent Computational Systems · doi:10.1109/raics61201.2024.10690032 · Cited by 7 · see citing works
MECBench: A Framework for Benchmarking Multi-Access Edge Computing Platforms
Naman, O., Qadi, H., Karsten, M., & Al-Kiswany, S. (2023) · IEEE International Conference on Edge Computing and Communications (EDGE) · doi:10.1109/edge60047.2023.00024 · Cited by 6 · see citing works
Hardware Resource Analysis in Distributed Training with Edge Devices
Park, S., Lee, J., & Kim, H. (2019) · Electronics · doi:10.3390/electronics9010028 · Cited by 6 · see citing works
Turtiainen, H., Costin, A., Lahtinen, T., Sintonen, L., & Hämäläinen, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03870 · Cited by 6 · see citing works
Gandhi, H., & Ribeiro, V. J. (2022) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets53615.2022.9668511 · Cited by 5 · see citing works
Ghasemi, M. H., Lucía, Ó., & Lucia, S. (2020) · IEEE Access · doi:10.1109/access.2020.2977087 · Cited by 5 · see citing works
Edge Computing for Artificial Intelligence
Wang, X., Han, Y., Leung, V. C. M., Niyato, D. T., Yan, X., & Chen, X. (2020) · Springer eBooks · doi:10.1007/978-981-15-6186-3_7 · Cited by 5 · see citing works
AI2: a novel explainable machine learning framework using an NLP interface
Dessureault, J.-S., & Massicotte, D. (2023) · International Conference on Machine Learning Technologies (ICMLT) · doi:10.1145/3589883.3589884 · Cited by 5 · see citing works
Evaluation of Embedded Platforms for Lower Limb Prosthesis with Visual Sensing CapabilitiesPreprint
da Silva, R. L., Starliper, N., Zhong, B., Huang, H., & Lobaton, E. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15224 · Cited by 5 · see citing works
Guo, Y., Qian, C., Song, J., & Yu, W. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-326572-3.00013-9 · Cited by 4 · see citing works
Thinking and Prospect of Power Chip Specificity
Ma, F., Li, M., Dong, X., Wang, B., Zhou, Y., Li, J., Feng, L., & Mohamed, M. A. (2021) · International Journal of Photoenergy · doi:10.1155/2021/1512629 · Cited by 4 · see citing works
Understanding the Runtime Overheads of Deep Learning Inference on Edge Devices
Ma, X., Li, G., Liu, L., Liu, H., Liu, L., & Feng, X. (2021) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom52081.2021.00061 · Cited by 4 · see citing works
Creating Robust Deep Neural Networks with Coded Distributed Computing for IoT
Hadidi, R., Cao, J., Asgari, B., & Kim, H. (2023) · IEEE International Conference on Edge Computing and Communications (EDGE) · doi:10.1109/edge60047.2023.00029 · Cited by 4 · see citing works
Porting Rulex Machine Learning Software to the Raspberry Pi as an Edge Computing Device
Daher, A., Rizik, A., Marco Muselli, B., Chiblè, H., & Caviglia, D. D. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-3-030-66729-0_33 · Cited by 4 · see citing works
Quantitative Expression of Elderly Multi-Modal Emotions with Spoken Dialogue Agent and Edge AI
Chen, S., Ozono, H., Nakamura, M., & Yasuda, K. (2023) · IEEE Eurasian Conference on Educational Innovation (ECEI) · doi:10.1109/ecei57668.2023.10105327 · Cited by 3 · see citing works
Evaluation of Edge Platforms for Deep Learning in Computer Vision
Rasmussen, C. B., Lejbølle, A. R., Nasrollahi, K., & Moeslund, T. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68799-1_38 · Cited by 3 · see citing works
Craioveanu, M. G., & Stâmâtescu, G. (2025) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med64031.2025.11073407 · Cited by 2 · see citing works
Velasco-Montero, D., Fernández‐Berni, J., Carmona‐Galán, R., & Rodriguez-Vazquez, A. (2020) · International journal of electrical and computer engineering systems · doi:10.32985/ijeces.11.1.1 · Cited by 2 · see citing works
When Mobile Apps Going Deep: An Empirical Study of Mobile Deep Learning.Preprint
Xu, M., Liu, J., Liu, Y., Lin, F. X., Liu, Y., & Liu, X. (2018) · arXiv (Cornell University) · Cited by 2 · see citing works
Performance and Consistency Analysis for Distributed Deep Learning Applications
Jia, D., Saha, M. P., Bhimani, J., & Ningfang Mi, A. (2020) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/ipccc50635.2020.9391566 · Cited by 1 · see citing works
Inference at the Edge for Complex Deep Learning Applications with Multiple Models and Accelerators
Ashwanth, K. P., K, D. N. L., Dev Divyendh, D., Sreehari Krishna, S., Sundar, V. K., & Kumar, P. D. (2023) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt56998.2023.10306363 · Cited by 1 · see citing works
Energy-Efficient Development of ML-Enabled Systems: A Data-Centric Approach
Omar, R. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644974 · Cited by 1 · see citing works
Proactive Elastic Scheduling for Serverless Ensemble Inference Services
He, S., Feng, B., & Ding, Z. (2024) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws62655.2024.00121 · Cited by 1 · see citing works
Energy Efficiency of AI-Powered Components: A Comparative Study of Feature Selection Methods
Omar, R., & Muccini, H. (2024) · International Conference on ICT for Sustainability (ICT4S) · doi:10.1109/ict4s64576.2024.00019 · Cited by 1 · see citing works
DISAIN SISTEM OTOMASI SUHU RUANGAN PERTEMUAN DENGAN PENERAPAN TEKNIK MACHINE LEARNING
Pella, S. I., & L, H. F. (2019) · Media Elektro · doi:10.35508/jme.v0i0.1781 · Cited by 1 · see citing works
Suo, K., Nguyen, T. N., Shi, Y., He, J. S., & Hung, C. (2022) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/ipccc55026.2022.9894338 · Cited by 1 · see citing works
An Experimental Study of Deep Neural Networks on HPC Clusters
Buryak, D., Попова, Н. Н., Воеводин, В. В., Konkov, Y., Ivanov, O., Shaykhlislamov, D., & Fateev, I. D. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-36592-9_40
Location Estimation of RF Emitting Source Using Supervised Machine Learning Technique
Rahouma, K. H., & Mostafa, A. S. A. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-58669-0_3
Towards Applying Deep Learning to the Internet of Things: A Model and a Framework
Elnagar, S., & Osei-Bryson, K.-M. (2020) · Lecture notes in business information processing · doi:10.1007/978-3-030-63396-7_26
Introduzindo a qualidade de imagem como uma nova condição de particionamento de DNN na borda
Pacheco, R. G., & Couto, R. S. (2020) · doi:10.5753/sbrc.2020.12272
Securing Arm Platform: From Software-Based To Hardware-Based Approaches
Ning, Z. (2020) · Human Biology
Creating Robust Deep Neural Networks With Coded Distributed Computing for IoT SystemsPreprint
Hadidi, R., Cao, J., & Kim, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04447
Computer Vision-based Monitoring of Harvest Quality
Rasmussen, C. B. (2021) · doi:10.54337/aau468596652
Flet-Edge: A Full Life-cycle Evaluation Tool for deep learning framework on the Edge
Jiang, X., Zhang, X., & Peng, X. (2023) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads56603.2022.00094
Self-citationXu, L., & Shi, W. (2025) · Wiley eBooks · doi:10.1002/9781394285860.ch4
Self-citation
- Authors
- Zhang, X., Wang, Y., & Shi, W.
- Affiliations
- Wayne State University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1906.01878
- OpenAlex
- W2963947694
- Cited by
- 77
- References
- 0
- Citations by year
- 2025: 4 · 2024: 7 · 2023: 6 · 2022: 9 · 2021: 13 · 2020: 20 · 2019: 15 · 2018: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- edge computing, machine learning framework, performance comparison, TensorFlow, MXNet, PyTorch, TensorFlow Lite, inference latency, memory usage, energy consumption, artificial neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
