Deployment of Deep Neural Networks for Object Detection on Edge AI Devices with Runtime Optimization
Open accessAbstract
Deep neural networks have proven increasingly important for automotive scene understanding with new algorithms offering constant improvements of the detection performance. However, there is little emphasis on experiences and needs for deployment in embedded environments. We therefore perform a case study of the deployment of two representative object detection networks on an edge AI platform. In particular, we consider RetinaNet for image-based 2D object detection and PointPillars for LiDAR-based 3D object detection. We describe the modifications necessary to convert the algorithms from a PyTorch training environment to the deployment environment taking into account the available tools. We evaluate the runtime of the deployed DNN using two different libraries, TensorRT and Torch-Script. In our experiments, we observe slight advantages of TensorRT for convolutional layers and TorchScript for fully connected layers. We also study the trade-off between run-time and performance, when selecting an optimized setup for deployment, and observe that quantization significantly reduces the runtime while having only little impact on the detection performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Analysing information immediately at the edge reduces the quantity of information that must be transported over the network.
“This is accomplished by analyzing information immediately at the edge, which reduces the quantity of information that must be transported over the network [165].”
From Edge AI: A survey (2023) · cites this paper as [165]
The Abstract (p. 1) describes deploying object-detection networks on an edge AI platform, while the Introduction (p. 1) discusses optimizing inference runtime. Neither passage addresses whether processing data at the edge reduces the amount transported over a network; searches for network transport and communication found no such finding in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measuredWidespread availability of IoT gadgets, developments in parallel processing, and the advent of 5G have contributed to a solid foundation upon which generalised machine learning may be built.
“The widespread availability of IoT gadgets, developments in parallel processing, and the advent of 5G have all contributed to the establishment of a solid foundation upon which generalised machine learning may be built [165].”
From Edge AI: A survey (2023) · cites this paper as [165]
The Introduction (pp. 1–2) discusses deploying object-detection networks on an edge AI device, not the foundations for generalised machine learning. Searches for IoT, 5G, and parallel processing found no passage connecting those developments to the claimed foundation; the References (p. 7) list a work on parallel programming but do not make that claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · cited 2× · see citing works
“A recent survey of quantization techniques for DNN inference is given in [4].”
Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
Wu, H., Judd, P., Zhang, X., Isaev, M., & Micikevicius, P. (2020) · arXiv · doi:10.48550/arxiv.2004.09602 · Cited by 246 · see citing works
Flagged first, then by citations
Research works printed but not linked: 13
- [2][2] G. Bradski. The OpenCV Library. Dr. Dobb's Journal of Software Tools , 2000. 2 -
- [5][5] Juncong Fei, Wenbo Chen, Philipp Heidenreich, Sascha Wirges, and Christoph Stiller. SemanticVoxels: Sequential Fusion for 3D Pedestrian Detection using LiDAR Point Cloud and Semantic Segmentation. In IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) , pages 185-190, 2020. 5 -
- [6][6] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The KITTI dataset. The 7. International Journal of Robotics Research , 32(11):12311237, 2013. 3, 5 -
- [7][7] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition , pages 3354-3361. IEEE, 2012. 5 -
- [10][10] IEEE. IEEE Standard for Binary Floating-Point Arithmetic. ANSI/IEEE Std 754-1985 , pages 1-20, 1985. 2 -
- [11][11] Shinpei Kato, Shota Tokunaga, Yuya Maruyama, Seiya Maeda, Manato Hirabayashi, Yuki Kitsukawa, Abraham Monrroy, Tomohito Ando, Yusuke Fujii, and Takuya Azumi. Autoware on board: Enabling autonomous vehicles with embedded systems. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS) , pages 287296. IEEE, 2018. 5 -
- [16][16] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ ar. Focal Loss for Dense Object Detection. In 2017 IEEE International Conference on Computer Vision (ICCV) , pages 2999-3007, 2017. 1, 2, 3, 4, 5, 6, 7 -
- [17][17] David Luebke. Cuda: Scalable parallel programming for high-performance scientific computing. In 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro , pages 836-838, 2008. 2 -
- [18][18] Szymon Migacz. 8-bit Inference with TensorRT. In GPU Technology Conference , page 5, 2017. 2, 4 -
- [21][21] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 , 2019. 2 - · arXiv
- [22][22] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 652-660, 2017. 3 -
- [23][23] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, Andrew Y Ng, et al. ROS: an open-source robot operating system. In ICRA workshop on open source software . Kobe, Japan, 2009. 2 -
- [24][24] Han Vanholder. Efficient Inference with TensorRT. In GPU Technology Conference , volume 1, 2016. 2 -
Cited by
Sorted by citations
Singh, R., & Gill, S. S. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.02.004 · Cited by 572 · see citing works
- Authors
- Stacker, L., Fei, J., Heidenreich, P., Bonarens, F., Rambach, J., Stricker, D., & Stiller, C.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Stellantis (Netherlands) (NL)
- German Research Centre for Artificial Intelligence (DE)
- ORCID
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 1015-1022
- ISSN
- 2473-9936 (ISSN-L)
- OpenAlex
- W3195414078
- Cited by
- 29
- References
- 7
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 7 · 2023: 9 · 2022: 6
- FWCI
- 1.80
- Type
- conference paper
- Language
- en
- Open access
- Licence
- Locations
- IEEE International Conference on Computer Vision Workshops (publishedVersion)
- Repository KITopen (Karlsruhe Institute of Technology) (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- KITopen
- Funders
- Bundesministerium für Wirtschaft und Energie
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- deep neural network deployment, object detection, edge AI, RetinaNet, PointPillars, runtime optimization, TensorRT, model quantization, embedded systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
