Towards Real-Time Object Detection on Embedded Systems
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Convolutional neural network (CNN) based methods have achieved great success in image classification and object detection tasks. However, unlike the image classification task, object detection is much more computation-intensive and energy-consuming since a large number of possible object proposals need to be evaluated. Consequently, it is difficult for object detection methods to be integrated into embedded systems with limited computing resources and energy supply. In this paper, we propose a pipelined object detection implementation on the embedded platform. We present a comprehensive analysis of state-of-the-art object detection algorithms and select Fast R-CNN as a possible solution. Additional modifications on the Fast R-CNN method are made to fit the specific platform and achieve trade-off between speed and accuracy on embedded systems. Finally, a multi-stage pipelined implementation on the embedded CPU+GPU platform with duplicated module-parallelism is proposed to make full use of the limited computation resources. The proposed system is highly energy-efficient and close to real-time performance. In the first Low-Power Image Recognition Challenge (LPIRC), our system achieved the best result with mAP/Energy of 1.818e-2/ (W.h) on the embedded Jetson TK1 CPU+GPU platform.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“Thanks to the breakthrough in Image-Net Large-Scale Vision Recognition Challenge (ILSVRC) [2], Deep Neural Network (DNN), especially Convolutional Neural network (CNN), has led great advances in object classification tasks [3], [4].”
BackgroundApproximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · cited 4×
“Thanks to the breakthrough in Image-Net Large-Scale Vision Recognition Challenge (ILSVRC) [2], Deep Neural Network (DNN), especially Convolutional Neural network (CNN), has led great advances in object classification tasks [3], [4].”
MethodInfluentialSelective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656
Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · cited 2×
“[14], [15] for CNN and CNN-based vision tasks.”
BackgroundMethodEIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
Self-citationDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Going Deeper with Embedded FPGA Platform for Convolutional Neural Network
Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., Song, S., et al. (2016) · Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2847263.2847265 · Cited by 1,252
Self-citationBING: Binarized Normed Gradients for Objectness Estimation at 300fps
Cheng, M.-M., Zhang, Z., Lin, W.-Y., & Torr, P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.414 · Cited by 1,050
What Makes for Effective Detection Proposals?
Hosang, J., Benenson, R., Dollar, P., & Schiele, B. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2465908 · Cited by 727
14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks
Chen, Y.-H., Krishna, T., Emer, J., & Sze, V. (2016) · 2016 IEEE International Solid-State Circuits Conference (ISSCC) · doi:10.1109/isscc.2016.7418007 · Cited by 713
DeepID-Net: Deformable deep convolutional neural networks for object detection
Ouyang, W., Wang, X., Zeng, X., Qiu, S., Luo, P., Tian, Y., Li, H., Yang, S., Wang, Z., Loy, C.-C., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298854 · Cited by 430
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369
CNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367 · cited 1×
“[14], [15] for CNN and CNN-based vision tasks.”
MethodKrähenbühl, P., & Koltun, V. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_47 · Cited by 336
Self-supervised Monocular Road Detection in Desert Terrain
Dahlkamp, H., Kaehler, A., Stavens, D., Thrun, S., & Bradski, G. (2006) · Robotics: Science and Systems II · doi:10.15607/rss.2006.ii.005 · Cited by 329
Integrated CPU-GPU Power Management for 3D Mobile Games
Pathania, A., Jiao, Q., Prakash, A., & Mitra, T. (2014) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/2593069.2593151 · Cited by 145
Grauman, K., & Leibe, B. (2011) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00332ed1v01y201103aim011 · Cited by 126
Multi-camera object detection for robotics
Coates, A., & Ng, A. Y. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509644 · Cited by 67
Jarus, M., Varrette, S., Oleksiak, A., & Bouvry, P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40517-4_16 · Cited by 62
Zhao, Q., & Liu, Z. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.23 · Cited by 34
Research works printed but not linked: 1
- rao, 0, Compute with Tegra K1, Proc GPU Technology Conf
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“Applying various convolutional filters, CNN models can capture the high-level representation of the input data, making it most popular for computer vision tasks, e.g., image classification (e.g., AlexNet [18], VGG network [19], ResNet [20], and MobileNet [21]) and object detection (e.g., Fast R-CNN [22], YOLO [23], an…”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584
“Where storage and computing resources are limited, such as face recognition [215, 240], autonomous driving [24, 218], mobile phone [216, 248], industrial production [245], embedded systems [158, 181].”
BackgroundSon, H., Choi, H., Seong, H., & Kim, C. (2018) · Automation in Construction · doi:10.1016/j.autcon.2018.11.033 · Cited by 165
Deep learning: as the new frontier in high-throughput plant phenotyping
Arya, S., Sandhu, K. S., Singh, J., & Kumar, S. (2022) · Euphytica · doi:10.1007/s10681-022-02992-3 · Cited by 124
Metadata
- Authors
- Mao, H., Yao, S., Tang, T., Li, B., Yao, J., & Wang, Y.
- Affiliations
- Tsinghua University (CN)
- Huawei Technologies (China) (CN)
- Venue
- IEEE Transactions on Emerging Topics in Computing
- Published
- 2018-07-01
- Volume
- 6
- Issue
- 3
- Pages
- 417-431
- ISSN
- 2168-6750 (ISSN-L)
- 2376-4562
- OpenAlex
- W2515751428
- Cited by
- 94
- References
- 39
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 6 · 2023: 10 · 2022: 18 · 2021: 14 · 2020: 13 · 2019: 14 · 2018: 8
- FWCI
- 5.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Emerging Topics in Computing (publishedVersion)
- Funders
- 973 Project (2013CB329000)
- National Natural Science Foundation of China (61373026)
- Brain Inspired Computing Research, Tsinghua University (20141080934)
- Tsinghua University Initiative Scientific Research Program
- Beijing Municipal Institutions
- Huawei Technologies
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- real-time object detection, embedded systems, Fast R-CNN, energy efficiency, speed-accuracy trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.