Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
In recent years, demand has been increasing for target detection and tracking from aerial imagery via drones using onboard powered sensors and devices. We propose a very effective method for this application based on a deep learning framework. A state-of-the-art embedded hardware system empowers small flying robots to carry out the real-time onboard computation necessary for object tracking. Two types of embedded modules were developed: one was designed using a Jetson TX or AGX Xavier, and the other was based on an Intel Neural Compute Stick. These are suitable for real-time onboard computing power on small flying drones with limited space. A comparative analysis of current state-of-the-art deep learning-based multi-object detection algorithms was carried out utilizing the designated GPU-based embedded computing modules to obtain detailed metric data about frame rates, as well as the computation power. We also introduce an effective target tracking approach for moving objects. The algorithm for tracking moving objects is based on the extension of simple online and real-time tracking. It was developed by integrating a deep learning-based association metric approach with simple online and real-time tracking (Deep SORT), which uses a hypothesis tracking methodology with Kalman filtering and a deep learning-based association metric. In addition, a guidance system that tracks the target position using a GPU-based algorithm is introduced. Finally, we demonstrate the effectiveness of the proposed algorithms by real-time experiments with a small multi-rotor drone.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



















Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Real time object detection and tracking is implemented and benchmarking results are obtained using embedded hardware such as Jetson TX/AGX Xavier and Intel Neural Compute Stick wherein RCNN is employed on the embedded hardware for object detection [26].”
From Object detection using YOLO: challenges, architectural successors, datasets and applications (2023) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Object Detection With Deep Learning: A Review
Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876865 · Cited by 5,545 · cited 1×
“The changes in YOLOv2 are briefly discussed below: • The changes in YOLOv2 are briefly discussed below: YOLO provides real-time processing with high accuracy, but it has higher localization errors and recall response than other region-based detector algorithms [39].”
BackgroundSimple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433
Deep Neural Networks for Object Detection
Szegedy, C., Toshev, A., & Erhan, D. (2013) · neural information processing systems · Cited by 1,662
Survey of Pedestrian Detection for Advanced Driver Assistance Systems
Gerónimo, D., López, A. M., Sappa, A. D., & Graf, T. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.122 · Cited by 974
MARS: A Video Benchmark for Large-Scale Person Re-Identification
Zheng, L., Bie, Z., Sun, Y., Wang, J., Su, C., Wang, S., & Tian, Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_52 · Cited by 965
Application of Deep Learning for Object Detection
Pathak, A. R., Pandey, M., & Rautaray, S. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.05.144 · Cited by 505
Development of advanced driver assistance systems with vehicle hardware-in-the-loop simulations
Gietelink, O., Ploeg, J., De Schutter, B., & Verhaegen, M. (2006) · Vehicle System Dynamics · doi:10.1080/00423110600563338 · Cited by 271
Detection, prediction, and avoidance of dynamic obstacles in urban environments
Ferguson, D., Darms, M., Urmson, C., & Kolski, S. (2008) · 2008 IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2008.4621214 · Cited by 131
Deep Learning in Object Recognition, Detection, and Segmentation
Wang, X. (2016) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000071 · Cited by 128
Embedded Real-Time Object Detection for a UAV Warning System
Tijtgat, N., Van Ranst, W., Volckaert, B., Goedeme, T., & De Turck, F. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.247 · Cited by 117
Sensor and object recognition technologies for self-driving cars
Hirz, M., & Walzel, B. (2018) · Computer-Aided Design and Applications · doi:10.1080/16864360.2017.1419638 · Cited by 86
Vision-Based Target Finding and Inspection of a Ground Target Using a Multirotor UAV System
Hinas, A., Roberts, J., & Gonzalez, F. (2017) · Sensors · doi:10.3390/s17122929 · Cited by 55
Autonomous Target Detection and Localization Using Cooperative Unmanned Aerial Vehicles
Yoon, Y., Gruber, S. F., Krakow, L. W., & Pack, D. J. (2008) · Lecture notes in control and information sciences · doi:10.1007/978-3-540-88063-9_12 · Cited by 13
Foundations of Python Network Programming
Rhodes, B., & Goerzen, J. (2014) · Apress eBooks · doi:10.1007/978-1-4302-5855-1 · Cited by 13
Research works printed but not linked: 4
- Tsang, S.-H. (2019, February 24). Review: YOLOv2 & YOLO9000—You Only Look Once (Object Detection). Available online: towardsdatascience.com/review-YOLOv2-YOLO9000-you-only-look-once-object-detection-7883d2b02a65.
- Szegedy, C., Reed, S., Erhan, D., Anguelov, D., and Ioffe, S. (2014). Scalable, high-quality object detection. arXiv.
- Qi, H., Zhang, Z., Xiao, B., Hu, H., Cheng, B., Wei, Y., and Dai, J. (2017, January 28). Deformable convolutional networks–coco detection and segmentation challenge 2017 entry. Proceedings of the ICCV COCO Challenge Workshop, Venice, Italy.
- Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Object detection using YOLO: challenges, architectural successors, datasets and applications
Diwan, T., Anirudh, G., & Tembhurne, J. V. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13644-y · Cited by 1,549
“Real time object detection and tracking is implemented and benchmarking results are obtained using embedded hardware such as Jetson TX/AGX Xavier and Intel Neural Compute Stick wherein RCNN is employed on the embedded hardware for object detection [26].”
MethodLiterature review: efficient deep neural networks techniques for medical image analysis
Abdou, M. A. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-06960-9 · Cited by 312
UAV in the advent of the twenties: Where we stand and what is next
Nex, F., Armenakis, C., Cramer, M., Cucci, D. A., Gerke, M., Honkavaara, E., Kukko, A., Persello, C., & Skaloud, J. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.12.006 · Cited by 309
A Survey of Object Detection for UAVs Based on Deep Learning
Tang, G., Ni, J., Zhao, Y., Gu, Y., & Cao, W. (2023) · Remote Sensing · doi:10.3390/rs16010149 · Cited by 244
Metadata
- Authors
- Hossain, S., & Lee, D.-J.
- Affiliations
- Kunsan National University (KR)
- Corresponding author
- Lee, Deok-jin
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2019-07-31
- Volume
- 19
- Issue
- 15
- Pages
- 3371
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC6695703
- OpenAlex
- W2957927798
- Cited by
- 184
- References
- 28
- Citations by year
- 2026: 4 · 2025: 18 · 2024: 26 · 2023: 35 · 2022: 38 · 2021: 39 · 2020: 21 · 2019: 3
- FWCI
- 11.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- multi-target detection, object tracking, aerial imagery, unmanned aerial vehicles, multi-rotor drones, embedded computing, Jetson, DeepSORT, Kalman filter
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.