An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, dynamic moving backgrounds and complex environments complicate the estimation of ground vehicle speeds [12].”
From A Systematic Review of Drone Based Road Traffic Monitoring System (2022) · cites this paper as [12]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“YOLOv3 [31] is a good detector, which outperforms some state-of-the-art methods like faster regions with convolutional neural networks (RCNN) with ResNet [35] and single shot multibox detector (SSD) [36], while still running significantly faster.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 1×
“YOLOv3 [31] is a good detector, which outperforms some state-of-the-art methods like faster regions with convolutional neural networks (RCNN) with ResNet [35] and single shot multibox detector (SSD) [36], while still running significantly faster.”
Method(2016) · Cited by 31,590
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128 · cited 1×
“YOLOv3 [31] is a good detector, which outperforms some state-of-the-art methods like faster regions with convolutional neural networks (RCNN) with ResNet [35] and single shot multibox detector (SSD) [36], while still running significantly faster.”
MethodSimple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433
Two-Frame Motion Estimation Based on Polynomial Expansion
Farnebäck, G. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-45103-x_50 · Cited by 2,971 · cited 1×
“In our system, the GPU-accelerated Gunnar Farneback algorithm [43] is adopted to extract image matching points.”
MethodViBe: A Universal Background Subtraction Algorithm for Video Sequences
Barnich, O., & Van Droogenbroeck, M. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2101613 · Cited by 1,885
Learning Non-maximum Suppression
Hosang, J., Benenson, R., & Schiele, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.685 · Cited by 736 · cited 1×
“Then, non-maximus suppression (NMS) [37] is used to select the final detection {x, y, w, h} from multiple overlapping candidate proposals, x, y represents the coordinates of the upper left corner of the detection bounding box, and w and h represent the width and height of the bounding box.”
MethodHigh-Speed tracking-by-detection without using image information
Bochinski, E., Eiselein, V., & Sikora, T. (2017) · 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078516 · Cited by 707
Chen, L., Ai, H., Zhuang, Z., & Shang, C. (2018) · 2018 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486597 · Cited by 530
Bae, S.-H., & Yoon, K.-J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.159 · Cited by 410
A survey of unmanned aerial vehicles (UAVs) for traffic monitoring
Kanistras, K., Martins, G., Rutherford, M. J., & Valavanis, K. P. (2013) · 2013 International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2013.6564694 · Cited by 322 · cited 1×
“For the aforementioned reasons, UAV-based intelligent monitoring systems have attracted more and more attention [21], and the vehicle speed estimation technology in airborne videos has gradually developed.”
BackgroundReal-Time Bidirectional Traffic Flow Parameter Estimation From Aerial Videos
Ke, R., Li, Z., Kim, S., Ash, J., Cui, Z., & Wang, Y. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2595526 · Cited by 186 · cited 2×
“Many methods recover the pixel scale by setting reference marks or ground control points [24,25,30], or by determining the flying height and camera internal parameters [23].”
MethodA Video-Based System for Vehicle Speed Measurement in Urban Roadways
Luvizon, D. C., Nassu, B. T., & Minetto, R. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2606369 · Cited by 165 · cited 1×
“The existing vehicle speed estimation system includes buried induction loop technology [4], radar technology [5], ultrasonic technology [6,7], laser technology [8], video-based technology [2], and so on [9–11].”
MethodVehicle Classification and Speed Estimation Using Combined Passive Infrared/Ultrasonic Sensors
Odat, E., Shamma, J. S., & Claudel, C. G. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2727224 · Cited by 148 · cited 1×
“The existing vehicle speed estimation system includes buried induction loop technology [4], radar technology [5], ultrasonic technology [6,7], laser technology [8], video-based technology [2], and so on [9–11].”
MethodVehicle Speed Estimation Using Acoustic Wave Patterns
Cevher, V., Chellappa, R., & McClellan, J. H. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.2005750 · Cited by 116
Evaluating the accuracy of vehicle tracking data obtained from Unmanned Aerial Vehicles
Guido, G., Gallelli, V., Rogano, D., & Vitale, A. (2016) · International Journal of Transportation Science and Technology · doi:10.1016/j.ijtst.2016.12.001 · Cited by 98 · cited 1×
“[26] presented a methodology used to extract data from a traffic stream though UAV application.”
MethodLai, A. H. S., & Yung, N. H. C. (2000) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/6979.880965 · Cited by 81
Airborne Video Registration and Traffic-Flow Parameter Estimation
Shastry, A. C., & Schowengerdt, R. A. (2005) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2005.858621 · Cited by 80 · cited 1×
“Many methods recover the pixel scale by setting reference marks or ground control points [24,25,30], or by determining the flying height and camera internal parameters [23].”
MethodHybrid Camera Array-Based UAV Auto-Landing on Moving UGV in GPS-Denied Environment
Yang, T., Ren, Q., Zhang, F., Xie, B., Ren, H., Li, J., & Zhang, Y. (2018) · Remote Sensing · doi:10.3390/rs10111829 · Cited by 78 · cited 5×
“First, there are vibration and blur in the images acquired by UAVs, and the airborne video environment is complex, as shown in Figure 1d.”
BackgroundInfluentialSelf-citationReal-Time Ground Vehicle Detection in Aerial Infrared Imagery Based on Convolutional Neural Network
Liu, X., Yang, T., & Li, J. (2018) · Electronics · doi:10.3390/electronics7060078 · Cited by 57 · cited 1×
“However, there are still many difficulties in vehicle speed estimation in airborne videos [27], as shown in Figure 1.”
BackgroundSelf-citationAn algorithm for automatic vehicle speed detection using video camera
Wu, J., Liu, Z., Li, J., Gu, C., Si, M., & Tan, F. (2009) · International Conference on Computer Science & Education (ICCSE) · doi:10.1109/iccse.2009.5228496 · Cited by 56 · cited 1×
“Most video-based speed estimation systems are based on fixed cameras [17].”
BackgroundSelf-citationHussain, T. M., Baig, A. M., Saadawi, T., & Ahmed, S. (1995) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.406637 · Cited by 53
Big Data Technology and Its Analysis of Application in Urban Intelligent Transportation System
Liu, Y. (2018) · International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) · doi:10.1109/icitbs.2018.00012 · Cited by 47 · cited 1×
“Therefore, improving traffic conditions has become an important issue for traffic management in various countries, and the intelligent transportation system (ITS) has been rapidly developed [1].”
BackgroundVehicle speed estimation by license plate detection and tracking
Luvizon, D. C., Nassu, B. T., & Minetto, R. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854869 · Cited by 46 · cited 1×
“(2) The method based on vehicle tracking [15,16]: As the vehicle is moving, its position in the image captured by the camera will change continuously.”
MethodTwo-camera based accurate vehicle speed measurement using average speed at a fixed point
Llorca, D. F., Salinas, C., Jimenez, M., Parra, I., Morcillo, A. G., Izquierdo, R., Lorenzo, J., & Sotelo, M. (2016) · 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795963 · Cited by 41 · cited 1×
“A very interesting project is mentioned in [19], which is a novel two-camera-based vehicle speed detection system.”
BackgroundVehicle Extraction and Speed Detection from Digital Aerial Images
Yamazaki, F., Liu, W., & Vu, T. T. (2008) · IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium · doi:10.1109/igarss.2008.4779606 · Cited by 40 · cited 1×
“[22] suggested deriving the velocity of cars by exploiting their shadows.”
BackgroundPanoramic UAV Surveillance and Recycling System Based on Structure-Free Camera Array
Yang, T., Li, Z., Zhang, F., Xie, B., Li, J., & Liu, L. (2019) · IEEE Access · doi:10.1109/access.2019.2900167 · Cited by 38 · cited 1×
“With the rapid development of UAVs in recent years [20], a method of traffic data acquisition using UAVs equipped with cameras has gradually emerged.”
MethodSelf-citationResearch of vehicle speed detection algorithm in video surveillance
Wang, J. (2016) · International Conference on Audio, Language and Image Processing · doi:10.1109/icalip.2016.7846573 · Cited by 33 · cited 1×
“For example, in [18], researchers presented a vehicle speed estimation algorithm for video surveillance.”
BackgroundVisual Detail Augmented Mapping for Small Aerial Target Detection
Li, J., Dai, Y., Li, C., Shu, J., Li, D., Yang, T., & Lu, Z. (2018) · Remote Sensing · doi:10.3390/rs11010014 · Cited by 25 · cited 1×
“Moreover, there can be many vehicles in a scene and their sizes are small, as shown in Figure 1c, so it is very difficult to locate the target accurately [29].”
BackgroundSelf-citationToward validation of freeway loop detector speed measurements using transit probe data
El-Geneidy, A., & Bertini, R. L. (2005) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2004.1399001 · Cited by 21 · cited 1×
“The existing vehicle speed estimation system includes buried induction loop technology [4], radar technology [5], ultrasonic technology [6,7], laser technology [8], video-based technology [2], and so on [9–11].”
MethodAlefs, B. G., & Schreiber, D. (2007) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2007.4357689 · Cited by 21 · cited 1×
“(2) The method based on vehicle tracking [15,16]: As the vehicle is moving, its position in the image captured by the camera will change continuously.”
MethodLaser-based large detection area speed measurement methods and systems
Musayev, E. (2007) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2007.03.007 · Cited by 19 · cited 1×
“The existing vehicle speed estimation system includes buried induction loop technology [4], radar technology [5], ultrasonic technology [6,7], laser technology [8], video-based technology [2], and so on [9–11].”
MethodDRONE-BASED TRAFFIC FLOW ESTIMATION AND TRACKING USING COMPUTER VISION
de Bruin, A., & Booysen, M. J. (2015) · SUNScholar (Stellenbosch University) · doi:10.13140/rg.2.1.1024.6888 · Cited by 15
Car speed estimation method for UAV images
Moranduzzo, T., & Melgani, F. (2014) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2014.6947604 · Cited by 15 · cited 2×
“[23] developed an approach for vehicle speed estimation by finding correspondence between scale-invariant feature transform (SIFT) features across video frames.”
BackgroundMethodNighttime Foreground Pedestrian Detection Based on Three-Dimensional Voxel Surface Model
Li, J., Zhang, F., Wei, L., Yang, T., & Lu, Z. (2017) · Sensors · doi:10.3390/s17102354 · Cited by 15
Self-citationSato, Y. (2002) · Vehicle Navigation and Information Systems Conference (VNIS) · doi:10.1109/vnis.1994.396860 · Cited by 9
Vehicle Speed Estimation Based on Sensor Networks and Signal Correlation Measurement
Zhang, W., Tan, G., & Ding, N. (2014) · Communications in computer and information science · doi:10.1007/978-3-642-54522-1_1 · Cited by 8
The Speed Detection Algorithm Based on Video Sequences
Wang, L., & Junfang, S. (2012) · International Conference on Computer Science and Service System · doi:10.1109/csss.2012.62 · Cited by 5 · cited 1×
“At present, there are two main methods used at home and abroad [12], as follows: (1) The speed estimation method based on the virtual loop [13,14]: The method is similar to the induction coil speed estimation method, the advantage of this method is the shortened processing time.”
MethodCar speed measurement based on ultrasonic Doppler's ground speed sensors
Lobur, M., & Darnobyt, Y. (2011) · International Conference on Experience of Designing and Applications of CAD Systems in Microelectronics · Cited by 5
Algorithm of Vehicle Speed Detection in Unmanned Aerial Vehicle Videos
Zhang, X., Chang, Y., Li, L., & Guo, J. (2014) · Transportation Research Board 93rd Annual MeetingTransportation Research Board · Cited by 4
Research works printed but not linked: 2
- Couto, M.S., Monteiro, J.L., and Santos, J.A. Improving virtual loop sensor accuracy for 2d motion detection. Proceedings of the 2002 Proceedings of the Bi World Automation Congress, Shanghai, China, 10–14 June 2002.
- Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
Cited by
Zhang, Z., & Zhu, L. (2023) · Drones · doi:10.3390/drones7060398 · Cited by 419
Butilă, E. V., & Boboc, R. G. (2022) · Remote Sensing · doi:10.3390/rs14030620 · Cited by 206
“Regarding the variables that were taken into account for the evaluation of the proposed system, some authors used parameters such as speed [60,64,78,85], traffic density [66,73], vehicle counting [77,92], vehicle trajectory [80,91], and parameters related to the performance of developed method:…”
MethodVision‐based vehicle speed estimation: A survey
Llorca, D. F., Martínez, A. H., & Daza, I. G. (2021) · IET Intelligent Transport Systems · doi:10.1049/itr2.12079 · Cited by 156
“…67, 72-75, 79, 82, 84, 87, 88, 89, 98, 101, 107, 113, 128, 129], the size of a previously known object such as vehicles [27, 62, 78, 85, 90, 103, 105, 106, 110, 117] or license plates [60, 66, 68, 70, 80, 94, 133], and the length and frequency of the lane markings [14, 20, 22, 23, 26, 61, 65,…”
BackgroundMethodInfluentialA Systematic Review of Drone Based Road Traffic Monitoring System
Bisio, I., Garibotto, C., Haleem, H., Lavagetto, F., & Sciarrone, A. (2022) · IEEE Access · doi:10.1109/access.2022.3207282 · Cited by 139
“…object data association, which is the problem of matching the predicted BBoxes of existing vehicles with the detected BBoxes of vehicles in the current frame, so as to optimize the number of matches in the two sets of BBoxes, a Hungarian algorithm [129] is implemented in [3], [12], and [107].”
BackgroundMethodRoad Traffic Monitoring from UAV Images Using Deep Learning Networks
Byun, S.-W., Shin, I.-K., Moon, J., Kang, J., & Choi, S. (2021) · Remote Sensing · doi:10.3390/rs13204027 · Cited by 128
Metadata
- Authors
- Li, J., Chen, S., Zhang, F., Li, E., Yang, T., & Lu, Z.
- Affiliations
- Xidian University (CN)
- Northwestern Polytechnical University (CN)
- Corresponding author
- Li, Jing
- Yang, Tao
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2019-05-24
- Volume
- 11
- Issue
- 10
- Pages
- 1241
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2944848755
- Cited by
- 57
- References
- 41
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 6 · 2023: 12 · 2022: 7 · 2021: 11 · 2020: 5 · 2019: 3
- FWCI
- 2.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Remote Sensing (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61502364, 61672429, 61272288)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multi-vehicle tracking, vehicle detection, vehicle size estimation, downward-looking camera, AirSim simulation, traffic dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.