Analysis Based on Recent Deep Learning Approaches Applied in Real-Time Multi-Object Tracking: A Review
Open accessThe DOI matches its Crossref record; 94 of the 95 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
The deep learning technique has proven to be effective in the classification and localization of objects on the image or ground plane over time. The strength of the technique's features has enabled researchers to analyze object trajectories across multiple cameras for online multi-object tracking (MOT) systems. In the past five years, these technical features have gained a reputation in handling several real-time multiple object tracking challenges. This contributed to the increasing number of proposed deep learning methods (DLMs) and networks seen by the computer vision community. The technique efficiently handled various challenges in real-time MOT systems and improved overall tracking performance. However, it experienced difficulties in the detection and tracking of objects in overcrowded scenes and motion variations and confused appearance variations. Therefore, in this paper, we summarize and analyze the 95 contributions made in the past five years on deep learning-based online MOT methods and networks that rank highest in the public benchmark. We review their expedition, performance, advantages, and challenges under different experimental setups and tracking conditions. We also further categorize these methods and networks into four main themes: Online MOT Based Detection Quality and Associations, Real-Time MOT with High-Speed Tracking and Low Computational Costs, Modeling Target Uncertainty in Online MOT, and Deep Convolutional Neural Network (DCNN), Affinity and Data Association. Finally, we discuss the ongoing challenges and directions for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![FIGURE 1 Single Camera Multi-Object Tracking Overview [19].](https://figures.cochat.ai/paper-figures/58651e85fdac40a7aa6b0bf978677901/analysis-based-on-recent-deep-learning-approaches-applied-in-real-time-multi-figure-1.png)



![FIGURE 5 Object Tracking based on Quality Detections [76].](https://figures.cochat.ai/paper-figures/9a317c3925944da59223f1e8d6bf6cd5/analysis-based-on-recent-deep-learning-approaches-applied-in-real-time-multi-figure-5.png)











Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vehicle detection and classification using deep learning (DL) and multi-object tracking (MOT) on video streams obtained from a network of surveillance cameras have become the state of the art (SOTA) in the automation of intelligent transport systems (ITSs) [3].”
From Real-Time Vehicle Classification and Tracking Using a Transfer Learning-Improved Deep Learning Network (2022) · cites this paper as [3]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“The detection quality is significant to improve the tracker’s capabilities in handling the object’s appearance similarities, generating and associating the tracklets, reducing false object detections, calculating, grouping the similarity trajectories, and drifting [74], [75].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 1×
“The detection quality is significant to improve the tracker’s capabilities in handling the object’s appearance similarities, generating and associating the tracklets, reducing false object detections, calculating, grouping the similarity trajectories, and drifting [74], [75].”
BackgroundYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“This is supported by the results of a considerable number of detached detections that could not be associated and hence degraded detections’ quality response [2], [22], [23], [54]–[56], [59], [60].”
ResultPreferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Simple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286 · cited 2×
“Then [32], [41], [62]–[66], emerged to learn the control of the problem through graphic models and flow optimizations but experienced difficulties in handling the affinities.”
BackgroundMethodSimple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433 · cited 1×
“Bewley et al. [65] used the merits of single object tracking [83] to integrate the Kalman filter features with the Hungarian algorithm to find the association in visual tracks.”
MethodSiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks
Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., & Yan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00441 · Cited by 2,607 · cited 2×
“Then, Li et al. [53] incorporated template branch and bounding box regression into a Siamese regional proposed network (SRPN) to improve the detection speed.”
BackgroundMOT16: A Benchmark for Multi-Object Tracking
Milan, A., Leal-Taixe, L., Reid, I., Roth, S., & Schindler, K. (2016) · arXiv · doi:10.48550/arxiv.1603.00831 · Cited by 1,309 · cited 1×
“Then optimizes the sequential states by modeling the maximum posterior estimation from the conditional for all sequential states of all objects from the first frame to the last frame [25].”
MethodTowards Real-Time Multi-Object Tracking
Wang, Z., Zheng, L., Liu, Y., Li, Y., & Wang, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_7 · Cited by 1,195
VirtualWorlds as Proxy for Multi-object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.470 · Cited by 1,079 · cited 1×
“It also caused difficulties in target tracking and data association issues across multiple cameras [94], whereby each camera scene needed to be merged in with those of the different cameras on the network.”
BackgroundUA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking
Wen, L., Du, D., Cai, Z., Lei, Z., Chang, M.-C., Qi, H., Lim, J., Yang, M.-H., & Lyu, S. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.102907 · Cited by 804 · cited 3×
“[26] capitalized on this theorem by creating CLEAR MOT evaluation metrics that have been implemented in neoteric work on deep learning-based real-time MOT methods, multi-camera tracking techniques (MCTs), and DCNNs with the trackingby-detection (TBD) approach to track objects across mul-”
MethodInfluentialHigh-Speed tracking-by-detection without using image information
Bochinski, E., Eiselein, V., & Sikora, T. (2017) · 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078516 · Cited by 707 · cited 7×
“In contrast [2], [22]–[24], [46], [54]–[61], [82], classified and assessed the matches between detection and tracklets to quickly track the targets.”
BackgroundMethodResultInfluentialLearning to Track: Online Multi-object Tracking by Decision Making
Xiang, Y., Alahi, A., & Savarese, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.534 · Cited by 674 · cited 5×
“In this section, we used Table 1 and showed an overview of the DLMs that are integrated with CNN to increase the detection quality rate by breaking input video into frames [18], [47].”
BackgroundMethodInfluentialMOTS: Multi-Object Tracking and Segmentation
Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B. B. G., Geiger, A., & Leibe, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00813 · Cited by 658 · cited 2×
“In contrast [2], [22]–[24], [46], [54]–[61], [82], classified and assessed the matches between detection and tracklets to quickly track the targets.”
BackgroundMethodFeatures for Multi-target Multi-camera Tracking and Re-identification
Ristani, E., & Tomasi, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00632 · Cited by 556
Chen, L., Ai, H., Zhuang, Z., & Shang, C. (2018) · 2018 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486597 · Cited by 530
Online Multi-Target Tracking Using Recurrent Neural Networks
Milan, A., Rezatofighi, S. H., Dick, A., Reid, I., & Schindler, K. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11194 · Cited by 506
Learning by Tracking: Siamese CNN for Robust Target Association
Leal-Taixe, L., Canton-Ferrer, C., & Schindler, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.59 · Cited by 444 · cited 1×
“Leal-Taixe et al. [68] extended the technique into the Siamese network to learn matching features for MOTs and then determined the affinity score.”
MethodChu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., & Yu, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.518 · Cited by 406 · cited 2×
“[55] propose an object-specific particle filtering framework for real-time MOT evaluations.”
MethodGMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs
Roshan Zamir, A., Dehghan, A., & Shah, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_25 · Cited by 353
Deep Affinity Network for Multiple Object Tracking
Sun, S., AKHTAR, N., Song, H., Mian, A. S., & Shah, M. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929520 · Cited by 340 · cited 1×
“[48] suggest tackling object appearance and data association issueswith tracking-by-detection via DCNN.”
BackgroundSpatially supervised recurrent convolutional neural networks for visual object tracking
Ning, G., Zhang, Z., Huang, C., Ren, X., Wang, H., Cai, C., & He, Z. (2017) · 2017 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2017.8050867 · Cited by 267 · cited 3×
“[21] introduced a spatially recurrent convolutional neural network (SR-CNN) by extending the spatial and”
BackgroundDetect-and-Track: Efficient Pose Estimation in Videos
Girdhar, R., Gkioxari, G., Torresani, L., Paluri, M., & Tran, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00044 · Cited by 254
Multi-object Tracking with Quadruplet Convolutional Neural Networks
Son, J., Baek, M., Cho, M., & Han, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.403 · Cited by 247 · cited 3×
“To overcome the detection association problem, Son et al. [40] proposed using quadruplet CNN to learn and associate detections across video frames using appearance and motion cues.”
BackgroundMethodMulti-object Tracking with Neural Gating Using Bilinear LSTM
Kim, C., Li, F., & Rehg, J. M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_13 · Cited by 243 · cited 2×
“In contrast [2], [22]–[24], [46], [54]–[61], [82], classified and assessed the matches between detection and tracklets to quickly track the targets.”
BackgroundMethodOnline Multi-target Tracking with Strong and Weak Detections
Sanchez-Matilla, R., Poiesi, F., & Cavallaro, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_7 · Cited by 240 · cited 1×
“To manage these problems, Sanchez-Matilla et al. [51] used multiple detectors with high-end and low-end confi-dence values to improve tracking performance.”
BackgroundBayesian Multi-object Tracking Using Motion Context from Multiple Objects
Yoon, J. H., Yang, M.-H., Lim, J., & Yoon, K.-J. (2015) · 2015 IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2015.12 · Cited by 210 · cited 1×
“the single-camera technique [33] contributed to the lack of robustness and difficulties in handling long occlusions.”
BackgroundDeep Network Flow for Multi-object Tracking
Schulter, S., Vernaza, P., Choi, W., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.292 · Cited by 207 · cited 3×
“sion detections, generated tracklets, and contributed to inappropriate matching or mapping of the specific appearance information [6], [16], [17].”
BackgroundMethodMotion Segmentation & Multiple Object Tracking by Correlation Co-Clustering
Keuper, M., Tang, S., Andres, B., Brox, T., & Schiele, B. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2876253 · Cited by 201 · cited 1×
“To model object motion, Keuper et al. [57] and Chen and Ren [58] proposed a motion segmentation technique that combined bottom-up motion segmentation with top-down multiple object tracking.”
MethodSubgraph decomposition for multi-target tracking
Tang, S., Andres, B., Andriluka, M., & Schiele, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299138 · Cited by 193 · cited 3×
“Then, Tang et al. [54] introduced subgraph decomposition to improve the motion model for multiple object tracking through a finite set of hypothesis detections.”
BackgroundResultJoint tracking and segmentation of multiple targets
Milan, A., Leal-Taixe, L., Schindler, K., & Reid, I. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299178 · Cited by 185 · cited 3×
“This ineffectively localized multiple objects, extracted features, created bounding box regression detections, generated tracklets, and contributed to inappropriate matching or mapping of the specific appearance information [6], [16], [17].”
BackgroundMethodMono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filtering
Scheidegger, S., Benjaminsson, J., Rosenberg, E., Krishnan, A., & Granström, K. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500454 · Cited by 175
Multi-Sensor Multi-Object Tracking With the Generalized Labeled Multi-Bernoulli Filter
Vo, B.-N., Vo, B.-T., & Beard, M. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2946023 · Cited by 171 · cited 2×
“[86] implemented a multi-sensor generalization labeled multi-Bernoulli (GLMB) filter with two sensors to reduce uncertainty about object existence and state.”
BackgroundMethodBeyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking
Sharma, S., Ansari, J. A., Krishna Murthy, J., & Madhava Krishna, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461018 · Cited by 170
Exploit the Connectivity: Multi-Object Tracking with TrackletNet
Wang, G., Wang, Y., Zhang, H., Gu, R., & Hwang, J.-N. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3350853 · Cited by 165 · cited 2×
“This is supported by the results of a considerable number of detached detections that could not be associated and hence degraded detec-tions’ quality response [2], [22], [23], [54]–[56], [59], [60].”
BackgroundResultMulti-class Multi-object Tracking Using Changing Point Detection
Lee, B., Erdenee, E., Jin, S., Nam, M. Y., Jung, Y. G., & Rhee, P. K. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48881-3_6 · Cited by 164 · cited 1×
“[72] used a CNN-based detector and Lucas-Kande Tracker (LKT)-based motion to compute the likelihood of foreground regions as the detection response of different object classes.”
MethodSiamRPN++: Evolution of Siamese Visual Tracking with Very Deep NetworksPreprint
Li, B., Wu, W., Wang, Q., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11703 · Cited by 139 · cited 2×
“Then, Li et al. [53] incorporated template branch and bounding box regression into a Siamese regional proposed network (SRPN) to improve the detection speed.”
BackgroundOn pairwise costs for network flow multi-object tracking
Chari, V., Lacoste-Julien, S., Laptev, I., & Šivic, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299193 · Cited by 132
Computationally Efficient Multi-Agent Multi-Object Tracking With Labeled Random Finite Sets
Li, S., Battistelli, G., Chisci, L., Yi, W., Wang, B., & Kong, L. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2880704 · Cited by 127 · cited 2×
“8 seconds/frame), but it experienced moderate computational costs [34], complexity, difficulties in handling motion variation and annotated objects that are caused by over-segmentation in real-time MOT evaluations of public datasets (MOT2016 and MOT2017).”
BackgroundMethodMulti-target Tracking in Multiple Non-overlapping Cameras Using Fast-Constrained Dominant Sets
Tesfaye, Y. T., Zemene, E., Prati, A., Pelillo, M., & Shah, M. A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01180-6 · Cited by 114 · cited 1×
“[88] used a constrained domain set and three hierarchical layers to enhance tracking of individual object appearances in each camera.”
MethodOnline multi-person tracking using Integral Channel Features
Kieritz, H., Becker, S., Hübner, W., & Arens, M. (2016) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2016.7738059 · Cited by 102 · cited 3×
“The object detection from different camera views is associated with trajectory objects and tracked independently on each camera view [36].”
BackgroundMethodCollaborative Deep Reinforcement Learning for Multi-object Tracking
Ren, L., Lu, J., Wang, Z., Tian, Q. C., & Zhou, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_36 · Cited by 95 · cited 3×
“[77] proposed a deep prediction-decision network in a collaborative deep reinforcement learning (C-DRL) method that simultaneously detected and predicted objects under a unified network via deep reinforcement learning.”
BackgroundMethodFagot-Bouquet, L., Audigier, R., Dhome, Y., & Lerasle, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_47 · Cited by 94
A Baseline for 3D Multi-Object TrackingPreprint
Weng, X., & Kitani, K. (2019) · arXiv (Cornell University) · Cited by 93
Yoon, Y.-C., Boragule, A., Song, Y.-M., Yoon, K., & Jeon, M. (2018) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639078 · Cited by 91
Predicting the Where and What of Actors and Actions through Online Action Localization
Soomro, K., Idrees, H., & Shah, M. A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.290 · Cited by 88 · cited 1×
“The emergence and expansion of convolutional neural networks (CNNs) to DCNNs strengthened DLMs and tracking-by-detection (TBDs), thus contributing to discernible progress in online MOTs [3]–[7].”
BackgroundMulti-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC ProjectPreprint
Zhang, Z., Wu, J., Zhang, X., & Zhang, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09531 · Cited by 86 · cited 1×
“[52] introduced multi-camera multi-object tracking by hierarchical clustering into a Fast R-CNN framework.”
MethodTrack, Then Decide: Category-Agnostic Vision-Based Multi-Object Tracking
Ošep, A., Mehner, W., Voigtlaender, P., & Leibe, B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460975 · Cited by 77
Tracking Noisy Targets: A Review of Recent Object Tracking ApproachesPreprint
Fiaz, M., Mahmood, A., & Jung, S. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03098 · Cited by 66 · cited 2×
“Currently, researchers [5], [11], [18], [19] have summarized only the multi-object tracking literature predicated on general visual tracking and detection techniques based on experimental studies rather than concentrating on deep learning methods based on online MOT.”
BackgroundMethodKutschbach, T., Bochinski, E., Eiselein, V., & Sikora, T. (2017) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078517 · Cited by 64 · cited 2×
“5% was achieved where the distance between the object detected in the first frame and its detection in the next frame increased [49].”
BackgroundMethodMulti-Object Tracking with Correlation Filter for Autonomous Vehicle
Zhao, D., Fu, H., Xiao, L., Wu, T., & Dai, B. (2018) · Sensors · doi:10.3390/s18072004 · Cited by 60 · cited 1×
“[50] proposed a compressed DCNN feature-based correlation filter and used”
MethodTracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers
He, Z., Li, J., Liu, D., He, H., & Barber, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00141 · Cited by 59 · cited 1×
“They have impressively promoted tracking performance by reducing the misdetection rate with the integration of a tracking-by-detection paradigm [20]–[24].”
BackgroundVisual Object Tracking Based on Mean-shift and Particle-Kalman Filter
Iswanto, I. A., & Li, B. (2017) · Procedia Computer Science · doi:10.1016/j.procs.2017.10.010 · Cited by 43 · cited 2×
“The technique has capitalized on the foundation laid with SCT approaches [35].”
MethodVisual Object Detection and Tracking using YOLO and SORT
Bathija, A., & Sharma, G. (2019) · Cited by 43
Multi-scale object candidates for generic object tracking in street scenes
Ošep, A., Hermans, A., Engelmann, F., Klostermann, D., Mathias, M., & Leibe, B. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487487 · Cited by 42
Multi-sensor multi-object tracking of vehicles using high-resolution radars
Scheel, A., Knill, C., Reuter, S., & Dietmayer, K. (2016) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2016.7535442 · Cited by 42 · cited 1×
“[71] suggested implementing the Monte Carlo algorithm with a multi-Bernoulli filter to handle the association measurements between objects.”
MethodStructural Constraint Data Association for Online Multi-object Tracking
Yoon, J. H., Lee, C., Yang, M., & Yoon, K. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1087-1 · Cited by 42
Fast and Robust Object Tracking Using Tracking Failure Detection in Kernelized Correlation Filter
Shin, J., Kim, H., Kim, D., & Paik, J. (2020) · Applied Sciences · doi:10.3390/app10020713 · Cited by 42 · cited 3×
“In contrast [2], [22]–[24], [46], [54]–[61], [82], classified and assessed the matches between detection and tracklets to quickly track the targets.”
BackgroundMethodLearning Non-Uniform Hypergraph for Multi-Object Tracking
Wen, L., Du, D., Li, S., Bian, X., & Lyu, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018981 · Cited by 41
Towards Multi-Object Detection and Tracking in Urban Scenario under Uncertainties
Kampker, A., Sefati, M., Rachman, A. S. A., Kreisköther, K., & Campoy, P. (2018) · International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS) · doi:10.5220/0006706101560167 · Cited by 40 · cited 1×
“Kampker et al. [64] presented a real-time framework for multi-object detection and maneuver-aware tracking for 3D LIDAR applications to tackle object uncertainty in cluttered urban environments.”
BackgroundA Unified Approach for Multi-Object Triangulation, Tracking and Camera Calibration
Houssineau, J., Clark, D. E., Ivekovič, Š., Lee, C. S., & Franco, J. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2523454 · Cited by 36
Human tracking over camera networks: a review
Hou, L., Wan, W., Hwang, J., Muhammad, R., Yang, M., & Han, K. (2017) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-017-0482-z · Cited by 36
Self-citationOnline CNN-based multiple object tracking with enhanced model updates and identity association
Gan, W., Wang, S., Lei, X., Lee, M., & Kuo, C. ‐. J. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.05.008 · Cited by 29 · cited 1×
“[63] use the merits of [55] to develop an online MOT approach to handle the drift and identity (ID) switches caused by occlusions and integration among targets.”
MethodOnline Multi-Object Tracking Using Selective Deep Appearance Matching
Yoon, Y.-C., Song, Y.-M., Yoon, K., & Jeon, M. (2018) · IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) · doi:10.1109/icce-asia.2018.8552105 · Cited by 27 · cited 1×
“Yoon et al. [70] designed an appearance matching network for robust online multiple object tracking to solve the computational bottleneck and affinity issues.”
BackgroundOnline multi-object tracking with efficient track drift and fragmentation handling
Ju, J., Kim, D., Ku, B., Han, D. K., & Ko, H. (2017) · Journal of the Optical Society of America A · doi:10.1364/josaa.34.000280 · Cited by 26 · cited 1×
“They used a data-driven mechanism by [8], [17], [27], [36]–[38], [40], [67]–[73], [89], [90] to learn the affinity models for data association and replaced the handcrafted features with”
MethodMulti-Object Tracking by Flying Cameras Based on a Forward-Backward Interaction
Carletti, V., Greco, A., Saggese, A., & Vento, M. (2018) · IEEE Access · doi:10.1109/access.2018.2864672 · Cited by 26 · cited 2×
“The network learned the object movement, size, speed, and direction [80] and predicted the next step of”
BackgroundMethodMulti-Person Tracking Based on Faster R-CNN and Deep Appearance Features
Khan, G., Tariq, Z., & Khan, M. U. G. (2019) · IntechOpen eBooks · doi:10.5772/intechopen.85215 · Cited by 26 · cited 3×
“Then [32], [41], [62]–[66], emerged to learn the control of the problem through graphic models and flow optimizations but experienced difficulties in handling the affinities.”
BackgroundMethodRobust multi-object tracking with semantic color correlation
Al-Shakarji, N., Seetharaman, G. S., Bunyak, F., & Palaniappan, K. (2017) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078507 · Cited by 24 · cited 1×
“It incorporated LSTM and enforced an end-to-end spatial-temporal regression with a single evaluation to enhance efficiency and effectiveness [81] by spatially glimpsing on various regions and regressing on the heat maps.”
MethodMulti-Vehicle Tracking via Real-Time Detection Probes and a Markov Decision Process Policy
Zou, Y. F., Zhang, W., Weng, W., & Meng, Z. (2019) · Sensors · doi:10.3390/s19061309 · Cited by 24 · cited 2×
“[89] used the established platform to update appearance information based on template matching rather than the learning-based approach.”
MethodMulti-object tracking with inter-feedback between detection and tracking
Tian, S., Yuan, F., & Xia, G.-S. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.07.028 · Cited by 23 · cited 3×
“Therefore, this has made DLMsmore resourceful in promoting the accuracy of motion prediction and the performance of bipartite matching between tracklets and detection [1], [10], [47].”
BackgroundMethodDiscriminative and Efficient Label Propagation on Complementary Graphs for Multi-Object Tracking
Amit Kumar, K. C., Jacques, L., & Vleeschouwer, C. D. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2533391 · Cited by 22
A comparative study on multi-object tracking methods for sports events
Moon, S.-W., Lee, J., Nam, D.-W., Kim, H., & Kim, W. (2017) · International Conference on Advanced Communication Technology · doi:10.23919/icact.2017.7890221 · Cited by 22 · cited 3×
“In the past five years, deep learning-based online multi-object tracking (MOT) paradigms have been inferior to sparse principal component analysis [1], [2].”
BackgroundData Fusion-Based Multi-Object Tracking for Unconstrained Visual Sensor Networks
Jiang, X., Fang, Z., Xiong, N. N., Gao, Y., Huang, B., Zhang, J., Yu, L., & Harrington, P. (2018) · IEEE Access · doi:10.1109/access.2018.2812794 · Cited by 21 · cited 3×
“Jiang et al. [8] analyzed object trajectories across multiple cameras to allow synthesis data and security analysis of images in various scenarios.”
BackgroundMethodWu, C.-W., Zhong, M.-T., Tsao, Y., Yang, S.-W., Chen, Y.-K., & Chien, S.-Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.184 · Cited by 18 · cited 3×
“It enables the enhancement of trackers to track multiple objects in a video frame sequence based on the detection quality [27].”
BackgroundMethodMulti-Object tracking using multi-channel part appearance representation
Anh, N. T. L., Khan, F. M., Negin, F., & Brémond, F. (2017) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078552 · Cited by 16
Learning Spatio-Temporal Information for Multi-Object Tracking
Wei, J., Yang, M., & Liu, F. (2017) · IEEE Access · doi:10.1109/access.2017.2686482 · Cited by 15 · cited 2×
“[13] developed the learning framework to address the issues in tracking and misdetections under heavy object interactions.”
BackgroundModel Update Strategies about Object Tracking: A State of the Art Review
Wang, D., Fang, W., Chen, W., Sun, T., & Chen, A. T. (2019) · Electronics · doi:10.3390/electronics8111207 · Cited by 15 · cited 2×
“In this section, we used Table 1 and showed an overview of the DLMs that are integrated with CNN to increase the detection quality rate by breaking input video into frames [18], [47].”
BackgroundMethodMulti-camera object tracking using surprisal observations in visual sensor networks
Bhuvana, V. P., Schranz, M., Regazzoni, C. S., Rinner, B., Tonello, A. M., & Huemer, M. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0347-x · Cited by 14 · cited 1×
“However, it provides a one-sided view and cannot provide multiple views due to its limitations in handling rotations, scaling, affinity distortions, quick movements, similarities, and occlusions [28], [29].”
BackgroundOnline Learned Siamese Network with Auto-Encoding Constraints for Robust Multi-Object Tracking
Liu, P., Li, X., Liu, H., & Fu, Z. (2019) · Electronics · doi:10.3390/electronics8060595 · Cited by 14 · cited 2×
“Zhu et al. [41] and Liu et al. [76] tried to address the problems by combining appearance and motion models.”
BackgroundOnline Multiple View Tracking: Targets Association Across Cameras
Le, Q. C., Conte, D., & Hidane, M. (2018) · HAL · Cited by 10
Ray, K. S., & Chakraborty, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02672 · Cited by 8 · cited 4×
“The non-static surface gives the impression that motion in the background pixels affects the detection quality [3], [12], [29].”
BackgroundMethodInfluentialMulti-appearance segmentation and extended 0-1 programming for dense small object tracking
Chen, L., & Ren, M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206168 · Cited by 7 · cited 2×
“This led to a failure in handling objects’ appearance variations [2], [23], [58] and high IDS (7,318) with a low tracking accuracy (33.4”
BackgroundMethodInfluentialWeb-Based Object Tracking Using Collaborated Camera Network
Azad, A. K. M., & Misbahuddin, M. (2018) · Advances in Internet of Things · doi:10.4236/ait.2018.82002 · Cited by 7 · cited 1×
“[ 31] further contributed to enhance the detector’s performance by labeling the objects on the output of the maximal classifiers.”
BackgroundVisual object tracking challenges revisited: VOT vs. OTB
Sun, B., Zuo, Z., Wusheng, L., Du, L., & Lu, Q. (2018) · PLoS ONE · doi:10.1371/journal.pone.0203188 · Cited by 7 · cited 3×
“The non-static surface gives the impression that motion in the background pixels affects the detection quality [3], [12], [29].”
BackgroundProblem Analysis of Multiple Object Tracking System: A Critical Review
Islam, M. Z., Islam, M. S., & Rana, M. S. (2015) · IJARCCE · doi:10.17148/ijarcce.2015.41183 · Cited by 6
Real-time tracking of multiple objects with locally adaptive correlation filters
Ruchay, A. N., Kober, V. I., & Chernoskulov, I. E. (2017) · International Conference on Information Technology and Nanotechnology (ITNT) · doi:10.18287/1613-0073-2017-1901-214-218 · Cited by 6 · cited 7×
“[23], [58] and high IDS (7,318) with a low tracking accuracy (33.”
BackgroundMethodResultInfluentialInter-Camera Person Tracking in Non-overlapping Networks
Kumar, K. A. S., Ramakrishnan, K. R., & Rathna, G. N. (2017) · International Conference on Distributed Smart Cameras · doi:10.1145/3131885.3131912 · Cited by 6 · cited 2×
“They used a data-driven mechanism by [8], [17], [27], [36]–[38], [40], [67]–[73], [89], [90] to learn the affinity models for data association and replaced the handcrafted features with a real-time MOT framework, such as Siamese CNN.”
BackgroundMethodReinforcement Learning Based Querying in Camera Networks for Efficient Target Tracking
Sharma, A., Anand, S., & Kaul, S. K. (2021) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v29i1.3522 · Cited by 6 · cited 1×
“[93] used a camera selection policy to select the candidate camera where the target is likely to appear by fording the ReID queries during the target transition.”
MethodUse of Gradient-Based Shadow Detection for Estimating Environmental Illumination Distribution
Lee, S., & Hong, H.-K. (2018) · Applied Sciences · doi:10.3390/app8112255 · Cited by 5 · cited 1×
“These limitations led to degraded overall detector performance, and Lee and Hong [30] incorporated separate detectors and classifiers for several different viewpoints to improve the detector performance.”
BackgroundMartín-Nieto, R., García‐Martín, Á., Martínez, J. M., & SanMiguel, J. C. (2018) · Sensors · doi:10.3390/s18124385 · Cited by 4 · cited 2×
“Houssineau et al [9] proposed a new online scheme for evaluating ReID algorithms for object tracking aiming to improve the target ReID process within a camera at different times.”
MethodMulti-Object Model-Free Tracking with Joint Appearance and Motion Inference
Liu, C., Yao, R., Rezatofighi, S. H., Reid, I. D., & Shi, Q. (2017) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2017.8227468 · Cited by 3 · cited 1×
“Bewley et al. [65] used the merits of single object tracking [83] to integrate the Kalman filter features with the Hungarian algorithm to find the association in visual tracks.”
MethodThe Application of Neural Network in Multiple Object Tracking
Wang, J., Zeng, X., Luo, W., & An, W. (2018) · DEStech Transactions on Computer Science and Engineering · doi:10.12783/dtcse/csse2018/24504 · Cited by 2 · cited 1×
“In this case, the current most deep learning-based online MOT systems introduced the DCNN and tracking-by-detection (TBD) paradigm to solve the problem of associating a target with multiple potential views [39].”
BackgroundObject Detection and Tracking Using Spatiotemporal Wavelet Domain Markov Random Field in Video
Li, X., & Liu, D. (2009) · International Conference on Computational Intelligence and Security (CIS) · doi:10.1109/cis.2009.13 · Cited by 2
Multi-Object Tracking Based on HOG Template Matching and Non-maximum Convergence Algorithm
Xiong, L., Zhang, X., jianlan, L. I. A. O., & Yang, G. (2017) · International Journal of Signal Processing Image Processing and Pattern Recognition · doi:10.14257/ijsip.2017.10.1.23 · Cited by 1 · cited 1×
“Although the proposed approach increased the target detection and association across the video frames, it failed to track and differentiate small objects [79] with similarities under complex scenes on a real-time MOT dataset (VOT2016 and MOT2016) and achieved moderate tracking accuracy (51.2”
BackgroundInfluential
Research works printed but not linked: 4
- patel, 2018, Multiple object detection and tracking: A survey, International Journal of Applied Science and Engineering Research, 6, 809
- tiwari, 2017, A review of detection and tracking of object from image and video sequences, Int J Comput Intell Res, 13, 745
- bai, 2017, Multi-object tracking and identification via particle filtering over sets, Proc 20th Int Conf Inf Fusion (Fusion), 10
- zhu, 2018, Online Multi-Object Tracking with Dual Matching Attention Networks 15th European Conference, 379
Cited by
Neupane, B., Horanont, T., & Aryal, J. (2022) · Sensors · doi:10.3390/s22103813 · Cited by 85
“Vehicle detection and classification using deep learning (DL) and multi-object tracking (MOT) on video streams obtained from a network of surveillance cameras have become the state of the art (SOTA) in the automation of intelligent transport systems (ITSs) [3].”
BackgroundTransformers in Single Object Tracking: An Experimental Survey
Kugarajeevan, J., Thanikasalam, K., Ramanan, A., & Fernando, S. (2023) · IEEE Access · doi:10.1109/access.2023.3298440 · Cited by 101
“object tracking [1, 2] approaches based on the number”
BackgroundMulti-object tracking: a systematic literature review
Hassan, S., Mujtaba, G., Rajput, A., & Fatima, N. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17297-3 · Cited by 90
Multiple Object Tracking in Deep Learning Approaches: A Survey
Park, Y., Dang, L. M., Lee, S., Han, D., & Moon, H. (2021) · Electronics · doi:10.3390/electronics10192406 · Cited by 69
“reviewed MOT papers during the last 5 years [13].”
BackgroundLöffler, K., & Mikut, R. (2022) · IEEE Access · doi:10.1109/access.2022.3192880 · Cited by 50
Metadata
- Authors
- Kalake, L., Wan, W., & Hou, L.
- Affiliations
- Shanghai University (CN)
- Huangshan University (CN)
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 32650-32671
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3132576305
- Cited by
- 68
- References
- 95
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 20 · 2023: 15 · 2022: 10 · 2021: 3
- FWCI
- 4.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Science and Technology Commission of Shanghai Municipality (18510760300)
- Anhui Natural Science Foundation (1908085MF178)
- China Postdoctoral Science Foundation (2020M681264)
- Anhui Excellent Young Talents Support Program
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multi-target tracking, real-time tracking, deep learning, online tracking, data association, target uncertainty, crowded scenes, appearance variation, deep convolutional neural networks, low computational cost, tracking benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.