The DOI matches its Crossref record; 263 of the 263 references it cites were found in a registry or an index.
Deep Learning for Visual Tracking: A Comprehensive Survey
Open accessAbstract
Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given the ill-posed nature of the problem and its popularity in a broad range of real-world scenarios, a number of large-scale benchmark datasets have been established, on which considerable methods have been developed and demonstrated with significant progress in recent years – predominantly by recentdeep learning(DL)-based methods. This survey aims to systematically investigate the current DL-based visual tracking methods, benchmark datasets, and evaluation metrics. It also extensively evaluates and analyzes the leading visual tracking methods. First, the fundamental characteristics, primary motivations, and contributions of DL-based methods are summarized from nine key aspects of: network architecture, network exploitation, network training for visual tracking, network objective, network output, exploitation of correlation filter advantages, aerial-view tracking, long-term tracking, and online tracking. Second, popular visual tracking benchmarks and their respective properties are compared, and their evaluation metrics are summarized. Third, the state-of-the-art DL-based methods are comprehensively examined on a set of well-established benchmarks of OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, and VisDrone2019. Finally, by conducting critical analyses of these state-of-the-art trackers quantitatively and qualitatively, their pros and cons under various common scenarios are investigated. It may serve as a gentle use guide for practitioners to weigh when and under what conditions to choose which method(s). It also facilitates a discussion on ongoing issues and sheds light on promising research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–263 of 263
Deep convolutional particle filter for visual tracking
Mozhdehi, R. J., & Medeiros, H. P. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296963 · Cited by 31
VisDrone-SOT2020: The Vision Meets Drone Single Object Tracking Challenge Results
Fan, H., Wen, L., Du, D., Zhu, P., Hu, Q., Ling, H., Shah, M. A., Wang, B., Dong, B., Yuan, D., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66823-5_44 · Cited by 30
Self-citationCell Tracking with Deep Learning and the Viterbi Algorithm
Hernandez, D. E., Chen, S. W., Hunter, E. E., Steager, E. B., & Kumar, V. (2018) · International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS) · doi:10.1109/marss.2018.8481231 · Cited by 29
Robust Visual Tracking via Hierarchical Particle Filter and Ensemble Deep Features
Li, S., Zhao, S., Cheng, B. H., Zhao, E., & Chen, J. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2889457 · Cited by 29
Correlation Filter Selection for Visual Tracking Using Reinforcement Learning
Xie, Y., Xiao, J., Huang, K., Thiyagalingam, J., & Zhao, Y. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2889488 · Cited by 29
Zhao, F., Wang, J., Wu, Y., & Tang, M. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2856540 · Cited by 27
Training-Set Distillation for Real-Time UAV Object Tracking
Li, F., Fu, C., Lin, F., Li, Y., & Lu, P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197252 · Cited by 26
End-to-End Feature Integration for Correlation Filter Tracking With Channel Attention
Li, D., Wen, G., Kuai, Y., & Porikli, F. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2877008 · Cited by 23
Channel Pruning for Visual Tracking
Che, M., Wang, R., Lu, Y., Li, Y., Zhi, H., & Xiong, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11009-3_3 · Cited by 23
Fu, C.-H., Huang, Z.-Y., Li, Y., Duan, R., & Lu, P. (2019) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros40897.2019.8967674 · Cited by 23
Real-Time Deep Tracking via Corrective Domain Adaptation
Li, H., Wang, X., Shen, F., Li, Y., Porikli, F., & Wang, M. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2923639 · Cited by 21
Surrounding-aware correlation filter for UAV tracking with selective spatial regularization
Fu, C., Xiong, W., Lin, F., & Yue, Y. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.107324 · Cited by 21
Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking
Gao, J. G., Hu, W., & Lu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00741 · Cited by 21
TracKlinic: Diagnosis of Challenge Factors in Visual Tracking
Fan, H., Yang, F., Chu, P., Lin, Y., Yuan, L., & Ling, H. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00101 · Cited by 19
TraX: The visual Tracking eXchange protocol and library
Zajc, L. Č. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.02.036 · Cited by 18
Deep Convolutional Particle Filter with Adaptive Correlation Maps for Visual Tracking
Mozhdehi, R. J., Reznichenko, Y., Siddique, A., & Medeiros, H. P. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451069 · Cited by 18
Generating Reliable Online Adaptive Templates for Visual Tracking
Guo, J., Xu, T., Jiang, S., & Shen, Z. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451440 · Cited by 18
SMART: Joint Sampling and Regression for Visual Tracking
Gao, J., Zhang, T., & Xu, C. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2904434 · Cited by 18
Visual Tracking via Auto-Encoder Pair Correlation Filter
Cheng, X., Zhang, Y., Zhou, L., & Zheng, Y. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2019.2913815 · Cited by 18
Learning target-aware correlation filters for visual tracking
Li, D., Wen, G., Kuai, Y., Xiao, J., & Porikli, F. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.11.036 · Cited by 16
Memory Mechanisms for Discriminative Visual Tracking Algorithms With Deep Neural Networks
Wang, L., Zhang, Y., Wang, J., & Zhang, Y. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2900506 · Cited by 16
Deep Position-Sensitive Tracking
Zha, Y., Ku, T., Li, Y., & Zhang, P. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2922125 · Cited by 16
DSNet: Deep and Shallow Feature Learning for Efficient Visual Tracking
Wu, Q., Yan, Y., Liang, Y., Liu, Y., & Wang, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20873-8_8 · Cited by 16
Gate connected convolutional neural network for object tracking
Thanikasalam, K., Fookes, C., Sridharan, S., Ramanan, A., & Pinidiyaarachchi, U. A. J. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296753 · Cited by 14
Spatial–temporal adaptive feature weighted correlation filter for visual tracking
Du, F., Liu, P., Zhao, W., & Tang, X. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.05.013 · Cited by 14
Tracking of Moving Objects With Regeneration of Object Feature Points
Лычков, И. И., Alfimtsev, A., & Сакулин, С. А. (2018) · Global Smart Industry Conference (GloSIC) · doi:10.1109/glosic.2018.8570061 · Cited by 14
Visual Tracking with Convolutional Neural Network
Zhang, L., & Suganthan, P. N. (2015) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.362 · Cited by 13
When correlation filters meet fully-convolutional Siamese networks for distractor-aware tracking
Kuai, Y., Wen, G., & Li, D. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.03.002 · Cited by 13
Online object tracking via motion-guided convolutional neural network (MGNet)
Gan, W., Lee, M., Wu, C.-H., & Kuo, C. ‐. J. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.03.016 · Cited by 13
Multiple Context Features in Siamese Networks for Visual Object Tracking
Morimitsu, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11009-3_6 · Cited by 13
Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking
Gao, J. G., Lu, Y., Qi, X., Kou, Y., Li, B., Li, L., Yu, S., & Hu, W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3156977 · Cited by 13
Learning a Robust Society of Tracking Parts Using Co-occurrence Constraints
Burceanu, E., & Leordeanu, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11009-3_9 · Cited by 12
Rotation-Aware Discriminative Scale Space Tracking
Marvasti-Zadeh, S. M., Ghanei-Yakhdan, H., & Kasaei, S. (2019) · Iranian Conference on Electrical Engineering · doi:10.1109/iraniancee.2019.8786548 · Cited by 12
Self-citationFu, C., He, Y., Lin, F., & Xiong, W. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04716-x · Cited by 12
Marvasti-Zadeh, S. M., Ghanei-Yakhdan, H., & Kasaei, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05586-z · Cited by 12
Self-citationGuo, J., & Xu, T. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2749458 · Cited by 11
Cross-View Contextual Relation Transferred Network for Unsupervised Vehicle Tracking in Drone Videos
Song, W., Li, S., Chang, T., Hao, A., Zhao, Q., & Qin, H. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093382 · Cited by 10
Vision-Based Tracking for Mobile Augmented Reality
Ababsa, F., Maïdi, M., Didier, J.-Y., & Mallem, M. (2008) · Studies in computational intelligence · doi:10.1007/978-3-540-78502-6_12 · Cited by 9
Siamese recurrent architecture for visual tracking
Xu, X., Ma, B., Chang, H., & Chen, X. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296462 · Cited by 9
Multi-Task Hierarchical Feature Learning for Real-Time Visual Tracking
Kuai, Y., Wen, G., & Li, D. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2883593 · Cited by 9
Efficient tracking with distinctive target colors and silhouette
Xiao, C., & Yılmaz, A. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900048 · Cited by 8
Robust object tracking by interleaving variable rate color particle filtering and deep learning
Akok, B., Gürkan, F., Kaplan, O., & Günsel, B. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296966 · Cited by 8
Learning spatio-temporal context via hierarchical features for visual tracking
Cao, Y., Ji, H., Zhang, W., & Xue, F. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.04.010 · Cited by 8
Efficient Multi-level Correlating for Visual Tracking
Ma, Y., Yuan, C., Gao, P., & Wang, F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20873-8_29 · Cited by 8
Long-term object tracking based on siamese network
Dai, K., Wang, Y., & Yan, X. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296961 · Cited by 7
Deep feature tracking based on interactive multiple model
Tang, F., Lu, X., Zhang, X., Hu, S., & Zhang, H. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.12.035 · Cited by 7
Flow Guided Short-Term Trackers with Cascade Detection for Long-Term Tracking
Wu, H., Yang, X., Yang, Y., & Liu, G. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00026 · Cited by 7
Wang, X., Li, H., Li, Y., Porikli, F., & Wang, M. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296363 · Cited by 6
Region-based fully convolutional siamese networks for robust real-time visual tracking
Yang, L., Jiang, P., Wang, F., & Wang, X. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296746 · Cited by 6
Occlusion‐robust object tracking based on the confidence of online selected hierarchical features
Liu, M., Jin, C., Yang, B., Cui, X., & Kim, H. (2018) · IET Image Processing · doi:10.1049/iet-ipr.2018.5454 · Cited by 6
WAEF: Weighted Aggregation with Enhancement Filter for Visual Object Tracking
Rout, L., Mishra, D., & Gorthi, R. K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11009-3_4 · Cited by 6
TracKlinic: Diagnosis of Challenge Factors in Visual TrackingPreprint
Fan, H., Yang, F., Chu, P., Yuan, L., & Ling, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07959 · Cited by 6
Fine-Grained Motion Representation For Template-Free Visual Tracking
Shuang, K., Huang, Y., Sun, Y., Cai, Z., & Guo, H. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093517 · Cited by 6
Adaptive exploitation of pre-trained deep convolutional neural networks for robust visual tracking
Marvasti-Zadeh, S. M., Ghanei-Yakhdan, H., & Kasaei, S. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10382-x · Cited by 6
Self-citationDensely Connected Discriminative Correlation Filters for Visual Tracking
Peng, C., Liu, F., Yang, J., & Kasabov, N. K. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2836360 · Cited by 5
Robust Visual Tracking in Low-Resolution Sequence
Lin, Z., & Yuan, C. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451826 · Cited by 5
Adaptive Aggregation of Arbitrary Online Trackers with a Regret Bound
Song, H., Suehiro, D., & Uchida, S. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093613 · Cited by 5
Integrating Multi-Level Convolutional Features for Correlation Filter Tracking
Liu, G., & Liu, G. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451425 · Cited by 4
Chen, W., Niu, B., Gu, H., et al. (2018) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/infoct.2018.8356856 · Cited by 3
i-Siam: Improving Siamese Tracker with Distractors Suppression and Long-Term Strategies
Tan, W. R., & Lai, S. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00013 · Cited by 3
Effective fusion of deep multitasking representations for robust visual tracking
Marvasti-Zadeh, S. M., Ghanei-Yakhdan, H., Kasaei, S., Nasrollahi, K., & Moeslund, T. B. (2021) · The Visual Computer · doi:10.1007/s00371-021-02304-1 · Cited by 3
Self-citationFlow Guided Siamese Network for Visual Tracking
Wang, G., Liu, B., Li, W., et al. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451324 · Cited by 2
Marvasti-Zadeh, S. M., Ghanei-Yakhdan, H., & Kasaei, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.02932 · Cited by 2
Self-citation
Research works printed but not linked: 11
- 10.3390/s18010112 · doi:10.3390/s18010112
- 10.1007/978-3-540-76856-2_46 · doi:10.1007/978-3-540-76856-2_46
- 10.1109/ICIP.2018.8451332 · doi:10.1109/icip.2018.8451332
- Pu, Deep attentive tracking via reciprocative learning, 1931
- Jiang, Deep collaborative tracking networks, 87
- Ma, Multi-scale recurrent tracking via pyramid recurrent network and optical flow, 242
- 10.1109/CVPR42600.2020.00661 · DOI · Paper page
- 10.1109/ICRA40945.2020.9196943 · DOI
- Li, BUAA-PRO: A tracking dataset with pixel-level annotation, 249
- Li, Explicit inductive bias for transfer learning with convolutional networks, 2825
- Yoon, Lifelong learning with dynamically expandable networks, 1
Cited by
Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00803 · Cited by 1,588
“Many efforts have been done in recent years [23, 28], however, designing a high-accuracy and real-time tracker is still a challenging task.”
BackgroundDetection and tracking meet drones challenge
Zhu, P., Wen, L., Du, D., Bian, X., Fan, H., Hu, Q., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119563 · Cited by 1,336
AiATrack: Attention in Attention for Transformer Visual Tracking
Gao, S., Zhou, C., Ma, C., Wang, X., & Yuan, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20047-2_9 · Cited by 351
V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving
Li, Y., Ma, D., An, Z., Wang, Z., Zhong, Y., Chen, S., & Feng, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3192802 · Cited by 321
High-Performance Long-Term Tracking With Meta-Updater
Dai, K., Zhang, Y., Wang, D., Li, J., Lu, H., & Yang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00633 · Cited by 310
Metadata
- Authors
- Marvasti-Zadeh, S. M., Cheng, L., Ghanei-Yakhdan, H., & Kasaei, S.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2022-05
- Volume
- 23
- Issue
- 5
- Pages
- 3943-3968
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2990187711
- Cited by
- 405
- References
- 263
- FWCI
- 26.54
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- University of Alberta Start-Up Grant, the NSERC Discovery (RGPIN-2019-04575)
- University of Alberta–Huawei Joint Innovation Collaboration Grants
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.