Once for All: A Two-Flow Convolutional Neural Network for Visual Tracking
Open accessAbstract
The main challenges of visual object tracking arise from the arbitrary appearance of the objects that need to be tracked. Most existing algorithms try to solve this problem by training a new model to regenerate or classify each tracked object. As a result, the model needs to be initialized and retrained for each new object. In this paper, we propose to track different objects in an object-independent approach with a novel two-flow convolutional neural network (YCNN). The YCNN takes two inputs (one is an object image patch, the other is a larger searching image patch), then outputs a response map which predicts how likely and where the object would appear in the search patch. Unlike the object-specific approaches, the YCNN is actually trained to measure the similarity between the two image patches. Thus, this model will not be limited to any specific object. Furthermore, the network is end-to-end trained to extract both shallow and deep dedicated convolutional features for visual tracking. And once properly trained, the YCNN can be used to track all kinds of objects without further training and updating. As a result, our algorithm is able to run at a very high speed of 45 frames-per-second. The effectiveness of the proposed algorithm can also be proved by the experiments on two popular data sets: OTB-100 and VOT-2014.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deep feature-based tracking methods have higher accuracy.
“Deep featurebased tracking methods have gained the attention of the tracking community because of their higher accuracy [3,4].”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [3]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Yılmaz, A., Javed, O., & Shah, M. A. (2006) · ACM Computing Surveys · doi:10.1145/1177352.1177355 · Cited by 4,807 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Online Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138 · see citing works
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371 · cited 1× · see citing works
“The experiments are conducted on OTB-100 with 100 sequences [25], OTB-50 with 50 sequences [25], and VOT-2014 with 25 sequences [26].”
MethodIncremental Learning for Robust Visual Tracking
Ross, D. A., Lim, J., Lin, R.-S., & Yang, M.-H. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0075-7 · Cited by 3,061 · see citing works
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
Learning Spatially Regularized Correlation Filters for Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.490 · Cited by 2,171 · see citing works
Accurate Scale Estimation for Robust Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.65 · Cited by 2,145 · see citing works
Robust object tracking with online multiple instance learning
Babenko, B., Yang, M.-H., & Belongie, S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.226 · Cited by 2,007 · see citing works
Struck: Structured output tracking with kernels
Hare, S., Saffari, A., & Torr, P. H. S. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126251 · Cited by 1,831 · see citing works
Staple: Complementary Learners for Real-Time Tracking
Bertinetto, L., Valmadre, J., Golodetz, S., Mikšík, O., & Torr, P. H. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.156 · Cited by 1,825 · see citing works
Hierarchical Convolutional Features for Visual Tracking
Ma, C., Huang, J., Yang, X., & Yang, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.352 · Cited by 1,811 · see citing works
Visual Tracking: An Experimental Survey
Smeulders, A. W. M., Chu, D. M., Cucchiara, R., Calderara, S., Dehghan, A., & Shah, M. A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.230 · Cited by 1,527 · see citing works
Siamese Instance Search for Tracking
Tao, R., Gavves, E., & Smeulders, A. W. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.158 · Cited by 1,208 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization
Zhang, J., Ma, S., & Sclaroff, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_13 · Cited by 1,150 · see citing works
Semi-supervised On-Line Boosting for Robust Tracking
Gräbner, H., Leistner, C., & Bischof, H. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88682-2_19 · Cited by 1,095 · see citing works
Visual Tracking with Fully Convolutional Networks
Wang, L., Ouyang, W., Wang, X., & Lu, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.357 · Cited by 1,001 · see citing works
Robust object tracking via sparsity-based collaborative model
Zhong, W., Lu, H., & Yang, M.-H. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247882 · Cited by 990 · see citing works
Real time robust L1 tracker using accelerated proximal gradient approach
Bao, C., Wu, Y., Ling, H., & Ji, H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247881 · Cited by 839 · see citing works
Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural NetworkPreprint
Hong, S., You, T., Kwak, S., & Han, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.06796 · Cited by 784 · see citing works
Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., & Yang, M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.466 · Cited by 735 · see citing works
Fast Visual Tracking via Dense Spatio-temporal Context Learning
Zhang, K., Lei Zhang, A., Liu, Q., Zhang, D. D., & Yang, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_9 · Cited by 681 · see citing works
The Visual Object Tracking VOT2014 Challenge Results
Kristan, M., Pflugfelder, R. P., Leonardis, A., Matas, J., Zajc, L. Č., Nebehay, G., Vojíř, T., Fernández, G., Lukežič, A., Dimitriev, A., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_14 · Cited by 606 · see citing works
Transfer Learning Based Visual Tracking with Gaussian Processes Regression
Gao, J. G., Ling, H., Hu, W., & Xing, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_13 · Cited by 452 · see citing works
Notice of Removal: The Visual Object Tracking VOT2013 Challenge Results
Kristan, M., Pflugfelder, R. P., Leonardis, A., Matas, J., Porikli, F., Zajc, L. Č., Nebehay, G., Fernandez, G., Vojíř, T., Gatt, A., et al. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.20 · Cited by 310 · see citing works
Robust Visual Tracking using 1 Minimization
Mei, X., & Ling, H. (2009) · Cited by 286 · see citing works
STCT: Sequentially Training Convolutional Networks for Visual Tracking
Wang, L., Ouyang, W., Wang, X., & Lu, H. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.153 · Cited by 259 · see citing works
Li, H., Li, Y., & Porikli, F. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.56 · Cited by 138 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Deep visual tracking: Review and experimental comparison
Li, P., Wang, D., Wang, L., & Lu, H. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.11.007 · Cited by 517 · see citing works
A survey on online learning for visual tracking
Abbass, M. Y., Kwon, K.-C., Kim, N., Abdelwahab, S. A., El-Samie, F. E. A., & Khalaf, A. A. M. (2021) · The Visual Computer · doi:10.1007/s00371-020-01848-y · Cited by 64 · see citing works
Fast and Robust Object Tracking Using Tracking Failure Detection in Kernelized Correlation Filter
Shin, J., Kim, H., Kim, D., & Paik, J. (2020) · Applied Sciences · doi:10.3390/app10020713 · Cited by 42 · see citing works
Multiple Cues-Based Robust Visual Object Tracking Method
Khan, B., Jalil, A., Ali, A., Alkhaledi, K., Mehmood, K., Cheema, K. M., Murad, M., Tariq, H., & El-Sherbeeny, A. M. (2022) · Electronics · doi:10.3390/electronics11030345 · Cited by 5 · see citing works
“The solution to kernelized ridge regression is given by [3] as per Equation (1).”
BackgroundMethod
- Authors
- Chen, K., & Tao, W.
- Affiliations
- Huazhong University of Science and Technology (CN)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2018-12
- Volume
- 28
- Issue
- 12
- Pages
- 3377-3386
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W2424629859
- Cited by
- 94
- References
- 34
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 5 · 2022: 6 · 2021: 12 · 2020: 16 · 2019: 19 · 2018: 16 · 2017: 13 · 2016: 2
- FWCI
- 7.27
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61772213, 61371140, 2015CFA062, 2015BAA133, 2017010201010121)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- visual object tracking, patch similarity, response map, end-to-end learning, real-time tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
