High Performance Visual Tracking with Siamese Region Proposal Network
Abstract
Visual object tracking has been a fundamental topic in recent years and many deep learning based trackers have achieved state-of-the-art performance on multiple benchmarks. However, most of these trackers can hardly get top performance with real-time speed. In this paper, we propose the Siamese region proposal network (Siamese-RPN) which is end-to-end trained off-line with large-scale image pairs. Specifically, it consists of Siamese subnetwork for feature extraction and region proposal subnetwork including the classification branch and regression branch. In the inference phase, the proposed framework is formulated as a local one-shot detection task. We can pre-compute the template branch of the Siamese subnetwork and formulate the correlation layers as trivial convolution layers to perform online tracking. Benefit from the proposal refinement, traditional multi-scale test and online fine-tuning can be discarded. The Siamese-RPN runs at 160 FPS while achieving leading performance in VOT2015, VOT2016 and VOT2017 real-time challenges.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
(2016) · Cited by 31,590 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · see citing works
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
R-FCN: Object Detection via Region-based Fully Convolutional NetworksPreprint
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 5,135 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406 · see citing works
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371 · cited 3× · see citing works
“OTB2015 [37] contains 100 sequences that are collected from commonly used tracking sequences.”
BackgroundMethodOne-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029 · see citing works
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
ECO: Efficient Convolution Operators for Tracking
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.733 · Cited by 2,805 · see citing works
Learning Spatially Regularized Correlation Filters for Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.490 · Cited by 2,171 · see citing works
Accurate Scale Estimation for Robust Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.65 · Cited by 2,145 · see citing works
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
Danelljan, M., Robinson, A., Shahbaz Khan, F., & Felsberg, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_29 · Cited by 1,853 · see citing works
Staple: Complementary Learners for Real-Time Tracking
Bertinetto, L., Valmadre, J., Golodetz, S., Mikšík, O., & Torr, P. H. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.156 · Cited by 1,825 · see citing works
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504 · see citing works
Siamese Instance Search for Tracking
Tao, R., Gavves, E., & Smeulders, A. W. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.158 · Cited by 1,208 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
Discriminative Correlation Filter with Channel and Spatial Reliability
Lukezic, A., Vojir, T., Zajc, L. C., Matas, J., & Kristan, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.515 · Cited by 1,153 · see citing works
The Visual Object Tracking VOT2015 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.79 · Cited by 895 · see citing works
Self-citationThe Visual Object Tracking VOT2016 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin, L., Vojír, T., Häger, G., Lukežič, A., Fernandez, G., et al. (2016) · ECCV Workshops · doi:10.1007/978-3-319-48881-3_54 · Cited by 776 · see citing works
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
Real, E., Shlens, J., Mazzocchi, S., Pan, X., & Vanhoucke, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.789 · Cited by 580 · see citing works
CREST: Convolutional Residual Learning for Visual Tracking
Song, Y., Ma, C., Gong, L., Zhang, J., Lau, R. W. H., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.279 · Cited by 572 · see citing works
Multiple People Tracking by Lifted Multicut and Person Re-identification
Tang, S., Andriluka, M., Andres, B., & Schiele, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.394 · Cited by 558 · see citing works
Learning feed-forward one-shot learnersPreprint
Bertinetto, L., Henriques, J. F., Valmadre, J., Torr, P. H. S., & Vedaldi, A. (2016) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1606.05233 · Cited by 393 · see citing works
Parallel Tracking and Verifying: A Framework for Real-Time and High Accuracy Visual Tracking
Fan, H., & Ling, H. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.585 · Cited by 330 · see citing works
End-to-End Flow Correlation Tracking with Spatial-Temporal Attention
Zhu, Z. D., Wu, W., Zou, W., & Yan, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00064 · Cited by 321 · see citing works
Self-citationThe Visual Object Tracking VOT2017 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.230 · Cited by 319 · see citing works
Self-citationDCFNet: Discriminant Correlation Filters Network for Visual TrackingPreprint
Wang, Q., Gao, J. G., Xing, J., Zhang, M., & Hu, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04057 · Cited by 305 · see citing works
On-Road Pedestrian Tracking Across Multiple Driving Recorders
Lee, K.-H., & Hwang, J. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2455418 · Cited by 139 · see citing works
UCT: Learning Unified Convolutional Networks for Real-Time Visual Tracking
Zhu, Z. D., Huang, G., Zou, W., Du, D., & Huang, C. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.231 · Cited by 82 · see citing works
Self-citationRe3 : Real-Time Recurrent Regression Networks for Object Tracking.Preprint
Gordon, D., Farhadi, A., & Fox, D. (2017) · arXiv (Cornell University) · Cited by 26 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks
Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., & Yan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00441 · Cited by 2,607 · see citing works
Self-citationChen, X., Yan, B., Zhu, J., Wang, D., Yang, X., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00803 · Cited by 1,588 · see citing works
“For most of the popular trackers (such as SiamFC [1], SiamRPN [22], and ATOM [9]), correlation plays a critical role in integrating the template or target information into the regions of interest (ROI).”
BackgroundMethodInfluentialLearning Spatio-Temporal Transformer for Visual Tracking
Yan, B., Peng, H., Fu, J., Wang, D., & Lu, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01028 · Cited by 1,280 · see citing works
“Most of offline Siamese trackers [2, 26, 25, 60, 29] belong to the spatial-only ones, which consider the object tracking as a template-matching between the initial template and the current search region.”
BackgroundMethodDeeper and Wider Siamese Networks for Real-Time Visual Tracking
Zhang, Z., & Peng, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00472 · Cited by 1,160 · see citing works
Siamese Box Adaptive Network for Visual Tracking
Chen, Z., Zhong, B., Li, G., Zhang, S., & Ji, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00670 · Cited by 1,052 · see citing works
SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking
Guo, D., Wang, J., Cui, Y., Wang, Z., & Chen, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00630 · Cited by 984 · see citing works
SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines
Xu, Y., Wang, Z., Li, Z., Yuan, Y., & Yu, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6944 · Cited by 974 · see citing works
Probabilistic Regression for Visual Tracking
Danelljan, M., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00721 · Cited by 798 · see citing works
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793 · see citing works
Deformable Siamese Attention Networks for Visual Object Tracking
Yu, Y., Xiong, Y., Huang, W., & Scott, M. R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00676 · Cited by 495 · see citing works
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking
Dendorfer, P., Os̆ep, A., Milan, A., Schindler, K., Cremers, D., Reid, I., Roth, S., & Leal-Taixé, L. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01393-0 · Cited by 374 · see citing works
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark
Fan, H., Bai, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Harshit, Huang, M., Liu, J., et al. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01387-y · Cited by 360 · see citing works
GradNet: Gradient-Guided Network for Visual Object Tracking
Li, P., Chen, B., Ouyang, W., Wang, D., Yang, X., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00626 · Cited by 348 · see citing works
D3S – A Discriminative Single Shot Segmentation Tracker
Lukezic, A., Matas, J., & Kristan, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00716 · Cited by 300 · see citing works
GlobalTrack: A Simple and Strong Baseline for Long-Term Tracking
Huang, L., Zhao, X., & Huang, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6758 · Cited by 279 · see citing works
LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search
Yan, B., Peng, H., Wu, K., Wang, D., Fu, J., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01493 · Cited by 277 · see citing works
‘Skimming-Perusal’ Tracking: A Framework for Real-Time and Robust Long-Term Tracking
Yan, B., Zhao, H., Wang, D., Lu, H., & Yang, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00247 · Cited by 218 · see citing works
Tracking by Instance Detection: A Meta-Learning Approach
Wang, G., Luo, C., Sun, X., Xiong, Z., & Zeng, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00632 · Cited by 206 · see citing works
Siam R-CNN: Visual Tracking by Re-Detection
Voigtlaender, P., Luiten, J., Torr, P. H. S., & Leibe, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00661 · Cited by 115 · see citing works
Learning Multi-Object Tracking and Segmentation From Automatic Annotations
Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulo, S. R., & Kontschieder, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00688 · Cited by 82 · see citing works
A survey on online learning for visual tracking
Abbass, M. Y., Kwon, K.-C., Kim, N., Abdelwahab, S. A., El-Samie, F. E. A., & Khalaf, A. A. M. (2021) · The Visual Computer · doi:10.1007/s00371-020-01848-y · Cited by 64 · see citing works
Towards Accurate Pixel-wise Object Tracking by Attention Retrieval
Zhang, Z., Li, B., Hu, W., & Peng, H. (2020) · arXiv · doi:10.48550/arxiv.2008.02745 · Cited by 32 · see citing works
Self-citation
- Authors
- Li, B., Yan, J., Wu, W., Zhu, Z., & Hu, X.
- Affiliations
- Beihang University (CN)
- Tsinghua University (CN)
- Sensetime (China) (CN)
- University of Chinese Academy of Sciences (CN)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 8971-8980
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2799058067
- Cited by
- 3,023
- References
- 37
- Citations by year
- 2026: 148 · 2025: 347 · 2024: 441 · 2023: 517 · 2022: 514 · 2021: 469 · 2020: 372 · 2019: 206 · 2018: 8 · 2017: 1
- FWCI
- 79.51
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- visual object tracking, Siamese network, region proposal, real-time tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
