Learning Attentions: Residual Attentional Siamese Network for High Performance Online Visual Tracking
Abstract
Offline training for object tracking has recently shown great potentials in balancing tracking accuracy and speed. However, it is still difficult to adapt an offline trained model to a target tracked online. This work presents a Residual Attentional Siamese Network (RASNet) for high performance object tracking. The RASNet model reformulates the correlation filter within a Siamese tracking framework, and introduces different kinds of the attention mechanisms to adapt the model without updating the model online. In particular, by exploiting the offline trained general attention, the target adapted residual attention, and the channel favored feature attention, the RASNet not only mitigates the over-fitting problem in deep network training, but also enhances its discriminative capacity and adaptability due to the separation of representation learning and discriminator learning. The proposed deep architecture is trained from end to end and takes full advantage of the rich spatial temporal information to achieve robust visual tracking. Experimental results on two latest benchmarks, OTB-2015 and VOT2017, show that the RASNet tracker has the state-of-the-art tracking accuracy while runs at more than 80 frames per second.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Squeeze-and-Excitation NetworksPreprint
Hu, J., Shen, L., & Sun, G. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1709.01507 · Cited by 37,845 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · see citing works
CONDENSATION—Conditional Density Propagation for Visual Tracking
Isard, M., & Blake, A. (1998) · International Journal of Computer Vision · doi:10.1023/a:1008078328650 · Cited by 4,886 · see citing works
Yılmaz, A., Javed, O., & Shah, M. A. (2006) · ACM Computing Surveys · doi:10.1145/1177352.1177355 · Cited by 4,807 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Online Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138 · see citing works
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073 · see citing works
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406 · see citing works
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371 · cited 7× · see citing works
“By extensively training large deep networks on large datasets offline and aggressively learning the target sequence online, those trackers have obtained record-breaking results on all recent benchmarks [51, 52] and challenges [15, 16, 28].”
BackgroundMethodInfluentialLearning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
ECO: Efficient Convolution Operators for Tracking
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.733 · Cited by 2,805 · see citing works
MatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444 · see citing works
Accurate Scale Estimation for Robust Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.65 · Cited by 2,145 · see citing works
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
Danelljan, M., Robinson, A., Shahbaz Khan, F., & Felsberg, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_29 · Cited by 1,853 · see citing works
Hierarchical Convolutional Features for Visual Tracking
Ma, C., Huang, J., Yang, X., & Yang, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.352 · Cited by 1,811 · see citing works
Visual Tracking: An Experimental Survey
Smeulders, A. W. M., Chu, D. M., Cucchiara, R., Calderara, S., Dehghan, A., & Shah, M. A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.230 · Cited by 1,527 · see citing works
Learning Background-Aware Correlation Filters for Visual Tracking
Galoogahi, H. K., Fagg, A., & Lucey, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.129 · Cited by 1,270 · see citing works
Siamese Instance Search for Tracking
Tao, R., Gavves, E., & Smeulders, A. W. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.158 · Cited by 1,208 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
Discriminative Correlation Filter with Channel and Spatial Reliability
Lukezic, A., Vojir, T., Zajc, L. C., Matas, J., & Kristan, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.515 · Cited by 1,153 · see citing works
Multiple object tracking: A literature review
Luo, W., Xing, J., Milan, A., Zhang, X.-Q., Liu, W., Zhao, X., & Kim, T.-K. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2020.103448 · Cited by 1,095 · see citing works
Self-citationConvolutional Features for Correlation Filter Based Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.84 · Cited by 1,002 · see citing works
Visual Tracking with Fully Convolutional Networks
Wang, L., Ouyang, W., Wang, X., & Lu, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.357 · Cited by 1,001 · see citing works
Olshausen, B., Anderson, C., & Van Essen, D. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-11-04700.1993 · Cited by 908 · see citing works
Learning Dynamic Siamese Network for Visual Object Tracking
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., & Wang, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.196 · Cited by 904 · see citing works
The Visual Object Tracking VOT2015 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.79 · Cited by 895 · see citing works
Self-citationOnline Tracking by Learning Discriminative Saliency Map with Convolutional Neural NetworkPreprint
Hong, S., You, T., Kwak, S., & Han, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.06796 · Cited by 784 · see citing works
The Visual Object Tracking VOT2016 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin, L., Vojír, T., Häger, G., Lukežič, A., Fernandez, G., et al. (2016) · ECCV Workshops · doi:10.1007/978-3-319-48881-3_54 · Cited by 776 · see citing works
Self-citationA survey of appearance models in visual object tracking
Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., & Hengel, A. V. D. (2013) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2508037.2508039 · Cited by 705 · see citing works
Self-citationCREST: Convolutional Residual Learning for Visual Tracking
Song, Y., Ma, C., Gong, L., Zhang, J., Lau, R. W. H., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.279 · Cited by 572 · see citing works
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.159 · Cited by 440 · see citing works
Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., & Yu, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.518 · Cited by 406 · see citing works
Attentional Correlation Filter Network for Adaptive Visual Tracking
Choi, J., Chang, H. J., Yun, S., Fischer, T., Demiris, Y., & Choi, J. Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.513 · Cited by 319 · see citing works
The Visual Object Tracking VOT2017 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.230 · Cited by 319 · see citing works
Self-citationDCFNet: Discriminant Correlation Filters Network for Visual TrackingPreprint
Wang, Q., Gao, J. G., Xing, J., Zhang, M., & Hu, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04057 · Cited by 305 · see citing works
Self-citationLearning Spatial-Aware Regressions for Visual Tracking
Sun, C., Wang, D., Lu, H., & Yang, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00934 · Cited by 226 · see citing works
Keyframe-based tracking for rotoscoping and animation
Agarwala, A., Hertzmann, A., Salesin, D., et al. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015764 · Cited by 211 · see citing works
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos
Du, W., Wang, Y., & Qiao, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.402 · Cited by 199 · see citing works
Good Features to Correlate for Visual Tracking
Gündoğdu, E., & Alatan, A. A. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2806280 · Cited by 160 · see citing works
Correlation Filters with Weighted Convolution Responses
He, Z., Fan, Y., Zhuang, J., Dong, Y., & Bai, H. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.233 · Cited by 133 · see citing works
Discriminative Spatial Attention for Robust Tracking
Fan, J., Wu, Y. N., & Dai, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_35 · Cited by 96 · see citing works
Hand Posture Recognition Using Finger Geometric Feature
Liu, L., Xing, J., Ai, H., et al. (2014) · Cited by 67 · see citing works
Self-citationRobust Object Tracking Based on Temporal and Spatial Deep Networks
Zhu, T., Xing, J., Wang, Q., Lang, C., Feng, S., & Jin, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.130 · Cited by 62 · see citing works
Self-citationJoint Scale-Spatial Correlation Tracking with Adaptive Rotation Estimation
Zhang, M.-D., Xing, J., Gao, J., Shi, X., Wang, Q., & Hu, W.-M. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccvw.2015.81 · Cited by 58 · see citing works
Self-citationRobust visual tracking using joint scale-spatial correlation filters
Zhang, M., Xing, J., Gao, J. G., & Hu, W. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351044 · Cited by 37 · see citing works
Self-citationRole of Spatiotemporal Oriented Energy Features for Robust Visual Tracking in Video Surveillance
Emami, A., Dadgostar, F., Bigdeli, A., & Lovell, B. (2012) · 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance · doi:10.1109/avss.2012.64 · Cited by 27 · see citing works
Multi-view vehicle detection and tracking in crossroads
Liu, L., Xing, J., & Ai, H. (2011) · doi:10.1109/acpr.2011.6166688 · Cited by 14 · see citing works
Self-citationLearning Spatial-Aware Regressions for Visual TrackingPreprint
Sun, C., Wang, D., Lu, H., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07457 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [7][7] Z. Cui, S. Xiao, J. Feng, and S. Yan. Recurrently targetattending tracking. In IEEE Conference on Computer Vision and Pattern Recognition , June 2016. 2, 7 -
- [22][22] J. Henriques, R. Caseiro, P. Martins, and J. Batista. Highspeed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence , 37(3):583-596, 2015. 1, 3, 7 -
- [26][26] M. Jaderberg, K. Simonyan, A. Zisserman, and k. kavukcuoglu. Spatial transformer networks. In Advances in Neural Information Processing Systems , pages 2017-2025, 2015. 2 -
- [29][29] H. Li, Y. Li, and F. Porikli. Deeptrack: Learning discriminative feature representations online for robust visual tracking. IEEE Transactions on Image Processing , 25(4):1834-1848, 2016. 2 -
- [40][40] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations , 2015. 2 -
- [46][46] J. Valmadre, L. Bertinetto, J. F. Henriques, A. Vedaldi, and P. H. Torr. End-to-end representation learning for correlation filter based tracking. In IEEE Conference on Computer Vision and Pattern Recognition , 2017. 2, 3, 4, 7 -
Cited by
Sorted by citations
SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks
Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., & Yan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00441 · Cited by 2,607 · see citing works
Self-citationChen, X., Yan, B., Zhu, J., Wang, D., Yang, X., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00803 · Cited by 1,588 · see citing works
Siamese Box Adaptive Network for Visual Tracking
Chen, Z., Zhong, B., Li, G., Zhang, S., & Ji, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00670 · Cited by 1,052 · see citing works
Probabilistic Regression for Visual Tracking
Danelljan, M., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00721 · Cited by 798 · see citing works
Deformable Siamese Attention Networks for Visual Object Tracking
Yu, Y., Xiong, Y., Huang, W., & Scott, M. R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00676 · Cited by 495 · see citing works
Tracking by Instance Detection: A Meta-Learning Approach
Wang, G., Luo, C., Sun, X., Xiong, Z., & Zeng, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00632 · Cited by 206 · see citing works
Siam R-CNN: Visual Tracking by Re-Detection
Voigtlaender, P., Luiten, J., Torr, P. H. S., & Leibe, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00661 · Cited by 115 · see citing works
Learning Target-Oriented Dual Attention for Robust RGB-T Tracking
Yang, R., Zhu, Y., Wang, X., Li, C., & Tang, J. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803528 · Cited by 45 · see citing works
Towards Accurate Pixel-wise Object Tracking by Attention Retrieval
Zhang, Z., Li, B., Hu, W., & Peng, H. (2020) · arXiv · doi:10.48550/arxiv.2008.02745 · Cited by 32 · see citing works
Self-citation
- Authors
- Wang, Q., Teng, Z., Xing, J., Gao, J., Hu, W., & Maybank, S.
- Affiliations
- University of Chinese Academy of Sciences (CN)
- Chinese Academy of Sciences (CN)
- Birkbeck, University of London (GB)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 4854-4863
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2797812763
- Cited by
- 604
- References
- 53
- Citations by year
- 2026: 7 · 2025: 29 · 2024: 47 · 2023: 82 · 2022: 114 · 2021: 107 · 2020: 107 · 2019: 99 · 2018: 11 · 2017: 1
- FWCI
- 24.41
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- BIROn (Birkbeck, University of London) (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- attention mechanism, correlation filter, channel attention, overfitting reduction, high-speed tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
