Deformable Siamese Attention Networks for Visual Object Tracking
Open accessAbstract
Siamese-based trackers have achieved excellent performance on visual object tracking. However, the target template is not updated online, and the features of target template and search image are computed independently in a Siamese architecture. In this paper, we propose Deformable Siamese Attention Networks, referred to as SiamAttn, by introducing a new Siamese attention mechanism that computes deformable self-attention and cross-attention. The self-attention learns strong context information via spatial attention, and selectively emphasizes interdependent channel-wise features with channel attention. The crossattention is capable of aggregating rich contextual interdependencies between the target template and the search image, providing an implicit manner to adaptively update the target template. In addition, we design a region refinement module that computes depth-wise cross correlations between the attentional features for more accurate tracking. We conduct experiments on six benchmarks, where our method achieves new state-of-the-art results, outperforming recent strong baseline, SiamRPN++, by 0.464 to 0.537 and 0.415 to 0.470 EAO on VOT 2016 and 2018.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · see citing works
Self-citationDual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Online Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138 · see citing works
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406 · see citing works
Kalal, Z., Mikolajczyk, K., & Matas, J. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.239 · Cited by 3,357 · see citing works
High Performance Visual Tracking with Siamese Region Proposal Network
Li, B., Yan, J., Wu, W., Zhu, Z., & Hu, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00935 · Cited by 3,023 · see citing works
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
ECO: Efficient Convolution Operators for Tracking
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.733 · Cited by 2,805 · see citing works
SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks
Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., & Yan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00441 · Cited by 2,607 · see citing works
A Benchmark and Simulator for UAV Tracking
Mueller, M., Smith, N., & Ghanem, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_27 · Cited by 1,944 · see citing works
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
Danelljan, M., Robinson, A., Shahbaz Khan, F., & Felsberg, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_29 · Cited by 1,853 · see citing works
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking
Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., & Ling, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00552 · Cited by 1,742 · cited 4× · see citing works
“We use ResNet-50, pre-trained on ImageNet [9], as the backbone, and the whole networks are then fine-tuned on the training sets of COCO [27], YouTube-VOS [39], LaSOT [10] and TrackingNet [30].”
BackgroundMethodInfluentialATOM: Accurate Tracking by Overlap Maximization
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00479 · Cited by 1,591 · see citing works
Distractor-Aware Siamese Networks for Visual Object Tracking
Zhu, Z. D., Wang, Q., Li, B., Wu, W., Yan, J., & Hu, W. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_7 · Cited by 1,575 · see citing works
Fast Online Object Tracking and Segmentation: A Unifying Approach
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00142 · Cited by 1,554 · see citing works
Learning Discriminative Model Prediction for Tracking
Bhat, G., Danelljan, M., Van Gool, L., & Timofte, R. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00628 · Cited by 1,515 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild
Müller, M., Bibi, A., Giancola, S., Alsubaihi, S., & Ghanem, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_19 · Cited by 1,192 · see citing works
Deeper and Wider Siamese Networks for Real-Time Visual Tracking
Zhang, Z., & Peng, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00472 · Cited by 1,160 · see citing works
Learning Dynamic Siamese Network for Visual Object Tracking
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., & Wang, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.196 · Cited by 904 · see citing works
The Sixth Visual Object Tracking VOT2018 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Bhat, G., Lukežič, A., Eldesokey, A., et al. (2018) · ECCV Workshops · doi:10.1007/978-3-030-11009-3_1 · Cited by 805 · see citing works
The Visual Object Tracking VOT2016 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin, L., Vojír, T., Häger, G., Lukežič, A., Fernandez, G., et al. (2016) · ECCV Workshops · doi:10.1007/978-3-319-48881-3_54 · Cited by 776 · see citing works
YouTube-VOS: Sequence-to-Sequence Video Object Segmentation
Xu, N., Yang, L., Fan, Y., Yang, J., Yue, D., Liang, Y., Price, B. L., Cohen, S. D., & Huang, T. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_36 · Cited by 618 · see citing works
Wang, Q., Teng, Z., Xing, J., Gao, J., Hu, W., & Maybank, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00510 · Cited by 604 · see citing works
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
Real, E., Shlens, J., Mazzocchi, S., Pan, X., & Vanhoucke, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.789 · Cited by 580 · see citing works
Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking
Fan, H., & Ling, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00814 · Cited by 477 · see citing works
Learning the Model Update for Siamese Trackers
Zhang, L., Gonzalez-Garcia, A., Weijer, J. V. D., Danelljan, M., & Khan, F. S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00411 · Cited by 475 · see citing works
Learning Aberrance Repressed Correlation Filters for Real-Time UAV Tracking
Huang, Z., Fu, C., Li, Y., Lin, F., & Lu, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00298 · Cited by 436 · see citing works
Learning Dynamic Memory Networks for Object Tracking
Yang, T., & Chan, A. B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_10 · Cited by 382 · see citing works
GradNet: Gradient-Guided Network for Visual Object Tracking
Li, P., Chen, B., Ouyang, W., Wang, D., Yang, X., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00626 · Cited by 348 · see citing works
End-to-End Flow Correlation Tracking with Spatial-Temporal Attention
Zhu, Z. D., Wu, W., Zou, W., & Yan, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00064 · Cited by 321 · see citing works
SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking
Wang, G., Luo, C., Xiong, Z., & Zeng, W. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00376 · Cited by 269 · see citing works
Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking
Xu, T., Feng, Z., Wu, X., & Kittler, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00804 · Cited by 232 · see citing works
On-Road Pedestrian Tracking Across Multiple Driving Recorders
Lee, K.-H., & Hwang, J. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2455418 · Cited by 139 · see citing works
Deep Meta Learning for Real-Time Target-Aware Visual Tracking
Choi, J., Kwon, J., & Lee, K. M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00100 · Cited by 117 · see citing works
Deeper and Wider Siamese Networks for Real-Time Visual TrackingPreprint
Zhang, Z., & Peng, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01660 · Cited by 110 · see citing works
Distractor-aware Siamese Networks for Visual Object TrackingPreprint
Zhu, Z. D., Wang, Q., Li, B., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.06048 · Cited by 85 · see citing works
Hand Posture Recognition Using Finger Geometric Feature
Liu, L., Xing, J., Ai, H., et al. (2014) · Cited by 67 · see citing works
TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the WildPreprint
Müller, M., Bibi, A., Giancola, S., Alsubaihi, S., & Ghanem, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10794 · Cited by 53 · see citing works
GradNet: Gradient-Guided Network for Visual Object TrackingPreprint
Li, P., Chen, B., Ouyang, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06800 · Cited by 49 · see citing works
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in VideoPreprint
Real, E., Shlens, J., Mazzocchi, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.00824 · Cited by 45 · see citing works
Learning Aberrance Repressed Correlation Filters for Real-Time UAV TrackingPreprint
Huang, Z., Fu, C., Li, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.02231 · Cited by 40 · see citing works
YouTube-VOS: Sequence-to-Sequence Video Object SegmentationPreprint
Xu, N., Yang, L., Fan, Y., Yang, J., Yue, D., Liang, Y., Price, B. L., Cohen, S. D., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00461 · Cited by 24 · see citing works
Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object TrackingPreprint
Xu, T., Feng, Z., Wu, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13242 · Cited by 22 · see citing works
SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object TrackingPreprint
Wang, G., Luo, C., Xiong, Z., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04452 · Cited by 17 · see citing works
Learning the Model Update for Siamese TrackersPreprint
Zhang, L., González-García, A., van de Weijer, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00855 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00803 · Cited by 1,588 · see citing works
“SiamAttn [46] explores both self-attention and cross branch attention to improve the discriminative ability of target features before applying depth-wise cross correlation.”
BackgroundMethodInfluentialLearning Spatio-Temporal Transformer for Visual Tracking
Yan, B., Peng, H., Fu, J., Wang, D., & Lu, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01028 · Cited by 1,280 · see citing works
- Authors
- Yu, Y., Xiong, Y., Huang, W., & Scott, M. R.
- Affiliations
- MSIGHT Technologies (China) (CN)
- ORCID
- Huang, Weilin
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 6727-6736
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035211844
- Cited by
- 495
- References
- 55
- Citations by year
- 2026: 18 · 2025: 62 · 2024: 102 · 2023: 114 · 2022: 128 · 2021: 64 · 2020: 7
- FWCI
- 24.14
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- visual object tracking, cross-attention, channel attention, spatial attention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
