Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking
Open accessAbstract
Discriminative Correlation Filters (DCF) are efficient in visual tracking but suffer from unwanted boundary effects. Spatially Regularized DCF (SRDCF) has been suggested to resolve this issue by enforcing spatial penalty on DCF coefficients, which, inevitably, improves the tracking performance at the price of increasing complexity. To tackle online updating, SRDCF formulates its model on multiple training images, further adding difficulties in improving efficiency. In this work, by introducing temporal regularization to SRDCF with single sample, we present our spatialtemporal regularized correlation filters (STRCF). The STRCF formulation can not only serve as a reasonable approximation to SRDCF with multiple training samples, but also provide a more robust appearance model than SRDCF in the case of large appearance variations. Besides, it can be efficiently solved via the alternating direction method of multipliers (ADMM). By incorporating both temporal and spatial regularization, our STRCF can handle boundary effects without much loss in efficiency and achieve superior performance over SRDCF in terms of accuracy and speed. Compared with SRDCF, STRCF with hand-crafted features provides a 5× speedup and achieves a gain of 5.4% and 3.6% AUC score on OTB-2015 and Temple-Color, respectively. Moreover, STRCF with deep features also performs favorably against state-of-the-art trackers and achieves an AUC score of 68.3% on OTB-2015.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
STRCF decreases computational cost while keeping promising performance.
“In order to decrease the computational cost while keeping the promising performance, a spatial-temporal regularized correlation filter tracking scheme (STRCF) is proposed in [28].”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
The proposed tracking scheme achieved the highest mean distance precision of 0.889 at threshold 20 pixels, outperforming LCT, MACF, MKCF, and STRCF.
“The proposed tracking scheme outperforms the other state-of-the-art algorithms by achieving the highest mean of 0.889.”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
STRCF achieved a mean overlap success rate of 0.799.
“while the remaining three trackers, i.e., STRCF [28], LCT [21], and MKCF [5], have a mean overlap success rate of 0.799, 0.717, and 0.645, respectively.”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
STRCF achieved a mean center location error of 44.89 pixels.
“The STRCF [28] scheme achieved the mean center location error of 44.89”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
The proposed tracker, along with MACF and STRCF, tracks the target successfully until the end of the car1 video sequence.
“Our proposed tracker, along with MACF [23] and STRCF [28], tracks the target successfully until the end of the video sequence.”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
In the suitcase_ce sequence, MKCF shows similar performance to the proposed algorithm, whereas MACF, STRCF, and LCT were unable to track the correct object.
“MKCF [5] showed a similar performance to our proposed algorithm on this sequence, whereas MACF [23], STRCF [28], and LCT [21] were unable to track the correct object.”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
In the microbike_ce sequence, STRCF and MACF fail to track the object in frame 500, while the proposed tracker tracks the target correctly until the end of the video sequence.
“While STRCF [28] and MACF [23] fail to track the object in frame number 500, our proposed tracker tracks the target correctly until the end of the video sequence.”
From Multiple Cues-Based Robust Visual Object Tracking Method (2022) · cites this paper as [28]
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406 · see citing works
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371 · cited 14× · see citing works
“One can see that DeepSTRCF ranks the second and outperforms DeepSRDCF with a margin of 5.2% on OTB-2015.”
BackgroundMethodInfluentialSelf-citationEckstein, J., & Bertsekas, D. P. (1992) · Mathematical Programming · doi:10.1007/bf01581204 · Cited by 2,922 · see citing works
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
ECO: Efficient Convolution Operators for Tracking
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.733 · Cited by 2,805 · see citing works
Learning Spatially Regularized Correlation Filters for Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.490 · Cited by 2,171 · see citing works
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
Danelljan, M., Robinson, A., Shahbaz Khan, F., & Felsberg, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_29 · Cited by 1,853 · see citing works
Staple: Complementary Learners for Real-Time Tracking
Bertinetto, L., Valmadre, J., Golodetz, S., Mikšík, O., & Torr, P. H. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.156 · Cited by 1,825 · see citing works
Hierarchical Convolutional Features for Visual Tracking
Ma, C., Huang, J., Yang, X., & Yang, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.352 · Cited by 1,811 · see citing works
Self-citationOnline Passive-Aggressive Algorithms
Crammer, K., Dekel, O., Keshet, J., Shalev‐Shwartz, S., & Singer, Y. (2003) · Cited by 1,698 · see citing works
Adaptive Color Attributes for Real-Time Visual Tracking
Danelljan, M., Khan, F. S., Felsberg, M., & van de Weijer, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.143 · Cited by 1,541 · see citing works
Petersen, K. B., & Pedersen, M. S. (2012) · doi:10.5281/zenodo.8002926 · Cited by 1,508 · see citing works
A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration
Li, Y., & Zhu, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_18 · Cited by 1,461 · see citing works
Discriminative Scale Space Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2609928 · Cited by 1,315 · see citing works
Learning Background-Aware Correlation Filters for Visual Tracking
Galoogahi, H. K., Fagg, A., & Lucey, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.129 · Cited by 1,270 · see citing works
MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization
Zhang, J., Ma, S., & Sclaroff, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_13 · Cited by 1,150 · see citing works
Convolutional Features for Correlation Filter Based Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.84 · Cited by 1,002 · see citing works
Visual Tracking with Fully Convolutional Networks
Wang, L., Ouyang, W., Wang, X., & Lu, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.357 · Cited by 1,001 · see citing works
Long-term correlation tracking
Ma, C., Yang, X., Zhang, C., & Yang, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299177 · Cited by 980 · see citing works
Self-citationOnline Passive-Aggressive Algorithms
CrammerKoby, DekelOfer, KeshetJoseph, Shalev-ShwartzShai, & SingerYoram (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248566 · Cited by 914 · see citing works
The Visual Object Tracking VOT2016 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin, L., Vojír, T., Häger, G., Lukežič, A., Fernandez, G., et al. (2016) · ECCV Workshops · doi:10.1007/978-3-319-48881-3_54 · Cited by 776 · see citing works
Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., & Yang, M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.466 · Cited by 735 · see citing works
Self-citationEncoding Color Information for Visual Tracking: Algorithms and Benchmark
Liang, P., Blasch, E., & Ling, H. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2482905 · Cited by 726 · see citing works
Large Margin Object Tracking with Circulant Feature Maps
Wang, M., Liu, Y., & Huang, Z. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.510 · Cited by 665 · see citing works
Context-Aware Correlation Filter Tracking
Mueller, M. F., Smith, N., & Ghanem, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.152 · Cited by 662 · see citing works
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.159 · Cited by 440 · see citing works
Multi-task Correlation Particle Filter for Robust Object Tracking
Zhang, T., Xu, C., & Yang, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.512 · Cited by 433 · see citing works
Self-citationReal-time part-based visual tracking via adaptive correlation filters
Liu, T., Wang, G., & Yang, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299124 · Cited by 402 · see citing works
Correlation filters with limited boundaries
Galoogahi, H. K., Sim, T., & Lucey, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299094 · Cited by 362 · see citing works
The Visual Object Tracking VOT2017 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.230 · Cited by 319 · see citing works
Self-citationModeling and Propagating CNNs in a Tree Structure for Visual TrackingPreprint
Nam, H., Baek, M., & Han, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07242 · Cited by 280 · see citing works
Multi-channel Correlation Filters
Galoogahi, H. K., Sim, T., & Lucey, S. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.381 · Cited by 266 · see citing works
Visual Object Tracking Performance Measures Revisited
Zajc, L. Č., Leonardis, A., & Kristan, M. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2520370 · Cited by 178 · see citing works
Structural Correlation Filter for Robust Visual Tracking
Liu, S., Zhang, T., Cao, X., & Xu, C. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.467 · Cited by 165 · see citing works
Target Response Adaptation for Correlation Filter Tracking
Bibi, A., Mueller, M. F., & Ghanem, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_25 · Cited by 161 · see citing works
Recurrently Target-Attending Tracking
Cui, Z., Xiao, S., Feng, J., & Yan, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.161 · Cited by 153 · see citing works
Learning Support Correlation Filters for Visual Tracking
Zuo, W., Wu, X., Lin, L., Lei Zhang, A., & Yang, M. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2829180 · Cited by 121 · see citing works
Self-citationIntegrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation
Li, F., Yao, Y., Li, P., Zhang, D. D., Zuo, W., & Yang, M. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.234 · Cited by 117 · see citing works
Self-citationLearning Support Correlation Filters for Visual TrackingPreprint
Zuo, W., Wu, X., Lin, L., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06032 · Cited by 26 · see citing works
Self-citationRobust and real-time deep tracking via multi-scale domain adaptation
Wang, X., Li, H., Li, Y., Shen, F., & Porikli, F. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019450 · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking
Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., & Ling, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00552 · Cited by 1,742 · see citing works
“…deep features ( e.g ., HCFT [37] and ECO [7]) and regularization techniques ( e.g ., BACF [15], SRDCF [10], CSRDCF [36], Staple CA [40] and STRCF [29]), ensemble trackers ( e.g ., PTAV [13], LCT [38], MEEM [56] and TLD [24]), sparse trackers ( e.g ., L1APG [2] and ASLA [23]), other…”
BackgroundLaSOT: A High-quality Large-scale Single Object Tracking Benchmark
Fan, H., Bai, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Harshit, Huang, M., Liu, J., et al. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01387-y · Cited by 360 · see citing works
LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search
Yan, B., Peng, H., Wu, K., Wang, D., Fu, J., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01493 · Cited by 277 · see citing works
Siam R-CNN: Visual Tracking by Re-Detection
Voigtlaender, P., Luiten, J., Torr, P. H. S., & Leibe, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00661 · Cited by 115 · see citing works
Multiple Cues-Based Robust Visual Object Tracking Method
Khan, B., Jalil, A., Ali, A., Alkhaledi, K., Mehmood, K., Cheema, K. M., Murad, M., Tariq, H., & El-Sherbeeny, A. M. (2022) · Electronics · doi:10.3390/electronics11030345 · Cited by 5 · see citing works
“The last row of Table 4 shows the mean center location error of the proposed tracker, LCT [21], MACF [23], MKCF [5], and STRCF [28].”
BackgroundMethodResultInfluential
- Authors
- Li, F., Tian, C., Zuo, W., Zhang, L., & Yang, M.-H.
- Affiliations
- Harbin Institute of Technology (CN)
- University of California, Merced (US)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 4904-4913
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964069521
- Cited by
- 935
- References
- 44
- Citations by year
- 2026: 29 · 2025: 86 · 2024: 100 · 2023: 127 · 2022: 169 · 2021: 162 · 2020: 158 · 2019: 93 · 2018: 11
- FWCI
- 31.46
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61671182, 61471146)
- Hong Kong Polytechnic University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- visual object tracking, discriminative correlation filter, spatial regularization, temporal regularization, edge effects, alternating direction method of multipliers, deep features, appearance model
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
