End-to-End Representation Learning for Correlation Filter Based Tracking
Open accessAbstract
The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations. It is well suited to object tracking because its formulation in the Fourier domain provides a fast solution, enabling the detector to be re-trained once per frame. Previous works that use the Correlation Filter, however, have adopted features that were either manually designed or trained for a different task. This work is the first to overcome this limitation by interpreting the Correlation Filter learner, which has a closed-form solution, as a differentiable layer in a deep neural network. This enables learning deep features that are tightly coupled to the Correlation Filter. Experiments illustrate that our method has the important practical benefit of allowing lightweight architectures to achieve state-of-the-art performance at high framerates.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Self-citationFully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · see citing works
Self-citationOnline Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138 · see citing works
Visual object tracking using adaptive correlation filters
Bolme, D., Beveridge, J. R., Draper, B. A., & Lui, Y. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539960 · Cited by 3,406 · see citing works
Wu, Y., Lim, J., & Yang, M.-H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388226 · Cited by 3,371 · see citing works
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Nam, H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.465 · Cited by 2,806 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1× · see citing works
“When the solution to the optimization problem is obtained iteratively, an alternative is to treat the iterations as a Recurrent Neural Network, and to explicitly unroll a fixed number of iterations [36].”
MethodSelf-citationLearning Spatially Regularized Correlation Filters for Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.490 · Cited by 2,171 · see citing works
Accurate Scale Estimation for Robust Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.65 · Cited by 2,145 · see citing works
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
Danelljan, M., Robinson, A., Shahbaz Khan, F., & Felsberg, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_29 · Cited by 1,853 · see citing works
Staple: Complementary Learners for Real-Time Tracking
Bertinetto, L., Valmadre, J., Golodetz, S., Mikšík, O., & Torr, P. H. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.156 · Cited by 1,825 · see citing works
Self-citationHierarchical Convolutional Features for Visual Tracking
Ma, C., Huang, J., Yang, X., & Yang, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.352 · Cited by 1,811 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
Convolutional Features for Correlation Filter Based Visual Tracking
Danelljan, M., Hager, G., Khan, F. S., & Felsberg, M. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.84 · Cited by 1,002 · see citing works
Long-term correlation tracking
Ma, C., Yang, X., Zhang, C., & Yang, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299177 · Cited by 980 · see citing works
Encoding Color Information for Visual Tracking: Algorithms and Benchmark
Liang, P., Blasch, E., & Ling, H. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2482905 · Cited by 726 · see citing works
Learning by Tracking: Siamese CNN for Robust Target Association
Leal-Taixe, L., Canton-Ferrer, C., & Schindler, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.59 · Cited by 444 · see citing works
Learning feed-forward one-shot learnersPreprint
Bertinetto, L., Henriques, J. F., Valmadre, J., Torr, P. H. S., & Vedaldi, A. (2016) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1606.05233 · Cited by 393 · see citing works
Self-citationCorrelation filters with limited boundaries
Galoogahi, H. K., Sim, T., & Lucey, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299094 · Cited by 362 · see citing works
Transferring Rich Feature Hierarchies for Robust Visual TrackingPreprint
Wang, N., Li, S., Gupta, A. K., & Yeung, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.04587 · Cited by 293 · see citing works
Matrix Backpropagation for Deep Networks with Structured Layers
Ionescu, C., Vantzos, O., & Sminchisescu, C. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.339 · Cited by 287 · see citing works
Gould, S. J., Fernando, B., Cherian, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.05447 · Cited by 114 · see citing works
Maximum Margin Correlation Filter: A New Approach for Localization and Classification
Rodriguez, A., Boddeti, V. N., Kumar, B. V. K. V., et al. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2220151 · Cited by 94 · see citing works
Differentiation of the Cholesky decompositionPreprint
Murray, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07527 · Cited by 19 · see citing works
Deep Learning of Appearance Models for Online Object TrackingPreprint
Zhai, M., Roshtkhari, M. J., & Mori, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02568 · Cited by 12 · see citing works
Zero-aliasing correlation filters
Fernandez, J. A., & Kumar, B. V. K. V. (2013) · International symposium on image and signal processing and analysis/ISPA ... · doi:10.1109/ispa.2013.6703722 · Cited by 10 · see citing works
Once for All: a Two-flow Convolutional Neural Network for Visual TrackingPreprint
Chen, K., & Tao, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.07507 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- J. A. Fernandez and B. Vijayakumar. Zero-aliasing correla- tion filters. In International Symposium on Image and Signal Processing and Analysis 2013, pages 101-106, 2013. 2
- M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pflugfelder, L. Čehovin, T. Vojír, G. Häger, A. Lukežič, G. Fernández, et al. The Visual Object Tracking VOT2016 challenge results. In ECCV Workshops. Springer, 2016. 5
- Y. Li and J. Zhu. A scale adaptive kernel correlation filter tracker with feature integration. In ECCV, pages 254-265, 2014. 2, 7
- D. Maclaurin, D. Duvenaud, and R. P. Adams. Gradient- based hyperparameter optimization through reversible learn- ing. In ICML, 2015. 2
- R. Tao, E. Gavves, and A. W. M. Smeulders. Siamese in- stance search for tracking. In CVPR, pages 1420-1429, 2016. 1, 2, 7
- J. Valmadre, S. Sridharan, and S. Lucey. Learning detectors quickly with stationary statistics. In ACCV, pages 99-114. Springer, 2014. 3
- O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for one shot learning. In NIPS, pages 3630-3638, 2016. 2
- M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus. De- convolutional networks. In CVPR, pages 2528-2535, 2010.
Cited by
Sorted by citations
Detection and tracking meet drones challenge
Zhu, P., Wen, L., Du, D., Bian, X., Fan, H., Hu, Q., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119563 · Cited by 1,336 · see citing works
Learning Spatio-Temporal Transformer for Visual Tracking
Yan, B., Peng, H., Fu, J., Wang, D., & Lu, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01028 · Cited by 1,280 · see citing works
SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking
Guo, D., Wang, J., Cui, Y., Wang, Z., & Chen, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00630 · Cited by 984 · see citing works
Probabilistic Regression for Visual Tracking
Danelljan, M., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00721 · Cited by 798 · see citing works
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
Deep visual tracking: Review and experimental comparison
Li, P., Wang, D., Wang, L., & Lu, H. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.11.007 · Cited by 517 · see citing works
GradNet: Gradient-Guided Network for Visual Object Tracking
Li, P., Chen, B., Ouyang, W., Wang, D., Yang, X., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00626 · Cited by 348 · see citing works
GlobalTrack: A Simple and Strong Baseline for Long-Term Tracking
Huang, L., Zhao, X., & Huang, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6758 · Cited by 279 · see citing works
- Authors
- Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., & Torr, P. H. S.
- Affiliations
- University of Oxford (GB)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 5000-5008
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2608627404
- Cited by
- 274
- References
- 29
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 13 · 2023: 20 · 2022: 35 · 2021: 42 · 2020: 59 · 2019: 59 · 2018: 33 · 2017: 3
- FWCI
- 12.31
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/N019474/1, EP/M013774/1, EP/L024683/1, EP/M013774/, ERC-2012-AdG 321162-HELIOS, Seebibyte EP/M013774/1)
- FP7 Ideas: European Research Council (321162)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- correlation filter, object tracking, deep features, Fourier domain, lightweight tracker
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
