CoChat
Sign Up
Cited by 1,554Video Surveillance and Tracking MethodsAdvanced Neural Network ApplicationsVisual Attention and Saliency Detection

Fast Online Object Tracking and Segmentation: A Unifying Approach

Open access
Wang, Qiang, Zhang, Li, Bertinetto, Luca, Hu, Weiming, Torr, Philip H.S.·2019·2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr.2019.00142
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach. Our method, dubbed SiamMask, improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task. Once trained, SiamMask solely relies on a single bounding box initialisation and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second. Despite its simplicity, versatility and fast speed, our strategy allows us to establish a new state-of-the-art among real-time trackers on VOT-2018, while at the same time demonstrating competitive performance and the best speed for the semi-supervised video object segmentation task on DAVIS-2016 and DAVIS-2017.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr.2019.00142

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Previous models have to improve nonlinear representation ability by introducing fashion structures, using additional modules, or designing effective online updaters.

    “Thus, previous models have to improve the nonlinear representation ability by introducing fashion structures [21, 44, 48], using additional modules [7, 46, 13], designing effective online updaters [2, 47, 10], to name a few.”

    From Transformer Tracking (2021) · cites this paper as [47]

    The Two variants section says the authors augment existing trackers with a segmentation branch, and the Mask representation section describes refinement modules used to produce a more accurate object mask. This supports the cited example of adding modules, but the paper does not establish that those modules improve nonlinear representation ability specifically. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • LTMU B takes the combination of ATOM and SiamMask as the short-term tracker, MDNet as the verifier, and GlobalTrack as the global detector.

    “To be specific, LTMU B takes the combination of ATOM [9] and SiamMask [50] as the short-term tracker, MDNet [37] as the verifier, and GlobalTrack [19] as the global detector.”

    From Learning Spatio-Temporal Transformer for Visual Tracking (2021) · cites this paper as [50]

  • Previous long-term tracking methods usually consist of multiple components, such as base trackers, target verification modules, and global detectors.

    “Specifically, previous methods usually consist of multiple components, such as base trackers [9, 50], target verification modules [21], and global detectors [40, 19].”

    From Learning Spatio-Temporal Transformer for Visual Tracking (2021) · cites this paper as [50]

    The cited paper’s Related Work section discusses online classifiers, correlation filters, and Siamese trackers, but does not describe global detectors as components of previous long-term tracking methods. Searches for “global detector” and “long-term tracking re-detection target verification” found no passage addressing that attributed point; no page number was available in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 48

  1. L. Bao, B. Wu, and W. Liu. Cnn in mrf: Video object seg- mentation via inference in a cnn-based higher-order spatio- temporal mrf. In IEEE Conference on Computer Vision and Pattern Recognition, 2018. 2, 3, 6
  2. L. Bertinetto, J. F. Henriques, J. Valmadre, P. H. S. Torr, and A. Vedaldi. Learning feed-forward one-shot learners. In Ad- vances in Neural Information Processing Systems, 2016. 3
  3. D. S. Bolme, J. R. Beveridge, B. A. Draper, and Y. M. Lui. Visual object tracking using adaptive correlation filters. In IEEE Conference on Computer Vision and Pattern Recogni- tion, 2010. 2
  4. S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool. One-shot video object seg- mentation. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 7
  5. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully con- nected crfs. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2018. 5, 11
  6. J. Cheng, Y.-H. Tsai, W.-C. Hung, S. Wang, and M.-H. Yang. Fast and accurate online video object segmentation via track- ing parts. In IEEE Conference on Computer Vision and Pat- tern Recognition, 2018. 2, 3, 6, 7
  7. J. Cheng, Y.-H. Tsai, S. Wang, and M.-H. Yang. Segflow: Joint learning for video object segmentation and optical flow. In IEEE International Conference on Computer Vision, 2017. 3, 7
  8. D. Comaniciu, V. Ramesh, and P. Meer. Real-time tracking of non-rigid objects using mean shift. In IEEE Conference on Computer Vision and Pattern Recognition, 2000. 2
  9. M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg. Learn- ing spatially regularized correlation filters for visual track- ing. In IEEE International Conference on Computer Vision, 2015. 2, 5
  10. A. He, C. Luo, X. Tian, and W. Zeng. Towards a better match in siamese network based visual object tracker. In European Conference on Computer Vision workshops, 2018. 2, 6, 7
  11. A. He, C. Luo, X. Tian, and W. Zeng. A twofold siamese network for real-time object tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 2018. 3
  12. K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r- cnn. In IEEE International Conference on Computer Vision, 2017. 4
  13. Y.-T. Hu, J.-B. Huang, and A. G. Schwing. Videomatch: Matching based video object segmentation. In European Conference on Computer Vision, 2018. 2, 3
  14. V. Jampani, R. Gadde, and P. V. Gehler. Video propagation networks. In IEEE Conference on Computer Vision and Pat- tern Recognition, 2017. 2, 3, 7
  15. A. Khoreva, R. Benenson, E. Ilg, T. Brox, and B. Schiele. Lucid data dreaming for object tracking. In IEEE Con- ference on Computer Vision and Pattern Recognition work- shops, 2017. 2, 3, 6
  16. H. Kiani Galoogahi, T. Sim, and S. Lucey. Multi-channel correlation filters. In IEEE International Conference on Computer Vision, 2013. 2
  17. H. Kiani Galoogahi, T. Sim, and S. Lucey. Correlation filters with limited boundaries. In IEEE Conference on Computer Vision and Pattern Recognition, 2015. 2
  18. M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pflugfelder, L. Čehovin, T. Vojír, G. Häger, A. Lukežič, G. Fernández, et al. The visual object tracking vot2016 chal- lenge results. In European Conference on Computer Vision, 2016. 1, 3, 5
  19. M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pfugfelder, L. C. Zajc, T. Vojir, G. Bhat, A. Lukezic, A. Eldesokey, G. Fernandez, and et al. The sixth visual object tracking vot-2018 challenge results. In European Conference on Computer Vision workshops, 2018. 1, 3, 5, 8, 12
  20. B. Li, J. Yan, W. Wu, Z. Zhu, and X. Hu. High performance visual tracking with siamese region proposal network. In IEEE Conference on Computer Vision and Pattern Recogni- tion, 2018. 2, 3, 4, 5, 7
  21. F. Li, C. Tian, W. Zuo, L. Zhang, and M.-H. Yang. Learn- ing spatial-temporal regularized correlation filters for visual tracking. In IEEE Conference on Computer Vision and Pat- tern Recognition, 2018. 2, 6, 7
  22. X. Li and C. C. Loy. Video object segmentation with joint re-identification and attention-aware mask propagation. In European Conference on Computer Vision, 2018. 2, 3, 6
  23. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. In European Conference on Com- puter Vision, 2014. 5
  24. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, 2015. 4
  25. A. Lukezic, T. Vojir, L. C. Zajc, J. Matas, and M. Kristan. Discriminative correlation filter with channel and spatial reli- ability. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2, 5, 6, 7
  26. K.-K. Maninis, S. Caelles, Y. Chen, J. Pont-Tuset, L. Leal- Taixé, D. Cremers, and L. Van Gool. Video object segmen- tation without temporal information. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. 2, 3, 6
  27. N. Märki, F. Perazzi, O. Wang, and A. Sorkine-Hornung. Bi- lateral space video segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. 2, 3, 6
  28. O. Miksik, J.-M. Pérez-Rúa, P. H. Torr, and P. Pérez. Roam: a rich object appearance model with application to rotoscop- ing. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1
  29. F. Perazzi. Video Object Segmentation. PhD thesis, ETH Zurich, 2017. 1, 3, 6
  30. F. Perazzi, A. Khoreva, R. Benenson, B. Schiele, and A. Sorkine-Hornung. Learning video object segmentation from static images. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2, 3, 6, 7
  31. F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In IEEE Conference on Computer Vision and Pattern Recog- nition, 2017. 1, 3, 6, 7, 8, 13
  32. F. Perazzi, O. Wang, M. Gross, and A. Sorkine-Hornung. Fully connected object proposals for video segmentation. In IEEE International Conference on Computer Vision, 2015. 3
  33. P. Pérez, C. Hue, J. Vermaak, and M. Gangnet. Color-Based Probabilistic Tracking. In European Conference on Com- puter Vision, 2002. 2
  34. P. O. Pinheiro, R. Collobert, and P. Dollár. Learning to seg- ment object candidates. In Advances in Neural Information Processing Systems, 2015. 2, 4
  35. P. O. Pinheiro, T.-Y. Lin, R. Collobert, and P. Dollár. Learn- ing to refine object segments. In European Conference on Computer Vision, 2016. 4, 7, 11
  36. J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine- Hornung, and L. Van Gool. The 2017 davis chal- lenge on video object segmentation. arXiv preprint arXiv:1704.00675, 2017. 6, 8, 13 · arXiv
  37. A. W. Smeulders, D. M. Chu, R. Cucchiara, S. Calderara, A. Dehghan, and M. Shah. Visual tracking: An experimental survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014. 1, 3
  38. R. Tao, E. Gavves, and A. W. Smeulders. Siamese instance search for tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. 2
  39. Y.-H. Tsai, M.-H. Yang, and M. J. Black. Video segmenta- tion via object flow. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. 2, 3, 6
  40. J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, and P. H. S. Torr. End-to-end representation learning for correla- tion filter based tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2
  41. J. Valmadre, L. Bertinetto, J. F. Henriques, R. Tao, A. Vedaldi, A. Smeulders, P. H. S. Torr, and E. Gavves. Long-term tracking in the wild: A benchmark. In European Conference on Computer Vision, 2018. 1
  42. L. Wen, D. Du, Z. Lei, S. Z. Li, and M.-H. Yang. Jots: Joint online tracking and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, 2015. 2, 3, 6
  43. S. Wug Oh, J.-Y. Lee, K. Sunkavalli, and S. Joo Kim. Fast video object segmentation by reference-guided mask propa- gation. In IEEE Conference on Computer Vision and Pattern Recognition, 2018. 2, 3, 7
  44. L. Yang, Y. Wang, X. Xiong, J. Yang, and A. K. Katsaggelos. Efficient video object segmentation via network modulation. In IEEE Conference on Computer Vision and Pattern Recog- nition, June 2018. 2, 3, 7
  45. T. Yang and A. B. Chan. Learning dynamic memory net- works for object tracking. In European Conference on Com- puter Vision, 2018. 2, 3
  46. D. Yeo, J. Son, B. Han, and J. H. Han. Superpixel-based tracking-by-segmentation using markov chains. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2
  47. J. S. Yoon, F. Rameau, J. Kim, S. Lee, S. Shin, and I. S. Kweon. Pixel-level matching for video object segmentation using convolutional neural networks. In IEEE International Conference on Computer Vision, 2017. 7
  48. Z. Zhu, Q. Wang, B. Li, W. Wu, J. Yan, and W. Hu. Distractor-aware siamese networks for visual object track- ing. In European Conference on Computer Vision, 2018. 2, 3, 5, 6, 7

Cited by

Sorted by citations

Authors
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., & Torr, P. H. S.
Affiliations
  • Institute of Automation (CN)
  • University of Oxford (GB)
  • Chinese Academy of Sciences (CN)
ORCID
Venue
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2019-06
Pages
1328-1338
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr.2019.00142
OpenAlex
W2963227409
Cited by
1,554
References
11
Citations by year
2026: 34 · 2025: 107 · 2024: 228 · 2023: 272 · 2022: 338 · 2021: 309 · 2020: 210 · 2019: 56
FWCI
74.41
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Royal Academy of Engineering
  • National Natural Science Foundation of China (U1636218, 61721004, 61751212)
  • Multidisciplinary University Research Initiative (EP/N019474/1)
  • Engineering and Physical Sciences Research Council (EP/N019474/1, EP/M013774/1, EP/M013774/, ERC-2012-AdG 321162-HELIOS, EP/I001107/2, Seebibyte EP/M013774/1)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
visual object tracking, video object segmentation, semi-supervised segmentation, Siamese network, real-time tracking, rotated bounding boxes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review