CoChat
Sign Up
Cited by 468Advanced Neural Network ApplicationsVisual Attention and Saliency DetectionMedical Image Segmentation Techniques

FEELVOS: Fast End-To-End Embedding Learning for Video Object Segmentation

Open access
Voigtlaender, Paul, Chai, Yuning, Schroff, Florian, Adam, Hartwig, Leibe, Bastian, Chen, Liang-Chieh·2019·2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr.2019.00971
Open this paper in CoChat — free accountOpen PDF

Abstract

Many of the recent successful methods for video object segmentation (VOS) are overly complicated, heavily rely on fine-tuning on the first frame, and/or are slow, and are hence of limited practical use. In this work, we propose FEELVOS as a simple and fast method which does not rely on fine-tuning. In order to segment a video, for each frame FEELVOS uses a semantic pixel-wise embedding together with a global and a local matching mechanism to transfer information from the first frame and from the previous frame of the video to the current frame. In contrast to previous work, our embedding is only used as an internal guidance of a convolutional network. Our novel dynamic segmentation head allows us to train the network, including the embedding, end-to-end for the multiple object segmentation task with a cross entropy loss. We achieve a new state of the art in video object segmentation without fine-tuning with a J&F measure of 71.5% on the DAVIS 2017 validation set. We make our code and models available at https://github.com/tensorflow/models/tree/master/research/feelvos.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr.2019.00971

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adopting a memory architecture to store object information from past frames is indispensable for accurately tracking objects in video.

    “To address the above challenges, adopting a memory architecture to store the object information from past frames is indispensable for accurately tracking objects in video [5,7,10,29,40,52,56,57,64,66,68].”

    From SAM2LONG: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree (2025) · cites this paper as [57]

    The Method section (page not supplied) says FEELVOS uses information from the first and previous frames to enable tracking, and the Ablation Study reports that previous-frame matching improves results. But the paper does not describe this as a memory architecture or establish that such an architecture is indispensable for accurate video tracking. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 32

  1. S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool. One-shot video object seg- mentation. In CVPR, 2017. 2, 7
  2. S. Caelles, A. Montes, K.-K. Maninis, Y. Chen, L. Van Gool, F. Perazzi, and J. Pont-Tuset. The 2018 DAVIS challenge on video object segmentation. arXiv preprint arXiv:1803.00557, 2018. 3 · arXiv
  3. L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Re- thinking atrous convolution for semantic image segmenta- tion. arXiv preprint arXiv:1706.05587, 2017. 6 · arXiv
  4. L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In ECCV, 2018. 3, 6
  5. J. Cheng, Y.-H. Tsai, W.-C. Hung, S. Wang, and M.-H. Yang. Fast and accurate online video object segmentation via track- ing parts. In CVPR, 2018. 1, 3, 6, 7
  6. H. Ci, C. Wang, and Y. Wang. Video object segmentation by learning location-sensitive embeddings. In ECCV, 2018. 2
  7. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR, 2009. 6
  8. A. Fathi, Z. Wojna, V. Rathod, P. Wang, H. O. Song, S. Guadarrama, and K. P. Murphy. Semantic instance segmentation via deep metric learning. arXiv preprint arXiv:1703.10277, 2017. 3, 4 · arXiv
  9. J. Han, L. Yang, D. Zhang, X. Chang, and X. Liang. Rein- forcement cutting-agent learning for video object segmenta- tion. In CVPR, 2018. 2
  10. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations. arXiv preprint arXiv:1704.04861, 2017. 6 · arXiv
  11. P. Hu, G. Wang, X. Kong, J. Kuen, and Y.-P. Tan. Motion- guided cascaded refinement network for video object seg- mentation. In CVPR, 2018. 2
  12. Y.-T. Hu, J.-B. Huang, and A. G. Schwing. Maskrnn: In- stance level video object segmentation. In NeurIPS, 2017. 2
  13. Y.-T. Hu, J.-B. Huang, and A. G. Schwing. Videomatch: Matching based video object segmentation. In ECCV, 2018. 1, 3, 4, 6, 7
  14. S. D. Jain and K. Grauman. Supervoxel-consistent fore- ground propagation in video. In ECCV, 2014. 6, 7
  15. A. Khoreva, R. Benenson, E. Ilg, T. Brox, and B. Schiele. Lucid data dreaming for object tracking. In arXiv preprint arXiv: 1703.09554, 2017. 2 · arXiv
  16. S. Li, B. Seybold, A. Vorobyov, A. Fathi, Q. Huang, and C.-C. J. Kuo. Instance embedding transfer to unsupervised video object segmentation. In CVPR, 2018. 3
  17. X. Li and C. Change Loy. Video object segmentation with joint re-identification and attention-aware mask propagation. In ECCV, 2018. 2, 6
  18. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Com- mon objects in context. In ECCV, 2014. 6
  19. J. Luiten, P. Voigtlaender, and B. Leibe. PReMVOS: Proposal-generation, refinement and merging for the davis challenge on video object segmentation 2018. The 2018 DAVIS Challenge on Video Object Segmentation -CVPR Workshops, 2018. 1, 2, 6
  20. J. Luiten, P. Voigtlaender, and B. Leibe. PReMVOS: Proposal-generation, refinement and merging for the YouTube-VOS challenge on video object segmentation 2018. The 1st Large-scale Video Object Segmentation Challenge - ECCV Workshops, 2018. 1, 2, 6
  21. F. Perazzi, A. Khoreva, R. Benenson, B. Schiele, and A. Sorkine-Hornung. Learning video object segmentation from static images. In CVPR, 2017. 2, 3, 7
  22. F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016. 1, 6
  23. T. Pohlen, A. Hermans, M. Mathias, and B. Leibe. Full- resolution residual networks for semantic segmentation in street scenes. In CVPR, 2017. 6
  24. J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine- Hornung, and L. Van Gool. The 2017 DAVIS chal- lenge on video object segmentation. arXiv preprint arXiv:1704.00675, 2017. 1, 6 · arXiv
  25. A. Prest, C. Leistner, J. Civera, C. Schmid, and V. Fer- rari. Learning object class detectors from weakly annotated video. In CVPR, 2012. 6, 7
  26. H. Qi, Z. Zhang, B. Xiao, H. Hu, B. Cheng, Y. Wei, and J. Dai. Deformable convolutional networks -coco detection and segmentation challenge 2017 entry. ICCV COCO Chal- lenge Workshop, 2017. 6
  27. P. Voigtlaender and B. Leibe. Online adaptation of convo- lutional neural networks for the 2017 DAVIS challenge on video object segmentation. The 2017 DAVIS Challenge on Video Object Segmentation -CVPR Workshops, 2017. 2, 6
  28. Z. Wu, C. Shen, and A. v. d. Hengel. Bridging category- level and instance-level semantic image segmentation. arXiv preprint arXiv:1605.06885, 2016. 6 · arXiv
  29. S. Wug Oh, J.-Y. Lee, K. Sunkavalli, and S. Joo Kim. Fast video object segmentation by reference-guided mask propa- gation. In CVPR, 2018. 1, 3, 5, 6, 7
  30. H. Xiao, J. Feng, G. Lin, Y. Liu, and M. Zhang. Monet: Deep motion exploitation for video object segmentation. In CVPR, 2018. 2
  31. N. Xu, L. Yang, Y. Fan, J. Yang, D. Yue, Y. Liang, B. Price, S. Cohen, and T. Huang. YouTube-VOS: Sequence-to- sequence video object segmentation. In ECCV, 2018. 3, 6
  32. L. Yang, Y. Wang, X. Xiong, J. Yang, and A. K. Katsaggelos. Efficient video object segmentation via network modulation. In CVPR, 2018. 1, 3, 6, 7

Cited by

Sorted by citations

Authors
Voigtlaender, P., Chai, Y., Schroff, F., Adam, H., Leibe, B., & Chen, L.-C.
Affiliations
  • RWTH Aachen University (DE)
  • Google (United States) (US)
ORCID
Venue
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2019-06
Pages
9473-9482
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr.2019.00971
OpenAlex
W2916797271
Cited by
468
References
5
Citations by year
2026: 10 · 2025: 27 · 2024: 67 · 2023: 80 · 2022: 112 · 2021: 93 · 2020: 62 · 2019: 17
FWCI
23.15
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RWTH Publications (RWTH Aachen) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
video object segmentation, cross-entropy loss, multi-object segmentation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review