CoChat
Sign Up
Cited by 354Advanced Neural Network ApplicationsVideo Surveillance and Tracking MethodsDomain Adaptation and Few-Shot Learning

Video Object Segmentation without Temporal Information

Open access
Maninis, K.-K., Caelles, S., Chen, Y., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., Van Gool, L.·2019·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2018.2838670
Open this paper in CoChat — free account

Abstract

Video Object Segmentation, and video processing in general, has been historically dominated by methods that rely on the temporal consistency and redundancy in consecutive video frames. When the temporal smoothness is suddenly broken, such as when an object is occluded, or some frames are missing in a sequence, the result of these methods can deteriorate significantly. This paper explores the orthogonal approach of processing each frame independently, i.e., disregarding the temporal information. In particular, it tackles the task of semi-supervised video object segmentation: the separation of an object from the background in a video, given its mask in the first frame. We present Semantic One-Shot Video Object Segmentation (OSVOS$^\mathrm {S}$S), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one shot). We show that instance-level semantic information, when combined effectively, can dramatically improve the results of our previous method, OSVOS. We perform experiments on two recent single-object video segmentation databases, which show that OSVOS$^\mathrm {S}$S is both the fastest and most accurate method in the state of the art. Experiments on multi-object video segmentation show that OSVOS$^\mathrm {S}$S obtains competitive results.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2018.2838670

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adopting a memory architecture to store object information from past frames is indispensable for accurately tracking objects in video.

    “To address the above challenges, adopting a memory architecture to store the object information from past frames is indispensable for accurately tracking objects in video [5,7,10,29,40,52,56,57,64,66,68].”

    From SAM2LONG: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree (2025) · cites this paper as [40]

    The paper makes the contrary point. On p. 1, it says its method produces accurate results while processing each frame independently, without temporal information from past frames; p. 2 reports accuracy results for that method. This contradicts the claim that storing information from past frames is indispensable. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 67

  1. [1][1] P. Arbel´ aez, M. Maire, C. Fowlkes, and J. Malik. Contour detection and hierarchical image segmentation. TPAMI , 33(5):898-916, 2011. 3, 5 -
  2. [2][2] G. Bertasius, J. Shi, and L. Torresani. High-for-low and low-forhigh: Efficient boundary detection from deep object features and its applications to high-level vision. In ICCV , 2015. 2 -
  3. [3][3] G. Bertasius, J. Shi, and L. Torresani. Semantic segmentation with boundary neural fields. In CVPR , 2016. 2 -
  4. [4][4] T. Brox and J. Malik. Object segmentation by long term analysis of point trajectories. In ECCV , 2010. 1, 8 -
  5. [5][5] S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taix´ e, D. Cremers, and L. Van Gool. One-shot video object segmentation. In CVPR , 2017. 3, 8 -
  6. [6][6] J. Chang, D. Wei, and J. W. Fisher III. A video representation using temporal superpixels. In CVPR , 2013. 2, 3 -
  7. [7][7] J. Dai, K. He, Y. Li, S. Ren, and J. Sun. Instance-sensitive fully convolutional networks. In ECCV , 2016. 2, 3, 9, 10 -
  8. [8][8] J. Dai, K. He, and J. Sun. Convolutional feature masking for joint object and stuff segmentation. In CVPR , 2015. 3 -
  9. [10][10] J. Dai, Y. Li, K. He, and J. Sun. R-FCN: Object detection via regionbased fully convolutional networks. In NIPS , 2016. 2 -
  10. [11][11] M. Dantone, J. Gall, G. Fanelli, and L. Van Gool. Real-time facial feature detection using conditional regression forests. In CVPR , 2012. 3 -
  11. [12][12] A. Faktor and M. Irani. Video segmentation by non-local consensus voting. In BMVC , 2014. 8 -
  12. [13][13] Q. Fan, F. Zhong, D. Lischinski, D. Cohen-Or, and B. Chen. Jumpcut: Non-successive mask transfer and interpolation for video cutout. ACM Trans. Graph. , 34(6), 2015. 2, 3 -
  13. [15][15] R. Girshick. Fast R-CNN. In ICCV , 2015. 1, 3 -
  14. [16][16] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. 1, 3 -
  15. [17][17] M. Grundmann, V. Kwatra, M. Han, and I. A. Essa. Efficient hierarchical graph-based video segmentation. In CVPR , 2010. 1, 2, 3 -
  16. [18][18] C. Hane, C. Zach, A. Cohen, R. Angst, and M. Pollefeys. Joint 3d scene reconstruction and class segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 97-104, 2013. 3 -
  17. [19][19] B. Hariharan, P. Arbel´ aez, R. Girshick, and J. Malik. Simultaneous detection and segmentation. In ECCV , 2014. 3 -
  18. [21][21] K. He, G. Gkioxari, P. Doll´ ar, and R. Girshick. Mask R-CNN. In ICCV , 2017. 2, 3, 5, 6, 9, 11 -
  19. [23][23] H. V. Hopwood. Living Pictures: Their History, Photo-Production and Practical Working . Optician & Photographic Trades Review, 1899. 1 -
  20. [24][24] S. Jain, B. Xiong, and K. Grauman. Fusionseg: Learning to combine motion and appearance for fully automatic segmentation of generic objects in videos. In CVPR , 2017. 8 -
  21. [25][25] S. D. Jain and K. Grauman. Supervoxel-consistent foreground propagation in video. In ECCV , 2014. 2, 8, 11 -
  22. [26][26] V. Jampani, R. Gadde, and P. V. Gehler. Video propagation networks. In CVPR , 2017. 2, 8, 9, 11 -
  23. [27][27] W.-D. Jang and C.-S. Kim. Online video object segmentation via convolutional trident network. In CVPR , 2017. 1 -
  24. [28][28] W.-D. Jang and C.-S. Kim. Online video object segmentation via convolutional trident network. In CVPR , 2017. 2, 8, 9, 11 -
  25. [29][29] A. Khoreva, R. Benenson, E. Ilg, T. Brox, and B. Schiele. Lucid data dreaming for object tracking. The 2017 DAVIS Challenge on Video Object Segmentation - CVPR Workshops , 2017. 8, 11, 13 -
  26. [30][30] Y. Koh and C.-S. Kim. Primary object segmentation in videos based on region augmentation and reduction. In CVPR , 2017. 8 -
  27. [31][31] I. Kokkinos. Pushing the boundaries of boundary detection using deep learning. In ICLR , 2016. 2, 4 -
  28. [32][32] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS , 2012. 1, 3, 5 -
  29. [33][33] T.-N. Le, K.-T. Nguyen, M.-H. Nguyen-Phan, T.-V. Ton, T.-A. N. (2), X.-S. Trinh, Q.-H. Dinh, V.-T. Nguyen, A.-D. Duong, A. Sugimoto, T. V. Nguyen, and M.-T. Tran. Instance re-identification flow for video object segmentation. The 2017 DAVIS Challenge on Video Object Segmentation - CVPR Workshops , 2017. 8, 11 -
  30. [34][34] K. Li, B. Hariharan, and J. Malik. Iterative instance segmentation. In CVPR , pages 3659-3667, 2016. 3 -
  31. [35][35] X. Li, Y. Qi, Z. Wang, K. Chen, Z. Liu, J. Shi, P. Luo, C. C. Loy, and X. Tang. Video object segmentation with re-identification. The 2017 DAVIS Challenge on Video Object Segmentation - CVPR Workshops , 2017. 8, 11, 13 -
  32. [36][36] Y. Li, H. Qi, J. Dai, X. Ji, and Y . Wei. Fully convolutional instance-aware semantic segmentation. In CVPR , 2017. 2, 3, 5, 6, 11 -
  33. [37][37] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. Zitnick. Microsoft COCO: Common Objects in Context. In ECCV , 2014. 8, 11 -
  34. [38][38] B. Liu, S. Gould, and D. Koller. Single image depth estimation from predicted semantic labels. In CVPR , 2010. 3 -
  35. [39][39] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed. SSD: Single shot multibox detector. In ECCV , 2016. 1 -
  36. [40][40] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR , 2015. 2, 3, 4, 5 -
  37. [41][41] K. Maninis, J. Pont-Tuset, P. Arbel´ aez, and L. V. Gool. Convolutional oriented boundaries: From image segmentation to high-level tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2017. 2, 3, 9 -
  38. [42][42] K. Maninis, J. Pont-Tuset, P. Arbel´ aez, and L. Van Gool. Convolutional oriented boundaries. In ECCV , 2016. 3, 4, 9 -
  39. [43][43] K. Maninis, J. Pont-Tuset, P. Arbel´ aez, and L. Van Gool. Deep retinal image understanding. In MICCAI , 2016. 2, 4, 5 -
  40. [44][44] R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille. The role of context for object detection and semantic segmentation in the wild. In CVPR , 2014. 5 -
  41. [45][45] V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML , pages 807-814, 2010. 4 -
  42. [46][46] N. Nicolas M¨ arki, F. Perazzi, O. Wang, and A. Sorkine-Hornung. Bilateral space video segmentation. In CVPR , 2016. 2, 3, 8, 9, 11 -
  43. [47][47] H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. In ICCV , 2015. 2, 3 -
  44. [48][48] A. Papazoglou and V. Ferrari. Fast object segmentation in unconstrained video. In ICCV , 2013. 8 -
  45. [49][49] F. Perazzi, A. Khoreva, R. Benenson, B. Schiele, and A.SorkineHornung. Learning video object segmentation from static images. In CVPR , 2017. 1, 2, 3, 8, 9, 11 -
  46. [50][50] F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR , 2016. 2, 7, 9 -
  47. [51][51] F. Perazzi, O. Wang, M. Gross, and A. Sorkine-Hornung. Fully connected object proposals for video segmentation. In ICCV , 2015. 2, 3, 8, 9 -
  48. [52][52] P. O. Pinheiro, T.-Y. Lin, R. Collobert, and P. Doll´ ar. Learning to refine object segments. In ECCV , 2016. 3 -
  49. [53][53] J. Pont-Tuset, P. Arbel´ aez, J. T. Barron, F. Marques, and J. Malik. Multiscale combinatorial grouping for image segmentation and object proposal generation. TPAMI , 2017. 3, 5 -
  50. [54][54] J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbel´ aez, A. Sorkine-Hornung, and L. Van Gool. The 2017 DAVIS challenge on video object segmentation. arXiv:1704.00675 , 2017. 2, 8, 9 - · arXiv
  51. [55][55] A. Prest, C. Leistner, J. Civera, C. Schmid, and V. Ferrari. Learning object class detectors from weakly annotated video. In CVPR , 2012. 2, 8, 11 -
  52. [56][56] S. A. Ramakanth and R. V. Babu. Seamseg: Video object segmentation using patch seams. In CVPR , 2014. 1, 2, 3 -
  53. [57][57] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS , 2015. 3 -
  54. [58][58] B. Romera-Paredes and P. H. S. Torr. Recurrent instance segmentation. In ECCV , 2016. 3 -
  55. [60][60] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV , 2015. 2, 3, 4 -
  56. [61][61] A. Shaban, A. Firl, A. Humayun, J. Yuan, X. Wang, P. Lei, N. Dhanda, B. Boots, J. M. Rehg, and F. Li. Multiple-instance video segmentation with sequence-specific object proposals. The 2017 DAVIS Challenge on Video Object Segmentation - CVPR Workshops , 2017. 8, 11 -
  57. [62][62] K. Simonyan and A. Zisserman. Very deep convolutional networks for - large-scale image recognition. In ICLR , 2015. 1, 3, 4, 5 -
  58. [64][64] M. Sun, P. Kohli, and J. Shotton. Conditional regression forests for human pose estimation. In CVPR , 2012. 3 -
  59. [65][65] A. M. Tekalp. Digital video processing . Prentice Hall Press, 2015. 1 -
  60. [66][66] P. Tokmakov, K. Alahari, and C. Schmid. Learning motion patterns in videos. In CVPR , 2017. 8 -
  61. [67][67] Y.-H. Tsai, M.-H. Yang, and M. J. Black. Video segmentation via object flow. In CVPR , 2016. 1, 8, 9, 11 -
  62. [68][68] J. R. Uijlings and V. Ferrari. Situational object boundary detection. In CVPR , 2015. 3 -
  63. [70][70] P. Voigtlaender and B. Leibe. Online adaptation of convolutional neural networks for video object segmentation. In BMVC , 2017. 2, 8, 9, 11 -
  64. [71][71] Y. Wang, Y.-q. Zhang, and J. Ostermann. Video Processing and Communications . Prentice Hall PTR, Upper Saddle River, NJ, USA, 1st edition, 2001. 1 -
  65. [72][72] S. Xie and Z. Tu. Holistically-nested edge detection. In ICCV , 2015. 2, 4 -
  66. [73][73] S. Xie and Z. Tu. Holistically-nested edge detection. International Journal of Computer Vision , pages 1-16, 2017. 3, 5 -
  67. [74][74] J. Yang, B. Price, S. Cohen, H. Lee, and M.-H. Yang. Object contour detection with a fully convolutional encoder-decoder network. In CVPR , 2016. 2, 3, 4 -

Cited by

Sorted by citations

Authors
Maninis, K.-K., Caelles, S., Chen, Y., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., & Van Gool, L.
Affiliations
  • ETH Zurich (CH)
  • Technical University of Munich (DE)
ORCID
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019-06-01
Volume
41
Issue
6
Pages
1515-1530
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2018.2838670
OpenAlex
W2754124089
Cited by
354
References
8
Citations by year
2026: 6 · 2025: 13 · 2024: 36 · 2023: 39 · 2022: 73 · 2021: 69 · 2020: 64 · 2019: 50 · 2018: 4
FWCI
23.40
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • Lirias (acceptedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion)
  • Lirias (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • EU Framework Programme for Research and Innovation Horizon 2020 (645331)
  • Swiss Commission for Technology and Innovation (19015.1 PFES-ES)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
video object segmentation, semi-supervised segmentation, foreground segmentation, semantic segmentation, fully convolutional network, occlusion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review