MOSE: A New Dataset for Video Object Segmentation in Complex Scenes
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Video object segmentation (VOS) aims at segmenting a particular object throughout the entire video clip sequence. The state-of-the-art VOS methods have achieved excellent performance (e.g., 90+% $\mathcal{J}$ & $\mathcal{F}$) on existing datasets. However, since the target objects in these existing datasets are usually relatively salient, dominant, and isolated, VOS under complex scenes has rarely been studied. To revisit VOS and make it more applicable in the real world, we collect a new VOS dataset called coMplex video Object SEgmentation (MOSE) to study the tracking and segmenting objects in complex scenarios. MOSE contains 2,149 video clips and 5,200 objects from 36 categories, with 431,725 high-quality object segmentation masks. The most notable feature of MOSE dataset is complex scenes with crowded and occluded objects. The target objects in the videos are commonly occluded by others and disappear in some frames. To analyze the proposed MOSE dataset, we benchmark 18 existing VOS methods under 4 different settings on the proposed MOSE dataset and conduct comprehensive comparisons. The experiments show that current VOS algorithms cannot well perceive objects in complex scenes. For example, under the semi-supervised VOS setting, the highest $\mathcal{J}$ & $\mathcal{F}$ by existing state-of-the-art VOS methods is only 59.4% on MOSE, much lower than their ∼90% $\mathcal{J}$ & $\mathcal{F}$ performance on DAVIS. The results reveal that although excellent performance has been achieved on existing benchmarks, there are unresolved challenges under complex scenes and more efforts are desired to explore these challenges in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Fast Online Object Tracking and Segmentation: A Unifying Approach
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00142 · Cited by 1,554 · cited 3×
“For example, SiamMask [86] applies a mask prediction branch on fully-convolutional Siamese object tracker to generate binary segmentation masks.”
MethodSelf-citationOne-Shot Video Object Segmentation
Caelles, S., Maninis, K.-K., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.565 · Cited by 927 · cited 2×
“Semi-supervised video object segmentation (or one-shot video object segmentation) [4] gives the first frame object mask and target to segment the target object throughout the remaining video frames.”
BackgroundOcclusion-Aware R-CNN: Detecting Pedestrians in a Crowd
Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S. Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_39 · Cited by 488
FEELVOS: Fast End-To-End Embedding Learning for Video Object Segmentation
Voigtlaender, P., Chai, Y., Schroff, F., Adam, H., Leibe, B., & Chen, L.-C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00971 · Cited by 468
XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model
Cheng, H. K., & Schwing, A. G. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19815-1_37 · Cited by 449
Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., & Yu, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.518 · Cited by 406
Online Multi-Object Tracking with Dual Matching Attention Networks
Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., & Yang, M.-H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_23 · Cited by 377 · cited 2×
“On the video domain, however, occlusion understanding is still underexplored with only several multi-object tracking works [16,55,96,113].”
Background
Cited by
SegGPT: Towards Segmenting Everything In Context
Wang, X., Zhang, X., Cao, Y., Wang, W., Shen, C., & Huang, T. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00110 · Cited by 310
Putting the Object Back into Video Object Segmentation
Cheng, H. K., Oh, S. W., Price, B., Lee, J.-Y., & Schwing, A. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00304 · Cited by 148
SAM2LONG: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree
Ding, S., Qian, R., Dong, X., Zhang, P., Zang, Y., Cao, Y., Guo, Y., Lin, D., & Wang, J. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01264 · Cited by 19
“To evaluate our method, we select nine standard VOS benchmarks: SA-V val and test [50], LVOS v1 [26], LVOS v2 [27], MOSE [17], VOST [54], PUMaVOS [4], DAVIS [47], and YTVOS [63].”
VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control
Bian, Y., Zhang, Z., Ju, X., Cao, M., Xie, L., Shan, Y., & Xu, Q. (2025) · Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers · doi:10.1145/3721238.3730673 · Cited by 12
“Context Mask Tokens”
Metadata
- Authors
- Ding, H., Liu, C., He, S., Jiang, X., Torr, P. H. S., & Bai, S.
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 20167-20177
- OpenAlex
- W4390873799
- Cited by
- 138
- References
- 15
- FWCI
- 15.69
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MOSE: A New Dataset for Video Object Segmentation in Complex Scenes. https://app.cochat.ai/papers/10.1109/iccv51070.2023.01850
- The paper
- Ding, H., Liu, C., He, S., Jiang, X., Torr, P. H. S., & Bai, S. (2023). MOSE: A New Dataset for Video Object Segmentation in Complex Scenes. 2023 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv51070.2023.01850
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.