CoChat
Sign Up
PreprintCited by 1,174Advanced Neural Network ApplicationsInfrastructure Maintenance and MonitoringDomain Adaptation and Few-Shot Learning

The Cityscapes Dataset for Semantic Urban Scene Understanding

Open access
Cordts, Marius, Omran, Mohamed, Ramos, Sebastian, Rehfeld, Timo, Enzweiler, Markus, Benenson, Rodrigo, Franke, Uwe, Roth, Stefan +1 more·2016·arXiv
DOI 10.48550/arxiv.1604.01685arXiv 1604.01685
Open this paper in CoChat — free accountOpen PDF

Abstract

Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations; 20000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1604.01685

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing segmentation datasets can hardly support LUSS due to the large image complexity but small data scale.

    “Existing segmentation datasets can hardly support LUSS due to the large image complexity but small data scale.”

    From Large-Scale Unsupervised Semantic Segmentation (2023) · cites this paper as [7]

    The Introduction states that existing urban-scene datasets are often small and do not fully capture the complexity of real-world inner-city traffic scenes, limiting progress in street-scene understanding. It does not extend that assessment to all segmentation datasets or specifically address LUSS. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 64

  1. P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marqués, and J. Ma- lik. Multiscale combinatorial grouping. In CVPR, 2014. 7, 8, ii
  2. S. Ardeshir, K. M. Collins-Sibley, and M. Shah. Geo- semantic segmentation. In CVPR, 2015. 4, ii
  3. G. J. Brostow, J. Fauqueur, and R. Cipolla. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, 30(2):88-97, 2009. 1, 3, 4, 7, ii
  4. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep con- volutional nets and fully connected CRFs. In ICLR, 2015. 5, 6, 7, ii, xii
  5. Y.-T. Chen, X. Liu, and M.-H. Yang. Multi-instance object segmentation with occlusion handling. In CVPR, 2015. 7
  6. J. Dai, K. He, and J. Sun. Convolutional feature masking for joint object and stuff segmentation. In CVPR, 2015. 6, 7
  7. P. Dollár, C. Wojek, B. Schiele, and P. Perona. Pedestrian detection: An evaluation of the state of the art. Trans. PAMI, 34(4):743-761, 2012. 1, 4
  8. M. Enzweiler and D. M. Gavrila. Monocular pedestrian de- tection: Survey and experiments. 31(12):2179-2195, 2009.
  9. M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL visual object classes challenge: A retrospective. IJCV, 111(1):98- 136, 2015. 1, 4, 5, 6, 7, ii
  10. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ra- manan. Object detection with discriminatively trained part- based models. Trans. PAMI, 32(9):1627-1645, 2010. 1
  11. U. Franke, D. Pfeiffer, C. Rabe, C. Knöppel, M. Enzweiler, F. Stein, and R. G. Herrtwich. Making Bertha see. In ICCV Workshops, 2013. 1
  12. A. Geiger, M. Lauer, C. Wojek, C. Stiller, and R. Urtasun. 3D traffic scene understanding from movable platforms. Trans. PAMI, 36(5):1012-1025, 2014. 1
  13. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The KITTI dataset. IJRR, 32(11), 2013. 1, 3, 4, ii
  14. R. Girshick. Fast R-CNN. In ICCV, 2015. 8, ii
  15. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich fea- ture hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014. 7
  16. F. Gueney and A. Geiger. Displets: Resolving stereo ambi- guities using object knowledge. In CVPR, 2015. 4, ii
  17. B. Hariharan, P. Arbeláez, R. B. Girshick, and J. Malik. Si- multaneous detection and segmentation. In ECCV, 2014. 7
  18. H. Hattori, V. N. Boddeti, K. M. Kitani, and T. Kanade. Learning scene-specific pedestrian detectors without real data. In CVPR, 2015. 7
  19. X. He and S. Gould. An exemplar-based CRF for multi- instance object segmentation. In CVPR, 2014. 7
  20. D. Hoiem, J. Hays, J. Xiao, and A. Khosla. Guest editorial: Scene understanding. IJCV, 2015. 1
  21. J. Hosang, R. Benenson, P. Dollár, and B. Schiele. What makes for effective detection proposals? Trans. PAMI, 38(4):814-830, 2015.
  22. H. Hu and B. Upcroft. Nonparametric semantic segmenta- tion for 3D street scenes. In IROS, 2013. 4, ii
  23. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012. 1
  24. A. Kundu, Y. Li, F. Dellaert, F. Li, and J. Rehg. Joint se- mantic segmentation and 3D reconstruction from monocular video. In ECCV, 2014. 4, ii
  25. L. Ladicky, J. Shi, and M. Pollefeys. Pulling things out of perspective. In CVPR, 2014. 4, ii
  26. B. Leibe, N. Cornelis, K. Cornelis, and L. Van Gool. Dy- namic 3D scene analysis from a moving vehicle. In CVPR, 2007. 1, ii
  27. B. Leibe, A. Leonardis, and B. Schiele. Robust object detec- tion with interleaved categorization and segmentation. IJCV, 77(1-3):259-289, 2008. 7
  28. G. Lin, C. Shen, A. van den Hengel, and I. Reid. Efficient piecewise training of deep structured models for semantic segmentation. In CVPR, 2016, to appear. 5, 6, 8, i, ii, xii, xv, xvii
  29. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Com- mon objects in context. In ECCV, 2014. 1, 4, 5, 7, ii
  30. Z. Liu, X. Li, P. Luo, C. C. Loy, and X. Tang. Semantic im- age segmentation via deep parsing network. In ICCV, 2015. 5, 6, ii, xii, xv, xvii
  31. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 1, 3, 5, 6, 8, i, xi, xiv, xvi
  32. I. Misra, A. Shrivastava, and M. Hebert. Watch and learn: Semi-supervised learning for object detectors from video. In CVPR, 2015. 7
  33. M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich. Feed- forward semantic segmentation with zoom-out features. In CVPR, 2015. 6
  34. R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fi- dler, R. Urtasun, and A. Yuille. The role of context for object detection and semantic segmentation in the wild. In CVPR, 2014. 1, 5, i, ii
  35. A. Oliva and A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV, 42(3):145-175, 2001. 1
  36. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Is object local- ization for free? Weakly-supervised learning with convolu- tional neural networks. In CVPR, 2015. 7
  37. G. Papandreou, L.-C. Chen, K. Murphy, and A. L. Yuille. Weakly-and semi-supervised learning of a DCNN for se- mantic image segmentation. In ICCV, 2015. 6, 7, 8, ii, xii, xv, xvii
  38. D. Pathak, P. Kraehenbuehl, and T. Darrell. Constrained con- volutional neural networks for weakly supervised segmenta- tion. In ICCV, 2015. 6
  39. D. Pathak, E. Shelhamer, J. Long, and T. Darrell. Fully con- volutional multi-class multiple instance learning. In ICLR, 2015. 6
  40. D. Pfeiffer, S. K. Gehrig, and N. Schneider. Exploiting the power of stereo confidences. In CVPR, 2013. 2
  41. P. H. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene parsing. In ICML, 2014. 6
  42. P. H. Pinheiro and R. Collobert. From image-level to pixel- level labeling with convolutional networks. In CVPR, 2015. 6
  43. J. Pont-Tuset and L. Van Gool. Boosting object proposals: From Pascal to COCO. In ICCV, 2015. 7
  44. S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: To- wards real-time object detection with region proposal net- works. In NIPS, 2015. 7
  45. H. Riemenschneider, A. Bódis-Szomorú, J. Weissenberg, and L. Van Gool. Learning where to classify in multi-view semantic segmentation. In ECCV. 2014. 4, ii
  46. H. Riemenschneider, S. Sternig, M. Donoser, P. M. Roth, and H. Bischof. Hough regions for joining instance localization and segmentation. In ECCV, 2012. 7
  47. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recog- nition challenge. IJCV, 115(3):211-252, 2015. 1, i, ii
  48. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Free- man. LabelMe: A database and web-based tool for image annotation. IJCV, 77(1-3):157-173, 2008. 2, i
  49. T. Scharwächter, M. Enzweiler, U. Franke, and S. Roth. Stix- mantics: A medium-level model for real-time semantic scene understanding. In ECCV, 2014. 1, 4
  50. S. Sengupta, E. Greveson, A. Shahrokni, and P. H. S. Torr. Semantic modelling of urban scenes. In ICRA, 2013. 4, 7, ii
  51. S. Sengupta, P. Sturgess, L. Ladicky, and P. H. S. Torr. Au- tomatic dense visual semantic mapping from street-level im- agery. In IROS, 2012. 4, ii
  52. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. OverFeat: Integrated recognition, localization and detection using convolutional networks. In ICLR, 2014. 1
  53. A. Sharma, O. Tuzel, and D. W. Jacobs. Deep hierarchical parsing for semantic segmentation. In CVPR, 2015. 6
  54. S. Song, S. P. Lichtenberg, and J. Xiao. Sun RGB-D: A RGB-D scene understanding benchmark suite. In CVPR, 2015. 4, ii
  55. J. Tighe and S. Lazebnik. Superparsing. IJCV, 101(2):329- 349, 2013. 4
  56. J. Tighe, M. Niethammer, and S. Lazebnik. Scene pars- ing with object instance inference using regions and per- exemplar detectors. IJCV, 112(2):150-171, 2015. 1, 7
  57. J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders. Selective search for object recognition. International journal of computer vision, 104(2):154-171, 2013. 7
  58. V. Vineet, O. Miksik, M. Lidegaard, M. Niessner, S. Golodetz, V. A. Prisacariu, O. Kahler, D. W. Murray, S. Izadi, P. Perez, and P. H. S. Torr. Incremental dense se- mantic stereo fusion for large-scale semantic scene recon- struction. In ICRA, 2015. 7, 8
  59. J. Xu, A. G. Schwing, and R. Urtasun. Learning to segment under various forms of weak supervision. In CVPR, 2015. 6
  60. J. Yao, S. Fidler, and R. Urtasun. Describing the scene as a whole: Joint object detection, scene classification and se- mantic segmentation. In CVPR, 2012. 7
  61. F. Yu and V. Koltun. Multi-scale context aggregation by di- lated convolutions. In ICLR, 2016, to appear. 6, 8, i, ii, xii, xv, xvii
  62. Z. Zhang, A. Schwing, S. Fidler, and R. Urtasun. Monocular object instance segmentation and depth ordering with CNNs. In ICCV, 2015. 4, 7, ii
  63. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr. Conditional ran- dom fields as recurrent neural networks. In ICCV, 2015. 5, 6, ii, xii, xv, xvii
  64. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014. 1

Cited by

Sorted by citations

Related

Authors
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B.
Affiliations
  • Technische Universität Darmstadt (DE)
  • Max Planck Society (DE)
  • Technische Universität Dresden (DE)
  • Daimler (Germany) (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1604.01685
arXiv
1604.01685
OpenAlex
W2953139137
Cited by
1,174
References
14
Citations by year
2026: 4 · 2025: 6 · 2024: 96 · 2023: 218 · 2022: 51 · 2021: 215 · 2020: 193 · 2019: 151 · 2018: 148 · 2017: 63 · 2016: 29
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
  • arXiv (Cornell University)
  • TIB Data Manager
Funders
  • Bundesministerium für Wirtschaft und Technologie
  • FP7 Ideas: European Research Council (307942)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Civil and Structural Engineering › Infrastructure Maintenance and Monitoring
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
cityscape, semantic segmentation, instance segmentation, pixel-level annotation, weakly supervised learning, benchmark dataset, urban street scene

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review