Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Deep learning has enabled impressive progress in the accuracy of semantic segmentation. Yet, the ability to estimate uncertainty and detect anomalies is key for safety-critical applications like autonomous driving. Existing uncertainty estimates have mostly been evaluated on simple tasks, and it is unclear whether these methods generalize to more complex scenarios. We present Fishyscapes, the first public benchmark for uncertainty estimation in the real-world task of semantic segmentation for urban driving. It evaluates pixel-wise uncertainty estimates towards the detection of anomalous objects in front of the vehicle. We adapt state-of-the-art methods to recent semantic segmentation models and compare approaches based on softmax confidence, Bayesian learning, and embedding density. Our results show that anomaly detection is far from solved even for ordinary situations, while our benchmark allows measuring advancements beyond the state-of-the-art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · cited 1×
“As evaluating the posterior over the weights is intractable in deep non-linear networks, recent works perform Monte-Carlo (MC) sampling with dropout [35] or ensembles [36].”
MethodOn calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Lee, K., Lee, K., Lee, H., & Shin, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03888 · Cited by 2,450
Deep Ordinal Regression Network for Monocular Depth Estimation
Fu, H., Gong, M., Wang, C., Batmanghelich, K., & Tao, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00214 · Cited by 1,991
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Bergmann, P., Batzner, K., Fauser, M., Sattlegger, D., & Steger, C. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01400-4 · Cited by 602
“Blum et al. (2019) recently introduced Fishyscapes, a dataset intended to benchmark semantic segmentation algorithms with respect to their ability to detect out-of-2 Fig.2 Example images for all five textures and ten object categories of the MVTec Anomaly Detection dataset.”
MethodEfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies
Batzner, K., Heckler, L., & König, R. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00020 · Cited by 386
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
Feng, D., Harakeh, A., Waslander, S. L., & Dietmayer, K. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3096854 · Cited by 316
Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization
Bergmann, P., Batzner, K., Fauser, M., Sattlegger, D., & Steger, C. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01578-9 · Cited by 286
Metadata
- Authors
- Blum, H., Sarlin, P.-E., Nieto, J., Siegwart, R., & Cadena, C.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2403-2412
- OpenAlex
- W3011680720
- Cited by
- 103
- References
- 15
- FWCI
- 2.78
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving. https://app.cochat.ai/papers/10.1109/iccvw.2019.00294
- The paper
- Blum, H., Sarlin, P.-E., Nieto, J., Siegwart, R., & Cadena, C. (2019). Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). https://doi.org/10.1109/iccvw.2019.00294
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.