CoChat
Sign Up
PreprintCited by 204Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningAutonomous Vehicle Technology and Safety

BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

Open access
Yu, Fisher, Chen, Haofeng, Wang, Xin, Xian, Wenqi, Chen, Yingying, Liu, Fangchen, Madhavan, Vashisht, Darrell, Trevor·2018·arXiv
DOI 10.48550/arxiv.1805.04687arXiv 1805.04687
Open this paper in CoChat — free accountOpen PDF

Abstract

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1805.04687

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Vistas, D2-City and BDD100k also contain images captured during different weather and illumination settings.

    “Vistas, D 2 -City and BDD100k also contain images captured during different weather and illumination settings.”

    From nuScenes: A Multimodal Dataset for Autonomous Driving (2020) · cites this paper as [85]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. S. Abu-El-Haija, N. Kothari, J. Lee, P. Natsev, G. Toderici, B. Varadarajan, and S. Vijayanarasimhan. Youtube-8m: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675, 2016. 2 · arXiv
  2. M. Aly. Real time detection of lane markers in urban streets. In Intelligent Vehicles Symposium, pages 7-12, 2008. 4
  3. F. Caba Heilbron, V. Escorcia, B. Ghanem, and J. Car- los Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 961-970, 2015. 2
  4. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3213-3223, 2016. 1, 2, 6
  5. P. Dollár, C. Wojek, B. Schiele, and P. Perona. Pedestrian detection: A benchmark. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 304-311. IEEE, 2009. 2, 10
  6. P. Dollár and C. L. Zitnick. Structured forests for fast edge detection. In ICCV, 2013. 7
  7. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) chal- lenge. International journal of computer vision, 88(2):303- 338, 2010. 1, 2
  8. T. Evgeniou and M. Pontil. Regularized multi-task learn- ing. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 109-117. ACM, 2004. 2
  9. J. Fritsch, T. Kuhnl, and A. Geiger. A new performance measure and evaluation benchmark for road detection algo- rithms. In Intelligent Transportation Systems-(ITSC), 2013 16th International IEEE Conference on, pages 1693-1700. IEEE, 2013. 4
  10. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1231-1237, 2013. 1, 2, 5, 10
  11. K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on com- puter vision, pages 2961-2969, 2017. 1, 7
  12. S. Lee, J. Kim, J. S. Yoon, S. Shin, O. Bailo, N. Kim, T.- H. Lee, H. S. Hong, S.-H. Han, and I. S. Kweon. VPGNet: Vanishing point guided network for lane and road marking detection and recognition. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 1965-1973. IEEE, 2017. 4
  13. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. In European conference on computer vision, pages 740-755. Springer, 2014. 1, 6
  14. W. Maddern, G. Pascoe, C. Linegar, and P. Newman. 1 year, 1000 km: The oxford robotcar dataset. IJ Robotics Res., 36(1):3-15, 2017. 2
  15. B. McCann, N. S. Keskar, C. Xiong, and R. Socher. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730, 2018. 1, 2 · arXiv
  16. G. Neuhold, T. Ollmann, S. R. Bulò, and P. Kontschieder. The mapillary vistas dataset for semantic understanding of street scenes. In International Conference on Computer Vi- sion (ICCV), 2017. 1, 2
  17. J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine- Hornung, and L. Van Gool. The 2017 davis chal- lenge on video object segmentation. arXiv preprint arXiv:1704.00675, 2017. · arXiv
  18. S.-A. Rebuffi, H. Bilen, and A. Vedaldi. Learning multiple visual domains with residual adapters. In Advances in Neural Information Processing Systems, pages 506-516, 2017. 1
  19. S.-A. Rebuffi, H. Bilen, and A. Vedaldi. Learning multiple visual domains with residual adapters. In Advances in Neural Information Processing Systems, pages 506-516, 2017. 1
  20. S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pages 91-99, 2015. 1, 6, 8, 12
  21. K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 2 · arXiv
  22. P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Pat- naik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, et al. Scal- ability in perception for autonomous driving: Waymo open dataset. arXiv, pages arXiv-1912, 2019. 5
  23. P. Voigtlaender, M. Krause, A. Osep, J. Luiten, B. B. G. Sekar, A. Geiger, and B. Leibe. Mots: Multi-object tracking and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7942- 7951, 2019. 1, 5, 8
  24. T. Wu and A. Ranganathan. A practical system for road marking detection and recognition. In Intelligent Vehicles Symposium, pages 25-30, 2012. 4
  25. H. Xu, Y. Gao, F. Yu, and T. Darrell. End-to-end learning of driving models from large-scale video datasets. arXiv preprint, 2017. 2, 6
  26. N. Xu, L. Yang, Y. Fan, D. Yue, Y. Liang, J. Yang, and T. Huang. Youtube-vos: A large-scale video object segmen- tation benchmark. arXiv preprint arXiv:1809.03327, 2018. 5 · arXiv
  27. F. Yu, V. Koltun, and T. Funkhouser. Dilated residual net- works. In Computer Vision and Pattern Recognition (CVPR), 2017. 6, 10
  28. F. Yu, A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 2 · arXiv
  29. F. Yu, D. Wang, E. Shelhamer, and T. Darrell. Deep layer ag- gregation. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 2403-2412, 2018. 3, 7, 8
  30. A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese. Taskonomy: Disentangling task transfer learn- ing. In The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), June 2018. 1, 2
  31. S. Zhang, R. Benenson, and B. Schiele. Citypersons: A di- verse dataset for pedestrian detection. In The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), July 2017. 2, 10
  32. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Advances in neural information processing sys- tems, pages 487-495, 2014. 2
  33. Y. Zhou, X. Wang, J. Jiao, T. Darrell, and F. Yu. Learning saliency propagation for semi-supervised instance segmenta- tion. In Computer Vision and Pattern Recognition (CVPR), 2020. 7 29.5 55.3 27.2 14.4 32.8 47.2 Non-City 24.9 48.6 22.1 11.7 28.1 40.7 Random 28.7 54.5 25.8 13.7 31.9 47.0 City Non-City 26.5 49.3 25.5 13.5 32.1 47.0 Non-City 24.3 46.0 22.4 13.3 30.0 42.0 Random 26.6 49.8 24.4 14.4 31.8 47.4 City Val 28.8 54.1 26.8 13.8 32.7 47.0 Non-City 24.9 48.3 22.2 11.8 28.7 41.2 Random 28.7 54.5 25.8 13.7 31.9 47.0 Daytime Daytime 30.6 56.0 28.7 16.4 35.6 50.7 Non-Daytime 25.9 49.6 23.2 12.8 30.7 42.7 Random 29.5 55.0 27.2 15.7 34.5 48.7 Daytime Non-Daytime 23.6 46.1 21.2 10.5 25.4 41.3 Non-Daytime 25.3 49.9 22.0 11.6 26.6 43.4 Random 26.0 50.9 22.5 12.8 27.4 45.1 Daytime Val 28.1 52.8 25.9 13.3 31.9 47.0 Non-Daytime 25.6 49.8 22.6 11.5 29.1 42.5 Random 28.7 54.5 25.8 13.7 31.9 47.0 Table 13: Full evaluation results of the domain discrepancy experiments with object detection.

Cited by

Sorted by citations

Authors
Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., & Darrell, T.
Affiliations
  • University of California, Berkeley (US)
  • Cornell University (US)
  • University of California San Diego (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1805.04687
arXiv
1805.04687
OpenAlex
W3016101116
Cited by
204
References
3
Citations by year
2025: 19 · 2024: 46 · 2023: 51 · 2022: 42 · 2021: 31 · 2020: 15
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords
BDD100K, driving data, autonomous driving, benchmark, image recognition, environmental diversity, geographic diversity, training strategies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review