The ApolloScape Open Dataset for Autonomous Driving and Its Application
Open accessAbstract
Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g., KITTI [2] or Cityscapes [3] , ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as joint 3D-2D segment labeling, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
NuScenes, BDD100K, ApolloScape, and SYNTHIA datasets offer diverse and extensive resources for exploring vehicle detection in challenging conditions.
“Among the noteworthy datasets, NuScenes [57], BDD100K [58], ApolloScape [59], and SYNTHIA [60] datasets offer diverse and extensive resources for exploring vehicle detection in challenging conditions.”
From Object Detection in Autonomous Vehicles under Adverse Weather: A Review of Traditional and Deep Learning Approaches (2024) · cites this paper as [59]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · see citing works
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · see citing works
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., & Lopez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.352 · Cited by 2,338 · see citing works
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
Neuhold, G., Ollmann, T., Bulo, S. R., & Kontschieder, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.534 · Cited by 1,434 · see citing works
Geometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803 · see citing works
TorontoCity: Seeing the World with a Million Eyes
Wang, S., Bai, M., Mattyus, G., Chu, H., Luo, W., Yang, B., Liang, J., Cheverie, J., Fidler, S., & Urtasun, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.327 · Cited by 186 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
“CamVid [8], Cityscapes [19], Mapillary Vistas [33], D(2)-City [11], BDD100k [85] and Apolloscape [41] released ever growing datasets with segmentation masks.”
BackgroundMethodInfluentialVision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · see citing works
“We construct a meta-dataset that includes the original datasets that were used in [32] (referred to as MIX 5 in that work) and extend it with with five additional datasets ([19, 45, 46, 48, 49]).”
MethodA Survey of Autonomous Driving: <i>Common Practices and Emerging Technologies</i>
Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020) · IEEE Access · doi:10.1109/access.2020.2983149 · Cited by 1,889 · see citing works
“ApolloScape is a very recent dataset that is not fully public yet [278].”
BackgroundACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding
Sakaridis, C., Dai, D., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01059 · Cited by 572 · see citing works
DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection
Yu, H., Luo, Y., Shu, M., Huo, Y., Yang, Z., Shi, Y., Guo, Z., Li, H., Hu, X., Yuan, J., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02067 · Cited by 577 · see citing works
Learning Depth with Convolutional Spatial Propagation Network
Cheng, X., Wang, P., & Yang, R. (2018) · arXiv · doi:10.48550/arxiv.1810.02695 · Cited by 345 · see citing works
Self-citationA2D2: Audi Autonomous Driving Dataset
Geyer, J., Kassahun, Y., Mahmudi, M., Ricou, X., Durgesh, R., Chung, A. S., Hauswald, L., Pham, V. H., Mühlegg, M., Dorn, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.06320 · Cited by 250 · see citing works
Canadian adverse driving conditions dataset
Pitropov, M., Garcia, D. E., Rebello, J., Smart, M., Wang, C., Czarnecki, K., & Waslander, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920979368 · Cited by 286 · see citing works
Tahir, N. U. A., Zhang, Z., Asim, M., Chen, J., & ELAffendi, M. (2024) · Algorithms · doi:10.3390/a17030103 · Cited by 145 · see citing works
Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey of Datasets and Methods
Gao, B., Pan, Y., Li, C., Geng, S., & Zhao, H. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3076844 · Cited by 105 · see citing works
Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy
Zhang, X., Li, J., Zhou, J., Zhang, S., Wang, J., Yuan, Y., Liu, J., & Li, J. (2025) · Automotive Innovation · doi:10.1007/s42154-024-00310-2 · Cited by 103 · see citing works
Benchmark Analysis of YOLO Performance on Edge Intelligence Devices
Feng, H., Mu, G., Zhong, S., Zhang, P., & Yuan, T. (2022) · Cryptography · doi:10.3390/cryptography6020016 · Cited by 100 · see citing works
Emerging trends in autonomous vehicle perception: Multimodal fusion for 3D object detection
Alaba, S. Y., Gurbuz, A. C., & Ball, J. E. (2024) · World Electric Vehicle Journal · doi:10.3390/wevj15010020 · Cited by 82 · see citing works
A Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96 · see citing works
Rizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58 · see citing works
Zhu, J., Li, X., Jin, P., Xu, Q., Sun, Z., & Song, X. (2020) · Sensors · doi:10.3390/s21010027 · Cited by 51 · see citing works
“With the advance of large automobile datasets like KITTI [5], APOLLO [6], and BDD100K [7], the precision of vehicle detectors have gained huge improvement from the onboard perspective.”
MethodSELMA: SEmantic Large-Scale Multimodal Acquisitions in Variable Weather, Daytime and Viewpoints
Testolina, P., Barbato, F., Michieli, U., Giordani, M., Zanuttigh, P., & Zorzi, M. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3257086 · Cited by 41 · see citing works
- Authors
- Huang, X., Wang, P., Cheng, X., Zhou, D., Geng, Q., & Yang, R.
- Affiliations
- Baidu (China) (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2020-10-01
- Volume
- 42
- Issue
- 10
- Pages
- 2702-2719
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2892614179
- Cited by
- 615
- References
- 11
- Citations by year
- 2026: 42 · 2025: 77 · 2024: 115 · 2023: 112 · 2022: 107 · 2021: 83 · 2020: 62 · 2019: 15 · 2018: 2
- FWCI
- 30.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Keywords
- autonomous driving, semantic segmentation, instance segmentation, 3D mapping, self-localization, sensor fusion, GPS/IMU, multi-task learning, driving scene understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
