When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets
Abstract
Data collection on public roads has been deemed a valuable activity along with the development of self-driving vehicles. The vehicle for data collection is typically equipped with a variety of sensors such as camera, LiDAR, radar, GPS, and IMU. The raw data of all sensors is logged on a disk while the vehicle is manually driven. The logged data can be subsequently used for training and testing different algorithms for autonomous driving, e.g., vehicle/pedestrian detection and tracking, SLAM, and motion estimation. Data collection is time-consuming and can sometimes be avoided by directly using existing datasets including sensor data collected by other researchers. A multitude of openly available datasets have been released to foster the research on automated driving. These datasets vary a lot in terms of traffic conditions, application focus, sensor setup, data format, size, tool support, and many other aspects. This paper presents an overview of 27 existing publicly available datasets containing data collected on public roads, compares each other from different perspectives, and provides guidelines for selecting the most suitable dataset for different purposes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632 · see citing works
Semantic object classes in video: A high-definition ground truth database
Brostow, G., Fauqueur, J., & Cipolla, R. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.04.005 · Cited by 1,400 · see citing works
Pedestrian detection: A benchmark
Dollár, P., Wojek, C., Schiele, B., & Perona, P. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206631 · Cited by 1,335 · see citing works
StereoScan: Dense 3d reconstruction in real-time
Geiger, A., Ziegler, J., & Stiller, C. (2011) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2011.5940405 · Cited by 1,092 · see citing works
Computer Vision for Autonomous Vehicles
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 664 · see citing works
Ford Campus vision and lidar data set
Pandey, G., McBride, J. R., & Eustice, R. M. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911400640 · Cited by 361 · see citing works
The 2005 DARPA Grand Challenge
Buehler, M., Singh, S., & Iagnemma, K. (2007) · Springer tracts in advanced robotics · doi:10.1007/978-3-540-73429-1 · Cited by 277 · see citing works
Cordts, M., Omran, M., Ramos, S., Scharwächter, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2015) · Cited by 271 · see citing works
The Málaga urban dataset: High-rate stereo and LiDAR in a realistic urban scenario
Blanco-Claraco, J.-L., Moreno-Dueñas, F.-Á., & González-Jiménez, J. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364913507326 · Cited by 262 · see citing works
Robust Multiperson Tracking from a Mobile Platform
Ess, A., Leibe, B., Schindler, K., & Gool, L. V. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.109 · Cited by 251 · see citing works
Learning a Driving SimulatorPreprint
Santana, E., & Hotz, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.01230 · Cited by 226 · see citing works
Towards 3D object recognition via classification of arbitrary object tracks
Teichman, A., Levinson, J., & Thrun, S. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5979636 · Cited by 173 · see citing works
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2017) · arXiv · doi:10.48550/arxiv.1704.05519 · Cited by 141 · see citing works
DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving
Alletto, S., Palazzi, A., Solera, F., Calderara, S., & Cucchiara, R. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.14 · Cited by 136 · see citing works
A new benchmark for vision-based cyclist detection
Li, X., Flohr, F. B., Yang, Y., Xiong, H., Braun, M., Pan, S., Li, K., & Gavrila, D. M. (2016) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2016.7535515 · Cited by 118 · see citing works
Joint 3D Estimation of Objects and Scene Layout
Geiger, A., Wojek, C., & Urtasun, R. (2011) · Cited by 97 · see citing works
Exploiting the Power of Stereo Confidences
Pfeiffer, D., Gehrig, S. K., & Schneider, N. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.45 · Cited by 90 · see citing works
Joint Attention in Autonomous Driving (JAAD)Preprint
Kotseruba, I., Rasouli, A., & Tsotsos, J. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.04741 · Cited by 87 · see citing works
Outdoor stereo camera system for the generation of real-world benchmark data sets
Meister, S., Jaehne, B., & Kondermann, D. (2012) · Optical Engineering · doi:10.1117/1.oe.51.2.021107 · Cited by 73 · see citing works
Stixmantics: A Medium-Level Model for Real-Time Semantic Scene Understanding
Scharwächter, T., Enzweiler, M., Franke, U., & Roth, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_35 · Cited by 58 · see citing works
How Much of Driving Is Preattentive?
Pugeault, N., & Bowden, R. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2487826 · Cited by 50 · see citing works
Towards Ubiquitous Autonomous Driving: The CCSAD Dataset
Guzmán, R., Hayet, J.-B., & Klette, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23192-1_49 · Cited by 40 · see citing works
A Multi-sensor Traffic Scene Dataset with Omnidirectional Video
Koschorrek, P., Piccini, T., Öberg, P. Å., Felsberg, M., Nielsen, L., & Mester, R. (2013) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2013.110 · Cited by 27 · see citing works
Performance Evaluation of Stereo and Motion Analysis on Rectified Image Sequences
Liu, Z. (2007) · ResearchSpace (University of Auckland) · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A survey of deep learning techniques for autonomous driving
Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) · Journal of Field Robotics · doi:10.1002/rob.21918 · Cited by 1,809 · see citing works
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425 · see citing works
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261 · see citing works
A Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96 · see citing works
Rizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58 · see citing works
- Authors
- Yin, H., & Berger, C.
- Affiliations
- Chalmers University of Technology (SE)
- University of Gothenburg (SE)
- ORCID
- Berger, Christian
- Venue
- 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1-8
- OpenAlex
- W2790504560
- Cited by
- 79
- References
- 27
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 17 · 2023: 6 · 2022: 12 · 2021: 10 · 2020: 12 · 2019: 11 · 2018: 3
- FWCI
- 7.90
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Intelligent Transportation Systems (ITSC) (publishedVersion)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- autonomous driving, driving data, sensor data, data comparison, vehicle detection, pedestrian detection, tracking, SLAM, motion estimation, LiDAR, camera, radar, GPS, inertial measurement unit
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
