Precise Synthetic Image and LiDAR (PreSIL) Dataset for Autonomous Vehicle Perception
Open accessAbstract
We introduce the Precise Synthetic Image and LiDAR (PreSIL) dataset for autonomous vehicle perception. Grand Theft Auto V (GTA V), a commercial video game, has a large detailed world with realistic graphics, which provides a diverse data collection environment. Existing works creating synthetic LiDAR data for autonomous driving with GTA V have not released their datasets, rely on an in-game raycasting function which represents people as cylinders, and can fail to capture vehicles past 30 metres. Our work creates a precise LiDAR simulator within GTA V which collides with detailed models for all entities no matter the type or position. The PreSIL dataset consists of over 50,000 frames and includes high-definition images with full resolution depth information, semantic segmentation (images), point-wise segmentation (point clouds), and detailed annotations for all vehicles and people. Collecting additional data with our framework is entirely automatic and requires no human annotation of any kind. We demonstrate the effectiveness of our dataset by showing an improvement of up to 5% average precision on the KITTI 3D Object Detection benchmark challenge when state-of-the-art 3D object detection networks are pre-trained with our data. The data and code are available at https://tinyurl.com/y3tb9sxy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · see citing works
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., & Lopez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.352 · Cited by 2,338 · see citing works
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943 · see citing works
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697 · see citing works
Self-citationThe Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174 · see citing works
Wu, B., Wan, A., Yue, X., & Keutzer, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462926 · Cited by 967 · see citing works
Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks?
Johnson‐Roberson, M., Barto, C., Mehta, R., Sridhar, S. N., Rosaen, K., & Vasudevan, R. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989092 · Cited by 649 · see citing works
Richter, S. R., Hayder, Z., & Koltun, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.243 · Cited by 434 · see citing works
Yue, X., Wu, B., Seshia, S. A., Keutzer, K., & Sangiovanni-Vincentelli, A. L. (2018) · International Conference on Multimedia Retrieval · doi:10.1145/3206025.3206080 · Cited by 235 · see citing works
Playing for Data: Ground Truth from Computer GamesPreprint
Richter, S. R., Vineet, V., Roth, S., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.02192 · Cited by 158 · see citing works
Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving CarsPreprint
Martinez, M., Sitawarin, C., Finch, K., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01397 · Cited by 57 · see citing works
Wu, B., Wan, A., Yue, X., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.07368 · Cited by 57 · see citing works
A LiDAR Point Cloud Generator: from a Virtual World to Autonomous DrivingPreprint
Yue, X., Wu, B., Seshia, S. A., Keutzer, K., & Sangiovanni-Vincentelli, A. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00103 · Cited by 51 · see citing works
Leveraging Pre-Trained 3D Object Detection Models for Fast Ground Truth Generation
Lee, J., Walsh, S., Harakeh, A., & Waslander, S. L. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569793 · Cited by 33 · see citing works
Self-citationSimulating LIDAR Point Cloud for Autonomous Driving using Real-world Scenes and Traffic Flows.Preprint
Fang, J., Yan, F., Zhao, T., Zhang, F., Zhou, D., Yang, R., Ma, Y., & Wang, L. (2018) · arXiv (Cornell University) · Cited by 32 · see citing works
Unlimited Road-scene Synthetic Annotation (URSA) Dataset
Angus, M., ElBalkini, M., Khan, S., Harakeh, A., Andrienko, O., Reading, C., Waslander, S. L., & Czarnecki, K. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569519 · Cited by 23 · see citing works
Self-citationLeveraging Pre-Trained 3D Object Detection Models For Fast Ground Truth GenerationPreprint
Lee, J., Walsh, S., Harakeh, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06072 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- M. Racinsky, "3d map estimation from a single rgb image," Master's thesis, Czech Technical University in Prague, Czech Republic, 2018.
Cited by
Sorted by citations
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425 · see citing works
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Roberts, M., Ramapuram, J., Ranjan, A., Kumar, A., Bautista, M. A., Paczan, N., Webb, R., & Susskind, J. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01073 · Cited by 358 · see citing works
- Authors
- Hurl, B., Czarnecki, K., & Waslander, S.
- Affiliations
- University of Waterloo (CA)
- University of Toronto (CA)
- Venue
- 2019 IEEE Intelligent Vehicles Symposium (IV)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 2522-2529
- ISSN
- 1931-0587 (ISSN-L)
- OpenAlex
- W2942740757
- Cited by
- 122
- References
- 22
- Citations by year
- 2026: 5 · 2025: 18 · 2024: 16 · 2023: 30 · 2022: 26 · 2021: 17 · 2020: 9 · 2019: 1
- FWCI
- 13.18
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Intelligent Vehicles Symposium (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- autonomous driving perception, Grand Theft Auto V, point cloud segmentation, semantic segmentation, 3D object detection, KITTI dataset, synthetic image dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
