V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting
Open accessAbstract
Utilizing infrastructure and vehicle-side information to track and forecast the behaviors of surrounding traffic participants can significantly improve decision-making and safety in autonomous driving. However, the lack of real-world sequential datasets limits research in this area. To address this issue, we introduce V2X-Seq, the first large-scale sequential V2X dataset, which includes data frames, trajectories, vector maps, and traffic lights captured from natural scenery. V2X-Seq comprises two parts: the sequential perception dataset, which includes more than 15,000 frames captured from 95 scenarios, and the trajectory forecasting dataset, which contains about 80,000 infrastructure-view scenarios, 80,000 vehicle-view scenarios, and 50,000 cooperative-view scenarios captured from 28 intersections' areas, covering 672 hours of data. Based on V2X-Seq, we introduce three new tasks for vehicle-infrastructure cooperative (VIC) autonomous driving: VIC3D Tracking, Online-VIC Forecasting, and Offline-VIC Forecasting. We also provide benchmarks for the introduced tasks. Find data, code, and more up-to-date information at https://github.com/AIR-THU/DAIR-V2X-Seq.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The main task of V2X-Seq SPD is VIC3D tracking, with evaluation metrics MOTA, MOTP and byte per second (BPS).
“The main task is VIC3D tracking, the metrics for evaluation are MOTA, MOTP and byte per second (BPS).”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
Reference [132] is the first large scale sequential V2X dataset from real world.
“Reference [132] is the first large scale sequential V2X dataset from real world.”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
V2X-Seq focuses on tracking and forecasting tasks, and contains a sequential perception dataset (SPD) and a trajectory forecasting dataset (TFD).
“It focuses on tracking and forecasting tasks, which contains: sequential perception dataset (SPD) and trajectory forecasting dataset (TFD).”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
For the forecasting task, the evaluation metrics are minADE, minFDE and MR.
“For forecasting task, the evaluation metrics are minADE, minFDE and MR.”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
Tracking tasks supported by TFD include Online-VIC trajectory Forecasting and Offline-VIC trajectory Forecasting.
“Tracking tasks supported by TFD include Online-VIC trajectory Forecasting and Offline-VIC trajectory Forecasting.”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
The SPD is built on the basis of DAIR-V2X-C.
“The SPD is built on the basis of DAIR-V2X-C.”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
Based on V2X-Seq, the team proposes a new middle fusion method: FF-Tracking, which can effectively complete the VIC3D tracking task with low latency.
“Based on V2X-Seq, the team proposes new middle fusion method: FF-Tracking, which can effectively complete the VIC3D tracking task with low latency.”
From Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy (2025) · cites this paper as [132]
Cites
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
Self-citationPointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
Xu, R., Xiang, H., Xia, X., Han, X., Li, J., & Ma, J. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812038 · Cited by 586 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Vehicle-to-Everything Communication in Intelligent Connected Vehicles: A Survey and Taxonomy
Zhang, X., Li, J., Zhou, J., Zhang, S., Wang, J., Yuan, Y., Liu, J., & Li, J. (2025) · Automotive Innovation · doi:10.1007/s42154-024-00310-2 · Cited by 103 · see citing works
- Authors
- Yu, H., Yang, W., Ruan, H., Yang, Z., Tang, Y., Gao, X., Hao, X., Shi, Y., Pan, Y., Sun, N., Song, J., Yuan, J., Luo, P., & Nie, Z.
- Affiliations
- University of Hong Kong (HK)
- Tsinghua University (CN)
- University of Chinese Academy of Sciences (CN)
- University of Science and Technology Beijing (CN)
- Beijing Institute of Technology (CN)
- Baidu (China) (CN)
- Beijing Connected and Autonomous Vehicles Technology Co., Ltd
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 5486-5495
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076547
- Cited by
- 182
- References
- 4
- Citations by year
- 2026: 49 · 2025: 83 · 2024: 48 · 2023: 2
- FWCI
- 56.73
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Baidu
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Building and Construction › Traffic Prediction and Management Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Traffic control and management
- Keywords
- autonomous driving, trajectory prediction, 3D tracking, traffic signals, intersections, vector map
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
