VirtualWorlds as Proxy for Multi-object Tracking Analysis
Abstract
Modern computer vision algorithms typically require expensive data acquisition and accurate manual labeling. In this work, we instead leverage the recent progress in computer graphics to generate fully labeled, dynamic, and photo-realistic proxy virtual worlds. We propose an efficient real-to-virtual world cloning method, and validate our approach by building and publicly releasing a new video dataset, called "Virtual KITTI", automatically labeled with accurate ground truth for object detection, tracking, scene and instance segmentation, depth, and optical flow. We provide quantitative experimental evidence suggesting that (i) modern deep learning algorithms pre-trained on real data behave similarly in real and virtual worlds, and (ii) pre-training on virtual data improves performance. As the gap between real and virtual worlds is small, virtual worlds enable measuring the impact of various weather and imaging conditions on recognition performance, all other things being equal. We show these factors may affect drastically otherwise high-performing deep models for tracking.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Virtual KITTI does not provide labels for the LiDAR pointclouds.
“Unfortunately, the dataset does not provide labels for the LiDAR pointclouds.”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [73]
Cites
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603 · see citing works
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918 · see citing works
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656 · see citing works
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876 · see citing works
A Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842 · see citing works
Bergstra, J., Yamins, D. L. K., & Cox, D. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 1,817 · see citing works
Global data association for multi-object tracking using network flows
Zhang, L., Li, Y., & Nevatia, R. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587584 · Cited by 945 · see citing works
Globally-optimal greedy algorithms for tracking a variable number of objects
Pirsiavash, H., Ramanan, D., & Fowlkes, C. C. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995604 · Cited by 790 · see citing works
Learning to Track: Online Multi-object Tracking by Decision Making
Yu, X., Alahi, A., & Savarese, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.534 · Cited by 684 · see citing works
Simulation as an engine of physical scene understanding
Battaglia, P. W., Hamrick, J. B., & Tenenbaum, J. B. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1306572110 · Cited by 681 · see citing works
Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera
Breitenstein, M. D., Reichlin, F., Leibe, B., Koller-Meier, E., & Gool, L. V. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.232 · Cited by 579 · see citing works
Continuous Energy Minimization for Multitarget Tracking
Milan, A., Roth, S., & Schindler, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.103 · Cited by 556 · see citing works
Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models
Aubry, M., Maturana, D., Efros, A. A., Russell, B. C., & Šivic, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.487 · Cited by 471 · see citing works
Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor
Choi, W. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.347 · Cited by 409 · see citing works
3D Traffic Scene Understanding From Movable Platforms
Geiger, A., Lauer, M., Wojek, C., Stiller, C., & Urtasun, R. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.185 · Cited by 397 · see citing works
Learning Deep Object Detectors from 3D Models
Peng, X., Sun, B., Ali, K., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.151 · Cited by 336 · see citing works
Motion Capture of Hands in Action Using Discriminative Salient Points
Ballan, L., Taneja, A., Gall, J., Gool, L. V., & Pollefeys, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_46 · Cited by 285 · see citing works
Teaching 3D geometry to deformable part models
Pepik, B., Stark, M. M., Gehler, P., & Schiele, B. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248075 · Cited by 209 · see citing works
Inferring 3D structure with a statistical image-based shape model
Grauman, Shakhnarovich, & Darrell (2003) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238408 · Cited by 197 · see citing works
Virtual and Real World Adaptation for Pedestrian Detection
Vázquez, D., López, A. M., Marín, J., Ponsa, D., & Gerónimo, D. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.163 · Cited by 177 · see citing works
Learning appearance in virtual scenarios for pedestrian detection
Marín, J., Vázquez, D., Gerónimo, D., & López, A. M. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540218 · Cited by 170 · see citing works
From Virtual to Reality: Fast Adaptation of Virtual Object Detectors to Real Domains
Sun, B., & Saenko, K. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.82 · Cited by 156 · see citing works
Back to the Future: Learning Shape Models from 3D CAD Data
Stark, M. M., Goesele, M., & Schiele, B. (2010) · British Machine Vision Conference (BMVC) · doi:10.5244/c.24.106 · Cited by 136 · see citing works
Learning scene-specific pedestrian detectors without real data
Hattori, H., Boddeti, V. N., Kitani, K., & Takeo Kanade, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299006 · Cited by 134 · see citing works
Picture: A probabilistic programming language for scene perception
Kulkarni, T. D., Kohli, P., Tenenbaum, J. B., & Mansinghka, V. K. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299068 · Cited by 133 · see citing works
OVVV: Using Virtual Worlds to Design and Evaluate Surveillance Systems
Taylor, G. R., Chosak, A. J., & Brewer, P. C. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383518 · Cited by 117 · see citing works
Differences between stereo and motion behaviour on synthetic and real-world stereo sequences
Vaudrey, T., Rabe, C., Klette, R., & Milburn, J. (2008) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz.2008.4762133 · Cited by 116 · see citing works
Data-Driven Scene Understanding from 3D Models
Satkin, S., Lin, J., & Hebert, M. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.128 · Cited by 80 · see citing works
Evaluation of image features using a photorealistic virtual world
Kaneva, B., Torralba, A. B., & Freeman, W. T. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126508 · Cited by 73 · see citing works
Multiple Object Tracking: A Review.Preprint
Luo, W., Zhao, X., & Kim, T. (2014) · arXiv (Cornell University) · Cited by 60 · see citing works
Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics ProgramsPreprint
Mansinghka, V. K., Kulkarni, T. D., Perov, Y., & Tenenbaum, J. B. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.0060 · Cited by 55 · see citing works
Learning Optimal Parameters For Multi-target Tracking
Wang, S., & Fowlkes, C. C. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.4 · Cited by 33 · see citing works
Broggi, A., Fascioli, A., Grisleri, P., Graf, T., & Meinecke, M.-M. (2006) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvpr.2005.495 · Cited by 32 · see citing works
Online Domain Adaptation for Multi-Object Tracking
Gaidon, A., & Vig, E. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.3 · Cited by 25 · see citing works
Self-citationHybrid Stochastic / Deterministic Optimization for Tracking Sports Players and Pedestrians
Collins, R. T., & Carr, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_20 · Cited by 25 · see citing works
Adaptation of Synthetic Data for Coarse-to-Fine Viewpoint Refinement
Busto, P. P., Liebelt, J., & Gall, J. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.14 · Cited by 19 · see citing works
Online, Real-Time Tracking Using a Category-to-Individual Detector
Hall, D. C., & Perona, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_24 · Cited by 12 · see citing works
Flagged first, then by citations
2 of the 41 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., & Gall, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00939 · Cited by 2,294 · see citing works
Learning from Simulated and Unsupervised Images through Adversarial Training
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., & Webb, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.241 · Cited by 1,831 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425 · see citing works
Multiple object tracking: A literature review
Luo, W., Xing, J., Milan, A., Zhang, X.-Q., Liu, W., Zhao, X., & Kim, T.-K. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2020.103448 · Cited by 1,095 · see citing works
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., & Birchfield, S. T. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00143 · Cited by 936 · see citing works
Learning from Synthetic Humans
Varol, G., Romero, J., Martín, X., Mahmood, N., Black, M. J., Laptev, I., & Schmid, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.492 · Cited by 915 · see citing works
Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00012 · Cited by 924 · see citing works
Data augmentation: A comprehensive survey of modern approaches
Mumuni, A., & Mumuni, F. (2022) · Array · doi:10.1016/j.array.2022.100258 · Cited by 891 · see citing works
Block-NeRF: Scalable Large Scene Neural View Synthesis
Tancik, M., Casser, V., Yan, X., Pradhan, S., Mildenhall, B., Srinivasan, P. P., Barron, J. T., & Kretzschmar, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00807 · Cited by 808 · see citing works
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763 · see citing works
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D
Liao, Y., Xie, J., & Geiger, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3179507 · Cited by 770 · see citing works
A Survey on 3D Object Detection Methods for Autonomous Driving Applications
Arnold, E. H., Al-Jarrah, O. Y., Dianati, M., Fallah, S., Oxtoby, D., & Mouzakitis, A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2892405 · Cited by 720 · see citing works
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
Dwibedi, D., Misra, I., & Hebert, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.146 · Cited by 687 · see citing works
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y. X., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 647 · see citing works
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
Prakash, A., Chitta, K., & Geiger, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00700 · Cited by 616 · see citing works
Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
Pohlen, T., Hermans, A., Mathias, M., & Leibe, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.353 · Cited by 591 · see citing works
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · see citing works
VisDA: The Visual Domain Adaptation ChallengePreprint
Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06924 · Cited by 667 · see citing works
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving
Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., & Geiger, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200245 · Cited by 528 · see citing works
BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks
Yao, Y., Luo, Z., Li, S., Zhang, J., Ren, Y., Zhou, L., Fang, T., & Quan, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00186 · Cited by 502 · see citing works
DF-Net: Unsupervised Joint Learning of Depth and Flow Using Cross-Task Consistency
Zou, Y., Luo, Z., & Huang, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_3 · Cited by 484 · see citing works
TartanAir: A Dataset to Push the Limits of Visual SLAM
Wang, W., Zhu, D., Wang, X., Hu, Y., Qiu, Y., Wang, C., Hu, Y., Kapoor, A., & Scherer, S. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341801 · Cited by 456 · see citing works
DSEC: A Stereo Event Camera Dataset for Driving Scenarios
Gehrig, M., Aarents, W., Gehrig, D., & Scaramuzza, D. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068942 · Cited by 444 · see citing works
GMFlow: Learning Optical Flow via Global Matching
Xu, H., Zhang, J., Cai, J., Rezatofighi, H., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00795 · Cited by 437 · see citing works
Richter, S. R., Hayder, Z., & Koltun, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.243 · Cited by 434 · see citing works
Augmented Reality Meets Computer Vision: Efficient Data Generation for Urban Driving Scenes
Alhaija, H. A., Mustikovela, S. K., Mescheder, L., Geiger, A., & Rother, C. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1070-x · Cited by 434 · see citing works
VirtualHome: Simulating Household Activities Via Programs
Puig, X., Ra, K., Boben, M., Li, J., Wang, T., Fidler, S., & Torralba, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00886 · Cited by 378 · see citing works
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Yu, J. J., Harley, A. W., & Derpanis, K. G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_1 · Cited by 377 · see citing works
KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving
Choi, Y., Kim, N., Hwang, S., Park, K., Yoon, J. S., An, K., & Kweon, I. S. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2791533 · Cited by 370 · see citing works
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Roberts, M., Ramapuram, J., Ranjan, A., Kumar, A., Bautista, M. A., Paczan, N., Webb, R., & Susskind, J. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01073 · Cited by 358 · see citing works
Street-view change detection with deconvolutional networks
Alcantarilla, P. F., Stent, S., Ros, G., Arroyo, R., & Gherardi, R. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9734-5 · Cited by 323 · see citing works
Alexopoulos, K., Nikolakis, N., & Chryssolouris, G. (2020) · International Journal of Computer Integrated Manufacturing · doi:10.1080/0951192x.2020.1747642 · Cited by 312 · see citing works
A Review of Deep Learning-Based Semantic Segmentation for Point Cloud
Zhang, J., Zhao, X., Chen, Z., & Lu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2958671 · Cited by 304 · see citing works
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape
Manhardt, F., Kehl, W., & Gaidon, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00217 · Cited by 305 · see citing works
Self-citationSceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?
McCormac, J., Handa, A., Leutenegger, S., & Davison, A. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.292 · Cited by 291 · see citing works
Learning Guided Convolutional Network for Depth Completion
Tang, J., Tian, F.-P., Feng, W., Li, J., & Tan, P. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3040528 · Cited by 294 · see citing works
Semantic segmentation using Vision Transformers: A survey
Thisanke, H., Deshan, C., Chamith, K., Seneviratne, S., Vidanaarachchi, R., & Herath, D. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106669 · Cited by 288 · see citing works
3D Recurrent Neural Networks with Context Fusion for Point Cloud Semantic Segmentation
Ye, X., Li, J., Huang, H., Du, L., & Zhang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_25 · Cited by 287 · see citing works
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Prakash, A., Boochoon, S., Brophy, M., Acuna, D., Cameracci, E., State, G., Shapira, O., & Birchfield, S. T. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794443 · Cited by 287 · see citing works
Dissecting Person Re-Identification From the Viewpoint of Viewpoint
Sun, X., & Zheng, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00070 · Cited by 284 · see citing works
SGM-Nets: Semi-Global Matching with Neural Networks
Seki, A., & Pollefeys, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.703 · Cited by 280 · see citing works
Atapour–Abarghouei, A., & Breckon, T. P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00296 · Cited by 278 · see citing works
Learning Motion Patterns in Videos
Tokmakov, P., Alahari, K., & Schmid, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.64 · Cited by 272 · see citing works
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
Gustafsson, F., Danelljan, M., & Schön, T. B. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00167 · Cited by 272 · see citing works
UnrealCV: Connecting Computer Vision to Unreal Engine
Qiu, W., & Yuille, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_75 · Cited by 267 · see citing works
Fake it till you make it: face analysis in the wild using synthetic data alone
Wood, E., Baltrušaitis, T., Hewitt, C., Dziadzio, S., Cashman, T. J., & Shotton, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00366 · Cited by 257 · see citing works
Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds
Engelmann, F., Kontogianni, T., Hermans, A., & Leibe, B. (2017) · RWTH Publications (RWTH Aachen) · Cited by 254 · see citing works
Joint Monocular 3D Vehicle Detection and Tracking
Hu, H.-N., Cai, Q.-Z., Wang, D., Lin, J., Sun, M., Kraehenbuehl, P., Darrell, T., & Yu, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00549 · Cited by 249 · see citing works
DrivingStereo: A Large-Scale Dataset for Stereo Matching in Autonomous Driving Scenarios
Yang, G., Song, X., Huang, C., Deng, Z., Shi, J., & Zhou, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00099 · Cited by 253 · see citing works
Are We Ready for Autonomous Drone Racing? The UZH-FPV Drone Racing Dataset
Delmerico, J., Cieslewski, T., Rebecq, H., Faessler, M., & Scaramuzza, D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793887 · Cited by 238 · see citing works
Chen, Y., Li, W., Chen, X., & Gool, L. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00194 · Cited by 236 · see citing works
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World
Manivasagam, S., Wang, S., Wong, K., Zeng, W., Sazanovich, M., Tan, S., Yang, B., Ma, W.-C., & Urtasun, R. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01118 · Cited by 241 · see citing works
Kubric: A scalable dataset generator
Greff, K., Belletti, F., Beyer, L., Doersch, C., Du, Y., Duckworth, D., Fleet, D. J., Gnanapragasam, D., Golemo, F., Herrmann, C., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00373 · Cited by 235 · see citing works
Deep Learning on Point Clouds and Its Application: A Survey
Liu, W., Sun, J., Li, W., Hu, T., & Wang, P. (2019) · Sensors · doi:10.3390/s19194188 · Cited by 225 · see citing works
Co-fusion: Real-time segmentation, tracking and fusion of multiple objects
Rünz, M., & Agapito, L. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989518 · Cited by 230 · see citing works
Meta-Sim: Learning to Generate Synthetic DatasetsPreprint
(2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 228 · see citing works
Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., & Markham, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00494 · Cited by 226 · see citing works
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
Kundu, A., Genova, K., Yin, X., Fathi, A., Pantofaru, C., Guibas, L. J., Tagliasacchi, A., Dellaert, F., & Funkhouser, T. A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01253 · Cited by 223 · see citing works
Guo, J., Kurup, U., & Shah, M. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2926042 · Cited by 233 · see citing works
Learning to Track with Object Permanence
Tokmakov, P., Li, J., Burgard, W., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01068 · Cited by 226 · see citing works
Self-citationLearning Monocular Depth by Distilling Cross-Domain Stereo Networks
Guo, X., Li, H., Yi, S., Ren, J., & Wang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_30 · Cited by 208 · see citing works
AADS: Augmented autonomous driving simulation using data-driven algorithms
Li, W., Pan, C., Zhang, R., Ren, J., Ma, Y., Fang, J., Yan, F., Geng, Q., Huang, X., Gong, H. J., et al. (2019) · Science Robotics · doi:10.1126/scirobotics.aaw0863 · Cited by 210 · see citing works
Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds
Engelmann, F., Kontogianni, T., Hermans, A., & Leibe, B. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.90 · Cited by 280 · see citing works
Auxiliary Tasks in Multi-task LearningPreprint
Liebel, L., & Körner, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06334 · Cited by 269 · see citing works
Structure-Guided Ranking Loss for Single Image Depth Prediction
Xian, K., Zhang, J., Wang, O., Mai, L., Lin, Z., & Cao, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00069 · Cited by 201 · see citing works
Cross-Domain Self-Supervised Multi-task Feature Learning Using Synthetic Imagery
Ren, Z., & Lee, Y. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00086 · Cited by 192 · see citing works
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
Zhao, S., Fu, H., Gong, M., & Tao, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01002 · Cited by 190 · see citing works
Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation
Lee, S., Kim, D.-W., Kim, N., & Jeong, S.-G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00018 · Cited by 188 · see citing works
A Survey on Deep Visual Place Recognition
Masone, C., & Barbara Caputo, B. (2021) · IEEE Access · doi:10.1109/access.2021.3054937 · Cited by 196 · see citing works
Multi-Modal 3D Object Detection in Autonomous Driving: A Survey
Wang, Y., Mao, Q., Zhu, H., Deng, J., Zhang, Y., Ji, J., Li, H., & Zhang, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01784-z · Cited by 214 · see citing works
What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?
Mayer, N., Ilg, E., Fischer, P., Hazirbas, C., Cremers, D., Dosovitskiy, A., & Brox, T. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1082-6 · Cited by 191 · see citing works
AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation
Kundu, J. N., Uppala, P. K., Pahuja, A., & Babu, R. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00281 · Cited by 184 · see citing works
Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation
Tremblay, J., To, T., & Birchfield, S. T. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00275 · Cited by 188 · see citing works
The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
Liang, J., Jiang, L., Murphy, K. P., Yu, T., & Hauptmann, A. G. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01052 · Cited by 183 · see citing works
Learning Joint 2D-3D Representations for Depth Completion
Chen, Y., Yang, B., Liang, M., & Urtasun, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01012 · Cited by 179 · see citing works
Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
Müller, M., Casser, V., Lahoud, J., Smith, N., & Ghanem, B. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1073-7 · Cited by 178 · see citing works
Li, X., Dong, Y., Peers, P., & Tong, X. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073641 · Cited by 174 · see citing works
Towards Real-Time Monocular Depth Estimation for Robotics: A Survey
Dong, X., Garratt, M., Anavatti, S. G., & Abbass, H. A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3160741 · Cited by 173 · see citing works
Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges
Siam, M., Elkerdawy, S., Jagersand, M., & Yogamani, S. (2017) · 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317714 · Cited by 178 · see citing works
Next-generation deep learning based on simulators and synthetic data
de Melo, C. M., Torralba, A. B., Guibas, L. J., DiCarlo, J. J., Chellappa, R., & Hodgins, J. K. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.11.008 · Cited by 173 · see citing works
WildDash - Creating Hazard-Aware Benchmarks
Zendel, O., Honauer, K., Murschitz, M., Steininger, D., & Domínguez, G. F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_25 · Cited by 164 · see citing works
Augmented LiDAR Simulator for Autonomous Driving
Fang, J., Zhou, D., Yan, F., Zhao, T., Zhang, F., Ma, Y., Wang, L., & Yang, R. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969927 · Cited by 163 · see citing works
Spatial Fusion GAN for Image Synthesis
Zhan, F., Zhu, H., & Lu, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00377 · Cited by 160 · see citing works
CGIntrinsics: Better Intrinsic Image Decomposition Through Physically-Based Rendering
Li, Z., & Snavely, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_23 · Cited by 158 · see citing works
Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems
Zhang, S., Li, J., Shi, L. X., Ding, M., Nguyen, D. C., Tan, W., Weng, J., & Han, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3324962 · Cited by 158 · see citing works
T$$^2$$Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
Zheng, C., Cham, T., & Cai, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_47 · Cited by 154 · see citing works
Simulating Content Consistent Vehicle Datasets with Attribute Descent
Yao, Y., Zheng, L., Yang, X., Naphade, M., & Gedeon, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_46 · Cited by 154 · see citing works
RigNet: Repetitive Image Guided Network for Depth Completion
Yan, Z., Wang, K., Li, X., Zhang, Z., Li, J., & Yang, J. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_13 · Cited by 151 · see citing works
Supervising the New with the Old: Learning SFM from SFM
Klodt, M., & Vedaldi, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_43 · Cited by 147 · see citing works
Restricted Deformable Convolution-Based Road Scene Semantic Segmentation Using Surround View Cameras
Deng, L., Yang, M. Z., Li, H., Li, T., Hu, B. Q., & Wang, C. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2939832 · Cited by 143 · see citing works
Neural RGB®D Sensing: Depth and Uncertainty From a Video Camera
Liu, C., Gu, J., Kim, K., Narasimhan, S. G., & Kautz, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01124 · Cited by 140 · see citing works
Photorealistic Image Synthesis for Object Instance Detection
Hodaň, T., Vineet, V., Gal, R., Shalev, E., Hanzelka, J., Connell, T., Urbina, P., Sinha, S. N., & Guenter, B. K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803821 · Cited by 145 · see citing works
Guerra, W., Tal, E. A., Murali, V., Ryou, G., & Karaman, S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968116 · Cited by 139 · see citing works
A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
Zhang, Y., David, P., Foroosh, H., & Gong, B. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2903401 · Cited by 142 · see citing works
RarePlanes: Synthetic Data Takes Flight
Shermeyer, J., Hossler, T., Etten, A. V., Hogan, D., Lewis, R., & Kim, D. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00025 · Cited by 142 · see citing works
Full Waveform LiDAR for Adverse Weather Conditions
Wallace, A. M., Halimi, A., & Buller, G. S. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2989148 · Cited by 143 · see citing works
Robust Dense Mapping for Large-Scale Dynamic Environments
Bârsan, I. A., Liu, P., Pollefeys, M., & Geiger, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462974 · Cited by 136 · see citing works
DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion
Shivakumar, S. S., Nguyen, T., Miller, I. D., Chen, S. W., Kumar, V., & Taylor, C. J. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917294 · Cited by 138 · see citing works
- Authors
- Gaidon, A., Wang, Q., Cabon, Y., & Vig, E.
- Affiliations
- Xerox (France) (FR)
- Arizona State University (US)
- Deutsches Zentrum für Luft- und Raumfahrt e. V. (DLR) (DE)
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 4340-4349
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2397830550
- Cited by
- 1,079
- References
- 41
- Citations by year
- 2026: 47 · 2025: 94 · 2024: 123 · 2023: 128 · 2022: 106 · 2021: 171 · 2020: 131 · 2019: 149 · 2018: 76 · 2017: 50 · 2016: 4
- FWCI
- 32.40
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- elib (German Aerospace Center) (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- virtual worlds, multi-target tracking, synthetic data, object detection, instance segmentation, depth estimation, optical flow, computer graphics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review