The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data
Open accessAbstract
We propose to harness the potential of simulation for semantic segmentation of real-world self-driving scenes in a domain generalization fashion. The segmentation network is trained without any information about target domains and tested on the unseen target domains. To this end, we propose a new approach of domain randomization and pyramid consistency to learn a model with high generalizability. First, we propose to randomize the synthetic images with styles of real images in terms of visual appearances using auxiliary datasets, in order to effectively learn domain-invariant representations. Second, we further enforce pyramid consistency across different "stylized" images and within an image, in order to learn domain-invariant and scale-invariant features, respectively. Extensive experiments are conducted on generalization from GTA and SYNTHIA to Cityscapes, BDDS, and Mapillary; and our method achieves superior results over the state-of-the-art techniques. Remarkably, our generalization results are on par with or even better than those obtained by state-of-the-art simulation-to-real domain adaptation methods, which access the target domain data at training time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 4×
“This is partially due to the community’s recent exploration to embodied vision [46, 62, 2], in which the perception has to be embodied and purposive for an agent to actively perceive and/or navigate through a physical environment [7, 10].”
BackgroundInfluentialSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. · doi:10.1109/icdar.2003.1227801 · Cited by 2,741
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1711.03213 · Cited by 2,656
Unsupervised Domain Adaptation by Backpropagation
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 2,521
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., & Lopez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.352 · Cited by 2,338
Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
Saito, K., Watanabe, K., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00392 · Cited by 2,161
(2013) · Cited by 2,056
Playing for Data: Ground Truth from Computer Games
Richter, S. R., Vineet, V., Roth, S., & Koltun, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_7 · Cited by 1,815
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533
Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training
Zou, Y., Yu, Z., Vijaya Kumar, B. V. K., & Wang, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_18 · Cited by 1,464
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
Neuhold, G., Ollmann, T., Bulo, S. R., & Kontschieder, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.534 · Cited by 1,434
Learning to Adapt Structured Output Space for Semantic Segmentation
Tsai, Y.-H., Hung, W.-C., Schulter, S., Sohn, K., Yang, M.-H., & Chandraker, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00780 · Cited by 1,430
Learning to Generalize: Meta-Learning for Domain Generalization
Li, D., Yang, Y., Song, Y.-Z., & Hospedales, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11596 · Cited by 1,251
Deeper, Broader and Artier Domain Generalization
Li, D., Yang, Y., Song, Y.-Z., & Hospedales, T. M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.591 · Cited by 1,237
Domain Generalization with Adversarial Feature Learning
Li, H., Pan, S. J., Wang, S., & Kot, A. C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00566 · Cited by 1,213
AutoAugment: Learning Augmentation Policies from Data
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2018) · arXiv · doi:10.48550/arxiv.1805.09501 · Cited by 1,106
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
Pan, X., Luo, P., Shi, J., & Tang, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_29 · Cited by 903
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., & Birchfield, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00143 · Cited by 884
Domain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
(2018) · arXiv (Cornell University) · Cited by 679
Domain Generalization for Object Recognition with Multi-task Autoencoders
Ghifary, M., Kleijn, W. B., Zhang, M., & Balduzzi, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.293 · Cited by 664
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460875 · Cited by 636
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
Hoffman, J., Wang, D., Yu, F., & Darrell, T. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02649 · Cited by 602
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578
Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
Sundermeyer, M., Marton, Z.-C., Durner, M., Brucker, M., & Triebel, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_43 · Cited by 564
Image to Image Translation for Domain Adaptation
Murez, Z., Kolouri, S., Kriegman, D., Ramamoorthi, R., & Kim, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00473 · Cited by 540
VisDA: The Visual Domain Adaptation Challenge
Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06924 · Cited by 504
Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
Zhang, Y., David, P., & Gong, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.223 · Cited by 494
Self-citationMetaReg: towards domain generalization using meta-regularization
Balaji, Y., Sankaranarayanan, S., & Chellappa, R. (2018) · neural information processing systems · Cited by 476
Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation
Sankaranarayanan, S., Balaji, Y., Jain, A., Lim, S. N., & Chellappa, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00395 · Cited by 472
Dataset Augmentation in Feature SpacePreprint
DeVries, T., & Taylor, G. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05538 · Cited by 461
Smart Augmentation Learning an Optimal Data Augmentation Strategy
Lemley, J., Bazrafkan, S., & Corcoran, P. (2017) · IEEE Access · doi:10.1109/access.2017.2696121 · Cited by 428
Das, A., Datta, S., Gkioxari, G., Lee, S., Parikh, D., & Batra, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00008 · Cited by 410
Fully Convolutional Adaptation Networks for Semantic Segmentation
Zhang, Y., Qiu, Z., Yao, T., Liu, D., & Mei, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00712 · Cited by 356
Self-citationConditional Generative Adversarial Network for Structured Domain Adaptation
Hong, W., Wang, Z., Yang, M., & Yuan, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00145 · Cited by 309
ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes
Chen, Y., Li, W., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00823 · Cited by 291
Das, A., Datta, S., Gkioxari, G., Lee, S., Parikh, D., & Batra, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00279 · Cited by 288
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Prakash, A., Boochoon, S., Brophy, M., Acuna, D., Cameracci, E., State, G., Shapira, O., & Birchfield, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794443 · Cited by 266
DCAN: Dual Channel-Wise Alignment Networks for Unsupervised Scene Adaptation
Wu, Z., Han, X., Lin, Y.-L., Uzunbas, M. G., Goldstein, T., Lim, S. N., & Davis, L. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_32 · Cited by 241
A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
Yue, X., Wu, B., Seshia, S. A., Keutzer, K., & Sangiovanni-Vincentelli, A. L. (2018) · Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval · doi:10.1145/3206025.3206080 · Cited by 208
Self-citationLearning Attributes Equals Multi-Source Domain Generalization
Gan, C., Yang, T., & Gong, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.17 · Cited by 189
Self-citationA Bayesian Data Augmentation Approach for Learning Deep Models
Tran, T. B., Pham, T. T., Carneiro, G., Palmer, L. J., & Reid, I. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10564 · Cited by 180
A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
Zhang, Y., David, P., Foroosh, H., & Gong, B. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2903401 · Cited by 146
Self-citationAPAC: Augmented PAttern Classification with Neural NetworksPreprint
Sato, I., Nishimura, H., & Yokoi, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.03229 · Cited by 141
Reshaping Visual Datasets for Domain Adaptation
Gong, B., Grauman, K. L., & Sha, F. (2013) · Cited by 131
Self-citationDomain Transfer Through Deep Activation Matching
Huang, H., Huang, Q., & Krähenbühl, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_36 · Cited by 111
Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
Zhu, X., Zhou, H., Yang, C., Shi, J., & Lin, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_35 · Cited by 110
Laplacian-Steered Neural Style Transfer
Li, S., Xu, X., Nie, L., & Chua, T.-S. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123425 · Cited by 103
Counterexample-Guided Data Augmentation
Dreossi, T., Ghosh, S., Yue, X., Keutzer, K., Sangiovanni-Vincentelli, A., & Seshia, S. A. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/286 · Cited by 60
Self-citationA LiDAR Point Cloud Generator: from a Virtual World to Autonomous DrivingPreprint
Yue, X., Wu, B., Seshia, S. A., Keutzer, K., & Sangiovanni-Vincentelli, A. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00103 · Cited by 51
Self-citation
Cited by
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
Zheng, Z., & Yang, Y. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01395-y · Cited by 545
Liu, Q., Chen, C., Qin, J., Dou, Q., & Heng, P.-A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00107 · Cited by 541
Metadata
- Authors
- Yue, X., Zhang, Y., Zhao, S., Sangiovanni-Vincentelli, A., Keutzer, K., & Gong, B.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2100-2110
- OpenAlex
- W2981540341
- Cited by
- 374
- References
- 66
- FWCI
- 13.72
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.