The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels
Open accessAbstract
Robots have to face challenging perceptual settings, including changes in viewpoint, lighting, and background. Current simulated reinforcement learning (RL) benchmarks such as DM Control provide visual input without such complexity, which limits the transfer of well-performing methods to the real world. In this paper, we extend DM Control with three kinds of visual distractions (variations in background, color, and camera pose) to produce a new challenging benchmark for vision-based control, and we analyze state of the art RL algorithms in these settings. Our experiments show that current RL methods for vision-based control perform poorly under distractions, and that their performance decreases with increasing distraction complexity, showing that new methods are needed to cope with the visual complexities of the real world. We also find that combinations of multiple distraction types are more difficult than a mere combination of their individual effects.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
All DeepMind Control Suite variants focus on visual generalisation and sample efficiency, require learning continuous control policies from pixels, and introduce visual distractors that the policy should be invariant to, either available during training or only present at deployment.
“All these environments focus on visual generalisation and sample efficiency, require learning continuous control policies from pixels and introduce visual distractors that the policy should be invariant to which are either available during training or only present at deployment.”
From A Survey of Zero-shot Generalisation in Deep Reinforcement Learning (2023) · cites this paper as Stone et al., 2021
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe hardest combinations of variations in Distracting Control Suite are unsolvable by current methods.
“the hardest combinations of which are unsolvable by current methods.”
From A Survey of Zero-shot Generalisation in Deep Reinforcement Learning (2023) · cites this paper as Stone et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · see citing works
The 2018 DAVIS Challenge on Video Object SegmentationPreprint
Caelles, S., Montes, A., Maninis, K.-K., Chen, Y., Gool, L. V., Perazzi, F., & Pont-Tuset, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00557 · Cited by 763 · see citing works
Learning Latent Dynamics for Planning from Pixels
Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., & Davidson, J. (2018) · arXiv · doi:10.48550/arxiv.1811.04551 · Cited by 708 · see citing works
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697 · see citing works
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618 · see citing works
Solving Rubik's Cube with a Robot Hand
OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.07113 · Cited by 602 · see citing works
CURL: Contrastive Unsupervised Representations for Reinforcement LearningPreprint
Srinivas, A., Laskin, M., & Abbeel, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04136 · Cited by 521 · see citing works
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. D. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1801.00690 · Cited by 497 · see citing works
Levine, S., & Koltun, V. (2013) · Cited by 481 · see citing works
Learning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 314 · see citing works
Reinforcement Learning with Augmented Data
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., & Srinivas, A. (2020) · arXiv · doi:10.48550/arxiv.2004.14990 · Cited by 299 · see citing works
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
Kostrikov, I., Yarats, D., & Fergus, R. (2020) · arXiv · doi:10.48550/arxiv.2004.13649 · Cited by 267 · see citing works
Self-Supervised Policy Adaptation during Deployment
Hansen, N., Jangir, R., Sun, Y., Alenyà, G., Abbeel, P., Efros, A. A., Pinto, L., & Wang, X.-L. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2007.04309 · Cited by 212 · see citing works
Learning Invariant Representations for Reinforcement Learning without Reconstruction
Zhang, A., McAllister, R., Calandra, R., Gal, Y., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.10742 · Cited by 165 · see citing works
Learning state representations with robotic priors
Jonschkowski, R., & Brock, O. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9459-7 · Cited by 158 · see citing works
Self-citationScaling data-driven robotics with reward sketching and batch reinforcement learning
Cabi, S., Colmenarejo, S. G., Novikov, A., Konyushova, K., Reed, S., Jeong, R., Żołna, K., Aytar, Y., Budden, D., Vecerík, M., et al. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.076 · Cited by 98 · see citing works
Natural Environment Benchmarks for Reinforcement LearningPreprint
Zhang, A., Wu, Y., & Pineau, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06032 · Cited by 36 · see citing works
Benchmarking Unsupervised Representation Learning for Continuous Control
Antonova, R., Devlin, S., Hofmann, K., & Kragić, D. (2020) · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [7][7] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, 'Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor,' in Proceedings of the 35th International Conference on Machine Learning , vol. 80, pp. 1861-1870, 2018. -
- [22][22] R. Rubinstein, 'The cross-entropy method for combinatorial and continuous optimization,' Methodology and computing in applied probability , vol. 1, no. 2, pp. 127-190, 1999. -
- [23][23] J. L. Ba, J. R. Kiros, and G. E. Hinton, 'Layer normalization,' arXiv preprint arXiv:1607.06450 , 2016. · arXiv
Cited by
Sorted by citations
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · see citing works
“, 2018) variants: DMC-Remastered, DMC-Generalisation Benchmark, Distracting Control Suite, Natural Environments (Grigsby & Qi, 2020; Hansen & Wang, 2021; Stone et al., 2021; Zhang et al., 2018a).”
BackgroundMethodInfluentialA Survey of Generalisation in Deep Reinforcement LearningPreprint
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv (Cornell University) · Cited by 63 · see citing works
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement LearningPreprint
Agarwal, R., Machado, M. C., Castro, P. S., & Bellemare, M. G. (2021) · International Conference on Learning Representations · doi:10.48550/arxiv.2101.05265 · Cited by 204 · see citing works
Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation
Xie, A., Lee, L., Xiao, T., & Finn, C. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611331 · Cited by 28 · see citing works
DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical RepresentationsPreprint
Deng, F., Jang, I., & Ahn, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14565 · Cited by 18 · see citing works
Contrastive Active InferencePreprint
Mazzaglia, P., Verbelen, T., & Dhoedt, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10083 · Cited by 17 · see citing works
Yang, R., Wang, J., Geng, Z., Ye, M., Ji, S., Li, B., & Wu, F. (2022) · Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539391 · Cited by 27 · see citing works
Environment Agnostic Representation for Visual Reinforcement learning
Choi, H., Lee, H., Jeong, S., & Min, D.-B. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.00031 · Cited by 20 · see citing works
Sequential action-induced invariant representation for reinforcement learning
Liang, D., Chen, Q., & Liu, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106579 · Cited by 9 · see citing works
TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer
Yamada, J., Rigter, M. P., Collins, J., & Posner, I. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610450 · Cited by 7 · see citing works
Intrinsically Motivated Self-supervised Learning in Reinforcement Learning
Zhao, Y., Du, C., Zhao, H., & Li, T. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812213 · Cited by 5 · see citing works
Liu, Q., Zhou, Q., Yang, R., & Wang, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26063 · Cited by 5 · see citing works
Learning task-relevant representations via rewards and real actions for reinforcement learning
Yuan, L., Lu, X., & Liu, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111788 · Cited by 4 · see citing works
Clustering-Driven State Embedding for Reinforcement Learning Under Visual Distractions
Wang, R., Cheng, Y., & Wang, X. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3449294 · Cited by 4 · see citing works
Learning Representations for Pixel-based Control: What Matters and Why?Preprint
Tomar, M., Mishra, U. A., Zhang, A., & Taylor, M. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07775 · Cited by 4 · see citing works
Pore, A., Muradore, R., & Dall’Alba, D. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801730 · Cited by 3 · see citing works
Unlocking Pixels for Reinforcement Learning via Implicit AttentionPreprint
Choromański, K., Jain, D., Yu, W., Song, X., Parker-Holder, J., Zhang, T., Likhosherstov, V., Pacchiano, A., Santara, A., Tang, Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04353 · Cited by 3 · see citing works
Robust Visual Imitation Learning with Inverse Dynamics Representations
Li, S., Wang, X., Zuo, R., Sun, K., Cui, L., Ding, J., Liu, P., & Ma, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29265 · Cited by 2 · see citing works
A Metric Space Perspective on Self-Supervised Policy Adaptation
Bodnar, C., Hausman, K., Dulac-Arnold, G., & Jonschkowski, R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068100 · Cited by 2 · see citing works
Self-citationDizon-Paradis, O., Wormald, S., Capecci, D., Bhandarkar, A., & Woodard, D. (2023) · doi:10.36227/techrxiv.23739099 · Cited by 1 · see citing works
Attention schema in neural agents
Liu, D., Bolotta, S., Zhu, H., Bengio, Y., & Dumas, G. (2023) · arXiv · doi:10.48550/arxiv.2305.17375 · Cited by 1 · see citing works
“Learning a rich and schematic model of one’s own attention will lead to better decision making, especially in contexts where there are many distractions and deep focus is hard to achieve (Stone et al., 2021).”
BackgroundFocus-Then-Decide: Segmentation-Assisted Reinforcement Learning
Chen, C., Xu, J., Liao, W., Ding, H., Zhang, Z., Yu, Y., & Zhao, R. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29002 · Cited by 1 · see citing works
Domain Adaptation of Visual Policies with a Single Demonstration
Wang, W., & Hager, G. D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610569 · Cited by 1 · see citing works
Wang, R., Cheng, Y., & Wang, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3556838 · Cited by 1 · see citing works
Invariance Through Latent Alignment
Yoneda, T., Yang, G., Walter, M. R., & Stadie, B. C. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.064 · Cited by 1 · see citing works
RL-DARTS: Differentiable Architecture Search for Reinforcement Learning.Preprint
Miao, Y., Song, X., Peng, D., Yue, S., Brevdo, E., & Faust, A. (2021) · arXiv (Cornell University) · Cited by 1 · see citing works
Learning in Sparse Rewards settings through Quality-Diversity algorithmsPreprint
Paolo, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.01027 · Cited by 1 · see citing works
A Geometric Perspective on Self-Supervised Policy AdaptationPreprint
Bodnar, C., Hausman, K., Dulac-Arnold, G., & Jonschkowski, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.07318
Self-citationRobust Robotic Control from Pixels using Contrastive Recurrent State-Space ModelsPreprint
Srivastava, N., Talbott, W. A., Lopez, M. B., Zhai, S., & Susskind, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01163
Dizon-Paradis, O. P., Wormald, S. E., Capecci, D. E., Bhandarkar, A., & Woodard, D. L. (2023) · doi:10.36227/techrxiv.23739099.v1 · Cited by 1 · see citing works
Zheng, J., Song, Y., Lin, G., Duan, J., Lin, H., & Li, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129139
- Authors
- Stone, A., Ramirez, O., Konolige, K., & Jonschkowski, R.
- Affiliations
- Google (United States) (US)
- ORCID
- Jonschkowski, Rico
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2101.02722
- OpenAlex
- W3118861865
- Cited by
- 31
- References
- 20
- Citations by year
- 2025: 1 · 2024: 10 · 2023: 5 · 2022: 5 · 2021: 9 · 2020: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- visual servoing, benchmark, visual distraction, background variation, color variation, robot control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
