Data Augmentation for Manipulation
Open accessAbstract
The success of deep learning depends heavily on the availability of large datasets, but in robotic manipulation there are many learning problems for which such datasets do not exist. Collecting these datasets is time-consuming and expensive, and therefore learning from small datasets is an important open problem. Within computer vision, a common approach to a lack of data is data augmentation. Data augmentation is the process of creating additional training examples by modifying existing ones. However, because the types of tasks and data differ, the methods used in computer vision cannot be easily adapted to manipulation. Therefore, we propose a data augmentation method for robotic manipulation. We argue that augmentations should be valid, relevant, and diverse. We use these principles to formalize augmentation as an optimization problem, with the objective function derived from physics and knowledge of the manipulation domain. This method applies rigid body transformations to trajectories of geometric state and action data. We test our method in two scenarios: 1) learning the dynamics of planar pushing of rigid cylinders, and 2) learning a constraint checker for rope manipulation. These two scenarios have different data and label types, yet in both scenarios, training on our augmented data significantly improves performance on downstream tasks. We also show how our augmentation method can be used on real-robot data to enable more data-efficient online learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It can be difficult to generate plausible interactions without prior knowledge of physics or causal dependencies, especially for new scenes, objects, or robots.
“however, it can be difficult to generate plausible interactions without prior knowledge of physics [35] or causal dependencies [41,42], especially for new scenes, objects, or robots.”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [35]
The Assumptions section (page unavailable) says valid augmentations generally rely on expert knowledge or heuristics, and specifies physical assumptions required by this method. This supports the difficulty of generating plausible interactions without prior physical knowledge, but the paper does not specifically establish that the difficulty is greater for new scenes, objects, or robots. The causal-dependencies clause has separate citations in the citing sentence. (LLM, conf=0.82)
Experimental — accuracy not yet measuredSeveral works have used offline data augmentation to increase the dataset size for learning policies.
“Several works have used offline data augmentation to increase the dataset size for learning policies [7, 35-45].”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [35]
The Cluttered Planar Pushing section reports increasing the training data from 3,000 to 75,000 time steps through augmentation. But the paper describes training a dynamics model and a constraint checker, not learning policies; that part of the cited claim is unsupported. (LLM, conf=0.86)
Experimental — accuracy not yet measuredOffline data augmentation can be effective for generating larger dataset for robot manipulation.
“Offline data augmentation can be effective for generating larger dataset for robot manipulation [7, 35-45]”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [35]
Because data augmentation is offline, it can be difficult to generate plausible interactions without prior knowledge of physics or causal dependencies, especially for new scenes, objects, or robots.
“However, because data augmentation is offline, it can be difficult to generate plausible interactions without prior knowledge of physics [35] or causal dependencies [41,42], especially for new scenes, objects, or robots.”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
Cited by 18
Research works printed but not linked: 17
- Gregory W. Benton, Marc Finzi, Pavel Izmailov, and Andrew Gordon Wilson. Learning Invariances in Neural Networks from Training Data. NeurIPS, 2020.
- Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. A Survey of Data Augmentation Approaches for NLP. Association for Computational Linguistics, 2021.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. NeurIPS, 2014.
- Ekin Dogus Cubuk and Barret Zoph and Dandelion Mané and Vijay Vasudevan and Quoc V. Le. Autoaugment: Learning augmentation policies from data. ArXiv, 2018.
- Oliver Kroemer, Scott Niekum, and George Konidaris. A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. JMLR, 2021.
- Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement Learning with Augmented Data. NeurIPS, 2020.
- Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B. Tenen- baum, Antonio Torralba, and Russ Tedrake. Propagation Networks for Model-Based Control Under Partial Obser- vation. ICRA, 2019.
- Dale McConachie, Thomas Power, Peter Mitrano, and Dmitry Berenson. Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces. IEEE Robotics and Automation Letters, 2020.
- David Navarro-Alarcón, Yun-Hui Liu, José Guadalupe Romero, and Peng Li. Model-Free Visually Servoed Deformation Control of Elastic Objects by Robot Ma- nipulators. IEEE Transactions on Robotics, 2013.
- Connor Shorten and Taghi M. Khoshgoftaar. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6:60, 2019.
- P.Y. Simard, D. Steinkraus, and J.C. Platt. Best practices for convolutional neural networks applied to visual doc- ument analysis. International Conference on Document Analysis and Recognition, 2003.
- Ahmet E. Tekden, Aykut Erdem, Erkut Erdem, Mert Imre, M. Yunus Seker, and Emre Ugur. Belief Regulated Dual Propagation Nets for Learning Action Effects on Groups of Articulated Objects. ICRA, 2020.
- Toan Tran, Trung Pham, Gustavo Carneiro, Lyle J. Palmer, and Ian D. Reid. A Bayesian Data Augmentation Approach for Learning Deep Models. NeurIPS, 2017.
- Yixuan Wang, Dale McConachie, and Dmitry Berenson. Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints. ICRA, 2021.
- Kuan-Ting Yu, Maria Bauzá, Nima Fazeli, and Alberto Rodriguez. More than a Million Ways to Be Pushed: A High-Fidelity Experimental Data Set of Planar Pushing. IROS, 2016.
- Sheng Zhong, Zhenyuan Zhang, Nima Fazeli, and Dmitry Berenson. TAMPC: A Controller for Escaping Traps in Novel Environments. IEEE Robotics and Automation Letters, 2021.
- Gao Ziyan, Armagan Elibol, and Nak Young Chong. Planar Pushing of Unknown Objects Using a Large-Scale Simulation Dataset and Few-Shot Learning. International Conference on Automation Science and Engineering (CASE), 2021.
- Authors
- Mitrano, P., & Berenson, D.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- arXiv
- 2205.02886
- OpenAlex
- W4283784507
- Cited by
- 18
- References
- 0
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 6 · 2023: 5
- FWCI
- 6.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Robotics: Science and Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1750489, 2113401)
- Toyota Research Institute
- Office of Naval Research (N00014-21-1-2118, N00014)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- data augmentation, manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
