The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
TossingBot: Learning to Throw Arbitrary Objects With Residual Physics
Open accessAbstract
We investigate whether a robot arm can learn to pick and throw arbitrary rigid objects into selected boxes quickly and accurately. Throwing has the potential to increase the physical reachability and picking speed of a robot arm. However, precisely throwing arbitrary objects in unstructured settings presents many challenges: from acquiring objects in grasps suitable for reliable throwing, to handling varying object-centric properties (e.g., mass distribution, friction, shape) and complex aerodynamics. In this work, we propose an end-to-end formulation that jointly learns to infer control parameters for grasping and throwing motion primitives from visual observations (RGB-D images of arbitrary objects in a bin) through trial and error. Within this formulation, we investigate the synergies between grasping and throwing (i.e., learning grasps that enable more accurate throws) and between simulation and deep learning (i.e., using deep networks to predict residuals on top of control parameters predicted by a physics simulator). The resulting system, TossingBot, is able to grasp and successfully throw arbitrary objects into boxes located outside its maximum reach range at 500+ mean picks per hour (600+ grasps per hour with 85% throwing accuracy); and generalizes to new objects and target locations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
(2010) · Cited by 14,134
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.058 · Cited by 1,268
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
Zeng, A., Song, S., Niessner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.29 · Cited by 1,128
Self-citationLearning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Self-citationLearning ambidextrous robot grasping policies
Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S., & Goldberg, K. (2019) · Science Robotics · doi:10.1126/scirobotics.aau4984 · Cited by 563
Residual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480
Automated construction of robotic manipulation programs
Takeo Kanade, A., Kuffner, J., & Diankov, R. (2010) · Cited by 458
Zeng, A., Song, S., Yu, K.-T., Donlon, E., Hogan, F. R., Bauza, M., Ma, D., Taylor, O., Liu, M., Romo, E., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461044 · Cited by 441
Self-citationClear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation
Sajjan, S., Moore, M., Pan, M., Nagaraja, G., Lee, J., Zeng, A., & Song, S. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197518 · Cited by 279
Self-citationExtrinsic dexterity: In-hand manipulation with external forces
Dafle, N. C., Rodriguez, A., Paolini, R., Tang, B., Srinivasa, S. S., Erdmann, M., Mason, M. T., Lundberg, I., Staab, H., & Fuhlbrigge, T. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907062 · Cited by 271
Self-citationDynamic Nonprehensile Manipulation: Controllability, Planning, and Experiments
Lynch, K. M., & Mason, M. T. (1999) · International Journal of Robotics Research · doi:10.1177/02783649922066079 · Cited by 270
Using inaccurate models in reinforcement learning
Abbeel, P., Quigley, M., & Ng, A. Y. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143845 · Cited by 207
Reinforcement Learning to adjust Robot Movements to New Situations
Kober, J., Öztop, E., & Peters, J. (2010) · Robotics: Science and Systems · doi:10.15607/rss.2010.vi.005 · Cited by 160
Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge
Morrison, D., Tow, A. W., McTaggart, M., Smith, R. S., Kelly-Boxall, N., Wade-McCue, S., Erskine, J., Grinover, R., Gurman, A., Hunn, T., et al. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463191 · Cited by 145
Automated Construction of Robotic Manipulation Programs
Diankov, R. (2010) · ProQuest Demo Repository · doi:10.1184/r1/6714905 · Cited by 144
Mason, M. T., & Lynch, K. (2002) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.1993.583093 · Cited by 139
Satish, V., Mahler, J., & Goldberg, K. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2895878 · Cited by 139
TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
Zeng, A., Song, S., Lee, J., Rodriquez, A., & A.Funkouser, T. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.004 · Cited by 130
Self-citationForm2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly
Zakka, K., Zeng, A., Lee, J., & Song, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196733 · Cited by 126
Self-citationDensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions
Xu, Z., Wu, J., Zeng, A., Tenenbaum, J. B., & Song, S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.046 · Cited by 122
Self-citationSilver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113
Deep predictive policy training using reinforcement learning
Ghadirzadeh, A., Maki, A., Kragic, D., & Bjorkman, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206046 · Cited by 113
Learning to See before Learning to Act: Visual Pre-training for Manipulation
Lin, Y.-C., Zeng, A., Song, S., Isola, P., & Lin, T.-Y. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197331 · Cited by 106
Self-citationAjay, A., Wu, J., Fazeli, N., Bauzá, M., Kaelbling, L. P., Tenenbaum, J. B., & Rodríguez, A. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593995 · Cited by 102
Reinforcement learning to adjust robot movements to new situations
Kober, J., Öztop, E., & Peters, J. (2011) · Max Planck Digital Library · doi:10.5591/978-1-57735-516-8/ijcai11-441 · Cited by 86
Aboaf, E. W., Atkeson, C. G., & Reinkensmeyer, D. J. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1988.12245 · Cited by 75
Combining learned and analytical models for predicting action effects.Preprint
Kloss, A., Stefan Schaal, A., & Bohg, J. (2017) · arXiv (Cornell University) · Cited by 47
Learning task error models for manipulation
Pástor, P., Kalakrishnan, M., Binney, J., Kelly, J. W., Righetti, L., Sukhatme, G. S., & Stefan Schaal, A. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630935 · Cited by 31
Optimal shape and motion planning for dynamic planar manipulation
Taylor, O., & Rodríguez, A. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9773-y · Cited by 29
Learning Visual Affordances for Robotic Manipulation
Zeng, A. (2019) · Cited by 23
Self-citationAdapting learned robotics behaviours through policy adjustment
Higuera, J. C. G., Meger, D., & Dudek, G. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989686 · Cited by 20
Data-Augmented Contact Model for Rigid Body SimulationPreprint
Jiang, Y., Sun, J., & Liu, C. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04019 · Cited by 19
TossingBot: Learning to Throw Arbitrary Objects with Residual PhysicsPreprint
Zeng, A., Song, S., Lee, J., Rodríguez, A., & Funkhouser, T. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.11239 · Cited by 13
Self-citation
Cited by
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies
Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C., & Hu, Z. (2022) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-022-03425-4 · Cited by 441
SAPIEN: A SimulAted Part-Based Interactive ENvironment
Xiang, F., Qin, Y., Mo, K., Xia, Y., Zhu, H., Liu, F., Liu, M., Jiang, H., Yuan, Y., Wang, H., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01111 · Cited by 406
A Review of Physics Simulators for Robotic Applications
Collins, J., Chand, S., Vanderkop, A., & Howard, D. (2021) · IEEE Access · doi:10.1109/access.2021.3068769 · Cited by 276
Florence, P., Lynch, C., Zeng, A., Ramirez, O., Wahid, A., Downs, L., Wong, A., Lee, J., Mordatch, I., & Tompson, J. (2021) · arXiv · doi:10.48550/arxiv.2109.00137 · Cited by 131
Self-citationBridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.063 · Cited by 94
Metadata
- Authors
- Zeng, A., Song, S., Lee, J., Rodriguez, A., & Funkhouser, T.
- Venue
- IEEE Transactions on Robotics
- Published
- 2020-08
- Volume
- 36
- Issue
- 4
- Pages
- 1307-1319
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2963092423
- Cited by
- 282
- References
- 40
- FWCI
- 20.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.