A non-rigid point and normal registration algorithm with applications to learning from demonstrations
Abstract
Recent work [1], [2], [3] has shown promising results in learning from demonstrations for the manipulation of deformable objects. Their approach finds a non-rigid registration between points in the demonstration scene and points in the test scene. This registration is then extrapolated and applied to the gripper motions in the demonstration scene to obtain the gripper motions for the test scene. If more than one demonstration is available, a quality score for the non-rigid registration is used to determine the best matching training scene. For many manipulation tasks, however, the gripper's direction of approach with respect to the objects' surface normals is important in order to succeed at the task. This prior work only registers points across scenes and does not register the surface normals, often leading to warps between scenes that are inappropriate for transfer of manipulation primitives. The main contributions of this paper are (i) An algorithm for non-rigid registration that considers both points and normals, and (ii) An evaluation of this registration approach in the context of learning from demonstrations for robotic manipulation. Our experiments, which consider an insertion task in simulation and also knot-tying and towel-folding executions in a PR2, show that incorporating normals results in improved performance and qualitatively better grasps.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Lee et al. extend TPS-RPM transfer to force control, and Lee et al. replace TPS-RPM with Coherent Point Drift to improve transferring quality.
“Lee et al. (2015a) [118] extend the work to force control, while Lee et al. (2015b) [119] replace TPS-RPM with Coherent Point Drift to improve the transferring quality.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [119]
Cites
A Method for Registration of 3-D Shapes
Besl, P., & Mckay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 22,163 · see citing works
Spline Models for Observational Data.
Müller, H., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,992 · see citing works
Spline Models for Observational Data
Wahba, G. (1990) · doi:10.1137/1.9781611970128 · Cited by 4,184 · see citing works
Efficient variants of the ICP algorithm
Rusinkiewicz, S., & Levoy, M. (2002) · Proceedings Third International Conference on 3-D Digital Imaging and Modeling · doi:10.1109/im.2001.924423 · Cited by 3,945 · see citing works
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227 · see citing works
A new point matching algorithm for non-rigid registration
Chui, H., & Rangarajan, A. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00009-2 · Cited by 1,560 · see citing works
Splines minimizing rotation-invariant semi-norms in Sobolev spaces
Duchon, J. (1977) · Lecture notes in mathematics · doi:10.1007/bfb0086566 · Cited by 1,397 · see citing works
Is imitation learning the route to humanoid robots?
Schaal, S. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01327-3 · Cited by 1,325 · see citing works
On Learning, Representing, and Generalizing a Task in a Humanoid Robot
Calinon, S., Guenter, F., & Billard, A. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2006.886952 · Cited by 1,090 · see citing works
Robot Programming by Demonstration
Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008) · Springer eBooks · doi:10.1007/978-3-540-30301-5_60 · Cited by 1,078 · see citing works
A new algorithm for non-rigid point matching
Chui, H., & Rangarajan, A. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854733 · Cited by 443 · see citing works
Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization
Schulman, J., Ho, J., Lee, A., Awwal, I., Bradlow, H., & Abbeel, P. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.031 · Cited by 442 · see citing works
Self-citationFully Automatic Registration of 3D Point Clouds
Makadia, A., Patterson, A., & Daniilidis, K. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.122 · Cited by 312 · see citing works
Learning from Demonstrations Through the Use of Non-rigid Registration
Schulman, J., Ho, J., Lee, C., & Abbeel, P. (2016) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-28872-7_20 · Cited by 133 · cited 5× · see citing works
“Abstract— Recent work [1], [2], [3] has shown promising results in learning from demonstrations for the manipulation of deformable objects.”
BackgroundInfluentialSelf-citationSchulman, J., Gupta, A., Venkatesan, S., Tayson-Frederick, M., & Abbeel, P. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696945 · Cited by 84 · see citing works
Self-citationA feature space for edgels in images with landmarks
Bookstein, F. L., & Green, W. D. K. (1993) · Journal of Mathematical Imaging and Vision · doi:10.1007/bf01248355 · Cited by 81 · see citing works
Kriging and splines with derivative information
Mardia, K. V. (1996) · Biometrika · doi:10.1093/biomet/83.1.207 · Cited by 81 · see citing works
Calinon, S., D'halluin, F., Caldwell, D. G., & Billard, A. G. (2009) · 2009 9th IEEE-RAS International Conference on Humanoid Robots · doi:10.1109/ichr.2009.5379592 · Cited by 73 · see citing works
Lee, A. X., Huang, S. H., Hadfield-Menell, D., Tzeng, E., & Abbeel, P. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6943185 · Cited by 32 · see citing works
Self-citationImage warping using derivative information
Mardia, K. V., & Little, J. A. (1994) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.179254 · Cited by 26 · see citing works
Flagged first, then by citations
1 of the 21 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405 · see citing works
Self-citationAn Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G. M., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 387 · see citing works
Self-citationTrack deformable objects from point clouds with structure preserved registration
Tang, T., & Tomizuka, M. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919841431 · Cited by 73 · see citing works
State estimation for deformable objects by point registration and dynamic simulation
Tang, T., Fan, Y., Lin, H.-C., & Tomizuka, M. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206058 · Cited by 41 · see citing works
Lee, A. X., Gupta, A., Lu, H. H., Levine, S., & Abbeel, P. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354120 · Cited by 34 · see citing works
Self-citationRobotic manipulation of deformable objects by tangent space mapping and non-rigid registration
Tang, T., Liu, C., Chen, W., & Tomizuka, M. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759418 · Cited by 25 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“Lee et al. (2015a) [118] extend the work to force control, while Lee et al. (2015b) [119] replace TPS-RPM with Coherent Point Drift to improve the transferring quality.”
MethodNon-rigid 3D Shape Registration Using an Adaptive Template
Dai, H., Pears, N. E., & Smith, W. A. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_5 · Cited by 15 · see citing works
Symmetric Shape Morphing for 3D Face and Head Modelling
Dai, H., Pears, N. E., Smith, W. A. P., & Duncan, C. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00023 · Cited by 7 · see citing works
Modelling of Orthogonal Craniofacial Profiles
Dai, H., Pears, N. E., & Duncan, C. (2017) · Journal of Imaging · doi:10.3390/jimaging3040055 · Cited by 4 · see citing works
Kadi, H. A., & Terzić, K. (2022) · Preprints.org · doi:10.20944/preprints202212.0305.v1 · Cited by 1 · see citing works
Learning robotic manipulation from user demonstrations
Rahmatizadeh, R. (2017) · Journal of International Crisis and Risk Communication Research
- Authors
- Lee, A. X., Goldstein, M. A., Barratt, S. T., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2015 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2015-05
- Pages
- 935-942
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1590340078
- Cited by
- 12
- References
- 21
- Citations by year
- 2023: 1 · 2022: 1 · 2019: 2 · 2018: 3 · 2017: 3 · 2016: 1 · 2015: 1
- FWCI
- 3.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Funders
- National Science Foundation
- Air Force Office of Scientific Research
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- non-rigid registration, surface normals, imitation learning, robotic manipulation, deformable objects, grasp planning, insertion tasks, knot tying, PR2 robot
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
