CoChat

Self-Supervised Correspondence in Visuomotor Policy Learning

Open access
Florence, Peter, Manuelli, Lucas, Tedrake, Russ · 2020 · IEEE Robotics and Automation Letters
Cited by 110

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Abstract

In this letter, we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has primarily used approaches such as autoencoding, pose-based losses, and end-to-end policy optimization in order to train the visual portion of visuomotor policies. We instead propose an approach using self-supervised dense visual correspondence training and show that this enables visuomotor policy learning with surprisingly high generalization performance with modest amounts of data. Using imitation learning, we demonstrate extensive hardware validation on challenging manipulation tasks with as few as 50 demonstrations. Our learned policies can generalize across classes of objects, react to deformable object configurations, and manipulate textureless symmetrical objects in a variety of backgrounds, all with closed-loop, real-time vision-based policies. Simulated imitation learning experiments suggest that correspondence training offers sample complexity and generalization benefits compared to autoencoding and end-to-end training.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/lra.2019.2956365

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 29

  1. 10.1109/ICRA.2014.6907062 · DOI · Paper page
  2. 10.1109/ICCV.2017.623 · DOI
  3. laskey, 0, DART: Noise injection for robust imitation learning, Proc Conf Robot Learn, 143
  4. 10.15607/RSS.2019.XV.020 · DOI
  5. 10.1109/ICRA.2017.7989384 · DOI · Paper page
  6. finn, 0, Guided cost learning: Deep inverse optimal control via policy optimization, Proc Int Conf Mach Learn, 49
  7. manuelli, 0, kPAM: Keypoint affordances for category-level robotic manipulation, Proc Int Symp Robot Res
  8. 10.1109/IROS.2016.7759578 · DOI · Paper page
  9. 10.1109/ICRA.2018.8461076 · DOI · Paper page
  10. agrawal, 0, Learning to poke by poking: Experiential learning of intuitive physics, Proc Adv Neural Inf Process Syst, 5074
  11. ebert, 2018, Visual foresight: Model-based deep reinforcement learning for vision-based robotic control, arXiv 1812 00568
  12. 10.1109/ICRA.2018.8462891 · DOI · Paper page
  13. 10.15607/RSS.2017.XIII.050 · DOI · Paper page
  14. kulkarni, 0, Unsupervised learning of object keypoints for perception and control, Proc Adv Neural Inf Process Syst
  15. 10.1007/978-3-540-30301-5_60 · DOI · Paper page
  16. 10.1109/ICRA.2018.8461249 · DOI · Paper page
  17. 10.1109/LRA.2016.2634089 · DOI · Paper page
  18. 10.1109/IROS.2017.8202141 · DOI · Paper page
  19. 10.1109/ICRA.2019.8793621 · DOI · Paper page
  20. kroemer, 2019, A review of robot learning for manipulation: Challenges, representations, and algorithms, arXiv 1907 03146
  21. florence, 0, Dense object nets: Learning dense visual object descriptors by and for robotic manipulation, Proc Conf Robot Learn, 373
  22. 10.1109/ICRA.2016.7487173 · DOI · Paper page
  23. 10.1109/LRA.2016.2633383 · DOI · Paper page
  24. levine, 2016, End-to-end training of deep visuomotor policies, J Mach Learn Res, 17, 1
  25. sieb, 0, Graph-structured visual imitation, Proc Conf Robot Learn
  26. 10.15607/RSS.2018.XIV.009 · DOI · Paper page
  27. 10.1109/ICRA.2016.7487517 · DOI · Paper page
  28. matas, 0, Sim-to-real reinforcement learning for deformable object manipulation, Proc Conf Robot Learn, 734
  29. 10.15607/RSS.2018.XIV.002 · DOI · Paper page

Cited by

See all 110

Metadata

Authors
Florence, P., Manuelli, L., & Tedrake, R.
Venue
IEEE Robotics and Automation Letters
Published
2020-04
Volume
5
Issue
2
Pages
492-499
ISSN
  • 2377-3766 (ISSN-L)
  • 2377-3774
OpenAlex
W2972997256
Cited by
110
References
10
FWCI
5.30
Type
journal article
Language
en
Open access
green
Funders
  • National Science Foundation (IIS-1427050)
  • Lockheed Martin Corporation (RPP2016-002)
  • Amazon Research Award Grant

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Self-Supervised Correspondence in Visuomotor Policy Learning. https://app.cochat.ai/papers/10.1109/lra.2019.2956365
The paper
Florence, P., Manuelli, L., & Tedrake, R. (2020). Self-Supervised Correspondence in Visuomotor Policy Learning. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2019.2956365

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.