CoChat

Interactive segmentation, tracking, and kinematic modeling of unknown 3D articulated objects

Open access
Katz, Dov, Kazemi, Moslem, Andrew Bagnell, J., Stentz, Anthony · 2013 · 2013 IEEE International Conference on Robotics and Automation

The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.

Abstract

We present an interactive perceptual skill for segmenting, tracking, and modeling the kinematic structure of 3D articulated objects. This skill is a prerequisite for general manipulation in unstructured environments. Robot-environment interactions are used to move an unknown object, creating a perceptual signal that reveals the kinematic properties of the object. The resulting perceptual information can then inform and facilitate further manipulation. The algorithm is computationally efficient, handles partial occlusions, and depends on little object motion; it only requires sufficient texture for visual feature tracking. We conducted experiments with everyday objects on a robotic manipulation platform equipped with an RGB-D sensor. The results demonstrate the robustness of the proposed method to lighting conditions, object appearance, size, structure, and configuration.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icra.2013.6631292

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Our manipulator interacts with an unknown articulated object (laptop). The proposed interactive perceptual skill segments the object, tracks its position, and models its kinematic structure.
Fig. 1 Our manipulator interacts with an unknown articulated object (laptop). The proposed interactive perceptual skill segments the object, tracks its position, and models its kinematic structure.CC-BY · From the PDF
Fig. 2 Objects that possess inherent degrees of freedom: drawer, train toy, cd box, book, checkerboard, papertowel, stroller, folder, and window. These degrees of freedom are closely related to the function of the object. Without prior knowledge, the degrees of freedom of an object cannot be extracted from visual information alone. They have to be discovered through interaction.
Fig. 2 Objects that possess inherent degrees of freedom: drawer, train toy, cd box, book, checkerboard, papertowel, stroller, folder, and window. These degrees of freedom are closely related to the function of the object. Without prior knowledge, the degrees of freedom of an object cannot be extracted from visual information alone. They have to be discovered through interaction.CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Fig. 4 Feature trajectories for a checkerboard. The two sides of the board move with respect to the background, and rotate with respect to each other.
Fig. 4 Feature trajectories for a checkerboard. The two sides of the board move with respect to the background, and rotate with respect to each other.CC-BY · From the PDF
Fig. 5 Left: motion trajectories of a drawer. Right: successful RANSAC line fitting to the relative motion.
Fig. 5 Left: motion trajectories of a drawer. Right: successful RANSAC line fitting to the relative motion.CC-BY · From the PDF
Fig. 6 Experimental results showing the process of segmenting a scene into its individual objects. Left to right: The first column shows the object before the interaction; the second column shows the object after the interaction; the third column shows the results of clustering the tracked features into rigid bodies; the fourth and fifth columns show the results of joint detection obtained for the first two rows.
Fig. 6 Experimental results showing the process of segmenting a scene into its individual objects. Left to right: The first column shows the object before the interaction; the second column shows the object after the interaction; the third column shows the results of clustering the tracked features into rigid bodies; the fourth and fifth columns show the results of joint detection obtained for the first two rows.CC-BY · From the PDF
Fig. 7 Additional experimental results showing the detected joint for more everyday objects: a checkerboard (revolute), a door (revolute), a paper-towel (revolute), a stroller (prismatic), a toy train (prismatic), and a window (prismatic). The detected joints are marked in green.
Fig. 7 Additional experimental results showing the detected joint for more everyday objects: a checkerboard (revolute), a door (revolute), a paper-towel (revolute), a stroller (prismatic), a toy train (prismatic), and a window (prismatic). The detected joints are marked in green.CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Image 10
CC-BY · From the PDF
Image 11
CC-BY · From the PDF
(c) Detecting a drawer's prismatic joint (10 trials, conditions vary)
(c) Detecting a drawer's prismatic joint (10 trials, conditions vary)CC-BY · From the PDF
Image 13
CC-BY · From the PDF
Fig. 9 To demonstrate repeatability we conducted experiments with two objects: door and drawer. Each experiment was repeated 10 times, while changing lighting, sensor position and orientation, and initial and final object configuration. Three examples of the detected joint (marked in green) are shown for each object. The plots show the angle between the detected axis and the floor. Door: mean = 88 . 43 ◦ , std. dev. = 0 . 016 , Drawer: mean = 177 . 3 ◦ , std.dev. 0 . 94 . Results are close to expectations ( 90 ◦ and 180 ◦ ). The difference ( ≤ 3 ◦ ) is due to misalignment of the sensor with the floor.
Fig. 9 To demonstrate repeatability we conducted experiments with two objects: door and drawer. Each experiment was repeated 10 times, while changing lighting, sensor position and orientation, and initial and final object configuration. Three examples of the detected joint (marked in green) are shown for each object. The plots show the angle between the detected axis and the floor. Door: mean = 88 . 43 ◦ , std. dev. = 0 . 016 , Drawer: mean = 177 . 3 ◦ , std.dev. 0 . 94 . Results are close to expectations ( 90 ◦ and 180 ◦ ). The difference ( ≤ 3 ◦ ) is due to misalignment of the sensor with the floor.CC-BY · From the PDF
Image 15
CC-BY · From the PDF
Fig. 11 Tracking the configuration of a checkerboard: the robot translates the object, rotates the right-half, then rotates it back. The plot shows the resulting change in configuration.
Fig. 11 Tracking the configuration of a checkerboard: the robot translates the object, rotates the right-half, then rotates it back. The plot shows the resulting change in configuration.CC-BY · From the PDF
Fig. 12 Experimental results showing a sequence of interactions with a firetruck toy and the corresponding joint detection results. Joint detection is complex here, as the revolute axes translate with the body of the truck. The sequence shows that joint detection remains reliable throughout the sequence.
Fig. 12 Experimental results showing a sequence of interactions with a firetruck toy and the corresponding joint detection results. Joint detection is complex here, as the revolute axes translate with the body of the truck. The sequence shows that joint detection remains reliable throughout the sequence.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 91

Metadata

Authors
Katz, D., Kazemi, M., Andrew Bagnell, J., & Stentz, A.
Affiliations
Carnegie Mellon University (US)
Venue
2013 IEEE International Conference on Robotics and Automation
Publisher
IEEE
Published
2013-05
Pages
5003-5010
ISSN
1050-4729 (ISSN-L)
OpenAlex
W2159461149
Cited by
74
References
17
Citations by year
2025: 1 · 2024: 3 · 2023: 6 · 2022: 6 · 2021: 6 · 2020: 6 · 2019: 7 · 2018: 5 · 2017: 7 · 2016: 7 · 2015: 10 · 2014: 6 · 2013: 4
FWCI
12.29
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • Figshare (submittedVersion, cc-by)
  • KiltHub Repository (submittedVersion)
  • KiltHub Repository
Funders
  • Defense Advanced Research Projects Agency
  • Army Research Laboratory (W911NF, W911NF-10-2-0016)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
interactive segmentation, object tracking, kinematic modeling, articulated objects, robotic manipulation, unknown object, RGB-D sensing, visual feature tracking, partial occlusion, robot-environment interaction, perceptual skills

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.