An integrated approach to visual perception of articulated objects
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
We present an integrated approach for perception of unknown articulated objects. To robustly perceive objects and understand interactions, our method tightly integrates pose tracking, shape reconstruction, and the estimation of their kinematic structure. The key insight of our method is that these sub-problems complement each other: for example, tracking is greatly facilitated by knowing the shape of the object, whereas the shape and the kinematic structure can be more easily reconstructed if the motion of the object is known. Our combined method leverages these synergies to improve the performance of perception. We analyze the proposed method in average cases and difficult scenarios using a variety of rigid and articulated objects. The results show that our integrated solution achieves better results than solutions for the individual problems. This demonstrates the benefits of approaching robot perception problems in an integrated manner.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 2×
“Other methods [6, 17] build on top of KinectFusion [15].”
BackgroundMethodSegmentation of Moving Objects by Long Term Video Analysis
Ochs, P., Malik, J., & Brox, T. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.242 · Cited by 585
Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds
Papon, J., Abramov, A., Schoeler, M., & Worgotter, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.264 · Cited by 545 · cited 1×
“assuming surface continuity as exploited by conditional random fields [9] graph-cuts, [10], and supervoxel region growing [11], or exploiting object location as in active segmentation [12].”
BackgroundEfficient moving object segmentation algorithm using background registration technique
Chien, S.-Y., Ma, S.-Y., & Chen, L.-G. (2002) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2002.800516 · Cited by 355 · cited 1×
“To detect which points moved due to object motion, image differencing [13, 14] can be applied.”
BackgroundManipulator and object tracking for in-hand 3D object modeling
Krainin, M., Henry, P., Ren, X., & Fox, D. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911403178 · Cited by 166 · cited 2×
“Partial object views can be obtained using image segmentation and pose information by controlling camera and object motion [1].”
BackgroundRGB-D object tracking: A particle filter approach on GPU
Choi, C., & Christensen, H. I. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696485 · Cited by 117 · cited 1×
“On the other hand, to track the pose of an object, methods commonly rely on knowledge of the object’s shape and its segmentation in the image [2, 3].”
Interactive segmentation for manipulation in unstructured environments
Kenney, J., Buckley, T., & Brock, O. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152393 · Cited by 99 · cited 1×
“To detect which points moved due to object motion, image differencing [13, 14] can be applied.”
BackgroundSelf-citationA Probabilistic Framework for Learning Kinematic Models of Articulated Objects
Sturm, J., Stachniss, C., & Burgard, W. (2011) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3229 · Cited by 96 · cited 1×
“Based on the tracked poses one can infer the kinematic structure restricting their relative motion [8].”
BackgroundProbabilistic object tracking using a range camera
Wuthrich, M., Pastor, P., Kalakrishnan, M., Bohg, J., & Schaal, S. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696810 · Cited by 79 · cited 2×
“We distinguish two main approaches to visual tracking: based on a known shape model of the object [2], and based on sufficient texture using point features [4] or dense optical flow [5, 7].”
MethodTracking a depth camera: Parameter exploration for fast ICP
Pomerleau, F., Magnenat, S., Colas, F., Liu, M., & Siegwart, R. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6094861 · Cited by 79 · cited 2×
“The tracker exploits this segmentation to refine the pose estimate of Online IP using ICP [19].”
MethodA Probabilistic Framework for Learning Kinematic Models of Articulated Objects
Sturm, J., Stachniss, C., & Burgard, W. (2012) · Cited by 78
Tracking a Depth Camera: Parameter Exploration for Fast ICP
Pomerleau, F., Magnenat, S., Colas, F., Liu, M., & Siegwart, R. (2011) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-010025266 · Cited by 72
Ikeuchi, K. (2021) · Springer eBooks · doi:10.1007/978-3-030-63416-2_300238 · Cited by 63
Directional Joint Bilateral Filter for Depth Images
Le, A. V., Jung, S., & Won, C. S. (2014) · Sensors · doi:10.3390/s140711362 · Cited by 58 · cited 1×
“We preprocess the raw depth images by applying a joint bilateral filter [18] in order to fill the depth-missing areas based on their surrounding depth and color information.”
MethodMartin Martin, R., & Brock, O. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6942902 · Cited by 55 · cited 5×
“work we have shown that tracking can also greatly benefit from a known kinematic structure [4].”
BackgroundMethodSelf-citationSTAR3D: Simultaneous Tracking and Reconstruction of 3D Objects Using RGB-D Data
Ren, C. Y., Prisacariu, V. A., Murray, D. W., & Reid, I. D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.197 · Cited by 52 · cited 1×
“present a method to simultaneously track and reconstruct 3-D objects [16] by refining an initial primitive shape model; in contrast to our method it can only reconstruct and track one moving object and the initial location of this object must be manually provided.”
MethodToward online 3-D object segmentation and mapping
Herbst, E. V., Henry, P., & Fox, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907318 · Cited by 45 · cited 1×
“Other methods [6, 17] build on top of KinectFusion [15].”
MethodActive segmentation for robotics
Mishra, A., Aloimonos, Y., & Fermuller, C. (2009) · 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2009.5354325 · Cited by 42 · cited 1×
“assuming surface continuity as exploited by conditional random fields [9] graph-cuts, [10], and supervoxel region growing [11], or exploiting object location as in active segmentation [12].”
BackgroundUnsupervised Dense Object Discovery, Detection, Tracking and Reconstruction
Ma, L., & Sibley, G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_6 · Cited by 22 · cited 2×
“Other methods [6, 17] build on top of KinectFusion [15].”
MethodEfficient Dense 3D Rigid-Body Motion Segmentation in RGB-D Video
Stückler, J., & Behnke, S. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.51 · Cited by 17 · cited 4×
“suggest a method which combines object tracking, segmentation, and reconstruction using an EM algorithm [5].”
BackgroundMethodInfluentialDang, T. H., Leemput, K. V., Verschoren, A., & Laukens, K. (2014) · Springer eBooks · doi:10.1007/978-0-387-31439-6_100233 · Cited by 16
Cited by
Co-fusion: Real-time segmentation, tracking and fusion of multiple objects
Runz, M., & Agapito, L. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989518 · Cited by 227
“[12] propose a combined approach for estimating pose, shape, and the kinematic structure of articulated objects based on motion segmentation.”
BackgroundInteractive perception: Leveraging action in perception and perception in action
Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., & Sukhatme, G. S. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2721939 · Cited by 227
“7 ision) 56, 44, 49], [109] , [49], [109] II 48] APR AP M P 7 ision) EU - I [57] CNS PM M P 7 ision) DOM - II [58] APR AP G P 7 ision) EU Esti- gmen- tation [43] CNS O 7 P 7 ision) SOM Esti- I [100], [103, 95, 110] CNS 7 95, 110] [100] P 7 ision) dynamics [103] SDM [99, 110] EU 99, 110] [100] Esti- II 102, 98, 101] AP…”
Self-citation(2021) · Journal of Machine Learning Research · Cited by 224
“text parameters to accurately model the degrees of freedom. Passive perception can be used to estimate the kinematic chains of articulated objects, especially if the object is being moved by a person [258,278,299,42]. Interactive perception is also well suited for estimating the articulation model parameters and the r…”
BackgroundCategory-Level Articulated Object Pose Estimation
Li, X., Wang, H., Yi, L., Guibas, L. J., Abbott, A. L., & Song, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00376 · Cited by 196
Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation
Mittal, M., Hoeller, D., Farshidian, F., Hutter, M., & Garg, A. (2022) · 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981779 · Cited by 86
“A common approach is estimating the object’s kinematic structure from passive observations [20]– [23] or through interactive perception [24], [25].”
Background
Metadata
- Authors
- Martin-Martin, R., Hofer, S., & Brock, O.
- Affiliations
- Technische Universität Berlin (DE)
- ORCID
- Brock, Oliver
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 5091-5097
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2419427454
- Cited by
- 41
- References
- 22
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 4 · 2022: 5 · 2021: 10 · 2020: 4 · 2019: 4 · 2018: 3 · 2017: 6 · 2016: 2
- FWCI
- 8.46
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- articulated objects, visual perception, pose tracking, shape recovery, robot perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.