Kineverse: A Symbolic Articulation Model Framework for Model-Agnostic Mobile Manipulation
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Service robots in the future need to execute abstract instructions such as “fetch the milk from the fridge”. To translate such instructions into actionable plans, robots require in-depth background knowledge. With regards to interactions with doors and drawers, robots require articulation models that they can use for state estimation and motion planning. Existing frameworks model articulated connections as abstract concepts such asprismatic, orrevolute, but do not provide a parameterized model of these connections for computation. In this letter, we introduce a novel framework that uses symbolic mathematical expressions to model articulated structures – robots and objects alike – in a unified and extensible manner. We provide a theoretical description of this framework, and the operations that are supported by its models, and introduce an architecture to exchange our models in robotic applications, making them as flexible as any other environmental observation. To demonstrate the utility of our approach, we employ our practical implementationKineversefor solving common robotics tasks from state estimation and mobile manipulation, and use it further in real-world mobile robot manipulation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Articulated object manipulation such as opening doors and drawers requires control of both the base and arm of the robot [20].”
From Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation (2023) · cites this paper as [20]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LaValle, S. M. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546877 · Cited by 5,211 · cited 1×
“Sampling-based trajectory planners such as RRT and PRM [14] require configuration space bounds to determine the space to sample from.”
LaValle, S. M. (2006) · Cited by 4,177
CasADi: a software framework for nonlinear optimization and optimal control
Andersson, J. A. E., Gillis, J., Horn, G., Rawlings, J. B., & Diehl, M. (2019) · Mathematical Programming Computation · doi:10.1007/s12532-018-0139-4 · Cited by 3,868 · cited 1×
“We use Casadi [21] as the backbone for our symbolic mathematical expressions.”
MethodPoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
CHOMP: Gradient optimization techniques for efficient motion planning
Ratliff, N., Zucker, M., Bagnell, J. A., & Srinivasa, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152817 · Cited by 992 · cited 1×
“Optimizationbased approaches, be it planners [15]–[17] or control [18, 19], require model differentiability.”
BackgroundAprilTag 2: Efficient and robust fiducial detection
Wang, J., & Olson, E. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759617 · Cited by 822 · cited 1×
“EKF Model Definition: To estimate the pose q of an articulated object, our EKF consumes a stream of observations Yt ⊆ SE(3) of the object’s individual parts, as they might be obtained from tracking markers [22, 23] or pose estimation networks [24].”
MethodGeneration of fiducial marker dictionaries using Mixed Integer Linear Programming
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J., & Medina-Carnicer, R. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.09.023 · Cited by 490 · cited 1×
“EKF Model Definition: To estimate the pose q of an articulated object, our EKF consumes a stream of observations Yt ⊆ SE(3) of the object’s individual parts, as they might be obtained from tracking markers [22, 23] or pose estimation networks [24].”
MethodFinding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization
Schulman, J., Ho, J., Lee, A., Awwal, I., Bradlow, H., & Abbeel, P. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.031 · Cited by 430
Logic-geometric programming: an optimization-based approach to combined task and motion planning
Toussaint, M. (2015) · Cited by 217
Foote, T. (2013) · 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA) · doi:10.1109/tepra.2013.6556373 · Cited by 200 · cited 1×
“With respect to distributing articulation models in a live robotic system, the tf library [20] addresses a similar problem.”
BackgroundCategory-Level Articulated Object Pose Estimation
Li, X., Wang, H., Yi, L., Guibas, L. J., Abbott, A. L., & Song, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00376 · Cited by 196
Manipulating articulated objects with interactive perception
Katz, D., & Brock, O. (2008) · 2008 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2008.4543220 · Cited by 135 · cited 1×
“While the earliest approach in this line of works [7] does not recover the full kinematic structure of the object, but only the likeliest pair-wise connections, later work [3, 8] recover the full kinematic graph of the model.”
BackgroundAertbeliën, E., & Schutter, J. D. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6942760 · Cited by 104 · cited 1×
“Optimizationbased approaches, be it planners [15]–[17] or control [18, 19], require model differentiability.”
BackgroundA Probabilistic Framework for Learning Kinematic Models of Articulated Objects
Sturm, J., Stachniss, C., & Burgard, W. (2011) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3229 · Cited by 96 · cited 4×
“For Cartesian Equilibrium Point control [12] as it is used in [3, 4, 8] only forward kinematics are required.”
BackgroundMethodInfluentialSelf-citationCOLLADA: Sailing the Gulf of 3D Digital Content Creation
Arnaud, R., & Barnes, M. C. (2006) · Cited by 86
Jain, A., & Kemp, C. C. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509445 · Cited by 78
A Probabilistic Framework for Learning Kinematic Models of Articulated Objects
Sturm, J., Stachniss, C., & Burgard, W. (2012) · Cited by 78
Self-citationActive articulation model estimation through interactive perception
Hausman, K., Niekum, S., Osentoski, S., & Sukhatme, G. S. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139655 · Cited by 77 · cited 2×
“To generate observations for an estimation, [4, 6, 8, 9] use simple heuristics for interactions.”
MethodArnaud, R., & Barnes, M. C. (2006) · A K Peters/CRC Press eBooks · doi:10.1201/b10630 · Cited by 77
Interactive segmentation, tracking, and kinematic modeling of unknown 3D articulated objects
Katz, D., Kazemi, M., Andrew Bagnell, J., & Stentz, A. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631292 · Cited by 74
A generalized framework for opening doors and drawers in kitchen environments
Rühr, T., Sturm, J., Pangercic, D., Beetz, M., & Cremers, D. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6224929 · Cited by 59 · cited 2×
“For Cartesian Equilibrium Point control [13] as it is used in [3, 4, 9] only forward kinematics are required.”
MethodSelf-citationUnderstanding the geometry of workspace obstacles in Motion Optimization
Ratliff, N., Toussaint, M., & Schaal, S. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139778 · Cited by 58
Learning Kinematic Feasibility for Mobile Manipulation Through Deep Reinforcement Learning
Honerkamp, D., Welschehold, T., & Valada, A. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3092685 · Cited by 57 · cited 1×
“The same is also a requirement for learning-based control approaches such as [13].”
BackgroundSelf-citationMartin Martin, R., & Brock, O. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6942902 · Cited by 55
Fang, Z., Bartels, G., & Beetz, M. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759590 · Cited by 28 · cited 1×
“Optimizationbased approaches, be it planners [15]–[17] or control [18, 19], require model differentiability.”
BackgroundSelf-citationInteractive Bayesian identification of kinematic mechanisms
Barragan, P. R., Kaelbling, L. P., & Lozano-Perez, T. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907126 · Cited by 23 · cited 1×
“To generate observations for an estimation, [4, 7, 9, 10] generate interactions for an estimation using simple interaction heuristics.”
MethodOpening a lockbox through physical exploration
Baum, M., Bernstein, M. A., Martín-Martín, R., Höfer, S., Kulick, J., Toussaint, M., Kacelnik, A., & Brock, O. (2017) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2017.8246913 · Cited by 20 · cited 4×
“To generate observations for an estimation, [4, 6, 8, 9] use simple heuristics for interactions.”
BackgroundMethodInfluentialActive exploration of joint dependency structures
Kulick, J., Otte, S., & Toussaint, M. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139549 · Cited by 19 · cited 1×
“use a model based on explicit change points and zones throughout which the DoF remains locked or unlocked, first introduced by [20], we opt to use a continuous model to”
MethodCoupled learning of action parameters and forward models for manipulation
Höfer, S., & Brock, O. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759573 · Cited by 14
(2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings
Cited by
Honerkamp, D., Welschehold, T., & Valada, A. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3284346 · Cited by 30
Self-citationSequential Manipulation Planning for Over-Actuated Unmanned Aerial Manipulators
Su, Y., Li, J., Jiao, Z., Wang, M., Chu, C., Li, H., Zhu, Y., & Liu, H. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341441 · Cited by 28
“In particular, consolidating the kinematics of the mobile base, the manipulator, and the manipulated object into one kinematics chain— constructing a VKC—emerges as an effective means; it plans the mobile manipulator as a whole, yielding more coordinated manipulations [16–19].”
BackgroundLearning Hierarchical Interactive Multi-Object Search for Mobile Manipulation
Schmalstieg, F., Honerkamp, D., Welschehold, T., & Valada, A. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3329619 · Cited by 21
Self-citationCategory-Independent Articulated Object Tracking with Factor Graphs
Heppert, N., Migimatsu, T., Yi, B., Chen, C., & Bohg, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982029 · Cited by 35
“In contrast to this non-parametric approach, [23] proposed a symbolic modeling language for arbitrary articulated objects.”
ResultLearning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects
Schiavi, G., Wulkop, P., Rizzi, G., Ott, L., Siegwart, R., & Chung, J. J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160747 · Cited by 27
Metadata
- Authors
- Rofer, A., Bartels, G., Burgard, W., Valada, A., & Beetz, M.
- Affiliations
- University of Bremen (DE)
- University of Freiburg (DE)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2022-04
- Volume
- 7
- Issue
- 2
- Pages
- 3372-3379
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W3199611723
- Cited by
- 10
- References
- 30
- Citations by year
- 2026: 2 · 2024: 2 · 2023: 4 · 2022: 2
- FWCI
- 0.81
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Albert-Ludwigs-Universität Freiburg
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Control and Systems Engineering › Robotic Mechanisms and Dynamics
- Keywords
- articulated structures, mobile manipulation, service robots, state estimation, motion planning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.