Shape2Pose: human-centric shape analysis
The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
As 3D acquisition devices and modeling tools become widely available there is a growing need for automatic algorithms that analyze the semantics and functionality of digitized shapes. Most recent research has focused on analyzing geometric structures of shapes. Our work is motivated by the observation that a majority of man-made shapes are designed to be used by people. Thus, in order to fully understand their semantics, one needs to answer a fundamental question: "how do people interact with these objects?" As an initial step towards this goal, we offer a novel algorithm for automatically predicting a static pose that a person would need to adopt in order to use an object. Specifically, given an input 3D shape, the goal of our analysis is to predict a corresponding human pose, including contact points and kinematic parameters. This is especially challenging for man-made objects that commonly exhibit a lot of variance in their geometric structure. We address this challenge by observing that contact points usually share consistent local geometric features related to the anthropometric properties of corresponding parts and that human body is subject to kinematic constraints and priors. Accordingly, our method effectively combines local region classification and global kinematically-constrained search to successfully predict poses for various objects. We also evaluate our algorithm on six diverse collections of 3D polygonal models (chairs, gym equipment, cockpits, carts, bicycles, and bipedal devices) containing a total of 147 models. Finally, we demonstrate that the poses predicted by our algorithm can be used in several shape analysis problems, such as establishing correspondences between objects, detecting salient regions, finding informative viewpoints, and retrieving functionally-similar shapes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“TSDF are currently a popular representation for vision-based SLAM in robotics, attracting increasing attention after the seminal work [183].”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [183]
In the citing paper
“Since range cameras carry their own light source, they also work in dark and untextured scenes, which enabled the achievement of remarkable SLAM results [183].”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [183]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Psychology of Everyday Things
Massaro, D. W., & Norman, D. A. (1990) · The American Journal of Psychology · doi:10.2307/1423268 · Cited by 3,466
Data-Driven Grasp Synthesis—A Survey
Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2289018 · Cited by 929
(2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79547-6 · Cited by 844
Lee, C. H., Varshney, A., & Jacobs, D. W. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073244 · Cited by 667
Consistent mesh partitioning and skeletonisation using the shape diameter function
Shapira, L., Shamir, A., & Cohen-Or, D. (2008) · The Visual Computer · doi:10.1007/s00371-007-0197-5 · Cited by 542
1Data-Driven Grasp Synthesis- A Survey
Bohg, J., Morales, A., Asfour, T., & Kragić, D. (2014) · MPG.PuRe (Max Planck Society) · Cited by 532
Salient geometric features for partial shape matching and similarity
Gal, R., & Cohen-Or, D. (2006) · ACM Transactions on Graphics · doi:10.1145/1122501.1122507 · Cited by 497
Learning 3D mesh segmentation and labeling
Kalogerakis, E., Hertzmann, A., & Singh, K. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778839 · Cited by 469
A probabilistic model for component-based shape synthesis
Kalogerakis, E., Chaudhuri, S., Koller, D., & Koltun, V. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185551 · Cited by 350
Self-citationSymmetry in 3D Geometry: Extraction and Applications
Mitra, N. J., Pauly, M., Wand, M., & Ceylan, D. (2013) · Computer Graphics Forum · doi:10.1111/cgf.12010 · Cited by 307
A planar-reflective symmetry transform for 3D shapes
Podolak, J., Shilane, P., Golovinskiy, A., Rusinkiewicz, S., & Funkhouser, T. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141923 · Cited by 297
Self-citationLee, C. H., Varshney, A., & Jacobs, D. W. (2005) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1186822.1073244 · Cited by 260
Learning part-based templates from large collections of 3D shapes
Kim, V. G., Li, W., Mitra, N. J., Chaudhuri, S., DiVerdi, S., & Funkhouser, T. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461933 · Cited by 230
Self-citationFrom 3D scene geometry to human workspace
Gupta, A., Satkin, S., Efros, A. A., & Hebert, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995448 · Cited by 224
Grabner, H., Gall, J., & Van Gool, L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995327 · Cited by 219
iWIRES: an analyze-and-edit approach to shape manipulation
Gal, R., Sorkine, O., Mitra, N. J., & Cohen-Or, D. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531339 · Cited by 212
Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering
Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., & Cohen-Or, D. (2011) · Proceedings of the 2011 SIGGRAPH Asia Conference · doi:10.1145/2024156.2024160 · Cited by 211
Probabilistic reasoning for assembly-based 3D modeling
Chaudhuri, S., Kalogerakis, E., Guibas, L., & Koltun, V. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964930 · Cited by 198
Self-citationConsistent segmentation of 3D models
Golovinskiy, A., & Funkhouser, T. (2009) · Computers & Graphics · doi:10.1016/j.cag.2009.03.010 · Cited by 190
Self-citationPhysically based grasping control from example
Pollard, N. S., & Zordan, V. B. (2005) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/1073368.1073413 · Cited by 179
Exploring collections of 3D models using fuzzy correspondences
Kim, V. G., Li, W., Mitra, N. J., DiVerdi, S., & Funkhouser, T. A. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185550 · Cited by 166
Self-citationData-Driven Grasp Synthesis Using Shape Matching and Task-Based Pruning
Li, Y., Fu, J. L., & Pollard, N. S. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.1033 · Cited by 166
Structure-aware shape processing
Mitra, N., Wand, M., Zhang, H. (., Cohen-Or, D., Kim, V., & Huang, Q.-X. (2013) · SIGGRAPH Asia 2013 Courses · doi:10.1145/2542266.2542267 · Cited by 162
Self-citationPerceptual models of viewpoint preference
Secord, A., Lu, J., Finkelstein, A., Singh, M., & Nealen, A. (2011) · ACM Transactions on Graphics · doi:10.1145/2019627.2019628 · Cited by 160
Distinctive regions of 3D surfaces
Shilane, P., & Funkhouser, T. A. (2007) · ACM Transactions on Graphics · doi:10.1145/1243980.1243981 · Cited by 156
Self-citationThe psychology of everyday things
Shackel, B. (1992) · Acta Psychologica · doi:10.1016/0001-6918(92)90062-i · Cited by 152
Robotic Grasping of Novel Objects
Saxena, A., Driemeyer, J., Kearns, J., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0156 · Cited by 152
Learning 3D mesh segmentation and labeling
Kalogerakis, E., Hertzmann, A., & Singh, K. S. (2010) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1833349.1778839 · Cited by 150
Schelling points on 3D surface meshes
Chen, X., Saparov, A., Pang, B., & Funkhouser, T. A. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185525 · Cited by 144
Self-citationUpright orientation of man-made objects
Fu, H., Cohen‐Or, D., Dror, G., & Sheffer, A. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360641 · Cited by 143
Joint shape segmentation with linear programming
Huang, Q., Koltun, V., & Guibas, L. J. (2011) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/2024156.2024159 · Cited by 137
Self-citationModeling 4D Human-Object Interactions for Event and Object Recognition
Wei, P., Zhao, Y., Zheng, N., & Zhu, S. C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.406 · Cited by 129
Deep Learning for Detecting Robotic Grasps
Lenz, I., Lee, H., & Saxena, A. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.012 · Cited by 126
Hallucinated Humans as the Hidden Context for Labeling 3D Scenes
Jiang, Y., Koppula, H., & Saxena, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.385 · Cited by 120
Goldfeder, C., & Allen, P. K. (2011) · Autonomous Robots · doi:10.1007/s10514-011-9228-1 · Cited by 120
Functional Object Class Detection Based on Learned Affordance Cues
Stark, M. M., Lies, P., Zillich, M., Wyatt, J., & Schiele, B. (2008) · Springer eBooks · doi:10.1007/978-3-540-79547-6_42 · Cited by 106
Scene Semantics from Long-Term Observation of People
Delaitre, V., Fouhey, D. F., Laptev, I., Šivic, J., Gupta, A., & Efros, A. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_21 · Cited by 102
A Metric for Comparing the Anthropomorphic Motion Capability of Artificial Hands
Feix, T., Romero, J., Ek, C. H., Schmiedmayer, H., & Kragić, D. (2012) · IEEE Transactions on Robotics · doi:10.1109/tro.2012.2217675 · Cited by 101
Robust realtime physics-based motion control for human grasping
Zhao, W., Zhang, J., Min, J., & Chai, J. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508412 · Cited by 100
People Watching: Human Actions as a Cue for Single View Geometry
Fouhey, D. F., Delaitre, V., Gupta, A., Efros, A. A., Laptev, I., & Šivic, J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33715-4_53 · Cited by 99
Exploration of continuous variability in collections of 3D shapes
Ovsjanikov, M., Li, W., Guibas, L. J., & Mitra, N. J. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964928 · Cited by 99
Self-citationAn optimization approach for extracting and encoding consistent maps in a shape collection
Huang, Q., Zhang, G., Gao, L., Hu, S., Butscher, A., & Guibas, L. J. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366186 · Cited by 94
Self-citationCo-hierarchical analysis of shape structures
van Kaick, O. M., Xu, K., Zhang, H., Wang, Y., Sun, S., Shamir, A., & Cohen‐Or, D. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461924 · Cited by 90
Learning Visual Object Categories for Robot Affordance Prediction
Sun, J., Moore, J. L., Bobick, A., & Rehg, J. M. (2009) · The International Journal of Robotics Research · doi:10.1177/0278364909356602 · Cited by 89
Fine-grained semi-supervised labeling of large shape collections
Huang, Q.-X., Su, H., & Guibas, L. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508364 · Cited by 72
Self-citationLearning Object Arrangements in 3D Scenes using Human Context
Yun, J., Lim, M., & Saxena, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6462 · Cited by 56
Upright orientation of man-made objects
Fu, H., Cohen‐Or, D., Dror, G., & Sheffer, A. (2008) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1399504.1360641 · Cited by 50
Learning Predictive Features in Affordance based Robotic Perception Systems
Fritz, G., Paletta, L., Breithaupt, R., Rome, E., & Dorffner, G. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.281720 · Cited by 45
Automatic preshaping for a dextrous hand from a simple description of objects
Bard, C., & Troccaz, J. (2002) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.1990.262507 · Cited by 44
Infinite Latent Conditional Random Fields for Modeling Environments through Humans
Yun, J., & Saxena, A. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.034 · Cited by 32
Hallucinating Humans for Learning Robotic Placement of Objects
Jiang, Y. P., & Saxena, A. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-00065-7_61 · Cited by 29
Global Optimization of Robotic Grasps
Gallegos, C. R., Porta, J. M., & Ros, L. (2011) · Robotics: Science and Systems · doi:10.15607/rss.2011.vii.012 · Cited by 20
Monocular depth perception and robotic grasping of novel objects
Ng, A. Y., & Saxena, A. (2009) · Cited by 18
A skeleton-based approach to grasp known objects with a humanoid robot
Przybylski, M., Wächter, M., Asfour, T., & Dillmann, R. (2012) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2012.6651547 · Cited by 12
Global Optimization of Robotic Grasps
Rosales, C., Porta, J. M., & Ros, L. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9481.003.0042 · Cited by 11
Infinite Latent Conditional Random Fields
Yun, J., & Saxena, A. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.42 · Cited by 5
(1977) · doi:10.4324/9781315740218-18
Research works printed but not linked: 1
- Hermans T. Rehg J. M. and Bobick A. 2011. Affordance prediction via learned object attributes. ICRA. Hermans T. Rehg J. M. and Bobick A. 2011. Affordance prediction via learned object attributes. ICRA .
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
A scalable active framework for region annotation in 3D shape collections
Yi, L., Kim, V. G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., & Guibas, L. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980238 · Cited by 1,231
Self-citation3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
Zeng, A., Song, S., Niessner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.29 · Cited by 1,128
Self-citationMo, K., Zhu, S., Chang, A. X., Yi, L., Tripathi, S., Guibas, L. J., & Su, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00100 · Cited by 667
Self-citation
Metadata
- Authors
- Kim, V. G., Chaudhuri, S., Guibas, L., & Funkhouser, T.
- Affiliations
- Stanford University (US)
- Princeton University (US)
- Venue
- ACM Transactions on Graphics
- Published
- 2014-07-27
- Volume
- 33
- Issue
- 4
- Pages
- 1-12
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2145429668
- Cited by
- 112
- References
- 59
- Citations by year
- 2026: 2 · 2024: 4 · 2023: 5 · 2022: 5 · 2021: 13 · 2020: 11 · 2019: 11 · 2018: 13 · 2017: 17 · 2016: 20 · 2015: 6 · 2014: 4
- FWCI
- 14.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Funders
- Division of Information and Intelligent Systems (DMS 1228304, CCF 1161480, IIS-1251217, CNS-0831374)
- Office of Naval Research (N00014-13-1-0341)
- Adobe Systems
- Division of Computer and Network Systems (DMS 1228304, CCF 1161480, IIS-1251217, CNS-0831374)
- Intel Corporation
- Division of Mathematical Sciences (DMS 1228304, CCF 1161480, IIS-1251217, CNS-0831374)
- Air Force Office of Scientific Research (FA9550-12-1-0372)
- Division of Computing and Communication Foundations (DMS 1228304, CCF 1161480, IIS-1251217, CNS-0831374)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Processing and 3D Reconstruction
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- human pose prediction, 3D shape analysis, object semantics, kinematic constraints, viewpoint selection, shape correspondence, man-made objects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.