The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Resolving 3D Human Pose Ambiguities With 3D Scene Constraints
Open accessAbstract
To understand and analyze human behavior, we need to capture humans moving in, and interacting with, the world. Most existing methods perform 3D human pose estimation without explicitly considering the scene. We observe however that the world constrains the body and vice-versa. To motivate this, we show that current 3D human pose estimation methods produce results that are not consistent with the 3D scene. Our key contribution is to exploit static 3D scene structure to better estimate human pose from monocular images. The method enforces Proximal Relationships with Object eXclusion and is called PROX. To test this, we collect a new dataset composed of 12 different 3D scenes and RGB sequences of 20 subjects moving in and interacting with the scenes. We represent human pose using the 3D human body model SMPL-X and extend SMPLify-X to estimate body pose using scene constraints. We make use of the 3D scene information by formulating two main constraints. The inter-penetration constraint penalizes intersection between the body model and the surrounding 3D scene. The contact constraint encourages specific parts of the body to be in contact with scene surfaces if they are close enough in distance and orientation. For quantitative evaluation we capture a separate dataset with 180 RGB frames in which the ground-truth body pose is estimated using a motion capture system. We show quantitatively that introducing scene constraints significantly reduces 3D joint error and vertex error. Our code and data are available for research at https://prox.is.tue.mpg.de.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“For the 2D detections we rely on OpenPose [12, 63, 70], which provides body, face and hands keypoints jointly for each person in an image.”
MethodScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,541
3D Semantic Parsing of Large-Scale Indoor Spaces
Armeni, I., Sener, O., Zamir, A. R., Jiang, H., Brilakis, I., Fischer, M., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.170 · Cited by 1,892
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationThe Visual Analysis of Human Movement: A Survey
Gavrila, D. M. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0716 · Cited by 1,782
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationSCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
AMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323
Self-citationHand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Romero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056
Vision-based human motion analysis: An overview
Poppe, R. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.016 · Cited by 835
Learning human activities and object affordances from RGB-D videos
Koppula, H. S., Gupta, R., & Saxena, A. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913478446 · Cited by 683
Detecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248010 · Cited by 649
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
Modeling mutual context of object and human pose in human-object interaction activities
Yao, B., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540235 · Cited by 573
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
Joo, H., Simon, T., & Sheikh, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00868 · Cited by 509
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 502
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
Tracking multiple humans in complex situations
Zhao, T., & Nevatia, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.73 · Cited by 490
A Simple Model of Bipedal Walking Predicts the Preferred Speed–Step Length Relationship
Kuo, A. D. (2001) · Journal of Biomechanical Engineering · doi:10.1115/1.1372322 · Cited by 444
A Statistical Model of Human Pose and Body Shape
Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., & Seidel, H. ‐. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01373.x · Cited by 390
State of the Art on 3D Reconstruction with RGB‐D Cameras
Zollhöfer, M., Stotko, P., Görlitz, A., Theobalt, C., Nießner, M., Klein, R., & Kolb, A. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13386 · Cited by 339
3D Human pose estimation: A review of the literature and analysis of covariates
Sarafianos, N., Boteanu, B., Ionescu, B., & Kakadiaris, I. A. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.09.002 · Cited by 323
Zanfir, A., Marinoiu, E., & Sminchisescu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00229 · Cited by 292
Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation
Tzionas, D., Ballan, L., Srikantha, A., Aponte, P., Pollefeys, M., & Gall, J. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0895-4 · Cited by 277
Self-citationOikonomidis, I., Kyriazis, N., & Argyros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126483 · Cited by 274
Co-fusion: Real-time segmentation, tracking and fusion of multiple objects
Runz, M., & Agapito, L. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989518 · Cited by 227
From 3D scene geometry to human workspace
Gupta, A., Satkin, S., Efros, A. A., & Hebert, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995448 · Cited by 224
Grabner, H., Gall, J., & Van Gool, L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995327 · Cited by 219
Markerless Motion Capture with unsynchronized moving cameras
Hasler, N., Rosenhahn, B., Thormahlen, T., Wand, M., Gall, J., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206859 · Cited by 175
Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
Naseer, M., Khan, S. S., & Porikli, F. (2018) · IEEE Access · doi:10.1109/access.2018.2886133 · Cited by 156
Understanding Everyday Hands in Action from RGB-D Images
Rogez, G., Supančič, J. S., & Ramanan, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.443 · Cited by 147
Savva, M., Chang, A. X., Hanrahan, P., Fisher, M., & Nießner, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925867 · Cited by 131
Activity-centric scene synthesis for functional 3D scene modeling
Fisher, M., Savva, M., Li, Y., Hanrahan, P., & Nießner, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818057 · Cited by 122
Hallucinated Humans as the Hidden Context for Labeling 3D Scenes
Yun, J., Koppula, H. S., & Saxena, A. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.385 · Cited by 118
Shape2Pose: human-centric shape analysis
Kim, V. G., Chaudhuri, S., Guibas, L., & Funkhouser, T. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601117 · Cited by 112
Scene Semantics from Long-Term Observation of People
Delaitre, V., Fouhey, D. F., Laptev, I., Šivic, J., Gupta, A., & Efros, A. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_21 · Cited by 102
Estimating 3D Motion and Forces of Person-Object Interactions From Monocular Video
Li, Z., Sedlář, J., Carpentier, J., Laptev, I., Mansard, N., & Šivic, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00884 · Cited by 101
Scalable 3D Tracking of Multiple Interacting Objects
Kyriazis, N., & Argyros, A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.438 · Cited by 98
SceneGrok: inferring action maps in 3D environments
Savva, M., Chang, A. X., Hanrahan, P., Fisher, M., & Nießner, M. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661230 · Cited by 97
Articulated distance fields for ultra-fast tracking of hands interacting
Taylor, J., Tankovich, V., Tang, D., Keskin, C., Kim, D., Davidson, P., Kowdle, A., & Izadi, S. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130853 · Cited by 91
Hand-Object Contact Force Estimation from Markerless Visual Tracking
Pham, T.-H., Kyriazis, N., Argyros, A., & Kheddar, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2759736 · Cited by 90
People Watching: Human Actions as a Cue for Single View Geometry
Fouhey, D. F., Delaitre, V., Gupta, A., Efros, A. A., Laptev, I., & Šivic, J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0710-z · Cited by 88
Categorizing object-action relations from semantic scene graphs
Aksoy, E. E., Abramov, A., Wörgötter, F., & Dellen, B. K. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509319 · Cited by 80
Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
Kyriazis, N., & Argyros, A. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.9 · Cited by 60
iMapper: interaction-guided scene mapping from monocular videos
Monszpart, Á., Guerrero, P., Ceylan, D., Yumer, E., & Mitra, N. J. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322961 · Cited by 58
Markerless motion capture of man-machine interaction
Rosenhahn, B., Schmaltz, C., Brox, T., Weickert, J., Cremers, D., & Seidel, H. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587520 · Cited by 50
Joint 3D Tracking of a Deformable Object in Interaction with a Hand
Tsoli, A., & Argyros, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_30 · Cited by 49
Dynamical Simulation Priors for Human Motion Tracking
Vondrak, M., Sigal, L., & Jenkins, O. C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.61 · Cited by 25
Cited by
Deep 3D human pose estimation: A review
Wang, J., Tan, S., Zhen, X., Xu, S., Zheng, F., He, Z., & Shao, L. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103225 · Cited by 399
HuMoR: 3D Human Motion Model for Robust Pose Estimation
Rempe, D., Birdal, T., Hertzmann, A., Yang, J., Sridhar, S., & Guibas, L. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01129 · Cited by 304
GRAB: A Dataset of Whole-Body Human Grasping of Objects
Taheri, O., Ghorbani, N., Black, M. J., & Tzionas, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_34 · Cited by 274
Self-citationKimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192
Metadata
- Authors
- Hassan, M., Choutas, V., Tzionas, D., & Black, M.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2282-2292
- OpenAlex
- W2969853371
- Cited by
- 274
- References
- 54
- FWCI
- 8.43
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.