Markerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Capturing the skeleton motion and detailed time-varying surface geometry of multiple, closely interacting peoples is a very challenging task, even in a multicamera setup, due to frequent occlusions and ambiguities in feature-to-person assignments. To address this task, we propose a framework that exploits multiview image segmentation. To this end, a probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Given the articulated template models of each person and the labeled pixels, a combined optimization scheme, which splits the skeleton pose optimization problem into a local one and a lower dimensional global one, is applied one by one to each individual, followed with surface estimation to capture detailed nonrigid deformations. We show on various sequences that our approach can capture the 3D motion of humans accurately even if they move rapidly, if they wear wide apparel, and if they are engaged in challenging multiperson motions, including dancing, wrestling, and hugging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
Automatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276467 · Cited by 495 · cited 1×
“While the joints are manually placed into each mesh, the skinning weights (cid:4) i;m are automatically computed using the approach [51].”
MethodAutomatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276467 · Cited by 455
Cited by
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
“3D models [7, 20, 24, 30, 35, 46, 54, 66, 72].”
Colyer, S. L., Evans, M., Cosker, D. P., & Salo, A. I. T. (2018) · Sports Medicine - Open · doi:10.1186/s40798-018-0139-y · Cited by 629
Panoptic Studio: A Massively Multiview System for Social Motion Capture
Joo, H., Liu, H., Tan, L., Gui, L., Nabbe, B., Matthews, I., Kanade, T., Nobuhara, S., & Sheikh, Y. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.381 · Cited by 546
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
Joo, H., Simon, T., & Sheikh, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00868 · Cited by 509
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Self-citationFunction4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors
Yu, T., Zheng, Z., Guo, K., Liu, P., Dai, Q., & Liu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00569 · Cited by 355
Self-citationMoSh: motion and shape capture from sparse markers
Loper, M., Mahmood, N., & Black, M. J. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661273 · Cited by 344
Panoptic Studio: A Massively Multiview System for Social Interaction Capture
Joo, H., Simon, T., Li, X., Liu, H., Tan, L., Gui, L., Banerjee, S., Godisart, T., Nabbe, B., Matthews, I., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2782743 · Cited by 344
Monocular Total Capture: Posing Face, Body, and Hands in the Wild
Xiang, D., Joo, H., & Sheikh, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01122 · Cited by 335
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction
Zheng, Z., Yu, T., Liu, Y., & Dai, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050505 · Cited by 320
Self-citationZanfir, A., Marinoiu, E., & Sminchisescu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00229 · Cited by 292
Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., & Liu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00761 · Cited by 255
Self-citationBody Parts Dependent Joint Regressors for Human Pose Estimation in Still Images
Dantone, M., Gall, J., Leistner, C., & Van Gool, L. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2318702 · Cited by 73
Self-citationDeep Neural Network-Based Cooperative Visual Tracking Through Multiple Micro Aerial Vehicles
Price, E., Lawless, G., Ludwig, R., Martinovic, I., Bulthoff, H. H., Black, M. J., & Ahmad, A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2850224 · Cited by 64
Metadata
- Authors
- Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H.-P., & Theobalt, C.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2013-11
- Volume
- 35
- Issue
- 11
- Pages
- 2720-2735
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2089384364
- Cited by
- 135
- References
- 4
- FWCI
- 11.15
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.