Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture From Images “In the Wild”
Open accessAbstract
We present the first method to perform automatic 3D pose, shape and texture capture of animals from images acquired in-the-wild. In particular, we focus on the problem of capturing 3D information about Grevy's zebras from a collection of images. The Grevy's zebra is one of the most endangered species in Africa, with only a few thousand individuals left. Capturing the shape and pose of these animals can provide biologists and conservationists with information about animal health and behavior. In contrast to research on human pose, shape and texture estimation, training data for endangered species is limited, the animals are in complex natural scenes with occlusion, they are naturally camouflaged, travel in herds, and look similar to each other. To overcome these challenges, we integrate the recent SMAL animal model into a network-based regression pipeline, which we train end-to-end on synthetically generated images with pose, shape, and background variation. Going beyond state-of-the-art methods for human shape and pose estimation, our method learns a shape space for zebras during training. Learning such a shape space from images using only a photometric loss is novel, and the approach can be used to learn shape in other settings with limited 3D supervision. Moreover, we couple 3D pose and shape prediction with the task of texture synthesis, obtaining a full texture map of the animal from a single image. We show that the predicted texture map allows a novel per-instance unsupervised optimization over the network features. This method, SMALST (SMAL with learned Shape and Texture) goes beyond previous work, which assumed manual keypoints and/or segmentation, to regress directly from pixels to 3D animal shape, pose and texture.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In ref. 110, the SMAL model was used in a DL approach to predict 3D shape and pose of the Grevy's zebra from images.
“In ref. 110, the SMAL model has been used in a DL approach to predict 3D shape and pose of the Grevy’s zebra from images.”
From Perspectives in machine learning for wildlife conservation (2022) · cites this paper as [110]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 1× · see citing works
“We generate images with random backgrounds by sampling background images from the COCO dataset [16].”
MethodHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · cited 2× · see citing works
“where Lphoto is the photometric loss, computed as the perceptual distance [32] between input image and image prediction.”
BackgroundMethodDeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · cited 1× · see citing works
“[20] use deep learning to track animal keypoints defined by a user on infrared images captured in the lab.”
BackgroundLoper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · see citing works
Self-citationHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1× · see citing works
“Methods focused on humans have the advantage of large datasets of images with ground truth 3D poses captured indoor with mocap systems [9].”
BackgroundEnd-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00744 · Cited by 1,899 · cited 1× · see citing works
“[11] exploit adversarial training, which uses decoupled datasets of 2D keypoints and 3D pose and shape to train on in-the-wild images in a weaklysupervised manner.”
BackgroundSelf-citationSCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489 · see citing works
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera
von Marcard, T., Henschel, R. D., Black, M. J., Rosenhahn, B., & Pons‐Moll, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_37 · Cited by 1,062 · cited 1× · see citing works
“d. One way for obtaining approximate 3D pose and shape from outdoor images is to use human annotators. Lassner et al. [15] build a dataset of human-rated high-quality 3D model fits. von Marcard et al. [31] introduce an in-the-wild dataset of human 3D pose and shape obtained by exploiting IMU sensors and video. Alternat…”
BackgroundSelf-citationKato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954 · cited 1× · see citing works
“We use the Neural Mesh Renderer (NMR) [14] for rendering the model and perspective projection.”
MethodLearning from Synthetic Humans
Varol, G., Romero, J., Martín, X., Mahmood, N., Black, M. J., Laptev, I., & Schmid, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.492 · Cited by 915 · cited 1× · see citing works
“[29] create SURREAL, a full synthetic dataset of 3D pose and shape.”
MethodSelf-citationThe space of human body shapes: reconstruction and parameterization from range scans
Allen, B., Curless, B., & Popović, Z. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882311 · Cited by 913 · cited 1× · see citing works
“t related to the goal of this paper. The estimation of 3D pose and shape from monocular images is largely based on low dimensional statistical models of the human body learned from thousands of scans [3,4,18]. Currently the most popular model is SMPL [18] and recent work on modeling animals also builds on its formulat…”
BackgroundLearning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00055 · Cited by 602 · cited 2× · see citing works
“lied to the full texture. Each loss is associated with a weight. Note that we include both a loss on the model parameters and a loss on silhouettes and projected keypoints. Similar to Pavlakos et al. [23], we observed that a per-vertex loss improves training compared with naive parameter regression. The network is imp…”
MethodInfluentialLearning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_23 · Cited by 540 · see citing works
Self-citationUnite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.500 · Cited by 533 · cited 1× · see citing works
“[15] build a dataset of human-rated high-quality 3D model fits.”
BackgroundSelf-citationNeural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498 · cited 1× · see citing works
“[22] exploit SMPL in conjunction with a bottom-up body part segmentation.”
MethodVideo Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons‐Moll, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00875 · Cited by 468 · cited 1× · see citing works
“[2] go further to create 3D textured models with approximate clothing shape from multiple video frames of a subject in a reference pose.”
BackgroundOpenDR: An Approximate Differentiable Renderer
Loper, M., & Black, M. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_11 · Cited by 458 · cited 1× · see citing works
“evy’s zebras creating models for 10 different subjects. For each zebra model we generated random images that differ in background, shape, pose, camera, and appearance. Models are rendered with OpenDR [17]. Examples of the images are in Figure3, top. Pose variation. For each zebra model we generated 1000 images with di…”
MethodSelf-citation3D Menagerie: Modeling the 3D Shape and Pose of Animals
Zuffi, S., Kanazawa, A., Jacobs, D. W., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.586 · Cited by 381 · cited 5× · see citing works
“Existing methods require manual annotation of the test images with keypoints and/or segmentation [33, 34] or clean imaging conditions where automatic segmentation is possible [5].”
MethodInfluentialSelf-citation3D Pose Regression Using Convolutional Neural Networks
Mahendran, S., Ali, H., & Vidal, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.73 · Cited by 210 · see citing works
Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences
Bogo, F., Black, M. J., Loper, M., & Romero, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.265 · Cited by 203 · see citing works
Self-citationSelf-supervised Learning of Motion CapturePreprint
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01337 · Cited by 198 · cited 1× · see citing works
“[28] exploit temporal consistency over video frames to train an end-to-end prediction model on images without ground truth 3D pose and shape.”
BackgroundLions and Tigers and Bears: Capturing Non-rigid, 3D, Articulated Shape from Images
Zuffi, S., Kanazawa, A., & Black, M. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00416 · Cited by 149 · cited 5× · see citing works
“nthetic data, we capture the texture of the animals from real images and render them with variability of background, pose, illumination and camera. This is obtained exploiting the recent SMALR method [33], which allows us to obtain accurate shape, pose, and texture of 10 animals by annotating only about 50 images. Fro…”
BackgroundMethodInfluentialSelf-citationWhat Shape Are Dolphins? Building 3D Morphable Models from 2D Images
Cashman, T. J., & Fitzgibbon, A. W. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.68 · Cited by 143 · cited 1× · see citing works
“g box detection, recognition, and instance segmentation are common [8]. There is very little work, however, on detecting 3D animal shape and pose. The seminal work on this from Cashman and Fitzgibbon [7] shows how to estimate the shape of dolphins from monocular images. Ntouskos et al. [21] model articulated animals a…”
BackgroundSelf-supervised learning of motion capture
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · neural information processing systems · Cited by 123 · see citing works
Indirect deep structured learning for 3D human body shape and pose prediction
Tan, V., Budvytis, I., & Cipolla, R. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.15 · Cited by 106 · see citing works
Indirect deep structured learning for 3D human body shape and pose prediction
Tan, V., Budvytis, I., & Cipolla, R. (2017) · Apollo (University of Cambridge) · doi:10.17863/cam.21421 · Cited by 103 · cited 1× · see citing works
“lation. Low-dimensional parametric models are particularly attractive for neural architectures as they can generate high quality meshes from a small number of network-generated parameters. Tan et al. [27] train a network that learns the mapping from silhouettes to pose and shape parameters of the SMPL model. Omran et…”
BackgroundSelf-Supervised Intrinsic Image DecompositionPreprint
Janner, M., Wu, J., Kulkarni, T. D., Yildirim, I., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03678 · Cited by 98 · cited 1× · see citing works
“Methods that also exploit photometric loss often are assisted by a segmentation [10], where the loss is computed only in the foreground region.”
MethodCreatures Great and SMAL: Recovering the Shape and Motion of Animals from Video
Biggs, B., Roddick, T., Fitzgibbon, A., & Cipolla, R. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-20873-8_1 · Cited by 89 · cited 2× · see citing works
“[5] fit the SMAL model to images automatically by training a joint detector on synthetically generated silhouettes.”
MethodAugmented Reality meets Deep Learning
Alhaija, H. A., Mustikovela, S. K., Mescheder, L., Geiger, A., & Rother, C. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.81 · Cited by 57 · cited 1× · see citing works
“we capture appearance from real images and use this data to render realistic samples with pose, shape and background variation. This approach of mixing real and synthetic is not novel: Alhaija et al. [1] use synthetic cars in a real scene; here we do the opposite: use real subjects against random backgrounds. We use t…”
MethodLearning 3D Deformation of Animals from 2D Images
Kanazawa, A., Kovalsky, S. Z., Basri, R., & Jacobs, D. W. (2016) · Computer Graphics Forum · doi:10.1111/cgf.12838 · Cited by 41 · cited 1× · see citing works
“[12] learn deformation models for cats and horses.”
BackgroundSelf-citationBalloon Shapes: Reconstructing and Deforming Objects with Volume from Images
Vicente, S., & Agapito, L. (2013) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2013.37 · Cited by 40 · cited 1× · see citing works
“to estimate the shape of dolphins from monocular images. Ntouskos et al. [21] model articulated animals as a collection of 3D primitives that they estimate from segmented images. Vincente and Agapito [30] show how to build a rough giraffe shape from two views. Kanazawa et al. [12] learn deformation models for cats and…”
BackgroundAugmented Reality Meets Deep Learning for Car Instance Segmentation in Urban Scenes
Alhaija, H., Mustikovela, S., Mescheder, L., Geiger, A., & Rother, C. (2017) · MPG.PuRe (Max Planck Society) · Cited by 40 · see citing works
Component-Wise Modeling of Articulated Objects
Ntouskos, V., Sanzari, M., Cafaro, B., Nardi, F., Natola, F., Pirri, F., & Ruiz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.268 · Cited by 18 · cited 1× · see citing works
“ittle work, however, on detecting 3D animal shape and pose. The seminal work on this from Cashman and Fitzgibbon [7] shows how to estimate the shape of dolphins from monocular images. Ntouskos et al. [21] model articulated animals as a collection of 3D primitives that they estimate from segmented images. Vincente and…”
BackgroundBackground modeling for generative image models
Schönborn, S., Egger, B., Forster, A., & Vetter, T. R. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.01.008 · Cited by 13 · cited 1× · see citing works
“computed only in the foreground region. In this work we do not assume any segmentation is available at test time, instead we render a full image prediction, which requires building a background model [26]. We estimate an average background color exploiting the manual segmentations of the images used to build the SMALR…”
MethodAnimated 3D Creatures from Single-view Video by Skeletal Sketching
Reinert, B., Ritschel, T., & Seidel, H. (2016) · MPG.PuRe (Max Planck Society) · doi:10.5555/3076132.3076159 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask R-CNN. In Proceedings of the International Conference on Computer Vision (ICCV), 2017. 3
- Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. Neu- ral 3d mesh renderer. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, 2018. 5
Cited by
Sorted by citations
Zero-1-to-3: Zero-shot One Image to 3D Object
Liu, R., Wu, R., Hoorick, B. V., Tokmakov, P., Zakharov, S., & Vondrick, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00853 · Cited by 888 · see citing works
“CAD models) or category-specific priors [37, 21, 36, 67, 68, 66, 25, 24].”
Perspectives in machine learning for wildlife conservation
Tuia, D., Kellenberger, B., Beery, S., Costelloe, B. R., Zuffi, S., Risse, B., Mathis, A., Mathis, M. W., van Langevelde, F., Burghardt, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-27980-y · Cited by 740 · see citing works
“In (114) the SMAL model has been used in a DL approach to predict 3D shape and pose of the Grevy’s zebra from images.”
MethodSelf-citationDeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547 · see citing works
Deep learning tools for the measurement of animal behavior in neuroscience
Mathis, M. W., & Mathis, A. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.10.008 · Cited by 450 · see citing works
“Recently, outstanding improvements have been made to capture shapes of animals from images (22, 45, 46).”
BackgroundA Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives
Mathis, A., Schneider, S., Lauer, J., & Mathis, M. W. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.017 · Cited by 231 · see citing works
“For some applications (like modeling soft tissue or Neuron 108, October 14, 2020 59 volume), full surface reconstructions are needed, and this area has seen tremendous progress in recent years (G€uler et al., 2018; Sanakoyeu et al., 2020; Zuffi et al., 2019).”
BackgroundReal-time, low-latency closed-loop feedback using markerless posture tracking
Kane, G. A., Lopes, G., Saunders, J. L., Mathis, A., & Mathis, M. W. (2020) · eLife · doi:10.7554/elife.61909 · Cited by 194 · see citing works
Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation
Joo, H., Neverova, N., & Vedaldi, A. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00015 · Cited by 189 · see citing works
“While similar techniques have been used in [60, 3, 73] to improve test-time performance, they have not been rigorously assessed against classical optimization approaches such as SMPLify, nor they have been used to generate pseudo-labels.”
MethodOpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association
Kreiss, S., Bertoni, L., & Alahi, A. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3124981 · Cited by 170 · see citing works
“Another line of work has extended the human SMPL model [41] to animals to learn simultaneously pose and shape of endangered animals [42], [43], [44].”
BackgroundSelf6D: Self-supervised Monocular 6D Object Pose Estimation
Wang, G., Manhardt, F., Shao, J., Ji, X., Navab, N., & Tombari, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_7 · Cited by 139 · see citing works
“Self-supervised learning approaches using neural rendering have also been proposed in the field of 3D object and human body reconstruction from single RGB images [57,20,42,1,64].”
BackgroundBANMo: Building Animatable 3D Neural Models from Many Casual Videos
Yang, G., Vo, M., Neverova, N., Ramanan, D., Vedaldi, A., & Joo, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00288 · Cited by 127 · see citing works
Learning From Synthetic Animals
Mu, J., Qiu, W., Hager, G. D., & Yuille, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01240 · Cited by 124 · see citing works
“Due to the lack of annotations, synthetic data has been widely used to address the problem [43, 3, 44, 45].”
BackgroundMethodMacaquePose: A Novel “In the Wild” Macaque Monkey Pose Dataset for Markerless Motion Capture
Labuguen, R. T., Matsumoto, J., Negrete, S. B., Nishimaru, H., Nishijo, H., Takada, M., Go, Y., Inoue, K., & Shibata, T. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2020.581154 · Cited by 120 · see citing works
“, 2017) or 3D shape of the subject (Biggs et al., 2018; Zuffi et al., 2019).”
BackgroundA-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and PosePreprint
Su, S.-Y., Yu, F., Zollhöfer, M., & Rhodin, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06199 · Cited by 123 · see citing works
“Closest to our approach in this category are the model fitting methods by [79] that textures and geometrically refines an untextured parametric quadruped model to zebra images and to [72] that uses optical flow to refine human pose.”
MethodGART: Gaussian Articulated Template Models
Lei, J., Wang, Y., Pavlakos, G., Liu, L., & Daniilidis, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01879 · Cited by 116 · see citing works
“Novel instances or more species can be captured with limited fidelity by deforming the template with image guidance [70, 94].”
BackgroundWho Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop
Biggs, B., Boyne, O., Charles, J., Fitzgibbon, A., & Cipolla, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_12 · Cited by 107 · see citing works
“Our network architecture is inspired by the model of 3D-Safari [32].”
BackgroundMethodAutolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors
Zakharov, S., Kehl, W., Bhargava, A., & Gaidon, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01224 · Cited by 106 · see citing works
Gosztolai, A., Günel, S., Ríos, V. L., Abrate, M. P., Morales, D., Rhodin, H., Fua, P., & Ramdya, P. P. (2021) · Nature Methods · doi:10.1038/s41592-021-01226-z · Cited by 98 · see citing works
Shape and Viewpoint Without Keypoints
Goel, S., Kanazawa, A., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_6 · Cited by 100 · see citing works
Self-citationFrom Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation
Li, C., & Lee, G. H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00153 · Cited by 101 · see citing works
“We also demonstrate the generalization capacity of our approach by directly testing on the Visual Domain Adaptation Challenge dataset (VisDA2019), the Zebra dataset [46] and the Animal-Pose dataset [6].”
BackgroundMethodInfluentialRealistic One-Shot Mesh-Based Head Avatars
Khakhulin, T., Sklyarova, V., Lempitsky, V., & Zakharov, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20086-1_20 · Cited by 91 · see citing works
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation
Mu, J., Qiu, W., Kortylewski, A., Yuille, A., Vasconcelos, N. M., & Wang, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01276 · Cited by 85 · see citing works
“One line of work leverages parametric mesh models [44, 39, 93, 5] to estimate shape and articulation for faces [76, 64, 68], hands[16], humans bodies [62, 4, 87, 35, 28, 58, 86], and animals [92, 29, 37, 91] by directly inferring shape and articulation parameters.”
BackgroundLASR: Learning Articulated Shape Reconstruction from a Monocular Video
Yang, G., Sun, D., Jampani, V., Vlasic, D., Cole, F. H., Chang, H., Ramanan, D., Freeman, W. T., & Liu, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01572 · Cited by 79 · see citing works
Emerging technologies for behavioral research in changing environments
Couzin, I. D., & Heins, C. (2022) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2022.11.008 · Cited by 80 · see citing works
Okura, F. (2022) · Ikushugaku zasshi · doi:10.1270/jsbbs.21074 · Cited by 69 · see citing works
“These types of representations are not limited to humans; extensions to animals, for example, are available for both skeletal pose representation (Mathis et al. 2018) and parametric shape representation (Zuffi et al. 2017, 2018, 2019).”
BackgroundArticulation-Aware Canonical Surface Mapping
Kulkarni, N. V., Gupta, A. K., Fouhey, D. F., & Tulsiani, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00053 · Cited by 69 · see citing works
“movement of parts, and a plethora of fitting based [4, 15] or prediction based [14, 36, 43] methods have been proposed to tackle this.”
BackgroundMethodAcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild
Joska, D., Clark, L., Muramatsu, N., Jericevich, R., Nicolls, F., Mathis, A., Mathis, M. W., & Patel, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561338 · Cited by 66 · see citing works
“An exciting approach is the SMAL method [20], [21], which leverages the 3D scans of toys of animals to obtain 3D shape and texture [11], [32], [33].”
MethodWormPose: Image synthesis and convolutional networks for pose estimation in C. elegans
Hebert, L., Ahamed, T., Costa, A. C., O’Shaughnessy, L., & Stephens, G. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008914 · Cited by 66 · see citing works
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
Wang, G., Manhardt, F., Liu, X., Ji, X., & Tombari, F. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3136301 · Cited by 69 · see citing works
“Self-supervised learning approaches using differentiable rendering have also been proposed in the field of 3D object and human body reconstruction from single RGB images [24], [78], [79], [80], [81].”
MethodBody language signals for rodent social communication
Ebbesen, C. L., & Froemke, R. C. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2021.01.008 · Cited by 53 · see citing works
3D Bird Reconstruction: A Dataset, Model, and Shape Recovery from a Single View
Badger, M. A., Wang, Y., Modh, A., Perkes, A. D., Kolotouros, N., Pfrommer, B. G., Schmidt, M. F., & Daniilidis, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_1 · Cited by 54 · see citing works
“[40] estimate 3D pose, shape, and texture of zebras in the wild by integrating the SMAL model into an end-to-end network regression pipeline.”
MethodWildlifeDatasets: An open-source toolkit for animal re-identification
Čermák, V., Picek, L., Adam, L., & Papafitsoros, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00585 · Cited by 60 · see citing works
“…[7] 2020 4736 46 ✗ ✗ ✓ ✗ SealID [36] 2022 2080 57 ✗ ✓ ✓ ✗ SeaTurtleID [38] 2022 7774 400 ✓ ✓ ✓ ✗ SeaTurtleID2022 [5] 2024 8729 438 ✓ ✓ ✓ ✗ SMALST [57] 2019 12850 10 ✗ ✗ ✓ ✗ StripeSpotter [30] 2011 820 45 ✗ ✓ ✓ ✗ WhaleSharkID [27] 2020 7693 543 ✓ ✓ ✓ ✗ ZindiTurtleRecall [4] 2022 12803 2265 ✗ ✓ ✓…”
BackgroundAnimal pose estimation: A closer look at the state-of-the-art, existing gaps and opportunities
Jiang, L., Lee, C., Teotia, D., & Ostadabbas, S. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103483 · Cited by 50 · see citing works
GIF: Generative Interpretable Faces
Ghosh, P. S., Gupta, P. S., Uziel, R., Ranjan, A., Black, M. J., & Bolkart, T. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00097 · Cited by 48 · see citing works
“While ‘objects’ like human bodies [46] or animals [72] potentially fulfill these requirements, it remains unclear how to apply GIF to general object categories.”
BackgroundSelf-citationMonocular Differentiable Rendering for Self-supervised 3D Object Detection
Beker, D., Kato, H., Morariu, M. A., Ando, T., Matsuoka, T., Kehl, W., & Gaidon, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_31 · Cited by 44 · see citing works
Leaving flatland: Advances in 3D behavioral measurement
Marshall, J. D., Li, T., Wu, J. H., & Dunn, T. (2022) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2022.02.002 · Cited by 46 · see citing works
“SMAL has since been used to build methods inferring shape in videos of wild zebras [50] and internet images of dogs [79,80], although surface mesh predictions can only be validated against relatively small sets of manually annotated keypoints.”
MethodExemplar Fine-Tuning for 3D Human Pose Fitting Towards In-the-Wild 3D Human Pose EstimationPreprint
Joo, H., Neverova, N., & Vedaldi, A. (2020) · arXiv (Cornell University) · Cited by 53 · see citing works
BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information
Ruegg, N., Zuffi, S., Schindler, K., & Black, M. J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00385 · Cited by 40 · see citing works
“[30] regress a zebra SMAL model from images by exploiting a texture map and learn a shape space for the Grevy’s zebra.”
MethodSelf-citationDeformation-Aware Unpaired Image Translation for Pose Estimation on Laboratory Animals
Li, S., Günel, S., Oštrek, M., Ramdya, P. P., Fua, P., & Rhodin, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01317 · Cited by 38 · see citing works
“[63] uses a model based algorithm, trains on synthetic renderings, and refines on real zebra photographs.”
MethodDOVE: Learning Deformable 3D Objects by Watching Videos
Wu, S., Jakab, T., Rupprecht, C., & Vedaldi, A. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01819-5 · Cited by 38 · see citing works
State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction
Tretschk, E., Kairanda, N., Mallikarjun, B. R., Dabral, R., Kortylewski, A., Egger, B., Habermann, M., Fua, P., Theobalt, C., & Golyanik, V. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14774 · Cited by 38 · see citing works
BITE: Beyond Priors for Improved Three-D Dog Pose Estimation
Rüegg, N., Tripathi, S., Schindler, K., Black, M. J., & Zuffi, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00856 · Cited by 35 · see citing works
“SMAL has been used to estimate 3D shape and pose of zebras [43] and dogs from monocular images [5, 21, 26] and RGBD data [19], while Biggs et al.”
MethodSelf-citationAnimal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Xu, J., Zhang, Y., Peng, J., Ma, W., Jesslen, A., Ji, P., Hu, Q., Zhang, J., Liu, Q., Wang, J., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00835 · Cited by 33 · see citing works
Plum, F., Bulla, R., Beck, H. K., Imirzian, N., & Labonte, D. (2023) · Nature Communications · doi:10.1038/s41467-023-42898-9 · Cited by 33 · see citing works
“Recent advancements in style-and domaintransfercouldbecombinedwithdataproducedby replicAnt to produce even stronger generalist 30,31,66 , or application speci fi c networks 47 – 49 .”
Beyond observation: Deep learning for animal behavior and ecological conservation
Saoud, L. S., Sultan, A., Elmezain, M., Heshmat, M., Seneviratne, L. D., & Hussain, I. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102893 · Cited by 37 · see citing works
CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes
Youwang, K., Ji-Yeon, K., & Oh, T.-H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20062-5_11 · Cited by 33 · see citing works
SeaTurtleID2022: A long-span dataset for reliable sea turtle re-identification
Adam, L., Čermák, V., Papafitsoros, K., & Picek, L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00699 · Cited by 31 · see citing works
“Evaluation of these methods is performed on bench-2014 mark databases, covering several animal groups like mammals [1, 30, 35, 46, 47, 52], reptiles [4, 16], and smaller organisms [10,20,40].”
MethodLuo, H., Xu, T. G., Jiang, Y., Zhou, C., Qiu, Q., Zhang, Y., Yang, W. H., Xu, L., & Jingyi Yu, A. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530086 · Cited by 34 · see citing works
“A series of works are proposed to capture and reconstruct animals like dogs [Biggs et al. 2020], horses [Zuffi et al. 2019], birds [Kanazawa et al. 2018] and other medium animals [Biggs et al. 2018; Cashman and Fitzgibbon 2012; Zuffi et al. 2017].”
BackgroundMethodThree-dimensional surface motion capture of multiple freely moving pigs using MAMMAL
An, L., Ren, J., Yu, T., Hai, T., Jia, Y., & Liu, Y. (2023) · Nature Communications · doi:10.1038/s41467-023-43483-w · Cited by 28 · see citing works
Birds of a Feather: Capturing Avian Shape Models from Images
Wang, Y., Kolotouros, N., Daniilidis, K., & Badger, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01450 · Cited by 28 · see citing works
“Recent methods that use articulated mesh models to extract these signals from images [7, 8, 54, 55] are poised to transform these fields.”
MethodReconstructing Animatable Categories from Videos
Yang, G., Wang, C., Reddy, N. D., & Ramanan, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01630 · Cited by 28 · see citing works
LoTE-Animal: A Long Time-span Dataset for Endangered Animal Behavior Understanding
Liu, D., Hou, J., Huang, S., Liu, J., He, Y., Zheng, B., Ning, J., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01836 · Cited by 28 · see citing works
“Zuffi Silvia et al. [82] and Mu Jiteng et al. [51] also attempted to use synthetically generated animals for pose estimation.”
Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble
Yao, C.-H., Hung, W.-C., Li, Y., Rubinstein, M., Yang, M., & Jampani, V. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00470 · Cited by 25 · see citing works
Unsupervised behaviour analysis and magnification (uBAM) using deep learning
Brattoli, B., Büchler, U., Dorkenwald, M., Reiser, P., Filli, L., Helmchen, F., Wahl, A.-S., & Ommer, B. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00326-x · Cited by 21 · see citing works
Xiao, S., Wang, Y., Perkes, A. D., Pfrommer, B., Schmidt, M. F., Daniilidis, K., & Badger, M. A. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01768-z · Cited by 19 · see citing works
PetFace: A Large-Scale Dataset and Benchmark for Animal Identification
Shinoda, R., & Shiohara, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_2 · Cited by 20 · see citing works
Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport
Saleh, M., Wu, S.-C., Cosmo, L., Navab, N., Busam, B., & Tombari, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01146 · Cited by 19 · see citing works
ShapeAdv: Generating Shape-Aware Adversarial 3D Point CloudsPreprint
Lee, K., Chen, Z., Yan, X., Urtasun, R., & Yumer, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11626 · Cited by 28 · see citing works
“and surface meshes [16, 23, 25, 27, 50, 51, 52, 70].”
PPR: Physically Plausible Reconstruction from Monocular Videos
Yang, G., Yang, S., Zhang, J. Z., Manchester, Z. R., & Ramanan, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00362 · Cited by 18 · see citing works
“A large body of work in human and animal reconstruction uses parametric models [31, 43, 61, 66, 82, 83], which are built from registered 3D scans of human or animals, and serve to recover 3D shapes given a single image or video at test time [1, 2, 22, 81].”
BackgroundThe Peril and Potential of XR-based Interactions with Wildlife
Pimentel, D. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3450378 · Cited by 18 · see citing works
“For example, emerging computer vision frameworks are capable of automatically extracting 3D textured models from images of wildlife [78].”
BackgroundFeng, J., He, C., Wang, G., & Wang, M. (2024) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2277 · Cited by 16 · see citing works
ScarceNet: Animal Pose Estimation with Scarce Annotations
Li, C., & Lee, G. H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01647 · Cited by 16 · see citing works
“Animal pose estimation, including both 2D [5, 16, 20] and 3D [4, 15, 37], has become an active research field in the last few years .”
BackgroundLearning monocular 3D reconstruction of articulated categories from motion
Kokkinos, F., & Kokkinos, I. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00178 · Cited by 16 · see citing works
“Still, for general articulated categories, such as horses or cows, the problem remains in its infancy due to both the lack of strong supervision [55] and the inherent challenge of representing and learning articulated deformations for general categories.”
BackgroundDiscovering Relationships between Object Categories via Universal Canonical Maps
Neverova, N., Sanakoyeu, A., Labatut, P., Novotný, D., & Vedaldi, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00047 · Cited by 16 · see citing works
hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern RecognitionPreprint
Li, C., Ghorbani, N., Broomé, S., Rashid, M., Black, M. J., Hernlund, E., Kjellström, H., & Zuffi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10102 · Cited by 21 · see citing works
“[19] propose an end-to-end reconstruction method also exploiting animal appearance.”
MethodSelf-citationWildlifeReID-10k: Wildlife Re-Identification Dataset with 10k Individual Animals
Adam, L., Čermák, V., Papafitsoros, K., & Picek, L. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00197 · Cited by 18 · see citing works
“…11, 44] and the corresponding review papers [43, 46, 47, 55] but also by the expanding number of publicly available datasets for animal re-identification that cover numerous animal groups, including primates [19, 60], carnivores [16, 31, 52], reptiles [3, 17], whales [14], or mammals [2, 51, 63].”
DigiDogs: Single-View 3D Pose Estimation of Dogs Using Synthetic Training Data
Shooter, M., Malleson, C., & Hilton, A. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00017 · Cited by 12 · see citing works
“[33] successfully achieved the automatic reconstruction of 3D pose, shape, and appearance of in-the-wild images of zebras.”
BackgroundRecent Advances in 3D Human Pose Estimation: From Optimization to Implementation and Beyond
Yan, J., Zhou, M., Pan, J., Yin, M., & Fang, B. (2022) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001422550035 · Cited by 12 · see citing works
Animal Pose Estimation Algorithm Based on the Lightweight Stacked Hourglass Network
Wenwen, Z., Xu, Y., Bai, R., & Li, L. (2022) · IEEE Access · doi:10.1109/access.2022.3231750 · Cited by 13 · see citing works
4D Atlas: Statistical Analysis of the Spatiotemporal Variability in Longitudinal 3D Shape Data
Laga, H., Padilla, M., Jermyn, I. H., Kurtek, S., Bennamoun, & Srivastava, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3163720 · Cited by 14 · see citing works
“This model, which has been extensively used in the literature, has also been adapted to other types of objects such as animals [14]”
BackgroundMethodSyDog: A Synthetic Dog Dataset for Improved 2D Pose EstimationPreprint
Shooter, M., Malleson, C., & Hilton, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.00249 · Cited by 16 · see citing works
“However, in recent years the generation of synthetic data has been an accelerator for machine learning [7, 17, 20, 23, 29].”
BackgroundAniMer: Animal Pose and Shape Estimation Using Family Aware Transformer
Lyu, J., Zhu, T., Gu, Y., Lin, L., Cheng, P., Liu, Y., Tang, X., & An, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01629 · Cited by 10 · see citing works
Chasing the cheetah: how field biomechanics has evolved to keep up with the fastest land animal
Shield, S., Muramatsu, N., da Silva, Z., & Patel, A. (2023) · Journal of Experimental Biology · doi:10.1242/jeb.245122 · Cited by 10 · see citing works
SAOR: Single-View Articulated Object Reconstruction
Aygün, M. M., & Aodha, O. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00988 · Cited by 10 · see citing works
“Although this line of work has led to impressive results for the human body [37], the results for deformable animal categories are lacking [4, 50, 71].”
BackgroundMethodInfluentialBiggs, B., Boyne, O., Charles, J., Fitzgibbon, A., & Cipolla, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8 · Cited by 13 · see citing works
Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
Sabathier, R., Mitra, N. J., & Novotný, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72986-7_16 · Cited by 9 · see citing works
Xu, T., Rao, J., Song, X., Feng, Z., & Wu, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02355-0 · Cited by 9 · see citing works
“One of the most representative approaches, High-a Resolution Network (HRNet) [32], achieves remarkable performance in human pose estimation by maintaining the low-to-high representations and fusing the multi-scale features, with a balanced perception of context details and content semantics.”
CreatureShop: Interactive 3D Character Modeling and Texturing From a Single Color Drawing
Zhang, C., Yang, L., Chen, N., Vining, N., Sheffer, A., Lau, F. C. M., Wang, G., & Wang, W. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3197560 · Cited by 9 · see citing works
“Recently, a category-specific learning-based automatic modeling and texturing method has been proposed [25] which estimates the pose, shape, and plausible texture of zebras in photographs; it is unclear how this method can be generalized to arbitrary drawings.”
MethodFrom Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose EstimationPreprint
Li, C., & Lee, G. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14843 · Cited by 9 · see citing works
“We also demonstrate the generalization capacity of our approach by directly testing on the Visual Domain Adaptation Challenge dataset (VisDA2019), the Zebra dataset [46] and the Animal-Pose dataset [6].”
BackgroundMethodInfluentialHaresh, S., Sun, X., Jiang, H., Chang, A. X., & Savva, M. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00043 · Cited by 10 · see citing works
“There has been much recent progress in dynamic 3D reconstruction targeting humans [5, 15, 26, 30], animals [49– 51], and objects [41, 45–47].”
BackgroundSHIC: Shape-Image Correspondences with No Keypoint Supervision
Shtedritski, A., Rupprecht, C., & Vedaldi, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_8 · Cited by 7 · see citing works
BARC: Breed-Augmented Regression Using Classification for 3D Dog Reconstruction from Images
Rueegg, N., Zuffi, S., Schindler, K., & Black, M. J. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01780-3 · Cited by 7 · see citing works
Self-citationTowards An Egocentric Framework for Rigid and Articulated Object Tracking in Virtual Reality
Taylor, C., McNicholas, R., & Cosker, D. P. (2020) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw50115.2020.00077 · Cited by 7 · see citing works
“Neural networks : Neural networks have been used successfully for tracking objects, as well as for predicting model parameters for rigid and non-rigid objects [10, 29, 30, 42].”
BackgroundMethodResultInfluentialEquine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation
Rashid, M., Broomé, S., Ask, K., Hernlund, E., Andersen, P. H., Kjellström, H., & Lee, Y. J. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00023 · Cited by 7 · see citing works
“Of note are [68] which uses a synthesized zebra dataset to train a network for predicting zebra pose, and [31] which develops a horse specific 3D shape basis and applies it to the downstream task of lameness detection.”
MethodUnsupervised Pose-aware Part Decomposition for Man-Made Articulated Objects
Kawana, Y., Mukuta, Y., & Harada, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20062-5_32 · Cited by 7 · see citing works
“Representative works rely on the use of category-specific template models as the shape and pose prior [21, 46, 3, 45, 19].”
MethodMoDA: Modeling Deformable 3D Objects from Casual Videos
Song, C., Wei, J., Chen, T., Chen, Y., Foo, C.-S., Liu, F., & Lin, G. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02310-5 · Cited by 7 · see citing works
“They are popular in building 3D shapes from images or videos [1, 2, 16, 74] and 3D human or animal generation tasks [48, 49, 61].”
DeepShapeKit: accurate 4D shape reconstruction of swimming fish
Wu, R., Deußen, O., & Li, L. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982097 · Cited by 7 · see citing works
“…SYN 3D hSMAL [18] horse SMAL MC 3D SMPLify [13] human SMPL CMU mocap 3D SMALify [25] animal SMAL mocap, P 3D SMALR [26] animal SMAL SYN 3D SMALST [27] zebra SMAL SYN 3D VIBE [28] human SMPL AMASS 3D WLDO [29] dog SMAL SD 3D CMR [30] bird ls CU 3D DOVE [31] bird li PO, M 3D NRSfM [32] any li P 3D…”
AWOL: Analysis WithOut Synthesis Using Language
Zuffi, S., & Black, M. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73636-0_1 · Cited by 6 · see citing works
Self-citationMounir, R., Shahabaz, A., Gula, R., Theuerkauf, J., & Sarkar, S. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01781-2 · Cited by 6 · see citing works
What is Point Supervision Worth in Video Instance Segmentation?
Huang, S., Huang, D.-A., Yu, Z., Lan, S., Radhakrishnan, S., Álvarez, J. M., Shrivastava, A., & Anandkumar, A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00273 · Cited by 6 · see citing works
“It has achieved increasing attention recently [9, 18, 32, 36, 42, 59, 64] due to its wide real-world applications such as video editing, 3D recon-* Work done during internship / affiliation with NVIDIA. struction [20, 28, 66], and view point estimation [27, 49].”
BackgroundSun, S., Zhu, Z., Dai, X., Zhao, Q., & Li, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69525-5_1 · Cited by 6 · see citing works
“Most recently, a learning-based approach called SMALST [33] integrates SMAL model into a regression network.”
MethodSkeleton Extraction for Articulated Objects With the Spherical Unwrapping Profiles
Zhao, Z., Xie, W., Zuo, B., & Wang, Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3239370 · Cited by 6 · see citing works
“However, the 3D mesh instances in [31], [44], [45] registered to this model are not adopted to train our network training because they contain less than 200 instancesintotal.”
BackgroundColorful 3d reconstruction from a single image based on deep learning
Zhu, Y., Zhang, Y., & Feng, Q. (2020) · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1145/3446132.3446157 · Cited by 6 · see citing works
“[26] present the first method that can automatically generate 3D pose, texture and shape of wild animals from images collected.”
MethodTransferring Dense Pose to Proximal Animal Classes
Sanakoyeu, A., Khalidov, V., McCarthy, M. S., Vedaldi, A., & Neverova, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00528 · Cited by 13 · see citing works
Niewiadomski, T., Yiannakidis, A., Cuevas-Velasquez, H., Sanyal, S., Black, M. J., Zuffi, S., & Kulits, P. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00795 · Cited by 5 · see citing works
“SMAL has been adopted to estimate shape and pose for zebras [66], to estimate the 3D pose and shape of dogs [4, 5, 43] from 2D datasets with keypoints and silhouette annotations, and to create dog avatars [45].”
Self-citationGeometric self-supervision for monocular 3D animal pose estimation
Dai, X., Li, S., Zhao, Q., & Yang, H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111407 · Cited by 5 · see citing works
M2S2: A Multimodal Sensor System for Remote Animal Motion Capture in the Wild
Vally, A., Maswoswere, G., Bowden, N., Paine, S., Amayo, P., Markham, A., & Patel, A. (2025) · IEEE Sensors Letters · doi:10.1109/lsens.2025.3542233 · Cited by 7 · see citing works
“It has successfully been employed in animal pose inference using both multiview [20] and monocular camera data [21].”
Semantic Keypoint Extraction for Scanned Animals using Multi-Depth-Camera Systems
Falque, R., Vidal‐Calleja, T., & Alempijevic, A. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160307 · Cited by 5 · see citing works
“Similar methods have been applied to various animals [21] and refined for specific breeds such as lions [22] and zebras [23].”
MethodKant, Y., Siarohin, A., Güler, R. A., Chai, M., Ren, J., Tulyakov, S., & Gilitschenski, I. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00842 · Cited by 6 · see citing works
Pose‐to‐Motion: Cross‐Domain Motion Retargeting with Pose Prior
Zhao, Q., Li, P., Wang, Y., Sorkine‐Hornung, O., & Wetzstein, G. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15170 · Cited by 5 · see citing works
Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction
Leung, B., Ho, C.-H., & Vasconcelos, N. M. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00453 · Cited by 5 · see citing works
“[20] requires the unsupervised part-based video reconstruction architecture proposed by the authors and [52] optimizes over a shape space specific to their architecture for zebra images.”
Background
- Authors
- Zuffi, S., Kanazawa, A., Berger-Wolf, T., & Black, M.
- Affiliations
- Istituto di Matematica Applicata e Tecnologie Informatiche (IT)
- University of California, Berkeley (US)
- University of Illinois Chicago (US)
- Max Planck Society (DE)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 5358-5367
- OpenAlex
- W2969812964
- Cited by
- 139
- References
- 36
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 25 · 2023: 23 · 2022: 22 · 2021: 29 · 2020: 19 · 2019: 2
- FWCI
- 20.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1514126)
- Nvidia
- Topics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- Grevy's zebra, in-the-wild images, threatened species, occlusion, camouflage, photometric loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
