Birds of a Feather: Capturing Avian Shape Models from Images
Open accessAbstract
Animals are diverse in shape, but building a deformable shape model for a new species is not always possible due to the lack of 3D data. We present a method to capture new species using an articulated template and images of that species. In this work, we focus mainly on birds. Although birds represent almost twice the number of species as mammals, no accurate shape model is available. To capture a novel species, we first fit the articulated template to each training sample. By disentangling pose and shape, we learn a shape space that captures variation both among species and within each species from image evidence. We learn models of multiple species from the CUB dataset, and contribute new species-specific and multi-species shape models that are useful for downstream reconstruction tasks. Using a low-dimensional embedding, we show that our learned 3D shape space better reflects the phylogenetic relationships among birds than learned perceptual features.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
3D shape models have recently been defined also for birds.
“3D shape models have been recently defined also for birds111.”
From Perspectives in machine learning for wildlife conservation (2022) · cites this paper as [111]
Cites
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190 · see citing works
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101 · see citing works
Inferring the historical patterns of biological evolution
Pagel, M. (1999) · Nature · doi:10.1038/44766 · Cited by 5,200 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · see citing works
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00744 · Cited by 1,899 · see citing works
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781 · see citing works
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,502 · see citing works
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489 · see citing works
Learning a model of facial shape and expression from 4D scans
Li, T., Bolkart, T., Black, M. J., Li, H., & Romero, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130813 · Cited by 1,226 · see citing works
Inferring evolutionary processes from phylogenies
PAGEL, M. (1997) · Zoologica Scripta · doi:10.1111/j.1463-6409.1997.tb00423.x · Cited by 1,106 · see citing works
Romero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056 · see citing works
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Kolotouros, N., Pavlakos, G., Black, M. J., & Daniilidis, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00234 · Cited by 1,046 · see citing works
Self-citationPhylogenetic signal and linear regression on species data: <i>Phylogenetic regression</i>
Revell, L. J. (2010) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2010.00044.x · Cited by 931 · see citing works
The space of human body shapes: reconstruction and parameterization from range scans
Allen, B., Curless, B., & Popović, Z. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882311 · Cited by 913 · see citing works
Tempo and mode in evolution: phylogenetic inertia, adaptation and comparative methods
Blomberg, S. P., & Garland, T. (2002) · Journal of Evolutionary Biology · doi:10.1046/j.1420-9101.2002.00472.x · Cited by 896 · see citing works
As-Rigid-As-Possible Surface Modeling
Sorkine‐Hornung, O., & Alexa, M. (2007) · doi:10.2312/sgp/sgp07/109-116 · Cited by 804 · see citing works
Recovering non-rigid 3D shape from image streams
Bregler, C., Hertzmann, A., & Biermann, H. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854941 · Cited by 777 · see citing works
The space of human body shapes
Allen, B., Curless, B., & Popović, Z. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882311 · Cited by 725 · see citing works
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00055 · Cited by 602 · see citing works
Self-citationStatistical methods for tomographic image reconstruction
Geman, S., & McClure, D. E. (1987) · Medical Entomology and Zoology · Cited by 586 · see citing works
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Kolotouros, N., Pavlakos, G., & Daniilidis, K. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00463 · Cited by 556 · see citing works
Self-citationLearning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_23 · Cited by 540 · see citing works
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
Joo, H., Simon, T., & Sheikh, Y. A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00868 · Cited by 498 · see citing works
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498 · see citing works
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons‐Moll, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00875 · Cited by 468 · see citing works
Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors
Torresani, L., Hertzmann, A., & Bregler, C. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70752 · Cited by 456 · see citing works
Proxy Anchor Loss for Deep Metric Learning
Kim, S., Kim, D.-W., Cho, M., & Kwak, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00330 · Cited by 437 · see citing works
Accelerating 3D deep learning with PyTorch3D
Johnson, J., Ravi, N., Reizenstein, J., Novotný, D., Tulsiani, S., Lassner, C., & Branson, S. (2020) · SIGGRAPH Asia Courses · doi:10.1145/3415263.3419160 · Cited by 413 · see citing works
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1186822.1073207 · Cited by 410 · see citing works
3D Menagerie: Modeling the 3D Shape and Pose of Animals
Zuffi, S., Kanazawa, A., Jacobs, D. W., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.586 · Cited by 381 · see citing works
Nealen, A., Igarashi, Y., Sorkine‐Hornung, O., & Alexa, M. (2006) · International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE) · doi:10.1145/1174429.1174494 · Cited by 353 · see citing works
ARCH: Animatable Reconstruction of Clothed Humans
Huang, Z., Xu, Y., Lassner, C., Li, H., & Tung, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00316 · Cited by 353 · see citing works
GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models
Xu, H., Băzăvan, E. G., Zanfir, A., Freeman, W. T., Sukthankar, R., & Sminchisescu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00622 · Cited by 347 · see citing works
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M. A., Bhatnagar, B. L., Theobalt, C., & Pons‐Moll, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00127 · Cited by 339 · see citing works
Home 3D body scans from noisy image and range data
Weiss, A., Hirshberg, D. A., & Black, M. J. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126465 · Cited by 306 · see citing works
Doyle, K., & Gordon, H. (1993) · Australian Veterinary Journal · doi:10.1111/j.1751-0813.1993.tb08058.x · Cited by 267 · see citing works
Category-specific object reconstruction from a single image
Kar, A., Tulsiani, S., Carreira, J., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298807 · Cited by 263 · see citing works
Towards 3D hand tracking using a deformable model
Heap, T., & Hogg, D. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1996.557255 · Cited by 223 · see citing works
The Feather Postures of Birds and the Problem of the Origin of Social Signals
Morris, D. (1956) · Behaviour · doi:10.1163/156853956x00264 · Cited by 222 · see citing works
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Habermann, M., Xu, W., Zollhöfer, M., Pons‐Moll, G., & Theobalt, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00510 · Cited by 218 · see citing works
Towards Accurate Marker-Less Human Shape and Pose Estimation over Time
Huang, Y., Bogo, F., Lassner, C., Kanazawa, A., Gehler, P., Romero, J., Akhter, I., & Black, M. J. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00055 · Cited by 217 · see citing works
Detailed Human Avatars from Monocular Video
Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons‐Moll, G. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00022 · Cited by 212 · see citing works
Demidenko, E. (1987) · Wiley series in probability and statistics · doi:10.1002/9781118651537.ch11 · Cited by 158 · see citing works
Lions and Tigers and Bears: Capturing Non-rigid, 3D, Articulated Shape from Images
Zuffi, S., Kanazawa, A., & Black, M. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00416 · Cited by 149 · see citing works
What Shape Are Dolphins? Building 3D Morphable Models from 2D Images
Cashman, T. J., & Fitzgibbon, A. W. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.68 · Cited by 143 · see citing works
Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach
Zhou, X., Zhu, M., Leonardos, S., & Daniilidis, K. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2605097 · Cited by 141 · see citing works
Self-citationThree-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture From Images “In the Wild”
Zuffi, S., Kanazawa, A., Berger-Wolf, T., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00546 · Cited by 139 · cited 3× · see citing works
“Recent methods that use articulated mesh models to extract these signals from images [7, 8, 54, 55] are poised to transform these fields.”
MethodAccelerating 3D Deep Learning with PyTorch3DPreprint
Ravi, N., Reizenstein, J., Novotný, D., Gordon, T., Lo, W., Johnson, J., & Gkioxari, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08501 · Cited by 122 · see citing works
Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop
Biggs, B., Boyne, O., Charles, J., Fitzgibbon, A., & Cipolla, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_12 · Cited by 107 · see citing works
Shape and Viewpoint Without Keypoints
Goel, S., Kanazawa, A., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_6 · Cited by 100 · see citing works
Statistics and Analysis of Shapes
Krim, H., & Yezzi, A. J. (2006) · Modeling and simulation in science, engineering & technology · doi:10.1007/0-8176-4481-4 · Cited by 76 · see citing works
Balloon Shapes: Reconstructing and Deforming Objects with Volume from Images
Vicente, S., & Agapito, L. (2013) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2013.37 · Cited by 40 · see citing works
Component-Wise Modeling of Articulated Objects
Ntouskos, V., Sanzari, M., Cafaro, B., Nardi, F., Natola, F., Pirri, F., & Ruiz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.268 · Cited by 18 · see citing works
Rphylopars: Phylogenetic Comparative Tools for Missing Data and Within-Species Variation
Goolsby, E. W., Bruggeman, J., & Ané, C. (2015) · doi:10.32614/cran.package.rphylopars · Cited by 16 · see citing works
Implicit Mesh Reconstruction from Unannotated Image CollectionsPreprint
Tulsiani, S., Kulkarni, N. V., & Gupta, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08504 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Perspectives in machine learning for wildlife conservation
Tuia, D., Kellenberger, B., Beery, S., Costelloe, B. R., Zuffi, S., Risse, B., Mathis, A., Mathis, M. W., van Langevelde, F., Burghardt, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-27980-y · Cited by 740 · see citing works
- Authors
- Wang, Y., Kolotouros, N., Daniilidis, K., & Badger, M.
- Affiliations
- University of Pennsylvania (US)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 14734-14744
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3181148101
- Cited by
- 28
- References
- 57
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 11 · 2023: 8 · 2022: 3
- FWCI
- 4.04
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1626008, 2038873, 1703319, 1934960)
- Topics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Processing and 3D Reconstruction
- Keywords
- deformable shape models, birds, 3D image reconstruction, phylogenetic relationships
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
