PANDA: Pose Aligned Networks for Deep Attribute Modeling
Open accessAbstract
We propose a method for inferring human attributes (such as gender, hair style, clothes style, expression, action) from images of people under large variation of viewpoint, pose, appearance, articulation and occlusion. Convolutional Neural Nets (CNN) have been shown to perform very well on large scale object recognition problems. In the context of attribute classification, however, the signal is often subtle and it may cover only a small part of the image, while the image is dominated by the effects of pose and viewpoint. Discounting for pose variation would require training on very large labeled datasets which are not presently available. Part-based models, such as poselets [4] and DPM [12] have been shown to perform well for this problem but they are limited by shallow low-level features. We propose a new method which combines part-based models and deep learning by training pose-normalized CNNs. We show substantial improvement vs. state-of-the-art methods on challenging attribute classification tasks in unconstrained settings. Experiments confirm that our method outperforms both the best part-based methods on this problem and conventional CNNs trained on the full bounding box of the person.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · see citing works
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
Self-citationWhat is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · see citing works
Self-citationStochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_25 · Cited by 2,097 · see citing works
Describing objects by their attributes
Farhadi, A., Endres, I., Hoiem, D., & Forsyth, D. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206772 · Cited by 1,917 · see citing works
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849 · see citing works
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · see citing works
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Stefan Harmeling, B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206594 · Cited by 1,563 · see citing works
Attribute and simile classifiers for face verification
Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459250 · Cited by 1,383 · see citing works
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325 · see citing works
Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,186 · see citing works
Self-citationPoselets: Body part detectors trained using 3D human pose annotations
Bourdev, L., & Malik, J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459303 · Cited by 961 · see citing works
Self-citationParikh, D., & Grauman, K. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126281 · Cited by 901 · see citing works
Pedestrian Detection with Unsupervised Multi-stage Feature Learning
Sermanet, P., Kavukcuoglu, K., Chintala, S., & LeCun, Y. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.465 · Cited by 753 · see citing works
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 633 · see citing works
Self-citationAutomatic Attribute Discovery and Characterization from Noisy Web Data
Berg, T. L., Berg, A. C., & Shih, J. J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_48 · Cited by 418 · see citing works
Detecting People Using Mutually Consistent Poselet Activations
Bourdev, L. D., Maji, S., Brox, T., & Malik, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_13 · Cited by 369 · see citing works
Self-citationDescribing people: A poselet-based approach to attribute classification
Bourdev, L. D., Maji, S., & Malik, J. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126413 · Cited by 334 · see citing works
Self-citationBuilding high-level features using large scale unsupervised learningPreprint
Le, Q. V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., & Ng, A. Y. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.6209 · Cited by 236 · see citing works
Self-citationDiscovering localized attributes for fine-grained recognition
Duan, K., Parikh, D., Crandall, D., & Grauman, K. L. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248089 · Cited by 231 · see citing works
Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
Zhang, N., Farrell, R. J., Iandola, F., & Darrell, T. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.96 · Cited by 196 · see citing works
Self-citationBirdlets: Subordinate categorization using volumetric primitives and pose-normalized appearance
Farrell, R. J., Oza, O., Zhang, N., Morariu, V. I., Darrell, T., & Davis, L. S. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126238 · Cited by 196 · see citing works
Self-citationPose pooling kernels for sub-category recognition
Zhang, N., Farrell, R. J., & Darrell, T. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248364 · Cited by 124 · see citing works
Self-citationHuman Attribute Recognition by Rich Appearance Dictionary
Joo, J., Wang, S., & Zhu, S. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.95 · Cited by 44 · see citing works
Deep Attribute NetworksPreprint
Chung, J., Lee, D.-H., Seo, Y., et al. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.2881 · Cited by 19 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Self-citationLearning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · see citing works
Self-citationDeep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747 · see citing works
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032 · see citing works
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133 · see citing works
Singh, K. K., & Lee, Y. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.381 · Cited by 724 · see citing works
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.55 · Cited by 768 · see citing works
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607 · see citing works
Striking the Right Balance With Uncertainty
Khan, S., Hayat, M., Zamir, S. W., Shen, J., & Shao, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00019 · Cited by 180 · see citing works
. Hyper-class augmented and regularized deep learning for fine-grained image classification
Xie, S., Yang, T., Wang, X., & Lin, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298880 · Cited by 169 · see citing works
Self-taught object localization with deep networks
Bazzani, L., Bergamo, A., Anguelov, D., & Torresani, L. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477688 · Cited by 140 · see citing works
- Authors
- Zhang, N., Paluri, M., Ranzato, M., Darrell, T., & Bourdev, L.
- Affiliations
- Meta (United States) (US)
- University of California, Berkeley (US)
- Venue
- 2014 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2014-06
- Pages
- 1637-1644
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2147414309
- Cited by
- 495
- References
- 30
- Citations by year
- 2026: 6 · 2025: 17 · 2024: 15 · 2023: 26 · 2022: 24 · 2021: 31 · 2020: 42 · 2019: 64 · 2018: 56 · 2017: 76 · 2016: 61 · 2015: 64 · 2014: 13
- FWCI
- 38.35
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1134072, 0905647, 1212798)
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- pose normalization, part-based models, person re-identification, viewpoint variation, occlusion, gender classification, clothing style, facial expressions, human action recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
