The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition
Open accessAbstract
Human faces in surveillance videos often suffer from severe image blur, dramatic pose variations, and occlusion. In this paper, we propose a comprehensive framework based on Convolutional Neural Networks (CNN) to overcome challenges in video-based face recognition (VFR). First, to learn blur-robust face representations, we artificially blur training data composed of clear still images to account for a shortfall in real-world video training data. Using training data composed of both still images and artificially blurred data, CNN is encouraged to learn blur-insensitive features automatically. Second, to enhance robustness of CNN features to pose variations and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which extracts complementary information from holistic face images and patches cropped around facial components. TBE-CNN is an end-to-end model that extracts features efficiently by sharing the low- and middle-level convolutional layers between the trunk and branch networks. Third, to further promote the discriminative power of the representations learnt by TBE-CNN, we propose an improved triplet loss function. Systematic experiments justify the effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves state-of-the-art performance on three popular video face databases: PaSC, COX Face, and YouTube Faces. With the proposed techniques, we also obtain the first place in the BTAS 2016 Video Person Recognition Evaluation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,822
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Bilinear CNN Models for Fine-Grained Visual Recognition
Lin, T.-Y., RoyChowdhury, A., & Maji, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.170 · Cited by 2,135
Deep Metric Learning Using Triplet Network
Hoffer, E., & Ailon, N. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24261-3_7 · Cited by 2,134
Deep Learning Face Representation by Joint Identification-Verification
Sun, Y., Wang, X., & Tang, X. (2014) · arXiv · doi:10.48550/arxiv.1406.4773 · Cited by 1,906
Learning Face Representation from Scratch
Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7923 · Cited by 1,521
Deep Convolutional Network Cascade for Facial Point Detection
Sun, Y., Wang, X., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.446 · Cited by 1,382
Face recognition in unconstrained videos with matched background similarity
Wolf, L., Hassner, T., & Maoz, I. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995566 · Cited by 1,382
Part-Based R-CNNs for Fine-Grained Category Detection
Zhang, N., Donahue, J., Girshick, R., & Darrell, T. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_54 · Cited by 1,150
Deeply learned face representations are sparse, selective, and robust
Sun, Y., Wang, X., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298907 · Cited by 939
Learning a convolutional neural network for non-uniform motion blur removal
Sun, J., Cao, W., Xu, Z., & Ponce, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298677 · Cited by 906
DeepID3: Face Recognition with Very Deep Neural Networks
Sun, Y. J., Liang, D., Wang, X., & Tang, X. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.00873 · Cited by 842
Deep Convolutional Neural Network for Image Deconvolution
Li, X., Ren, J., Liu, C., & Jia, J. (2014) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 836
Discriminative Learning of Deep Convolutional Feature Point Descriptors
Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., & Moreno-Noguer, F. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.22 · Cited by 804
Discriminative Deep Metric Learning for Face Verification in the Wild
Hu, J., Lu, J., & Tan, Y.-P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.242 · Cited by 682
Joint Deep Learning for Pedestrian Detection
Ouyang, W., & Wang, X. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.257 · Cited by 660
Part-Stacked CNN for Fine-Grained Visual Categorization
Huang, S., Xu, Z., Tao, D., & Zhang, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.132 · Cited by 468
Self-citationRobust Face Recognition via Multimodal Deep Face Representation
Ding, C., & Tao, D. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477042 · Cited by 449 · cited 1×
“We augment the CASIA-WebFace database by horizontal flipping and image jittering [58].”
MethodSelf-citationDeformable part models are convolutional neural networks
Girshick, R., Iandola, F., Darrell, T., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298641 · Cited by 411
Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition
Ding, C., Choi, J., Tao, D., & Davis, L. S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2462338 · Cited by 381
Self-citationPatch Alignment for Dimensionality Reduction
Zhang, T., Tao, D., Li, X., & Yang, J. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.212 · Cited by 381
Self-citationWolf, L., Hassner, T., & Taigman, Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.230 · Cited by 333
A Comprehensive Survey on Pose-Invariant Face Recognition
Ding, C., & Tao, D. (2016) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2845089 · Cited by 329
Self-citationRecognition of blurred faces using Local Phase Quantization
Ahonen, T., Rahtu, E., Ojansivu, V., & Heikkila, J. (2008) · 2008 19th International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761847 · Cited by 301
Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching
Harandi, M. T., Sanderson, C., Shirazi, S., & Lovell, B. C. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995564 · Cited by 290
Yang, W., Ouyang, W., Li, H., & Wang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.335 · Cited by 255
Targeting Ultimate Accuracy: Face Recognition via Deep Embedding
Liu, J., Deng, Y., Bai, T., Wei, Z., & Huang, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.07310 · Cited by 223
Face Recognition Using Sparse Approximated Nearest Points between Image Sets
Hu, Y., Mian, A. S., & Owens, R. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.283 · Cited by 189
The challenge of face recognition from digital point-and-shoot cameras
Beveridge, J. R., Bowyer, K. W., Flynn, P. J., Cheng, S., Phillips, P. J., Bolme, D. S., Draper, B. A., Givens, G. H., Lui, Y. M., Teli, M. N., et al. (2013) · 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2013.6712704 · Cited by 181
Chan, C. H., Tahir, M. A., Kittler, J., & Pietikäinen, M. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.199 · Cited by 180
Multi-manifold deep metric learning for image set classification
Lu, J., Wang, G., Deng, W., Moulin, P., & Zhou, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298717 · Cited by 178
A Benchmark and Comparative Study of Video-Based Face Recognition on COX Face Database
Huang, Z., Shan, S., Wang, R., Zhang, H., Lao, S., Kuerban, A., & Chen, X. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2493448 · Cited by 146
Image Set-Based Collaborative Representation for Face Recognition
Zhu, P., Zuo, W., Zhang, L., Shiu, S. C.-K., & Zhang, D. D. (2014) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2014.2324277 · Cited by 142
Dictionary-Based Face Recognition from Video
Chen, Y.-C., Patel, V. M., Phillips, P. J., & Chellappa, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_55 · Cited by 137
Eigen-PEP for Video Face Recognition
Li, H., Hua, G., Shen, X., Lin, Z., & Brandt, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16811-1_2 · Cited by 130
FACE RECOGNITION FROM VIDEO: A REVIEW
Barr, J., Bowyer, K. W., Flynn, P. J., & Biswas, S. (2012) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001412660024 · Cited by 123
A Compact and Discriminative Face Track Descriptor
Parkhi, O., Simonyan, K., Vedaldi, A., et al. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.219 · Cited by 120
1 Image Set based Collaborative Representation for Face Recognition
Zhu, P., Member, S., Zuo, W., Zhang, L., Shiu, S., & Zhang, D. D. (2014) · Cited by 113
Combining appearance and motion for face and gender recognition from videos
Hadid, A., & Pietikäinen, M. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.02.011 · Cited by 109
Recent Progress in Image DeblurringPreprint
Wang, R., & Tao, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.6838 · Cited by 108
Self-citationPose-Robust Recognition of Low-Resolution Face Images
Biswas, S., Aggarwal, G., Flynn, P. J., & Bowyer, K. W. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.68 · Cited by 104
Huang, Z., Wang, R., Shan, S., & Chen, X. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.03.011 · Cited by 93
Image sets alignment for Video-Based Face Recognition
Cui, Z., Shan, S., Zhang, H., Lao, S., & Chen, X. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247982 · Cited by 91
Facial Deblur Inference Using Subspace Analysis for Recognition of Blurred Faces
Nishiyama, M., Hadid, A., Takeshima, H., et al. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.203 · Cited by 91
Wan, L., Eigen, D., & Fergus, R. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298686 · Cited by 86
A Blur-Robust Descriptor with Applications to Face Recognition
Gopalan, R., Taheri, S., Turaga, P. K., et al. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.15 · Cited by 84
Hierarchical-PEP model for real-world face recognition
Li, H., & Hua, G. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299032 · Cited by 69
Fine-grained pose prediction, normalization, and recognitionPreprint
Zhang, N., Shelhamer, E., Gao, Y., & Darrell, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07063 · Cited by 49
Report on the FG 2015 Video Person Recognition Evaluation
Beveridge, J. R., Zhang, H., Draper, B. A., Flynn, P. J., Feng, Z., Huber, P., Kittler, J., Huang, Z., Li, S., Li, Y., et al. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163156 · Cited by 48
Self-citationDeformable Part Models with CNN Features
Savalle, P.-A., Tsogkas, S., Papandreou, G., & Kokkinos, I. (2014) · HAL · Cited by 44
Toward Large-Population Face Identification in Unconstrained Videos
Liu, L., Zhang, L., Liu, H., & Yan, S. (2014) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2319671 · Cited by 43
Video-to-video face matching: Establishing a baseline for unconstrained face recognition
Best-Rowden, L., Klare, B. F., Klontz, J. C., & Jain, A. K. (2013) · IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2013.6712699 · Cited by 36
Bicego, M., Grosso, E., & Tistarelli, M. (2005) · Lecture notes in computer science · doi:10.1007/11608288_16 · Cited by 24
An Efficient Joint Formulation for Bayesian Face Verification
Chen, D., Cao, X., Wipf, D., Wen, F., & Sun, J. Z. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2533383 · Cited by 20
Report on the BTAS 2016 Video Person Recognition Evaluation
Scheirer, W. J., Flynn, P. J., Ding, C., Guo, G., Štruc, V., Jazaery, M. A., Grm, K., Dobrišek, S., Tao, D., Zhu, Y. M., et al. (2016) · IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2016.7791198 · Cited by 19
Self-citationOne-to-many face recognition with bilinear CNNsPreprint
RoyChowdhury, A., Lin, T., Maji, S., & Miller, E. G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.01342 · Cited by 14
Video face matching using subset selection and clustering of probabilistic multi-region histograms
Mau, S., Shaokang Chen, A., Sanderson, C., & Lovell, B. C. (2010) · Cited by 8
A comparative study of video-based object recognition from an egocentric viewpoint
Shao, M., Tang, D., Liu, Y., & Kim, T. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.07.023 · Cited by 5
Cited by
Deep face recognition: A survey
Wang, M., & Deng, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.081 · Cited by 996
Wang, P., Fan, E., & Wang, P. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.07.042 · Cited by 847
How, D. N. T., Hannan, M. A., Hossain Lipu, M. S., & Ker, P. J. (2019) · IEEE Access · doi:10.1109/access.2019.2942213 · Cited by 812
A survey on deep learning based face recognition
Guo, G., & Zhang, N. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102805 · Cited by 485
Artificial intelligence in cyber security: research advances, challenges, and opportunities
Zhang, Z., Ning, H., Shi, F., Farha, F., Xu, Y., Xu, J., Zhang, F., & Choo, K.-K. R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09976-0 · Cited by 304
Metadata
- Authors
- Ding, C., & Tao, D.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2018-04-01
- Volume
- 40
- Issue
- 4
- Pages
- 1002-1014
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2480418144
- Cited by
- 441
- References
- 65
- FWCI
- 24.60
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.