The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Voxceleb: Large-scale speaker verification in the wild
Open accessAbstract
The objective of this work is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual dataset collected from open source media using a fully automated pipeline. Most existing datasets for speaker identification contain samples obtained under quite constrained conditions, and usually require manual annotations, hence are limited in size. We propose a pipeline based on computer vision techniques to create the dataset from open-source media. Our pipeline involves obtaining videos from YouTube; performing active speaker verification using a two-stream synchronization Convolutional Neural Network (CNN), and confirming the identity of the speaker using CNN based facial recognition. We use this pipeline to curate VoxCeleb which contains contains over a million ‘real-world’ utterances from over 6000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare different CNN architectures with various aggregation methods and training loss functions that can effectively recognise identities from voice under various conditions. The models trained on our dataset surpass the performance of previous works by a significant margin.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2016) · Cited by 31,590
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
Self-citationDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Proceedings European Conference on Computer Vision 2002
Heyden, A., Sparr, G., Nielsen, M., & Johansen, P. (2002) · Lund University Publications (Lund University) · Cited by 5,153
Speaker Verification Using Adapted Gaussian Mixture Models
Reynolds, D. A., Quatieri, T. F., & Dunn, R. B. (2000) · Digital Signal Processing · doi:10.1006/dspr.1999.0361 · Cited by 4,268
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036
Front-End Factor Analysis for Speaker Verification
Dehak, N., Kenny, P. J., Dehak, R., Dumouchel, P., & Ouellet, P. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2064307 · Cited by 3,549
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Self-citationVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Self-citationRobust text-independent speaker identification using Gaussian mixture speaker models
Reynolds, D. A., & Rose, R. C. (1995) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.365379 · Cited by 2,860
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461375 · Cited by 2,620
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.572 · Cited by 2,584
MatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444
SphereFace: Deep Hypersphere Embedding for Face Recognition
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.713 · Cited by 2,280
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256
Self-citationVoxCeleb: A Large-Scale Speaker Identification Dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-950 · Cited by 2,210
Self-citationLivescu, K., Bezman, A., Borges, N., Yung, L.-Y., Çetin, Ö., Frankel, J., King, S., Magimai.-Doss, M., Chi, X., & Lavoie, L. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,145
SWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134 · cited 1×
“The statistics for all the (Godfrey et al.,1992) Telephony (cid:1) 3114 33,039 POLYCOST (Hennebert et al., 2000) Telephony (cid:1) 133 1285 z ICSI Meeting Corpus (Janin et al., 2003) Meetings (cid:1) 53 922 Forensic Comparison (Morrison et al., 2015) Telephony @ 552 1264 ANDOSL (Millar et al.,1994)…”
BackgroundMS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_6 · Cited by 2,122 · cited 1×
“…cross-modal transfer from face to voice or vice versa (Nagrani et al., 2018b; 2018a), emotion recognition (Albanie et al., 2018) and training face recognition from video to complement existing face recognition datasets (Cao et al., 2018; Kemelmacher-Shlizerman et al., 2016; Guo et al., 2016).”
BackgroundProceedings IEEE International Conference on Acoustics Speech and Signal Processing
Zheng, F., McLaughlin, S., & Mulgrew, B. (1991) · Cited by 1,905
NetVLAD: CNN architecture for weakly supervised place recognitionPreprint
Arandjelović, R., Gronát, P., Torii, A., Pajdla, T., & Šivic, J. (2015) · arXiv (Cornell University) · Cited by 1,790
Deep Metric Learning via Lifted Structured Feature Embedding
Song, H. O., Xiang, Y., Jegelka, S., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.434 · Cited by 1,630
Proceedings of the British Machine Vision Conference 2014
(2014) · doi:10.5244/c.28 · Cited by 1,337
A simple neural network module for relational reasoning
Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., & Lillicrap, T. (2017) · arXiv · doi:10.48550/arxiv.1706.01427 · Cited by 1,281 · cited 1×
“It is inspired by the relation networks and their use in face comparisons (Santoro et al., 2017; Xie et al., 2018).”
BackgroundDARPA TIMIT: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallett, D. S., & Dahlgren, N. L. (1993) · doi:10.6028/nist.ir.4930 · Cited by 1,267
Deep neural networks for small footprint text-dependent speaker verification
Variani, E., Lei, X., McDermott, E., Moreno, I. L., & Gonzalez-Dominguez, J. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854363 · Cited by 1,035
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale
Kemelmacher-Shlizerman, I., Seitz, S. M., Miller, D., & Brossard, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.527 · Cited by 912
Generalized End-to-End Loss for Speaker Verification
Wan, L., Wang, Q., Papir, A., & Moreno, I. L. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462665 · Cited by 820
Deep Neural Network Embeddings for Text-Independent Speaker Verification
Snyder, D., Garcia-Romero, D., Povey, D., & Khudanpur, S. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-620 · Cited by 739
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689
Self-citationJanin, A., Baron, D., Edwards, J., Ellis, D., Gelbart, D., Morgan, N., Peskin, B., Pfau, T., Shriberg, E., Stolcke, A., et al. (2003) · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · doi:10.1109/icassp.2003.1198793 · Cited by 665
Proceedings of the International Conference on Acoustics Speech and Signal Processing
Bilbao, S., Arcas, K., & Chaigne, A. J. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · Cited by 645
Speaker Recognition by Machines and Humans: A tutorial review
Hansen, J. H. L., & Hasan, T. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2015.2462851 · Cited by 621
Ephrat, A., Mosseri, I., Lang, O., Dekel, T., Wilson, K., Hassidim, A., Freeman, W. T., & Rubinstein, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201357 · Cited by 620
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54184-6_6 · Cited by 537
Self-citationEnd-to-end text-dependent speaker verification
Heigold, G., Moreno, I., Bengio, S., & Shazeer, N. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472652 · Cited by 514
A novel scheme for speaker recognition using a phonetically-aware deep neural network
Lei, Y., Scheffer, N., Ferrer, L., & McLaren, M. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853887 · Cited by 496
Probabilistic Linear Discriminant Analysis
Ioffe, S. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_41 · Cited by 473
Audio-driven facial animation by joint end-to-end learning of pose and emotion
Karras, T., Aila, T., Laine, S., Herva, A., & Lehtinen, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073658 · Cited by 434
The Conversation: Deep Audio-Visual Speech Enhancement
Afouras, T., Chung, J. S., & Zisserman, A. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1400 · Cited by 430
Self-citationJoint Factor Analysis of Speaker and Session Variability: Theory and Algorithms
Kenny, P. J. (2006) · Cited by 414
Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing
Świętojański, P., & Renals, S. J. (2015) · Cited by 401
McCowan, I. A., Carletta, J., Kraaij, W., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Vasilis Karaiskos, B., et al. (2005) · University of Twente Research Information · Cited by 384
Utterance-level Aggregation for Speaker Recognition in the Wild
Xie, W., Nagrani, A., Chung, J. S., & Zisserman, A. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683120 · Cited by 327
Self-citationEmotion Recognition in Speech using Cross-Modal Transfer in the Wild
Albanie, S., Nagrani, A., Vedaldi, A., & Zisserman, A. (2018) · Proceedings of the 26th ACM international conference on Multimedia · doi:10.1145/3240508.3240578 · Cited by 290
Self-citationLearning and example selection for object and pattern detection
Sung, K.-K., & Poggio, T. (1996) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 260
The Speakers in the Wild (SITW) Speaker Recognition Database
McLaren, M., Ferrer, L., Castan, D., & Lawson, A. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-1129 · Cited by 240
Seeing Voices and Hearing Faces: Cross-Modal Biometric Matching
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00879 · Cited by 220
Self-citationYou said that?Preprint
Chung, J. S., Jamaludin, A., & Zisserman, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02966 · Cited by 152
Self-citationLearnable PINs: cross-modal embeddings for person identity
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_5 · Cited by 151
Self-citationDeep Speaker Embeddings for Short-Duration Speaker Verification
Bhattacharya, G., Alam, J., & Kenny, P. J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1575 · Cited by 140
GhostVLAD for Set-Based Face Recognition
Zhong, Y., Arandjelović, R., & Zisserman, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20890-5_3 · Cited by 114
Self-citationLearning to lip read words by watching videos
Chung, J. S., & Zisserman, A. (2018) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.02.001 · Cited by 106
Self-citationShon, S., Tang, H., & Glass, J. R. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639622 · Cited by 91
POLYCOST: A telephone-speech database for speaker recognition
Hennebert, J., Melin, H., Petrovska‐Delacrétaz, D., & Genoud, D. (2000) · Speech Communication · doi:10.1016/s0167-6393(99)00082-5 · Cited by 83
The Australian National Database of Spoken Language
Millar, J. B., Vonwiller, J. P., Harrington, J. M., & Dermody, P. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1994.389346 · Cited by 78
Xie, W., Shen, L., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_48 · Cited by 78
Self-citationA Novel Learnable Dictionary Encoding Layer for End-to-End Language Identification
Cai, W., Cai, Z., Zhang, X., Wang, X., & Li, M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462025 · Cited by 76
Son, J. S., & Zisserman, A. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.155 · Cited by 74
Self-citationResidual Enhanced Visual Vectors for on-device image matching
Chen, D. M., Tsai, S. S., Chandrasekhar, V., Gabriel Takacs, A., Chen, H., Vedantham, R., Grzeszczuk, R., & Girod, B. (2011) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2011.6190128 · Cited by 40
Analysis of Length Normalization in End-to-End Speaker Verification System
Cai, W., Chen, J., & Li, M. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-92 · Cited by 39
Deep bottleneck features for i-vector based text-independent speaker verification
Ghalehjegh, S. H., & Rose, R. C. (2015) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404844 · Cited by 28
NFI-FRITS: A forensic speaker recognition database and some first experiments
van der Vloed, D., Bouten, J. S., & van Leeuwen, D. A. (2014) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2014-2 · Cited by 24
Robust speech recognition in noise: an evaluation using the SPINE corpus
Hansen, J. H. L., Sarikaya, R., Yapanel, U. H., & Pellom, B. L. (2001) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2001-274 · Cited by 21
Finding Difficult Speakers in Automatic Speaker Recognition
Stoll, L. L. (2011) · eScholarship (California Digital Library) · Cited by 15
Cross-modal Supervision for Learning Active Speaker Detection in VideoPreprint
Chakravarty, P., & Tuytelaars, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08907 · Cited by 1
Cited by
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
SUPERB: Speech Processing Universal PERformance Benchmark
Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1775 · Cited by 681
Two-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471
Vggsound: A Large-Scale Audio-Visual Dataset
Chen, H., Xie, W., Vedaldi, A., & Zisserman, A. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053174 · Cited by 430
Self-citationEnd-to-End anti-spoofing with RawNet2
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414234 · Cited by 402
“When trained using the VoxCeleb2 database, RawNet2 gives a 20% relative improvement compared to an x-vector based system [19] whereas, using the expanded VoxCeleb1-E evaluation dataset protocol, it gives a 13% relative improvement compared to a Thin-ResNet-34 based system [20].”
Method
Metadata
- Authors
- Nagrani, A., Chung, J. S., Xie, W., & Zisserman, A.
- Venue
- Computer Speech & Language
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 60
- Pages
- 101027
- ISSN
- 0885-2308 (ISSN-L)
- OpenAlex
- W2981087920
- Cited by
- 660
- References
- 72
- FWCI
- 30.85
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.