VoxCeleb: A Large-Scale Speaker Identification Dataset
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
Abstract
Most existing datasets for speaker identification contain samples obtained under quite constrained conditions, and are usually hand-annotated, hence limited in size. The goal of this paper is to generate a large scale text-independent speaker identification dataset collected 'in the wild'. We make two contributions. First, we propose a fully automated pipeline based on computer vision techniques to create the dataset from open-source media. Our pipeline involves obtaining videos from YouTube; performing active speaker verification using a two-stream synchronization Convolutional Neural Network (CNN), and confirming the identity of the speaker using CNN based facial recognition. We use this pipeline to curate VoxCeleb which contains hundreds of thousands of 'real world' utterances for over 1,000 celebrities. Our second contribution is to apply and compare various state of the art speaker identification techniques on our dataset to establish baseline performance. We show that a CNN based architecture obtains the best performance for both identification and verification.
Overview
CoChat Summary
TL;DR
The paper introduces VoxCeleb, a fully automated, large-scale speaker identification dataset built from YouTube videos “in the wild,” with 1,251 celebrities and over 100,000 utterances. It also benchmarks speaker recognition methods and reports that a CNN performs best on both identification and verification.
Key findings
- VoxCeleb contains over 100,000 utterances for 1,251 celebrities; the dataset is gender balanced, with 55% male speakers.
- The automated collection pipeline uses active speaker verification and face verification and was tuned to be highly precise: precision = 1.000 for both stages, with recall = 0.613 for active speaker verification and 0.726 for face verification.
- For speaker identification, the CNN achieves 80.5% top-1 and 92.1% top-5 accuracy, compared with 49.0% / 56.6% for I-vectors + SVM and 60.8% / 75.6% for I-vectors + PLDA + SVM.
- For speaker verification, the best reported result is CNN + Embedding with C min det = 0.71 and EER = 7.8%, better than I-vectors + PLDA (0.73, 8.8%) and GMM-UBM (0.80, 15.0%).
Methods
- Built VoxCeleb from YouTube videos using a fully automated multi-stage pipeline: candidate POI list, video download, face tracking, active speaker verification via SyncNet, and face verification via VGG Face CNN.
- Dataset scale/statistics reported: 1,251 POIs, 690 male POIs, 36 / 18 / 8 videos per POI (max/avg/min), 250 / 123 / 45 utterances per POI (max/avg/min), and 145.0 / 8.2 / 4.0 s utterance length (max/avg/min).
- Trained and evaluated a CNN on spectrograms with minimal preprocessing; identification used a 1,251-way classification setup, while verification compared I-vectors/PLDA, GMM-UBM, and CNN embedding approaches.
Limitations
- The paper states that the pipeline is designed to be conservative, and that this means “a lot of the downloaded videos” are discarded to reduce label errors.
- The paper notes that the collection starts from a list of 2,622 identities from VGG Face and uses YouTube search with “interview” appended, which is an evident source of selection bias toward publicly available, interview-like media.
- The dataset is described as coming from open-source media / YouTube, so it is limited to speakers and contexts available there; this is evident from the text, though not stated as a formal limitation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 12 Not checked · 37 Own results and interpretations
In the paper
“| Face verification | [30] | 1.000 | 0.726 |”
What the sources say
- Not checked yetcites [30] [30] O. M. Parkhi, A. Vedaldi, and A. Zisserman, 'Deep face recognition,' in Proceedings of the British Machine Vision Conference , 2015. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike the SITW dataset, both audio and video for each speaker is released.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“| Active speaker verification | [31] | 1.000 | 0.613 |”
What the sources say
- Not checked yetcites [31] [31] P. Chakravarty and T. Tuytelaars, 'Cross-modal supervision for learning active speaker detection in video,' arXiv preprint arXiv:1603.08907 , 2016. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reduces the number of parameters from 319M in VGG-M to 67M in our network”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| # of videos per POI | 36 / 18 / 8 |”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
15 more claims
Create a free account to see all 50 claims and to verify the 16 references still open.
- Probably not a substantial claim. 9
- It may state two separate claims as one. 23
- The quote only partly states this claim. 5
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Meanwhile, the emergence of large-scale VSA datasets [28, 29, 30, 31, 32] promoted the further development of deep learning based VSA research.”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [31]
In the citing paper
“Although it is mainly built for speaker identification, it also can be used for VSG.”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [31]
In the citing paper
“It contains over 100k utterances from 1,251 celebrities.”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [31]
In the citing paper
“VoxCeleb1 [31] is a large-scale text-independent audio-visual dataset collected from open-source YouTube media.”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [31]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“The network is trained using batch normalisation [40] and all hyper-parameters (e.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831 · cited 1×
“Facial landmark positions are detected for each face detection using the regression tree based method of [33].”
MethodCNN architectures for large-scale audio classification
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952132 · Cited by 2,023
MatConvNet - Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4564 · Cited by 578 · cited 1×
“Our implementation is based on the deep learning toolbox MatConvNet [39] and trained on a NVIDIA TITAN X GPU.”
MethodCNN Architectures for Large-Scale Audio ClassificationPreprint
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.09430 · Cited by 217 · cited 1×
“This has led to a very recent shift from hand-crafted features to the domain of deep CNNs which can be applied to higher dimensional inputs [13, 14] and for speaker iden-tification [15].”
BackgroundCross-Modal Supervision for Learning Active Speaker Detection in Video
Chakravarty, P., & Tuytelaars, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_18 · Cited by 70 · cited 2×
“Precision-recall curves for both tasks on their respective benchmark datasets [30, 31] are shown in Figure 1 right, and the values at the operating point are given in Table 3.”
BackgroundCross-modal Supervision for Learning Active Speaker Detection in VideoPreprint
Chakravarty, P., & Tuytelaars, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08907 · Cited by 1 · cited 2×
“Precision-recall curves for both tasks on their respective benchmark datasets [30, 31] are shown in Figure 1 right, and the values at the operating point are given in Table 3.”
Background
Research works printed but not linked: 28
- [2][2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'ImageNet classification with deep convolutional neural networks,' in Advances in Neural Information Processing Systems , pp. 1106-1114, 2012. -
- [5][5] M. McLaren, L. Ferrer, D. Castan, and A. Lawson, 'The speakers in the wild (SITW) speaker recognition database,' INTERSPEECH , vol. 2016, 2016. -
- [6][6] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, S. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, and F. Li, 'Imagenet large scale visual recognition challenge,' International Journal of Computer Vision , 2015. -
- [7][7] D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, 'Speaker verification using adapted gaussian mixture models,' Digital signal processing , vol. 10, no. 1-3, pp. 19-41, 2000. -
- [8][8] D. A. Reynolds and R. C. Rose, 'Robust text-independent speaker identification using gaussian mixture speaker models,' IEEE transactions on speech and audio processing , vol. 3, no. 1, pp. 7283, 1995. -
- [10][10] N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, 'Front-end factor analysis for speaker verification,' IEEE Transactions on Audio, Speech, and Language Processing , vol. 19, no. 4, pp. 788-798, 2011. -
- [11][11] U. H. Yapanel, X. Zhang, and J. H. Hansen, 'High performance digit recognition in real car environments.,' in INTERSPEECH , 2002. -
- [12][12] J. H. Hansen, R. Sarikaya, U. H. Yapanel, and B. L. Pellom, 'Robust speech recognition in noise: an evaluation using the spine corpus.,' in INTERSPEECH , pp. 905-908, 2001. -
- [13][13] T. N. Sainath, R. J. Weiss, A. W. Senior, K. W. Wilson, and O. Vinyals, 'Learning the speech front-end with raw waveform CLDNNs,' in INTERSPEECH , pp. 1-5, 2015. -
- [15][15] Y. Lukic, C. Vogt, O. D¨ urr, and T. Stadelmann, 'Speaker identification and clustering using convolutional neural networks,' in IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP) , pp. 1-6, IEEE, 2016. -
- [16][16] D. van der Vloed, J. Bouten, and D. A. van Leeuwen, 'NFIFRITS: a forensic speaker recognition database and some first experiments,' in The Speaker and Language Recognition Workshop , 2014. -
- [17][17] J. Hennebert, H. Melin, D. Petrovska, and D. Genoud, 'POLYCOST: a telephone-speech database for speaker recognition,' Speech communication , vol. 31, no. 2, pp. 265-270, 2000. -
- [18][18] J. B. Millar, J. P. Vonwiller, J. M. Harrington, and P. J. Dermody, 'The Australian national database of spoken language,' in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , vol. 1, pp. I-97, IEEE, 1994. -
- [19][19] J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett, 'DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM. NIST speech disc 1-1.1,' NASA STI/Recon technical report , vol. 93, 1993. -
- [21][21] R. Woo, A. Park, and T. J. Hazen, 'The MIT Mobile Device Speaker Verification Corpus: Data collection and preliminary experiments,' The Speaker and Language Recognition Workshop , 2006. -
- [23][23] A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, A. Stolcke, et al. , 'The ICSI meeting corpus,' in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , vol. 1, IEEE, 2003. -
- [24][24] I. McCowan, J. Carletta, W. Kraaij, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, et al. , 'The AMI meeting corpus,' in International Conference on Methods and Techniques in Behavioral Research , vol. 88, 2005. -
- [25][25] P. Bell, M. J. Gales, T. Hain, J. Kilgour, P. Lanchantin, X. Liu, A. McParland, S. Renals, O. Saz, M. Wester, et al. , 'The MGB challenge: Evaluating multi-genre broadcast media recognition,' in IEEE Workshop on Automatic Speech Recognition and Understanding , pp. 687-693, IEEE, 2015. -
- [27][27] J. J. Godfrey, E. C. Holliman, and J. McDaniel, 'Switchboard: Telephone speech corpus for research and development,' in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , vol. 1, pp. 517-520, IEEE, 1992. -
- [28][28] W. M. Fisher, G. R. Doddington, and K. M. Goudie-Marshall, 'The DARPA speech recognition research database: specifications and status,' in Proc. DARPA Workshop on speech recognition , pp. 93-99, 1986. -
- [30][30] O. M. Parkhi, A. Vedaldi, and A. Zisserman, 'Deep face recognition,' in Proceedings of the British Machine Vision Conference , 2015. -
- [32][32] D. E. King, 'Dlib-ml: A machine learning toolkit,' The Journal of Machine Learning Research , vol. 10, pp. 1755-1758, 2009. -
- [34][34] J. S. Chung and A. Zisserman, 'Lip reading in the wild,' in Proceedings of the Asian Conference on Computer Vision , 2016. -
- [35][35] M. Everingham, J. Sivic, and A. Zisserman, 'Taking the bite out of automatic naming of characters in TV video,' Image and Vision Computing , vol. 27, no. 5, 2009. -
- [36][36] J. S. Chung and A. Zisserman, 'Out of time: automated lip sync in the wild,' in Workshop on Multi-view Lip-reading, ACCV , 2016. -
- [37][37] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, 'Return of the devil in the details: Delving deep into convolutional nets,' in Proceedings of the British Machine Vision Conference , 2014. -
- [38][38] S. Chopra, R. Hadsell, and Y. LeCun, 'Learning a similarity metric discriminatively, with application to face verification,' in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , vol. 1, pp. 539-546, IEEE, 2005. -
- [41][41] S. Ioffe, 'Probabilistic linear discriminant analysis,' in Proceedings of the European Conference on Computer Vision , pp. 531542, Springer, 2006.
Cited by
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461375 · Cited by 2,620
“5 we incorporate audio from the new VoxCeleb dataset [19] into both extractor and PLDA training lists.”
BackgroundMethodVoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256
Self-citationWavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
“Compared with the Fbank feature, the representations from WavLM Large achieve over 35% relative EER improvement on all three trials for the VoxCeleb1 evaluation set.”
BackgroundInfluentialDesplanques, B., Thienpondt, J., & Demuynck, K. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2650 · Cited by 1,568
Speaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T., & Lee, K. A. (2019) · arXiv · doi:10.48550/arxiv.1904.05441 · Cited by 688
“The x-vector model used to compute ASV scores required to compute the t-DCF is pretrained(3) with the Kaldi [20] VoxCeleb [21] recipe.”
MethodASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
“For X2Face, we have used the model, as well as pretrained weights, provided by the authors (in the original paper it was also trained and evaluated on the VoxCeleb1 dataset).”
MethodInfluentialVoxceleb: Large-scale speaker verification in the wild
Nagrani, A., Chung, J. S., Xie, W., & Zisserman, A. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2019.101027 · Cited by 660
“Duplicates are identified and removed as follows: each speech segment is represented by a 1024D vector using the model in Nagrani et al. (2017) as a feature extractor.”
BackgroundMethodInfluentialSelf-citationEgo4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630
Generative artificial intelligence: a systematic review and applications
Sengar, S. S., Hasan, A. B., Kumar, S., & Carroll, F. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20016-1 · Cited by 467
“The evaluation of video generative models often involves a set of widely recognized datasets: Voxcleb [94], HDTF [154], ClebV [142], Kinetics [59], UCF101 [123], and for specifically audio performance testing VCTK Corpus[132] and LibriSpeech [100].”
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y., Pino, J., et al. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-143 · Cited by 586
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585
“Each video is split in short clips as described in pre-processing of VoxCeleb dataset.”
MethodInfluentialTransfer learning from speaker verification to multispeaker text-to-speech synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Moreno, I. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.04558 · Cited by 519
“We made use of three additional training sets: (1) LibriSpeech Other, which contains 461 hours of speech from a set of 1,166 speakers disjoint from those in the clean subsets, (2) VoxCeleb [11], and (3) VoxCeleb2 [6] which contain 139K utterances from 1,211 speakers, and 1.”
MethodAttentive Statistics Pooling for Deep Speaker Embedding
Okabe, K., Koshinaka, T., & Shinoda, K. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-993 · Cited by 514
“, a Time-Delay Neural Network (TDNN) [9], Convolutional Neural Network (CNN) [7, 8], LSTM [10, 11], or Gated Recurrent Unit (GRU) [8].”
BackgroundMethodResultInfluentialSpeechBrain: A General-Purpose Speech Toolkit
Ravanelli, M., Parcollet, T., Plantinga, P., Rouhe, A., Cornell, S., Lugosch, L., Subakan, C., Dawalatabad, N., Heba, A., Zhong, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.04624 · Cited by 512
“Table 4 reports the performance achieved on a speaker verification task with models trained on VoxCeleb2 [35] and tested on VoxCeleb1-clean [34].”
MethodASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505
Vggsound: A Large-Scale Audio-Visual Dataset
Chen, H., Xie, W., Vedaldi, A., & Zisserman, A. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053174 · Cited by 430
“In contrast to these manually curated datasets, recent papers have demonstrated the possibility of collecting high-quality human speech datasets in an automated and scalable manner by using computer vision algorithms [11, 12, 13].”
ResultSelf-citationHierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss
Chen, L., Maddox, R. K., Duan, Z., & Xu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00802 · Cited by 437
“For example, in web videos [5, 24] (e.g., LRW and VoxCeleb datasets), speakers move significantly when they are talking.”
BackgroundMethodX2Face: A Network for Controlling Face Generation Using Images, Audio, and Pose Codes
Wiles, O., Koepke, A. S., & Zisserman, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_41 · Cited by 435
Self-citationTalking Face Generation by Adversarially Disentangled Audio-Visual Representation
Zhou, H., Liu, Y., Liu, Z., Luo, P., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019299 · Cited by 416
In Defence of Metric Learning for Speaker Recognition
Chung, J. S., Huh, J., Mun, S., Lee, M., Heo, H.-S., Choe, S., Ham, C., Jung, S., Lee, B.-J., & Han, I. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1064 · Cited by 383
Self-citationSpeaker recognition based on deep learning: An overview
Bai, Z., & Zhang, X.-L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.004 · Cited by 386
“1Although some work used all speakers in a given database for both training and test which is essentially regarded as a close-set speaker classification problem [15], most real world speaker recognition systems must be able to “enroll” and “test” new speakers dynamically.”
BackgroundMethodInfluentialASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., et al. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-8 · Cited by 380
Self-supervised speech representation learning: A review
Mohamed, A., Lee, H.-Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoff, K., Li, S.-W., Livescu, K., Maaløe, L., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3207050 · Cited by 372
“VoxCeleb1 [236] for SID/ASV (automatic speaker verification) tasks, FSC (Fluent Speech Commands) [237] for IC (intent classification), QUESST (QUESST 2014) [238] for QbE (query by example), LS En-Fr [239] and CoVoST-2 [240] for ST (speech translation), and ALFFA and OpenSLR-multi for multilingual ASR.”
MethodM2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., & Li, S.-N. (2022) · Proceedings of the 2022 International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531415 · Cited by 368
“To process our dataset, we use the pretrained model on VoxCeleb dataset [40], which contains speech videos from speakers spanning a wide range of different ethnic groups, accents, professions and ages, and reenact the faces in our real videos.”
MethodFlow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset
Zhang, Z., Li, L., Ding, Y., & Fan, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00366 · Cited by 355
“, Voxceleb[26, 8] is built for speaker identification and LRW[9] is built for word recognition, so they do not pay attention to the video resolution.”
BackgroundVoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
Wang, Q., Muckenhirn, H., Wilson, K., Sridhar, P., Wu, Z., Hershey, J. R., Saurous, R. A., Weiss, R. J., Jia, Y., & Moreno, I. L. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1101 · Cited by 358
Biometrics recognition using deep learning: a survey
Minaee, S., Abdolrashidi, A., Su, H., Bennamoun, M., & Zhang, D. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10237-x · Cited by 245
“VoxCeleb: The VoxCeleb dataset Nagrani et al. (2017) and VoxCeleb2 dataset Chung et al. (2018) are public datasets compiled from interview videos uploaded to YouTube.”
MethodMEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation
Wang, K., Wu, Q., Song, L., Yang, Z., Wu, W., Qian, C., He, R., Qiao, Y., & Loy, C. C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58589-1_42 · Cited by 328
“…Resolution #Clips SAVESS [23] A fundamental design choice for emotional talking-face corpus is the choice between (i) the in-the-wild approach [7,39], i.e. annotating videos gathered from sources such as the Internet and (ii) the controlled approach [29,33], i.e. recording coordinated…”
BackgroundZhang, W., Cun, X., Wang, X., Zhang, Y., Shen, X., Guo, Y., Shan, Y., & Wang, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00836 · Cited by 336
AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
Qian, K., Zhang, Y., Chang, S., Yang, X., & Hasegawa-Johnson, M. (2019) · arXiv · doi:10.48550/arxiv.1905.05879 · Cited by 336
“In our implementation, the speaker encoder is pre-trained on the combination of VoxCeleb1 (Nagrani et al., 2017) and Librispeech (Panayotov et al., 2015) corpora, where there are a total of 3549 speakers.”
MethodUtterance-level Aggregation for Speaker Recognition in the Wild
Xie, W., Nagrani, A., Chung, J. S., & Zisserman, A. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683120 · Cited by 327
Self-citationAn Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Michelsanti, D., Tan, Z.-H., Zhang, S.-X., Xu, Y., Yu, M., Yu, D., & Jensen, J. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3066303 · Cited by 316
“VoxCeleb [185] 2017 1,251 Continuous sentences Not specifiedb Not specifiedb Videos in the wild from [195] (690 males) (153,516 utterances, Youtube 352 hours) Challenging multi-speaker acoustic environments”
BackgroundExploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System
Cai, W., Chen, J., & Li, M. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-11 · Cited by 315
Speaker Recognition for Multi-speaker Conversations Using X-vectors
Snyder, D., Garcia-Romero, D., Sell, G., McCree, A., Povey, D., & Khudanpur, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683760 · Cited by 294
“The system is trained on a large subset of the combined VoxCeleb 1 [28] and VoxCeleb 2 [29] corpora sampled at 16 kHz.”
MethodEmotion Recognition in Speech using Cross-Modal Transfer in the Wild
Albanie, S., Nagrani, A., Vedaldi, A., & Zisserman, A. (2018) · Proceedings of the 26th ACM international conference on Multimedia · doi:10.1145/3240508.3240578 · Cited by 290
“VoxCeleb [50] video dataset – this is a large-scale dataset of emotionunlabelled speaking face-tracks obtained in the wild (section 4); (iii) we transfer supervision across modalities from faces to a speech, and then train a speech emotion recognition model using speaking facetracks (section 5); and, (iv) we demonstra…”
BackgroundMethodInfluentialSelf-citationMotion Representations for Articulated Animation
Siarohin, A., Woodford, O. J., Ren, J., Chai, M., & Tulyakov, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01344 · Cited by 245
“• VoxCeleb [21] consists of interview videos of different celebrities.”
BackgroundWireless Semantic Communications for Video Conferencing
Jiang, P., Wen, C.-K., Jin, S., & Li, G. Y. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221968 · Cited by 255
“1) Configurations: The VoxCeleb dataset [39] has considerable face videos of speakers.”
BackgroundSurvey on Deep Neural Networks in Speech and Vision Systems
Alam, M., Samad, M. D., Vidyaratne, L., Glandon, A., & Iftekharuddin, K. M. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.053 · Cited by 252
Contrastive Learning of General-Purpose Audio Representations
Saeed, A., Grangier, D., & Zeghidour, N. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413528 · Cited by 219
Fully Supervised Speaker Diarization
Zhang, A., Wang, Q., Zhu, Z., Paisley, J., & Wang, C. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683892 · Cited by 234
Thin-Plate Spline Motion Model for Image Animation
Zhao, J., & Zhang, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00364 · Cited by 222
“• VoxCeleb [20]: Videos of different celebrities talking downloaded from youtube cropped to 256*256 resolution according to the bounding boxes of the faces.”
A Survey of Audio Classification Using Deep Learning
Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023) · IEEE Access · doi:10.1109/access.2023.3318015 · Cited by 207
“The VoxCeleb dataset contains an extensive collection of over 100,000 utterances from 1,251 different celebrities [167].”
BackgroundSeeing Voices and Hearing Faces: Cross-Modal Biometric Matching
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00879 · Cited by 220
Self-citationWho is Real Bob? Adversarial Attacks on Speaker Recognition Systems
Chen, G., Chenb, S., Fan, L., Du, X., Zhao, Z., Song, F., & Liu, Y. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00004 · Cited by 198
PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering
Ren, Y., Li, G., Chen, Y., Li, T. H., & Liu, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01350 · Cited by 201
“We train our model using the VoxCeleb dataset [25] which contains 22496 talking-head videos extracted from YouTube videos.”
MethodCN-Celeb: A Challenging Chinese Speaker Recognition Dataset
Fan, Y., Kang, J. W., Li, L. T., Li, K. C., Chen, H. L., Cheng, S. T., Zhang, P. Y., Zhou, Z. Y., Cai, Y. Q., & Wang, D. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054017 · Cited by 186
“As in [15], an active speaker verification system was employed to verify if the speech was really spoken by the target person.”
BackgroundMethodInfluentialOC-FakeDect: Classifying Deepfakes Using One-class Variational Autoencoder
Khalid, H., & Woo, S. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00336 · Cited by 197
“Also, we only use the real and fake images dataset from FaceForensics++: it would be interesting to extend our approach and leverage additional real images from CelebA [20], FFHQ [16], and VoxCeleb [21], and further experiment to detect GAN-generated images.”
MethodVisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
Gao, R., & Grauman, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01524 · Cited by 166
Talking Face Generation by Conditional Recurrent Adversarial Network
Song, Y., Zhu, J., Li, D., Wang, A., & Qi, H. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/129 · Cited by 182
Depth-Aware Generative Adversarial Network for Talking Head Video Generation
Hong, F.-T., Zhang, L., Shen, L., & Xu, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00339 · Cited by 181
Nautsch, A., Wang, X., Evans, N., Kinnunen, T. H., Vestman, V., Todisco, M., Delgado, H., Sahidullah, M., Yamagishi, J., & Lee, K. A. (2021) · IEEE Transactions on Biometrics, Behavior, and Identity Science · doi:10.1109/tbiom.2021.3059479 · Cited by 185
“7325 speakers contained within the entire VoxCeleb2 corpus [21] and the development portion of the VoxCeleb1 corpus [22].”
BackgroundResNeXt and Res2Net Structures for Speaker Verification
Zhou, T., Zhao, Y., & Wu, J. (2021) · 2021 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383531 · Cited by 181
Deep Audio-visual Learning: A Survey
Zhu, H., Luo, M.-D., Wang, R., Zheng, A.-H., & He, R. (2021) · International Journal of Automation and Computing · doi:10.1007/s11633-021-1293-0 · Cited by 174
“Classes Length* Year GRID [119] Lab 34 33,000 2006 Lombard Grid [120] Lab 54 54,000 2018 TCD TIMIT [121] Lab 62 - 2015 Vid TIMIT [122] Lab 43 - 2009 RAVDESS [123] Lab 24 - 2018 SEWA [124] Lab 180 - 2017 Speech OuluVS [125] Lab 20 1000 2009 OuluVS2 [126] Lab 52 3640 2016 Voxceleb [127] Wild 1,251 154,516 2017 Voxceleb2…”
BackgroundInfluentialThe Second DIHARD Diarization Challenge: Dataset, Task, and Baselines
Ryant, N., Church, K., Cieri, C., Cristia, A., Du, J., Ganapathy, S., & Liberman, M. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1268 · Cited by 179
Landini, F., Profant, J., Diez, M., & Burget, L. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2021.101254 · Cited by 175
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
Yin, F., Zhang, Y., Cun, X., Cao, M., Fan, Y., Wang, X., Bai, Q., Wu, B., Wang, J., & Yang, Y. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19790-1_6 · Cited by 167
Text-Independent Speaker Verification Based on Triplet Convolutional Neural Network Embeddings
Zhang, C., Koishida, K., & Hansen, J. H. L. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2831456 · Cited by 183
“for deep neural network based speaker embedding system [18], [19], [20], [24], [25].”
BackgroundAudio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion
Wang, S., Li, L., Ding, Y., Fan, C., & Yu, X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/152 · Cited by 259
Speech2Face: Learning the Face Behind a Voice
Oh, T.-H., Dekel, T., Kim, C., Mosseri, I., Freeman, W. T., Rubinstein, M., & Matusik, W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00772 · Cited by 168
“We test our model both qualitatively and quantitatively on the AVSpeech dataset [14] and the VoxCeleb dataset [35].”
MethodJung, J.-W., Heo, H.-S., Kim, J.-H., Shim, H.-J., & Yu, H.-J. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1982 · Cited by 166
Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings
Cooper, E., Lai, C.-I., Yasuda, Y., Fang, F., Wang, X., Chen, N., & Yamagishi, J. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054535 · Cited by 164
“Data : Following [13], we trained speaker verification systems on VoxCeleb1+2 [26, 27].”
BackgroundMethodMERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01589 · Cited by 160
“[86] Arsha Nagrani, Joon Son Chung, and Andrew Zisserman.”
BackgroundMethodInfluentialCelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Zhu, H., Wu, W., Zhu, W., Jiang, L., Tang, S., Zhang, L., Liu, Z., & Loy, C. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_38 · Cited by 386
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 141
“The quantitative results of talking face generation on LRW [282] and VoxCeleb2 [281] datasets are shown in Table V.”
Zhang, Y., Park, D. S., Han, W., Qin, J., Gulati, A., Shor, J., Jansen, A., Xu, Y., Huang, Y., Wang, S., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3182537 · Cited by 161
“The benchmark includes a variety of tasks such as speech emotion recognition [85], [86], speaker identification [87], and language identification [88], but specifically excludes tasks that are focused on the meaning of words.”
BackgroundExploring wav2vec 2.0 on Speaker Verification and Language Identification
Fan, Z., Li, M., Zhou, S., & Xu, B. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1280 · Cited by 128
MarioNETte: Few-Shot Face Reenactment Preserving Identity of Unseen Targets
Ha, S., Kersner, M., Kim, B., Seo, S., & Kim, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6721 · Cited by 157
Ava Active Speaker: An Audio-Visual Dataset for Active Speaker Detection
Roth, J., Chaudhuri, S., Klejch, O., Marvin, R., Gallagher, A., Kaver, L., Ramaswamy, S., Stopczynski, A., Schmid, C., Xi, Z., et al. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053900 · Cited by 230
“It also facilitates mining suitable training data for modeling these tasks [10, 14, 17].”
MethodSpot the Conversation: Speaker Diarisation in the Wild
Chung, J. S., Huh, J., Nagrani, A., Afouras, T., & Zisserman, A. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2337 · Cited by 186
“The test set contains approximately 53 hours and will be released after the VoxCeleb Speaker Recognition Challenge in October 2020.”
BackgroundMethodInfluentialSelf-citationLearnable PINs: cross-modal embeddings for person identity
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_5 · Cited by 151
Self-citationBYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534474 · Cited by 141
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Ao, J., Wang, R., Zhou, L., Wang, C., Ren, S., Wu, Y., Liu, S., Ko, T., Li, Q., Zhang, Y., et al. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.393 · Cited by 144
Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition
Xiang, X., Wang, S., Huang, H., Qian, Y., & Yu, K. (2019) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc47483.2019.9023039 · Cited by 153
“The experiments are carried out on the VoxCeleb1 [19] and SITW [20] test set with most settings are the same, while detailed training data differ, which will be introduced in the corresponding sections.”
MethodJahangir, R., Teh, Y. W., Nweke, H. F., Mujtaba, G., Al-Garadi, M. A., & Ali, I. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114591 · Cited by 168
KoDF: A Large-scale Korean DeepFake Detection Dataset
Kwon, P., You, J., Nam, G., Park, S., & Chae, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01057 · Cited by 145
“We use the official code and the pretrained model [10] on VoxCelab dataset [41].”
MethodMFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification
Zhang, Y., Lv, Z., Wu, H., Zhang, S., Hu, P., Wu, Z., Lee, H.-Y., & Meng, H. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-563 · Cited by 132
Everybody’s Talkin’: Let Me Talk as You Want
Song, L., Wu, W., Qian, C., He, R., & Loy, C. C. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3146783 · Cited by 126
The Third DIHARD Diarization Challenge
Ryant, N., Singh, P., Krishnamohan, V., Varma, R., Church, K., Cieri, C., Du, J., Ganapathy, S., & Liberman, M. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1208 · Cited by 173
Towards Directly Modeling Raw Speech Signal for Speaker Verification Using CNNS
Muckenhirn, H., Magimai.-Doss, M., & Marcel, S. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462165 · Cited by 135
“These systems take as input either output of filterbanks [5, 6] or spectrograms [7, 8].”
BackgroundLarge Margin Softmax Loss for Speaker Verification
Liu, Y., He, L. J., & Liu, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2357 · Cited by 131
Survey of Deep Learning Paradigms for Speech Processing
Bhangale, K. B., & Kothandaraman, M. (2022) · Wireless Personal Communications · doi:10.1007/s11277-022-09640-y · Cited by 145
“VoxCeleb database is large-scale text-independent speaker recognition corpus consisting of 153,486 utterances for 1251 celebrities, extracted from YouTube videos [139].”
BackgroundWang, Y., Boumadane, A., & Heba, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02735 · Cited by 226
A Survey of Speaker Recognition: Fundamental Theories, Recognition Methods and Opportunities
Kabir, M. M., Mridha, M. F., Shin, J., Jahan, I., & Ohi, A. Q. (2021) · IEEE Access · doi:10.1109/access.2021.3084299 · Cited by 159
“Popular in the wild datasets are SITW [92] and VoxCeleb [127].”
BackgroundVillalba, J., Chen, N., Snyder, D., Garcia-Romero, D., McCree, A. V., Sell, G., Borgstrom, J., Garcia, L. P., Richardson, F. S., Dehak, R., et al. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.101026 · Cited by 136
Towards Learning a Universal Non-Semantic Representation of Speech
Shor, J., Jansen, A., Maor, R., Lang, O., Tuval, O., Quitry, F. D. C., Tagliasacchi, M., Shavitt, I., Emanuel, D., & Haviv, Y. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1242 · Cited by 123
“1 (Nagrani et al., 2017) 2 (Revay & Teschke, 2019) 3 (Kaggle) 4 (Ghaleb et al.”
BackgroundSelf Multi-Head Attention for Speaker Recognition
India, M., Safari, P., & Hernando, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2616 · Cited by 140
A Deep Neural Network Model for Speaker Identification
Ye, F., & Yang, J. J. (2021) · Applied Sciences · doi:10.3390/app11083603 · Cited by 129
Speaker Anonymization Using X-vector and Neural Waveform Models
Fang, F., Wang, X., Yamagishi, J., Echizen, I., Todisco, M., Evans, N., & Bonastre, J.-F. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-28 · Cited by 126
Neural Head Reenactment with Latent Pose Descriptors
Burkov, E., Pasechnik, I., Grigorev, A., & Lempitsky, V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01380 · Cited by 173
“The emergence of large unlabeled datasets of human videos such as [29, 4, 5] allows to learn latent pose-expression descriptors in an unsupervised way.”
BackgroundMethodCN-Celeb: Multi-genre speaker recognition
Li, L., Liu, R., Kang, J., Fan, Y. B., Cui, H., Cai, Y., Vipperla, R., Zheng, T. F., & Wang, D. (2022) · Speech Communication · doi:10.1016/j.specom.2022.01.002 · Cited by 162
HeadGAN: One-shot Neural Head Synthesis and Editing
Doukas, M. C., Zafeiriou, S., & Sharmanska, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01413 · Cited by 117
“For [45] and [34], we use the pretrained models provided by the authors, trained on VoxCeleb.”
MethodInfluentialVOXLINGUA107: A Dataset for Spoken Language Recognition
Valk, J., & Alumäe, T. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383459 · Cited by 268
“The VoxCeleb 1&2 [8, 9] corpora are popular free datasets for training speaker recognition models, containing automatically retrieved data from YouTube.”
BackgroundSelf-Supervised Representation Learning From Videos for Facial Action Unit Detection
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01118 · Cited by 137
“TCAE was trained on the union of VoxCeleb1 [28] and VoxCeleb2 [6] datasets.”
MethodAudio self-supervised learning: A survey
Liu, S., Mallol-Ragolta, A., Parada‐Cabaleiro, E., Qian, K., Jing, X., Kathan, A., Hu, B., & Schuller, B. W. (2022) · Patterns · doi:10.1016/j.patter.2022.100616 · Cited by 188
One-Shot Talking Face Generation from Single-Speaker Audio-Visual Correlation Learning
Wang, S., Li, L., Ding, Y., & Yu, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20154 · Cited by 168
Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification
Chen, Z., Chen, S., Wu, Y., Qian, Y., Wang, C., Liu, S., Qian, Y., & Zeng, M. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747814 · Cited by 112
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild
Cheng, K., Cun, X., Zhang, Y., Xia, M., Yin, F., Zhu, M., Wang, X., Wang, J., & Wang, N. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555399 · Cited by 217
“To perform semantic-guided expression reenactment, we train our network on the VoxCeleb [Nagrani et al. 2017] dataset with the pose and expression from [Deng et al.”
BackgroundMethodInfluential
Metadata
- Authors
- Nagrani, A., Chung, J. S., & Zisserman, A.
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 2616-2620
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2726515241
- Cited by
- 2,210
- References
- 4
- Citations by year
- 2026: 144 · 2025: 280 · 2024: 313 · 2023: 361 · 2022: 330 · 2021: 319 · 2020: 265 · 2019: 151 · 2018: 47
- FWCI
- 56.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M013774/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- speaker identification, speaker verification, VoxCeleb, text-independent speaker recognition, automated dataset generation, two-stream CNN, face recognition, YouTube videos
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.