Bias in Automated Speaker Recognition
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Automated speaker recognition uses data processing to identify speakers by their voice. Today, automated speaker recognition is deployed on billions of smart devices and in services such as call centres. Despite their wide-scale deployment and known sources of bias in related domains like face recognition and natural language processing, bias in automated speaker recognition has not been studied systematically. We present an in-depth empirical and analytical study of bias in the machine learning development workflow of speaker verification, a voice biometric and core task in automated speaker recognition. Drawing on an established framework for understanding sources of harm in machine learning, we show that bias exists at every development stage in the well-known VoxCeleb Speaker Recognition Challenge, including data generation, model building, and implementation. Most affected are female speakers and non-US nationalities, who experience significant performance degradation. Leveraging the insights from our findings, we make practical recommendations for mitigating bias in automated speaker recognition, and outline future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Racial disparities in automated speech recognition
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915768117 · Cited by 765
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689 · cited 1×
“After searching and downloading video clips for identified celebrities, further processing was done to track faces, identify active speakers and verify the speaker’s identity using the HOG-based face detector [20], Sync-Net [6] and VGG Face CNN [47] respectively.”
MethodSUPERB: Speech Processing Universal PERformance Benchmark
Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1775 · Cited by 681
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651
Speaker Recognition by Machines and Humans: A tutorial review
Hansen, J. H. L., & Hasan, T. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2015.2462851 · Cited by 621
Speaker recognition based on deep learning: An overview
Bai, Z., & Zhang, X.-L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.004 · Cited by 386 · cited 4×
“Several system properties beyond performance have been considered in recent years, such as robustness [2] and privacy [35].”
BackgroundInfluentialScheuerman, M. K., Paul, J. M., & Brubaker, J. R. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359246 · Cited by 252
Wachter, S., Mittelstadt, B., & Russell, C. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3792772 · Cited by 167 · cited 1×
“Bias in machine learning (ML) is a source of unfairness [29] that can have harmful consequences, such as discrimination [55].”
BackgroundClova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020
Heo, H. S., Lee, B.-J., Huh, J., & Chung, J. S. (2020) · arXiv · doi:10.48550/arxiv.2009.14153 · Cited by 90 · cited 2×
“The challenge has released two pre-trained baseline models [15] trained on the VoxCeleb 2 training set [32] with close to 1 million speech utterances of 5994 speakers.”
MethodDiversity and Inclusion Metrics in Subset Selection
Mitchell, M., Baker, D., Moorosi, N., Denton, R., Hutchinson, B., Hanna, A., Gebru, T., & Morgenstern, J. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375832 · Cited by 87
Research works printed but not linked: 41
- Martine Adda-Decker and Lori Lamel. 2005. Do speech recognizers prefer female speakers?INTERSPEECH (2005), 2205–2208. https://www.isca-speech.org/archive/interspeech_2005/addadecker05_interspeech.html
- Tolga Bolukbasi, Kai-wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker ? Debiasing Word Embeddings. In NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems. 4356 – 4364.
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of Machine Learning Research: Conference on Fairness, Accountability, and Transparency, Vol. 81. 1889–1896.
- 10.21437/Interspeech.2020-1064 · DOI · Paper page
- Gianni Fenu, Mirko Marras, Giacomo Medda, and Giacomo Meloni. 2021. Fair Voice Biometrics : Impact of Demographic Imbalance on Group Fairness in Speaker Recognition. (2021), 1892–1896.
- Sadaoki Furui. 1994. An Overview of Speaker Recognition Technology. In ESCA Workshop on Automatic Speaker Recognition, Identification and Verification. 1 – 9.
- 10.1016/j.csl.2019.101032 · DOI · Paper page
- Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems (2016), 3323–3331.
- Khaled Hechmi Trung Ngo Trong Ville Hautamaki and Tomi Kinnunen. 2021. VoxCeleb Enrichment for Age and Gender Recognition. (2021). http://arxiv.org/abs/2109.13510 · DOI
- Georg Heigold, Ignacio Moreno, Samy Bengio, and Noam Shazeer. 2016. End-to-End Text-Dependent Speaker Verification. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5115–5119.
- Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio and Emily Denton. 2020. Characterising Bias in Compressed Models. https://arxiv.org/abs/2010.03058 · arXiv
- Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner. 2016. Machine Bias. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- 10.1016/j.imavis.2013.10.001 · DOI
- 10.1109/ICB.2013.6613025 · DOI
- 10.5555/1577069.1755843 · doi:10.5555/1577069.1755843
- 10.1016/j.specom.2009.08.009 · DOI · Paper page
- Chao Li, Xiaokong Ma, Bing Jiang, Xiangang Li, Xuewei Zhang, Xiao Liu, Ying Cao, Ajay Kannan, and Zhenyao Zhu. 2017. Deep speaker: An end-to-end neural speaker embedding system. arXiv (2017).
- 10.21437/Interspeech.2020-2248 · DOI
- 10.3389/fbioe.2015.00126 · doi:10.3389/fbioe.2015.00126
- M McLaren L Ferrer D Castan and A Lawson. 2016. The Speakers in the Wild (SITW) speaker recognition database.. In Interspeech. pdfs.semanticscholar.org. https://pdfs.semanticscholar.org/3fe3/58a66359ee2660ec0d13e727eb8f3f0007c2.pdf
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv (2019).
- Arsha Nagrani Joon Son Chung Jaesung Huh Andrew Brown Ernesto Coto Weidi Xie Mitchell McLaren Douglas A Reynolds and Andrew Zisserman. 2020. VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge. (2020). http://arxiv.org/abs/2012.06867 · arXiv
- 10.1016/j.csl.2019.101027 · DOI · Paper page
- Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. Voxceleb: A large-scale speaker identification dataset. arXiv (2017), 2616–2620.
- 10.1016/j.csl.2019.06.001 · DOI
- 10.21437/Interspeech.2020-1815 · DOI
- NIST. 2019. NIST 2019 Speaker Recognition Evaluation Plan. 1 (2019), 1–7.
- NIST. 2020. NIST 2020 CTS Speaker Recognition Challenge Evaluation Plan. Technical Report. 1–8 pages.
- 10.21437/Interspeech.2016-523 · DOI
- Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep Face Recognition. In British Machine Vision Conference.
- 10.21437/Interspeech.2020-1249 · DOI
- Inioluwa Deborah Raji and Genevieve Fried. 2021. About Face: A Survey of Facial Recognition Evaluation. (2021). http://arxiv.org/abs/2102.00813 · arXiv
- Douglas A. Reynolds. 2002. An Overview of Automatic Speaker Recognition Technology. IEEE (2002).
- Lea Schönherr, Maximilian Golla, Thorsten Eisenhofer, Jan Wiele, Dorothea Kolossa, and Thorsten Holz. 2020. Unacceptable, where is my privacy? Exploring Accidental Triggers of Smart Speakers. (8 2020). http://arxiv.org/abs/2008.00508 · arXiv
- Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings(2015), 1–14.
- Rita Singh. 2019. Profiling Humans from their Voice. https://doi.org/10.1007/978-981-13-8403-5 · doi:10.1007/978-981-13-8403-5
- D Snyder, D Garcia-Romero, D Povey, and S Khudanpur. 2017. Deep Neural Network Embeddings for Text-Independent Speaker Verification.Interspeech (2017). https://www.isca-speech.org/archive/Interspeech_2017/pdfs/0620.PDF
- 10.1109/ICASSP.2018.8461375 · DOI · Paper page
- 10.21437/Interspeech.2017-1746 · DOI
- 10.1145/3485730.3493448 · DOI
- Wiebke Toussaint Akhil Mathur Fahim Kawsar and Aaron Yi Ding. 2022. Tiny always-on and fragile: Bias propagation through design choices in on-device machine learning workflows. (2022) 19 pages. http://arxiv.org/abs/2201.07677 · arXiv
Cited by
"I wouldn't say offensive but...": Disability-Centered Perspectives on Large Language Models
Gadiraju, V., Kane, S., Dev, S., Taylor, A., Wang, D., Denton, R., & Brewer, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3593989 · Cited by 155
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155
The life cycle of large language models in education: A framework for understanding sources of bias
Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13505 · Cited by 119
“Finally, aggregation bias can arise when a chosen model does not perform equally well across all subgroups, often because the data includes distinct subgroups that are treated uniformly instead of individually (Hutiri & Ding, 2022).”
BackgroundMancone, S., Diotaiuti, P., Valente, G., Corrado, S., Bellizzi, F., Vilarino, G. T., & Andrade, A. (2023) · Behavioral Sciences · doi:10.3390/bs13070550 · Cited by 14
FairSSD: Understanding Bias in Synthetic Speech Detectors
Singh Yadav, A. K., Bhagtani, K., Salvi, D., Bestagini, P., & Delp, E. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00445 · Cited by 12
“EER is also used in existing speech recognition work for bias study [30, 31].”
Method
Metadata
- Authors
- Hutiri, W. T., & Ding, A. Y.
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 230-247
- OpenAlex
- W4226371168
- Cited by
- 47
- References
- 12
- FWCI
- 10.79
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bias in Automated Speaker Recognition. https://app.cochat.ai/papers/10.1145/3531146.3533089
- The paper
- Hutiri, W. T., & Ding, A. Y. (2022). Bias in Automated Speaker Recognition. 2022 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3531146.3533089
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.