Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
Open accessAbstract
The study of label noise in sound event recognition has recently gained attention with the advent of larger and noisier datasets. This work addresses the problem of missing labels, one of the big weaknesses of large audio datasets, and one of the most conspicuous issues for AudioSet. We propose a simple and model-agnostic method based on a teacher-student framework with loss masking to first identify the most critical missing label candidates, and then ignore their contribution during the learning process. We find that a simple optimisation of the training label set improves recognition performance without additional computation. We discover that most of the improvement comes from ignoring a critical tiny portion of the missing labels. We also show that the damage done by missing labels is larger as the training set gets smaller, yet it can still be observed even when training with massive amounts of audio. We believe these insights can generalize to other large-scale datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing datasets for human vocal sounds have a relatively small number of vocal sound samples or noisy labels.
“However, existing datasets have a relatively small number of vocal sound samples or noisy labels.”
From Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition (2022) · cites this paper as [9]
The Introduction (page not provided) contrasts small sound-event datasets with larger datasets that have label-noise problems, supporting the general trade-off. It does not establish that datasets specifically for human vocal sounds have few vocal samples or noisy labels. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe FSD50K evaluation set has more accurate labels while AudioSet labels are relatively noisy.
“The reason why we use FSD50K rather than AudioSet as the base dataset is because FSD50K, especially its evaluation set, has more accurate labels [8] while labels of AudioSet are relatively noisy [6, 9, 10].”
From Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
(1966) · Medical Entomology and Zoology · Cited by 10,293 · see citing works
Signal detection theory and psychophysics
Ingleby, D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,616 · see citing works
Audio Set: An ontology and human-labeled dataset for audio events
Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952261 · Cited by 3,134 · see citing works
Self-citationCNN architectures for large-scale audio classification
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952132 · Cited by 2,605 · see citing works
Self-citationRevisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · see citing works
ESC: Dataset for Environmental Sound Classification
Piczak, K. J. (2015) · ACM Multimedia · doi:10.1145/2733373.2806390 · Cited by 2,195 · see citing works
Classification in the Presence of Label Noise: A Survey
Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777 · see citing works
Do Deep Nets Really Need to be Deep?Preprint
Ba, J. L., & Caruana, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6184 · Cited by 1,592 · see citing works
A Dataset and Taxonomy for Urban Sound Research
Salamon, J., Jacoby, C., & Bello, J. P. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2655045 · Cited by 1,355 · see citing works
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D. (2020) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2020.3030497 · Cited by 1,235 · see citing works
Confident Learning: Estimating Uncertainty in Dataset Labels
Northcutt, C., Jiang, L., & Chuang, I. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12125 · Cited by 795 · see citing works
Revisiting Unreasonable Effectiveness of Data in Deep Learning EraPreprint
Sun, C., Shrivastava, A., Singh, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02968 · Cited by 300 · see citing works
Computational Analysis of Sound Scenes and Events
(2017) · Springer eBooks · doi:10.1007/978-3-319-63450-0 · Cited by 272 · see citing works
Learning small-size DNN with output-distribution-based criteria
Li, J., Zhao, R., Huang, J.-T., & Gong, Y. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-432 · Cited by 252 · see citing works
On Issues of Instance Selection
Liu, H., & Motoda, H. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014056429969 · Cited by 191 · see citing works
Freesound Datasets: A Platform For The Creation Of Open Audio Datasets.
Fonseca, E., Pons, J., Favory, X., Font, F., Bogdanov, D., Ferraro, A., Oramas, S., Porter, A., & Serra, X. (2017) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · doi:10.5281/zenodo.1417158 · Cited by 171 · see citing works
Self-citationESC: Dataset for Environmental Sound Classification
Piczak, K. J. (2015) · Harvard Dataverse · doi:10.7910/dvn/ydeput · Cited by 158 · see citing works
Fonseca, E., Plakal, M., Font, F., Ellis, D. P. W., Favory, X., Pons, J., & Serra, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09902 · Cited by 131 · see citing works
Self-citationChime-home: A dataset for sound source recognition in a domestic environment
Foster, P., Sigtia, S., Krstulović, S., Barker, J., & Plumbley, M. D. (2015) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2015.7336899 · Cited by 89 · see citing works
Audio Tagging with Noisy Labels and Minimal Supervision
Fonseca, E., Plakal, M., Font, F., Ellis, D. P. W., & Serra, X. (2019) · Detection and Classification of Acoustic Scenes and Events Workshop (DCASE) · doi:10.33682/w13e-5v06 · Cited by 87 · see citing works
Self-citationPANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern RecognitionPreprint
Kong, Q., Cao, Y. P., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10211 · Cited by 69 · see citing works
A Deep Residual Network for Large-Scale Acoustic Scene Analysis
Ford, L., Tang, H., Grondin, F., & Glass, J. R. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2731 · Cited by 51 · see citing works
A Closer Look at Weak Label Learning for Audio Events
Shah, A., Kumar, A., Hauptmann, A. G., & Raj, B. (2018) · arXiv · doi:10.48550/arxiv.1804.09288 · Cited by 49 · see citing works
A Closer Look at Weak Label Learning for Audio EventsPreprint
Shah, A., Kumar, A., Hauptmann, A. G., & Raj, B. (2018) · arXiv (Cornell University) · doi:10.13140/rg.2.2.29936.76806 · Cited by 49 · see citing works
Sound Event Detection Using Point-Labeled Data
Kim, B., & Pardo, B. (2019) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2019.8937213 · Cited by 16 · see citing works
Learning Sound Event Classifiers from Web Audio with Noisy Labels
Fonseca, E., Plakal, M., Ellis, D. P. W., Font, F., Favory, X., & Serra, X. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683158 · Cited by 13 · see citing works
Self-citationLearning Sound Events from Webly Labeled Data
Kumar, A., Shah, A., Hauptmann, A., & Raj, B. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/384 · Cited by 11 · see citing works
Model-Agnostic Approaches To Handling Noisy Labels When Training Sound Event Classifiers
Fonseca, E., Font, F., & Serra, X. (2019) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2019.8937249 · Cited by 8 · see citing works
Self-citationSeCoST: Sequential Co-Supervision for Weakly Labeled Audio Event DetectionPreprint
Kumar, A., & Ithapu, V. K. (2019) · arXiv (Cornell University) · Cited by 7 · see citing works
Meire, M., Karsmakers, P., & Vuegen, L. (2019) · doi:10.33682/y8xs-0463 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
FSD50K: An Open Dataset of Human-Labeled Sound Events
Fonseca, E., Favory, X., Pons, J., Font, F., & Serra, X. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3133208 · Cited by 462 · see citing works
Self-citationSound Event Detection: A tutorial
Mesaros, A., Heittola, T., Virtanen, T. I., & Plumbley, M. D. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3090678 · Cited by 274 · see citing works
PSLA: Improving Audio Tagging With Pretraining, Sampling, Labeling, and Aggregation
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3120633 · Cited by 164 · see citing works
The Benefit of Temporally-Strong Labels in Audio Event Classification
Hershey, S., Ellis, D. P. W., Fonseca, E., Jansen, A., Liu, C., Moore, R. C., & Plakal, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414579 · Cited by 97 · see citing works
Self-citationVocalsound: A Dataset for Improving Human Vocal Sounds Recognition
Gong, Y., Yu, J., & Glass, J. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746828 · Cited by 43 · see citing works
“There are a few existing datasets for generic audio event classification that also contain human vocal sound samples such as AudioSet [4], FSD50K [8], ESC-50 [7], and DCASE [11].”
InfluentialWhat is the ground truth? Reliability of multi-annotator data for audio tagging
Martín-Morató, I., & Mesaros, A. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616087 · Cited by 27 · see citing works
Symptom Identification for Interpretable Detection of Multiple Mental Disorders on Social Media
Zhang, Z., Chen, S., Wu, M., & Zhu, K. Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.677 · Cited by 21 · see citing works
Jarray, N., Abbes, A. B., & Farah, I. R. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3168982 · Cited by 20 · see citing works
Solfeggio Teaching Method Based on MIDI Technology in the Background of Digital Music Teaching
Shen, S., & Wu, K. (2023) · International Journal of Web-Based Learning and Teaching Technologies · doi:10.4018/ijwltt.331085 · Cited by 12 · see citing works
Unsupervised contrastive learning of sound event representations
Fonseca, E., Ortego, D., McGuinness, K., O'Connor, N. E., & Serra, X. (2021) · Repositori digital de la UPF (Universitat Pompeu Fabra) · doi:10.1109/icassp39728.2021.9415009 · Cited by 46 · see citing works
Self-citationComputer Audition: From Task-Specific Machine Learning to Foundation Models
Triantafyllopoulos, A., Tsangko, I., Gebhard, A., Mesaros, A., Virtanen, T. I., & Schuller, B. W. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3593952 · Cited by 12 · see citing works
Lightweight Convolutional-iConformer for Sound Event Detection
Chan, T. K., & Chin, C. S. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3173582 · Cited by 10 · see citing works
Semi-Supervised Audio Classification with Partially Labeled Data
Gururani, S., & Lerch, A. (2021) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism52913.2021.00027 · Cited by 9 · see citing works
Detection of Multiple Mental Disorders from Social Media with Two-Stream Psychiatric Experts
Chen, S., Zhang, Z., Wu, M., & Zhu, K. Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.562 · Cited by 8 · see citing works
Self-Supervised Learning from Automatically Separated Sound Scenes
Fonseca, E., Jansen, A., Ellis, D. P. W., Wisdom, S., Tagliasacchi, M., Hershey, J. R., Plakal, M., Hershey, S., Moore, R. C., & Serra, X. (2021) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa52581.2021.9632739 · Cited by 6 · see citing works
Self-citationPhase-Aware Transformations in Variational Autoencoders for Audio Effects
Cámara, M., & Blanco, J. L. (2022) · Journal of the Audio Engineering Society · doi:10.17743/jaes.2022.0042 · Cited by 4 · see citing works
Ontology-aware Learning and Evaluation for Audio Tagging
Liu, H., Kong, Q., Liu, X., Mei, X., Wang, W., & Plumbley, M. D. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-979 · Cited by 4 · see citing works
Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging
Zhang, Z., Zhou, Z., Tang, H., Li, G., Wu, M., & Zhu, K. Q. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482097 · Cited by 3 · see citing works
Automatic re-labeling of Google AudioSet for improved quality of learned features and pre-training
Grzywalski, T., & Botteldooren, D. B. (2024) · Signal Processing: Algorithms, Architectures, Arrangements, and Applications · doi:10.23919/spa61993.2024.10715611 · Cited by 2 · see citing works
Multi-Label Audio Classification with a Noisy Zero-Shot Teacher
Braun, S., & Gamper, H. (2024) · International Workshop on Acoustic Signal Enhancement · doi:10.1109/iwaenc61483.2024.10693989 · Cited by 1 · see citing works
Progressive Teacher-Student Training Framework for Music Tagging
Lü, R., Zheng, B., Hai, J., Tao, F., Duan, Z., & Liu, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747342
Revamping Blood Vessel Edge-Buffer Labels: A Self-Correcting Region Supervision
Zuo, W., Wang, H., Pan, X., Lan, R., & Luo, X. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3358078
A Self-Improving Ensemble Learning Framework for Human Non-Speech Vocalization Classification
Wang, Y., Wu, G., & Shan, N. (2025) · IEEE Access · doi:10.1109/access.2025.3639444
Self-Labeling Sounds Using Optimal Transport
Harju, M., Font, F., & Mesaros, A. (2026) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2026.3659053
Bahadori, S., Hassanzadeh, A., & Vanani, I. R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6839366
- Authors
- Fonseca, E., Hershey, S., Plakal, M., Ellis, D. P. W., Jansen, A., & Moore, R. C.
- Affiliations
- Google (United States) (US)
- Venue
- IEEE Signal Processing Letters
- Published
- 2020
- Volume
- 27
- Pages
- 1235-1239
- ISSN
- 1070-9908 (ISSN-L)
- 1558-2361
- OpenAlex
- W3021910945
- Cited by
- 26
- References
- 33
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 3 · 2023: 3 · 2022: 6 · 2021: 10
- FWCI
- 2.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Signal Processing Letters (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Google Research
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- missing labels, sound event recognition, label noise, teacher-student models, AudioSet, training data size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
