The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Metrics for Polyphonic Sound Event Detection
Open accessAbstract
This paper presents and discusses various metrics proposed for evaluation of polyphonic sound event detection systems used in realistic situations where there are typically multiple sound sources active simultaneously. The system output in this case contains overlapping events, marked as multiple sounds detected as being active at the same time. The polyphonic system output requires a suitable procedure for evaluation against a reference. Metrics from neighboring fields such as speech recognition and speaker diarization can be used, but they need to be partially redefined to deal with the overlapping events. We present a review of the most common metrics in the field and the way they are adapted and interpreted in the polyphonic case. We discuss segment-based and event-based definitions of each metric and explain the consequences of instance-based and class-based averaging using a case study. In parallel, we provide a toolbox containing implementations of presented metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
Evaluating Learning Algorithms: A Classification Perspective
Japkowicz, N., & Shah, M. (2011) · doi:10.1017/cbo9780511921803 · Cited by 767
Environmental Sound Recognition With Time–Frequency Audio Features
Chu, S., Narayanan, S., & Kuo, C.-C. J. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2009.2017438 · Cited by 639
Detection and Classification of Acoustic Scenes and Events
Stowell, D., Giannoulis, D., Benetos, E., Lagrange, M., & Plumbley, M. D. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2428998 · Cited by 600
On the Stratification of Multi-label Data
Sechidis, K., Tsoumakas, G., & Vlahavas, I. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23808-6_10 · Cited by 432
Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement
Forman, G., & Scholz, M. (2010) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1882471.1882479 · Cited by 368
Events Detection for an Audio-Based Surveillance System
Clavel, C., Ehrette, T., & Richard, G. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521669 · Cited by 314
Robust sound event recognition using convolutional neural networks
Zhang, H. S., McLoughlin, I. V., & Song, Y. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178031 · Cited by 271
A Discriminative Model for Polyphonic Piano Transcription
Poliner, G. E., & Ellis, D. P. W. (2006) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2007/48317 · Cited by 266
Polyphonic sound event detection using multi label deep neural networks
Cakir, E., Heittola, T., Huttunen, H., & Virtanen, T. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280624 · Cited by 256
Self-citationAcoustic event detection in real life recordings
Mesaros, A., Heittola, T., Eronen, A., & Virtanen, T. I. (2010) · Cited by 241
Self-citationContext-dependent sound event detection
Heittola, T., Mesaros, A., Eronen, A., & Virtanen, T. (2013) · EURASIP Journal on Audio, Speech, and Music Processing · doi:10.1186/1687-4722-2013-1 · Cited by 217
Self-citationAutomatic Surveillance of the Acoustic Activity in Our Living Environment
Härmä, A., McKinney, M. F., & Skowronek, J. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521503 · Cited by 188
Reliable detection of audio events in highly noisy environments
Foggia, P., Petkov, N., Saggese, A., Strisciuglio, N., & Vento, M. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.06.026 · Cited by 172
A flexible framework for key audio effects detection and auditory context inference
Cai, R., Lu, L., Hanjalić, A., Zhang, H.-J., & Cai, L. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tsa.2005.857575 · Cited by 149
Polyphonic music transcription using note event modeling
Ryynänen, M. P., & Klapuri, A. P. (2006) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/aspaa.2005.1540233 · Cited by 140
Simultaneous Estimation of Chords and Musical Context From Audio
Mauch, M., & Dixon, S. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2032947 · Cited by 130
Peng, Y.-T., Lin, C., Sun, M., & Tsai, K. (2009) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2009.5202720 · Cited by 104
Exploiting spectro-temporal locality in deep learning based acoustic event detection
Espi, M., Fujimoto, M., Kinoshita, K., & Nakatani, T. (2015) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-015-0069-2 · Cited by 101
Chime-home: A dataset for sound source recognition in a domestic environment
Foster, P., Sigtia, S., Krstulović, S., Barker, J., & Plumbley, M. D. (2015) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2015.7336899 · Cited by 89
Acoustic Monitoring and Localization for Social Care
Goetze, S., Schröder, J., Gerlach, S., Hollosi, D., Appell, J.-E., & Wallhoff, F. (2012) · Journal of Computing Science and Engineering · doi:10.5626/jcse.2012.6.1.40 · Cited by 52
Acoustic event detection for multiple overlapping similar sources
Stowell, D., & Clayton, D. F. (2015) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2015.7336885 · Cited by 40
Cited by
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T., Schluter, J., Chang, S.-Y., & Sainath, T. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 855
Self-citationConvolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Cakir, E., Parascandolo, G., Heittola, T., Huttunen, H., & Virtanen, T. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2690575 · Cited by 604
Self-citationAdavanne, S., Politis, A., Nikunen, J., & Virtanen, T. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2018.2885636 · Cited by 534
Self-citationTUT database for acoustic scene classification and sound event detection
Mesaros, A., Heittola, T., & Virtanen, T. (2016) · 2016 24th European Signal Processing Conference (EUSIPCO) · doi:10.1109/eusipco.2016.7760424 · Cited by 517
Self-citationComputational bioacoustics with deep learning: a review and roadmap
Stowell, D. (2022) · PeerJ · doi:10.7717/peerj.13152 · Cited by 471
Metadata
- Authors
- Mesaros, A., Heittola, T., & Virtanen, T.
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2016-05-25
- Volume
- 6
- Issue
- 6
- Pages
- 162
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W2408239454
- Cited by
- 579
- References
- 22
- FWCI
- 56.32
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.