The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
Open accessAbstract
In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3-D) space. The proposed network takes a sequence of consecutive spectrogram time frames as input and maps it to two outputs in parallel. As the first output, the sound event detection (SED) is performed as a multi-label classification task on each time frame producing temporal activity for all the sound event classes. As the second output, localization is performed by estimating the 3-D Cartesian coordinates of the direction-of-arrival (DOA) for each sound event class using multi-output regression. The proposed method is able to associate multiple DOAs with respective sound event labels and further track this association with respect to time. The proposed method uses separately the phase and magnitude component of the spectrogram calculated on each audio channel as the feature, thereby avoiding any method- and array-specific feature extraction. The method is evaluated on five Ambisonic and two circular array format datasets with different overlapping sound events in anechoic, reverberant, and real-life scenarios. The proposed method is compared with two SED, three DOA estimation, and one SELD baselines. The results show that the proposed method is generic and applicable to any array structures, robust to unseen DOA values, reverberation, and low SNR scenarios. The proposed method achieved a consistently higher recall of the estimated number of DOAs across datasets in comparison to the best baseline. Additionally, this recall was observed to be significantly better than the best baseline method for a higher number of overlapping sound events.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Multiple emitter location and signal parameter estimation
Schmidt, R. (1986) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.1986.1143830 · Cited by 14,718 · cited 16×
“MUSIC requires a good estimate of the number of active sound sources in order to estimate their DOAs.”
BackgroundMethodInfluentialESPRIT-estimation of signal parameters via rotational invariance techniques
Roy, R., & Kailath, T. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.32276 · Cited by 7,289
Image method for efficiently simulating small-room acoustics
Allen, J. B., & Berkley, D. A. (1979) · The Journal of the Acoustical Society of America · doi:10.1121/1.382599 · Cited by 3,742
A Dataset and Taxonomy for Urban Sound Research
Salamon, J., Jacoby, C., & Bello, J. P. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2655045 · Cited by 1,309
Estimating animal population density using passive acoustics
Marques, T. A., Thomas, L., Martin, S. W., Mellinger, D. K., Ward, J. A., Moretti, D. J., Harris, D., & Tyack, P. L. (2013) · Biological Reviews · doi:10.1111/brv.12001 · Cited by 746
Robust Localization in Reverberant Rooms
DiBiase, J. H., Silverman, H. F., & Brandstein, M. S. (2001) · Digital Signal Processing · doi:10.1007/978-3-662-04619-7_8 · Cited by 678
The third ‘CHiME’ speech separation and recognition challenge: Dataset, task and baselines
Barker, J., Marxer, R., Vincent, E., & Watanabe, S. (2015) · 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404837 · Cited by 641
Environmental Sound Recognition With Time–Frequency Audio Features
Chu, S., Narayanan, S., & Kuo, C.-C. J. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2009.2017438 · Cited by 639
Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Cakir, E., Parascandolo, G., Heittola, T., Huttunen, H., & Virtanen, T. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2690575 · Cited by 604
Self-citationMetrics for Polyphonic Sound Event Detection
Mesaros, A., Heittola, T., & Virtanen, T. (2016) · Applied Sciences · doi:10.3390/app6060162 · Cited by 579
Self-citationDiBiase, J. H. (2000) · Cited by 468
Real-time passive source localization: a practical linear-correction least-squares approach
Huang, Y., Benesty, J., Elko, G. W., & Mersereati, R. M. (2001) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.966097 · Cited by 464
Ottersten, B., Viberg, M., Stoica, P., & Nehorai, A. (1993) · Springer Series in Information Sciences · doi:10.1007/978-3-642-77347-1_4 · Cited by 365
DCASE 2017 Challenge setup: Tasks, datasets and baseline system
Mesaros, A., Heittola, T., Diment, A., Elizalde, B., Shah, A., Emmanuel Vincent, A., Raj, B., & Virtanen, T. I. (2017) · HAL · Cited by 326
Self-citation(2009) · Wiley eBooks · Cited by 321
Recurrent neural networks for polyphonic sound event detection in real life recordings
Parascandolo, G., Huttunen, H., & Virtanen, T. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472917 · Cited by 319
Self-citationA learning-based approach to direction of arrival estimation in noisy and reverberant environments
Xiao, X., Zhao, S., Zhong, X., Jones, D. L., Chng, E. S., & Li, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178484 · Cited by 289
Broadband doa estimation using convolutional neural networks trained with noise signals
Chakrabarty, S., & Habets, E. A. P. (2017) · 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2017.8170010 · Cited by 286
Adavanne, S., Politis, A., & Virtanen, T. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553182 · Cited by 277
Self-citationValin, J.-M., Michaud, F., & Rouat, J. (2007) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2006.08.004 · Cited by 275
Robust sound event recognition using convolutional neural networks
Zhang, H. S., McLoughlin, I. V., & Song, Y. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178031 · Cited by 271
Polyphonic sound event detection using multi label deep neural networks
Cakir, E., Heittola, T., Huttunen, H., & Virtanen, T. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280624 · Cited by 256
Self-citationConvolutional Neural Networks for Distant Speech Recognition
Swietojanski, P., Ghoshal, A., & Renals, S. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2325781 · Cited by 253
Audio Surveillance: A Systematic Review
Crocco, M., Cristani, M., Trucco, A., & Murino, V. (2016) · ACM Computing Surveys · doi:10.1145/2871183 · Cited by 244
Acoustic event detection in real life recordings
Mesaros, A., Heittola, T., Eronen, A., & Virtanen, T. I. (2010) · Cited by 241
Self-citationWölfel, M., & McDonough, J. (2009) · Wiley eBooks · doi:10.1002/9780470714089 · Cited by 240
Audio Surveillance of Roads: A System for Detecting Anomalous Sounds
Foggia, P., Petkov, N., Saggese, A., Strisciuglio, N., & Vento, M. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2470216 · Cited by 237
Voice source localization for automatic camera pointing system in videoconferencing
Wang, H., & Chu, P. (2002) · 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1997.599595 · Cited by 211
Deep Neural Networks for Multiple Speaker Detection and Localization
He, W., Motlicek, P., & Odobez, J.-M. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461267 · Cited by 198
Sound event detection using spatial features and convolutional recurrent neural network
Adavanne, S., Pertila, P., & Virtanen, T. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952260 · Cited by 160
Self-citationSound Source Localization Using Deep Learning Models
Yalta, N., Nakadai, K., & Ogata, T. (2017) · Journal of Robotics and Mechatronics · doi:10.20965/jrm.2017.p0037 · Cited by 159
Takeda, R., & Komatani, K. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7471706 · Cited by 158
Furnas, B. J., & Callas, R. L. (2014) · Journal of Wildlife Management · doi:10.1002/jwmg.821 · Cited by 151
Sound Event Detection in Multichannel Audio Using Spatial and Harmonic FeaturesPreprint
Adavanne, S., Parascandolo, G., Pertila, P. E., Heittola, T., & Virtanen, T. I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02293 · Cited by 119
Self-citationTakeda, R., & Komatani, K. (2016) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2016.7846325 · Cited by 114
Duration-Controlled LSTM for Polyphonic Sound Event Detection
Hayashi, T., Watanabe, S., Toda, T., Hori, T., Roux, J. L., & Takeda, K. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2740002 · Cited by 101
Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks
Phan, H., Hertel, L., Maaß, M., & Mertins, A. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-123 · Cited by 94
Sound Source Localization in a Multipath Environment Using Convolutional Neural Networks
Ferguson, E. L., Williams, S. B., & Jin, C. T. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462024 · Cited by 94
A neural network based algorithm for speaker localization in a multi-room environment
Vesperini, F., Vecchiotti, P., Principi, E., Squartini, S., & Piazza, F. (2016) · Machine learning for signal processing ... · doi:10.1109/mlsp.2016.7738817 · Cited by 80
Classification of Spatial Audio Location and Content Using Convolutional Neural Networks
Hirvonen, T. (2015) · Journal of the Audio Engineering Society · Cited by 72
A report on sound event detection with different binaural featuresPreprint
Adavanne, S., & Virtanen, T. I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02997 · Cited by 55
Self-citationAdavanne, S., Politis, A., & Virtanen, T. I. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489542 · Cited by 43
Self-citationSound based localization and identification in industrial environments
Grobler, C., Kruger, C. P., Silva, B., et al. (2017) · Proceedings of the Annual Conference of the IEEE Industrial Electronics Society · doi:10.1109/iecon.2017.8217063 · Cited by 36
Multi-Speaker Localization Using Convolutional Neural Network Trained with NoisePreprint
Chakrabarty, S., & Habets, E. A. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04276 · Cited by 33
Multiple speaker tracking with the Factorial von Mises-Fisher Filter
Traa, J., & Smaragdis, P. (2014) · IEEE International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2014.6958891 · Cited by 26
Mesaros, A., Heittola, T., & Ellis, D. P. W. (2017) · Springer eBooks · doi:10.1007/978-3-319-63450-0_6 · Cited by 20
Zöhrer, M., & Pernkopf, F. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1238 · Cited by 16
Research works printed but not linked: 17
- lim, 0, Rare sound event detection using 1D convolutional recurrent neural networks, Proc Workshop Detection Classif Acoust Scenes Events
- 10.1109/ASPAA.2009.5346532 · doi:10.1109/aspaa.2009.5346532
- ottersten, 1993, Radar Array Processing (Springer Series in Information Sciences)
- 10.1121/1.4726012 · DOI
- yiwere, 2017, Distance estimation and localization of sound sources in reverberant conditions using deep neural networks, Int J Appl Eng Res, 12, 12384
- sun, 2017, Indoor sound source localization with probabilisitic neural network, IEEE Trans Ind Electron, 29, 6403
- roden, 0, On sound source localization of speech signals using deep neural networks, Proc Deutsche Jahrestagung für Akustik
- 10.1007/s11042-015-3105-4 · doi:10.1007/s11042-015-3105-4
- 10.1109/ICASSP.2014.6853670 · doi:10.1109/icassp.2014.6853670
- jeong, 0, Audio event detection using multiple-input convolutional neural network, Proc Workshop Detection Classif Acoust Scenes Events
- butko, 0, Two-source acoustic event detection and localization: Online implementation in a smart-room, Proc Euro Signal Process Conf, 1317
- 10.1121/1.4988711 · doi:10.1121/1.4988711
- teutsch, 2007, Modal Array Signal Processing Principles and Applications of Acoustic Wavefield Decomposition, 348
- dibiase, 2001, Microphone Arrays
- lu, 0, Bidirectional GRU for sound event detection, Proc Workshop Detection Classif Acoust Scenes Events
- zhou, 0, Sound event detection in multichannel audio LSTM network, Proc Workshop Detection Classif Acoust Scenes Events
- 10.1007/978-3-540-68585-2_34 · doi:10.1007/978-3-540-68585-2_34
Cited by
Machine learning in acoustics: Theory and applications
Bianco, M. J., Gerstoft, P., Traer, J., Ozanich, E., Roch, M. A., Gannot, S., & Deledalle, C.-A. (2019) · The Journal of the Acoustical Society of America · doi:10.1121/1.5133944 · Cited by 609
A survey of sound source localization with deep learning methods
Grumiaux, P.-A., Kitić, S., Girin, L., & Guérin, A. (2022) · The Journal of the Acoustical Society of America · doi:10.1121/10.0011809 · Cited by 347
Lenk, C., Hövel, P., Ved, K., Durstewitz, S., Meurer, T., Fritsch, T., Männchen, A., Küller, J., Beer, D., Ivanov, T., et al. (2023) · Nature Electronics · doi:10.1038/s41928-023-00957-5 · Cited by 70
Khalifa, Y., Donohue, C., Coyle, J. L., & Sejdic, E. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3000057 · Cited by 49
Metadata
- Authors
- Adavanne, S., Politis, A., Nikunen, J., & Virtanen, T.
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2019-03
- Volume
- 13
- Issue
- 1
- Pages
- 34-48
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2810934215
- Cited by
- 534
- References
- 50
- FWCI
- 29.03
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- H2020 European Research Council (637422)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.