CoChat

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

Open access
Lin, Tzu‐Hao, Tsao, Yu · 2020 · Remote Sensing in Ecology and Conservation
Cited by 65

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

A comprehensive assessment of ecosystem dynamics requires the monitoring of biological, physical and social changes. Changes that cannot be observed visually may be trackable acoustically through soundscape analysis. Soundscapes vary greatly depending on geophysical events, biodiversity and human activities. However, retrieving source‐specific information from geophony, biophony and anthropophony remains a challenging task, due to interference by simultaneous sound sources. Audio source separation is a technique that aims to recover individual sound sources when only mixtures are accessible. Here, we review techniques of monoaural audio source separation with the fundamental theories and assumptions behind them. Depending on the availability of prior information about the source signals, the task can be approached as a blind source separation or a model‐based source separation. Most blind source separation techniques depend on assumptions about the behaviour of the source signals, and their performance may deteriorate when the assumptions fail. Model‐based techniques generally do not require specific assumptions, and the models are directly learned from labelled data. With the recent advances of deep learning, the model‐based techniques can yield state‐of‐the‐art separation performance, accordingly facilitate content‐based audio information retrieval. Source separation techniques have been adopted in several ecoacoustic applications to evaluate the contributions from biodiversity and anthropogenic disturbance to soundscape dynamics. They can also be employed as nonlinear filters to improve the recognition of bioacoustic signals. To effectively retrieve ecological information from soundscapes, source separation is a crucial tool. We believe that the future integrations of ecological hypotheses and deep learning can realize a high‐performance source separation for ecoacoustics, and accordingly improve soundscape‐based ecosystem monitoring. Therefore, we outline a roadmap for applying source separation to assist in soundscape information retrieval and hope to promote cross‐disciplinary collaboration.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/rse2.141

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 90

  1. 10.1016/j.marpolbul.2011.04.038 · DOI
  2. 10.1016/j.patrec.2009.09.014 · DOI · Paper page
  3. 10.1016/j.ecoinf.2019.02.009 · DOI
  4. 10.1016/j.ecoinf.2019.01.008 · DOI
  5. 10.1016/j.ecoinf.2019.02.010 · DOI
  6. 10.1111/j.1365-2664.2011.01993.x · DOI · Paper page
  7. 10.22261/JEA.R1156L · DOI
  8. Bryan N. J. andG. J.Mysore.2013.Interactive refinement of supervised and semi‐supervised sound source separation estimates. Pp.883–887 in2013 IEEE International Conference on Acoustics Speech and Signal Processing. · doi:10.1109/icassp.2013.6637775
  9. 10.1109/MSP.2018.2874719 · doi:10.1109/msp.2018.2874719
  10. Chen Z. S.Watanabe H.Erdogan andJ. R.Hershey.2015.Speech enhancement and recognition using multi‐task learning of long short‐term memory recurrent neural networks. Pp.3274–3278inInterspeech 2015. · DOI
  11. 10.1016/0165-1684(94)90029-9 · DOI · Paper page
  12. 10.1098/rsos.160606 · DOI
  13. Du J. Y.Tu Y.Xu L.Dai andC.‐H.Lee.2014.Speech separation of a target speaker based on deep neural networks. Pp.473–477 in2014 12th International Conference on Signal Processing (ICSP). · DOI
  14. 10.1007/s10980-011-9635-x · DOI · Paper page
  15. 10.7717/peerj.2108 · DOI · Paper page
  16. Erdogan H. J. R.Hershey S.Watanabe andJ.Le Roux.2015.Phase‐sensitive and recognition‐boosted speech separation using deep recurrent neural networks. Pp.708–712 in2015 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI · Paper page
  17. Ewert S. andM. B.Sandler.Structured dropout for weak label and multi‐instance learning and its application to score‐informed source separation.Pp. 2277–2281 in .2017.IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI
  18. 10.1016/j.ecolind.2017.07.064 · DOI · Paper page
  19. Fan H.‐T. J.‐W.Hung X.Lu S.‐S.Wang andY.Tsao.2014.Speech enhancement using segmental nonnegative matrix factorization. Pp.4483–4487 in2014 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · doi:10.1109/icassp.2014.6854450
  20. 10.22261/JEA.TRZD5I · DOI
  21. 10.1109/TMM.2010.2098858 · DOI · Paper page
  22. Fu S.‐W. Y.Tsao X.Lu andH.Kawai.2017.Raw waveform‐based speech enhancement by fully convolutional networks. Pp.6–12 in2017 Asia‐Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). · DOI · Paper page
  23. Fu S.‐W., 2018, End‐to‐end waveform utterance enhancement for direct evaluation metrics optimization by fully convolutional neural networks, IEEE T. Audio Speech, 26, 1570
  24. 10.1121/1.4808713 · DOI · Paper page
  25. 10.1121/1.4919312 · doi:10.1121/1.4919312
  26. Hershey J. R., 2016, Deep clustering: discriminative embeddings for segmentation and separation
  27. 10.1126/science.1127647 · DOI · Paper page
  28. Hoyer P. O., 2004, Non‐negative matrix factorization with sparseness constraints, J. Mach. Learn. Res, 5, 1457
  29. Huang P.‐S. S. D.Chen P.Smaragdis andM.Hasegawa‐Johnson.2012.Singing‐voice separation from monaural recordings using robust principal component analysis. Pp.57–60in IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI · Paper page
  30. Huang P.‐S., 2015, Joint optimization of masks and deep recurrent neural networks for monaural source separation, IEEE T. Audio Speech, 23, 2136
  31. Hui L. M.Cai C.Guo L.He W.‐Q.Zhang andJ.Liu.2015.Convolutional maxout neural networks for speech separation. Pp.24‐27 in2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). · DOI
  32. 10.1016/j.camwa.2012.03.077 · doi:10.1016/j.camwa.2012.03.077
  33. Isik Y. J.Le Roux Z.Chen S.Watanabe andJ. R.Hershey.2016.Single‐channel multi‐speaker separation using deep clustering. Pp.545–549 inInterspeech2016. · DOI
  34. Jang G.‐J., 2003, A maximum likelihood approach to single‐channel source separation, J. Mach. Learn. Res., 4, 1365
  35. Kameoka H., 2018, Nonnegative matrix factorization with basis clustering using cepstral distance regularization, IEEE T. Audio Speech, 26, 1029
  36. Karamatlı E. A. T.Cemgil andS.Kırbız.2018.Weak label supervision for monaural source separation using non‐negative denoising variational autoencoders.ArXiv 1810.13104. · DOI
  37. 10.1016/j.ecoinf.2012.08.001 · DOI · Paper page
  38. 10.1016/S0169-5347(03)00071-5 · DOI · Paper page
  39. Kingma D. P., 2014, Auto‐encoding variational bayes. in
  40. Krause B. L., 1993, The niche hypothesis: a virtual symphony of animal sounds, the origins of musical expression and the health of habitats, Soundscape Newsl., 6, 4
  41. 10.1016/j.biocon.2016.01.013 · DOI · Paper page
  42. 10.1371/journal.pone.0055661 · DOI · Paper page
  43. 10.1016/j.patrec.2014.01.008 · DOI · Paper page
  44. Le Roux J. J. R.Hershey andF.Weninger.2015.Deep NMF for speech separation. Pp.66–70 in2015 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI
  45. 10.1038/44565 · DOI · Paper page
  46. Lin T.‐H. andY.Tsao.2018.Listening to the deep: Exploring marine soundscape variability by information retrieval techniques. Pp.1–6 in2018 OCEANS ‐ MTS/IEEE Kobe Techno‐Oceans (OTO). · doi:10.1109/oceanskobe.2018.8559307
  47. 10.1121/1.4816572 · doi:10.1121/1.4816572
  48. 10.1038/s41598-017-04790-7 · DOI · Paper page
  49. Lin T.‐H. Y.Tsao Y.‐H.Wang H.‐W.Yen andS.‐S.Lu.2017b.Computing biodiversity change via a soundscape monitoring network. Pp.128–133 in2017 Pacific Neighborhood Consortium Annual Conference and Joint Meetings (PNC). · DOI
  50. 10.1121/1.5034169 · DOI
  51. Lin T.‐H. H.‐T.Yang J.‐M.Huang C.‐J.Yao Y.‐S.Lien P.‐J.Wang et al.2019.Evaluating changes in the marine soundscape of an offshore wind farm via the machine learning‐based source separation. Pp.1–6 in2019 IEEE Underwater Technology (UT). · DOI
  52. Lostanlen V., 2019, Long‐distance detection of bioacoustic events with per‐channel energy normalization, ArXiv, 1911, 00417
  53. Lu X. Y.Tsao S.Matsuda andC.Hori.2013.Speech enhancement based on deep denoising autoencoder. Pp.436–440inInterspeech2013. · DOI · Paper page
  54. Maurer A., 2016, The benefit of multitask representation learning, J. Mach. Learn. Res., 17, 1
  55. 10.1098/rsos.160370 · DOI
  56. Molla M. K. I., 2008, The robustness and applicability of audio source separation from single mixtures, Acoust. Aust., 36, 2
  57. 10.3354/meps12813 · DOI
  58. 10.1007/s12304-017-9288-5 · DOI
  59. Nugraha A. A., 2016, Multichannel audio source separation with deep neural networks, IEEE T. Audio Speech, 24, 1652
  60. O'Grady P. andB.Pearlmutter.2006.Convolutive non‐negative matrix factorisation with a sparseness constraint. Pp. 427‐432 in2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing. · DOI
  61. 10.1371/journal.pone.0193345 · DOI · Paper page
  62. 10.1007/s11284-017-1509-5 · DOI · Paper page
  63. 10.1073/pnas.1424667112 · DOI · Paper page
  64. 10.1007/s13280-015-0708-y · doi:10.1007/s13280-015-0708-y
  65. Smaragdis P. B.Raj andM.Shashanka.2007.Supervised and semi‐supervised separation of sounds from single‐channel mixtures.Pp.414–421 inIndependent Component Analysis and Signal Separation (ICA'07). · DOI · Paper page
  66. 10.1109/MSP.2013.2297715 · DOI · Paper page
  67. Sobieraj I. Q.Kong andM. D.Plumbley.2017.Masked non‐negative matrix factorization for bird detection using weakly labeled data. Pp.1769–1773 in2017 25th European Signal Processing Conference (EUSIPCO). · DOI
  68. Stoller D. S.Ewert andS.Dixon.2018a.Wave‐u‐net: a multiscale neural network for end‐to‐end source separation. Pp.334–340 in19th International Society for Music Information Retrieval Conference (ISMIR).
  69. Stoller D. S.Ewert andS.Dixon.2018b.Adversarial semi‐supervised audio source separation applied to singing voice extraction.Pp. 2391–2395 in2018 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI
  70. Stöter F.‐R. A.Liutkus andN.Ito.2018.The 2018 signal separation evaluation campaign.Pp. 293–305 inInternational Conference on Latent Variable Analysis and Signal Separation. · doi:10.1007/978-3-319-93764-9_28
  71. Stowell D.2018.Computational bioacoustic scene analysis.Pp.303‐333 inComputational Analysis of Sound Scenes and Events T. Virtanen M. D. Plumbley and D. Ellis Eds. Berlin Germany: Springer. · DOI
  72. Stowell D., 2015, Denoising without access to clean data using a partitioned autoencoder, ArXiv, 1509, 05982
  73. Subakan Y. C. andP.Smaragdis.2018.Generative adversarial source separation. Pp.26–30 in2018 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI
  74. 10.1007/s12304-015-9248-x · DOI · Paper page
  75. 10.1371/journal.pone.0004065 · DOI · Paper page
  76. 10.20965/jrm.2017.p0213 · DOI
  77. 10.1109/ISCSLP.2014.6936615 · DOI
  78. Uhlich S. F.Giron andY.Mitsufuji.2015.Deep neural network based instrument extraction from music. Pp. 2135–2139 in 2015 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · DOI
  79. 10.1016/j.ecolind.2018.03.026 · DOI · Paper page
  80. Vaseghi S. V.2008.Advanced digital signal processing and noise reduction (Fourth Edition). John Wiley & Sons. · DOI · Paper page
  81. 10.1007/s10980-011-9636-9 · DOI · Paper page
  82. 10.1007/0-387-32845-9_9 · doi:10.1007/0-387-32845-9_9
  83. Wang D., 2017, Supervised speech separation based on deep learning: an overview, IEEE T. Audio Speech, 26, 1702
  84. Wang Z. andF.Sha.2014.Discriminative non‐negative matrix factorization for single‐channel speech separation. Pp.3749–3753 in2014 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). · doi:10.1109/icassp.2014.6854302
  85. 10.1109/TASL.2013.2250961 · DOI · Paper page
  86. Weninger F. J. L.Roux J. R.Hershey andS.Watanabe.2014.Discriminative NMF and its application to single‐channel source separation. Pp.865–869 inInterspeech 2014. · DOI
  87. 10.1109/93.556537 · DOI · Paper page
  88. Wu C.‐W. andA.Lerch.2015.Drum transcription using partially fixed non‐negative matrix factorization. Pp.1281‐1285 in2015 23rd European Signal Processing Conference (EUSIPCO). · DOI
  89. Xie J. M.Towsey J.Zhang X.Dong andP.Roe.2015.Application of image processing techniques for frog call classification. Pp.4190–4194 in2015 IEEE International Conference on Image Processing (ICIP). · DOI
  90. Xu Y., 2015, A regression approach to speech enhancement based on deep neural networks, IEEE T. Audio Speech, 23, 7

Cited by

Metadata

Authors
Lin, T., & Tsao, Y.
Venue
Remote Sensing in Ecology and Conservation
Publisher
Wiley
Published
2019-12-16
Volume
6
Issue
3
Pages
236-247
ISSN
2056-3485 (ISSN-L)
OpenAlex
W2996692619
Cited by
65
References
5
FWCI
4.98
Type
journal article
Language
en
Open access
gold
Funders
  • Asi@Connect (Asi@Connect-17-100)
  • JSPS KAKENHI (JP18H06491)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.