Classification of animal sounds in a hyperdiverse rainforest using convolutional neural networks with data augmentation
Open accessAbstract
To protect tropical forest biodiversity, we need to be able to detect it reliably, cheaply, and at scale. Automated detection of sound producing animals from passively recorded soundscapes via machine-learning approaches is a promising technique towards this goal, but it is constrained by the necessity of large training data sets. Using soundscapes from a tropical forest in Borneo and a Convolutional Neural Network model (CNN), we investigate i) the minimum viable training data set size for accurate prediction of call types (‘sonotypes’), and ii) the extent to which data augmentation and transfer learning can overcome the issue of small and imbalanced training data sets. We found that even relatively high sample sizes (>80 per sonotype) lead to mediocre accuracy, which however improved significantly with data augmentation and transfer learning, including at extremely small sample sizes (3 per sonotype), regardless of taxonomic group or call characteristics. Neither transfer learning nor data augmentation alone achieved high accuracy. Our results suggest that transfer learning and data augmentation could make the use of CNNs to classify species’ vocalizations feasible even for small soundscape-based projects with many rare species. Retraining our open-source model requires only basic programming skills which makes it possible for individual conservation initiatives to match their local context, in order to enable more evidence-informed management of biodiversity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A major bottleneck for machine learning approaches, including CNNs, is the need of large training datasets, particularly challenging in the hyper-diverse communities of tropical forests.
“A major bottleneck for machine learning approaches, including CNNs, is the need of large training datasets. This is particularly challenging in the hyper-diverse communities of tropical forests41.”
From Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests (2023) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · see citing works
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826 · see citing works
Harris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844 · see citing works
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · see citing works
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · see citing works
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400 · see citing works
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2015) · doi:10.1109/cvpr31182.2015 · Cited by 3,735 · see citing works
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558 · see citing works
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499 · see citing works
BirdNET: A deep learning solution for avian diversity monitoring
Kahl, S., Wood, C. M., Eibl, M., & Klinck, H. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101236 · Cited by 903 · see citing works
Hillebrand, H., Blasius, B., Borer, E. T., Chase, J. M., Downing, J., Eriksson, B. K., Filstrup, C. T., Harpole, W. S., Hodapp, D., Larsen, S., et al. (2017) · Journal of Applied Ecology · doi:10.1111/1365-2664.12959 · Cited by 804 · see citing works
Powers, R., & Jetz, W. (2019) · Nature Climate Change · doi:10.1038/s41558-019-0406-z · Cited by 677 · see citing works
Global forest loss disproportionately erodes biodiversity in intact landscapes
Betts, M. G., Wolf, C., Ripple, W. J., Phalan, B., Millers, K. A., Duarte, A., Butchart, S. H. M., & Levi, T. (2017) · Nature · doi:10.1038/nature23285 · Cited by 637 · see citing works
Ecoacoustics: the Ecological Investigation and Interpretation of Environmental Sound
Sueur, J., & Farina, A. (2015) · Biosemiotics · doi:10.1007/s12304-015-9248-x · Cited by 485 · see citing works
Stowell, D., Wood, M. D., Pamuła, H., Stylianou, Y., & Glotin, H. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13103 · Cited by 414 · see citing works
Six Common Mistakes in Conservation Priority Setting
Game, E. T., Kareiva, P., & Possingham, H. P. (2013) · Conservation Biology · doi:10.1111/cobi.12051 · Cited by 338 · see citing works
Guidelines for the use of acoustic indices in environmental research
Bradfer‐Lawrence, T., Gardner, N., Bunnefeld, L., Bunnefeld, N., Willis, S. G., & Dent, D. H. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13254 · Cited by 319 · see citing works
warbleR: an r package to streamline analysis of animal acoustic signals
Araya‐Salas, M., & Smith‐Vidaurre, G. (2016) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12624 · Cited by 305 · see citing works
Derryberry, E. P., Phillips, J. N., Derryberry, G. E., Blum, M. J., & Luther, D. (2020) · Science · doi:10.1126/science.abd5777 · Cited by 304 · see citing works
Knight, E. C., Hannah, K. C., Foley, G. J., Scott, C. D., Brigham, R. M., & Bayne, E. M. (2017) · Avian Conservation and Ecology · doi:10.5751/ace-01114-120214 · Cited by 230 · see citing works
Efficacy of extracting indices from large‐scale acoustic recordings to monitor biodiversity
Buxton, R. T., McKenna, M. F., Clapp, M. K., Meyer, E. W., Stabenau, E., Angeloni, L. M., Crooks, K. R., & Wittemyer, G. (2018) · Conservation Biology · doi:10.1111/cobi.13119 · Cited by 230 · see citing works
LeBien, J. G., Zhong, M., Campos‐Cerqueira, M., Velev, J. P., Dodhia, R. M., Ferres, J. L., & Aide, T. M. (2020) · Ecological Informatics · doi:10.1016/j.ecoinf.2020.101113 · Cited by 189 · see citing works
Data augmentation approaches for improving animal audio classification
Nanni, L., Maguolo, G., & Paci, M. (2020) · Ecological Informatics · doi:10.1016/j.ecoinf.2020.101084 · Cited by 183 · see citing works
(2010) · Journal of Cosmology · Cited by 179 · see citing works
Characterizing soundscapes across diverse ecosystems using a universal acoustic feature set
Sethi, S. S., Jones, N. S., Fulcher, B., Picinali, L., Clink, D. J., Klinck, H., Orme, C. D. L., Wrege, P. H., & Ewers, R. M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2004702117 · Cited by 171 · see citing works
Measuring sound detection spaces for acoustic animal sampling and monitoring
Darras, K., Pütz, P., Fahrurrozi, Rembold, K., & Tscharntke, T. (2016) · Biological Conservation · doi:10.1016/j.biocon.2016.06.021 · Cited by 164 · see citing works
Three critical factors affecting automated image species recognition performance for camera traps
Schneider, S., Greenberg, S., Taylor, G. W., & Kremer, S. C. (2020) · Ecology and Evolution · doi:10.1002/ece3.6147 · Cited by 161 · see citing works
Songs of Darwin's finches diverge when a new species enters the community
Grant, B. R., & Grant, P. R. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1015115107 · Cited by 152 · see citing works
Investigation of Different CNN-Based Models for Improved Bird Sound Classification
Xie, J., Hu, K., Zhu, M., Yu, J., & Zhu, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2957572 · Cited by 145 · see citing works
Data requirements and data sources for biodiversity priority area selection
Williams, P. H., Margules, C. R., & Hilbert, D. W. (2002) · Journal of Biosciences · doi:10.1007/bf02704963 · Cited by 138 · see citing works
A toolbox for animal call recognition
Towsey, M., Planitz, B. M., Nantes, A., et al. (2012) · Bioacoustics · doi:10.1080/09524622.2011.648753 · Cited by 132 · see citing works
Deep learning for environmental conservation
Lamba, A., Cassey, P., Segaran, R. R., & Koh, L. P. (2019) · Current Biology · doi:10.1016/j.cub.2019.08.016 · Cited by 131 · see citing works
Workflow and convolutional neural network for automated identification of animal sounds
Ruff, Z. J., Lesmeister, D. B., Appel, C. L., & Sullivan, C. M. (2021) · Ecological Indicators · doi:10.1016/j.ecolind.2021.107419 · Cited by 123 · see citing works
Species Richness (of Insects) Drives the Use of Acoustic Space in the Tropics
Aide, T. M., Hernández-Serna, A., Campos‐Cerqueira, M., Acevedo-Charry, O. A., & Deichmann, J. L. (2017) · Remote Sensing · doi:10.3390/rs9111096 · Cited by 117 · see citing works
The sound of a tropical forest
Burivalova, Z., Game, E. T., & Butler, R. A. (2019) · Science · doi:10.1126/science.aav1902 · Cited by 107 · see citing works
Zhong, M., LeBien, J. G., Campos‐Cerqueira, M., Dodhia, R. M., Ferres, J. L., Velev, J. P., & Aide, T. M. (2020) · Applied Acoustics · doi:10.1016/j.apacoust.2020.107375 · Cited by 105 · see citing works
Detecting small changes in populations at landscape scales: a bioacoustic site-occupancy framework
Wood, C. M., Popescu, V. D., Klinck, H., Keane, J. J., Gutiérrez, R. J., Sawyer, S. C., & Peery, M. Z. (2019) · Ecological Indicators · doi:10.1016/j.ecolind.2018.11.018 · Cited by 103 · see citing works
Handcrafted features and late fusion with deep learning for bird sound classification
Xie, J., & Zhu, M. (2019) · Ecological Informatics · doi:10.1016/j.ecoinf.2019.05.007 · Cited by 100 · see citing works
Automated identification of avian vocalizations with deep convolutional neural networks
Ruff, Z. J., Lesmeister, D. B., Duchac, L. S., Padmaraju, B. K., & Sullivan, C. M. (2019) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.125 · Cited by 98 · see citing works
Audio-based Bird Species Identification with Deep Convolutional Neural Networks.
Lasseck, M. (2018) · CLEF (Working Notes) · Cited by 69 · see citing works
Buřivalová, Z., Allnutt, T. F., Rademacher, D., Schlemm, A., Wilcove, D. S., & Butler, R. A. (2019) · Conservation Science and Practice · doi:10.1111/csp2.28 · Cited by 68 · see citing works
Self-citationConvolutional Neural Networks for Scops Owl Sound Classification
Hidayat, A. A., Cenggoro, T. W., & Pardamean, B. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2020.12.010 · Cited by 68 · see citing works
Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval
Lin, T., & Tsao, Y. (2019) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.141 · Cited by 63 · see citing works
Lin, T., Fang, S.-H., & Tsao, Y. (2017) · Scientific Reports · doi:10.1038/s41598-017-04790-7 · Cited by 56 · see citing works
Chen, X., Zhao, J., Chen, Y., Zhou, W., & Hughes, A. C. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.108269 · Cited by 52 · see citing works
High microphone signal-to-noise ratio enhances acoustic sampling of wildlife
Darras, K., Deppe, F., Fabian, Y., KARTONO, A. P., Angulo, A., Kolbrek, B., Mulyani, Y. A., & Prawiradilaga, D. M. (2020) · PeerJ · doi:10.7717/peerj.9955 · Cited by 50 · see citing works
Koranne, S. (2010) · Springer eBooks · doi:10.1007/978-1-4419-7719-9 · Cited by 45 · see citing works
A multi-view CNN-based acoustic classification system for automatic animal species identification
Xu, W., Zhang, X., Yao, L., Xue, W., & Wei, B. W. (2020) · Ad Hoc Networks · doi:10.1016/j.adhoc.2020.102115 · Cited by 44 · see citing works
Automated classification of bat echolocation call recordings with artificial intelligence
Tabak, M. A., Murray, K. L., Reed, A. M., Lombardi, J. A., & Bay, K. J. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101526 · Cited by 37 · see citing works
Yang, D., Li, T., Liu, M.-T., Li, X., & Chen, B. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101350 · Cited by 34 · see citing works
Rappaport, D. I., Royle, J. A., & Morton, D. C. (2020) · Ecological Indicators · doi:10.1016/j.ecolind.2020.106172 · Cited by 33 · see citing works
Convolutional Feature Vectors and Support Vector Machine for Animal Sound Classification
Ko, K., Park, S., & Ko, H. (2018) · doi:10.1109/embc.2018.8512408 · Cited by 32 · see citing works
Searfoss, A. M., Pino, J. C., & Creanza, N. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13368 · Cited by 25 · see citing works
Johnson, A., Saypanya, S., Hansel, T., et al. (2021) · Conservation Science and Practice · doi:10.1111/csp2.539 · Cited by 20 · see citing works
Fletcher, N. H. (2014) · Springer handbooks · doi:10.1007/978-1-4939-0755-7_19 · Cited by 14 · see citing works
Automated classification of bat echolocation call recordings with artificial intelligencePreprint
Tabak, M. A., Murray, K. L., Lombardi, J. A., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.23.449619 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Bradski, G. (2000). The OpenCV Library. Dr. Dobb's Journal of Software Tools.
- Kahl, Stefan, Stöter, Fabian-Robert, Hervé Goëau, Hervé Glotin, Planque, Robert, Willem-Pier Vellinga, Alexis, Joly, 2019. Overview of BirdCLEF 2019: large-scale bird recognition in soundscapes." In Working Notes of CLEF 2019-Conference and Labs of the Evaluation Forum, no, 2380. CEUR, pp. 1-9.
Cited by
Sorted by citations
Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests
Müller, J., Mitesser, O., Schaefer, H. M., Seibold, S., Busse, A., Kriegel, P., Rabl, D., Gelis, R., Arteaga, A., Freile, J., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-41693-w · Cited by 89 · see citing works
Self-citation
- Authors
- Sun, Y., Midori Maeda, T., Solís-Lemus, C., Pimentel-Alarcón, D., & Buřivalová, Z.
- Affiliations
- University of Wisconsin–Madison (US)
- Corresponding author
- Pimentel-Alarcón, Daniel
- Buřivalová, Zuzana
- Venue
- Ecological Indicators
- Publisher
- Elsevier BV
- Published
- 2022-12
- Volume
- 145
- Pages
- 109621
- ISSN
- 1470-160X (ISSN-L)
- OpenAlex
- W4307992680
- Cited by
- 49
- References
- 57
- Citations by year
- 2026: 10 · 2025: 19 · 2024: 12 · 2023: 7 · 2022: 1
- FWCI
- 4.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Ecological Indicators (publishedVersion, cc-by-nc-nd)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- U.S. Department of Energy (DE-SC0021016)
- College of Engineering, University of Wisconsin-Madison
- The Nature Conservancy
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Developmental Biology › Animal Vocal Communication and Behavior
- Physical Sciences › Environmental Science › Ecology › Marine animal studies overview
- Keywords
- animal sound classification, vocalizations, soundscape monitoring, passive acoustic monitoring, tropical rainforest, Borneo, hyperdiversity, limited training data, data augmentation, transfer learning, convolutional neural networks, rare species, biodiversity conservation, open-source models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
