Bird and whale species identification using sound images
Abstract
Image identification of animals is mostly centred on identifying them based on their appearance, but there are other ways images can be used to identify animals, including by representing the sounds they make with images. In this study, the authors present a novel and effective approach for automated identification of birds and whales using some of the best texture descriptors in the computer vision literature. The visual features of sounds are built starting from the audio file and are taken from images constructed from different spectrograms and from harmonic and percussion images. These images are divided into sub‐windows from which sets of texture descriptors are extracted. The experiments reported in this study using a dataset of Bird vocalisations targeted for species recognition and a dataset of right whale calls targeted for whale detection (as well as three well‐known benchmarks for music genre classification) demonstrate that the fusion of different texture features enhances performance. The experiments also demonstrate that the fusion of different texture features with audio features is not only comparable with existing audio signal approaches but also statistically improves some of the stand‐alone audio features. The code for the experiments will be publicly available at https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0 .
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Spectrograms and related tools turned natural history into something closer to experimental science, allowing researchers to measure fine structure of bird song, frog calls, and whale vocalizations rather than rely solely on the ear.
“These tools turned natural history into something closer to experimental science. Researchers could now measure the fine structure of bird song, frog calls, and whale vocalizations rather than rely solely on the ear”
From Listening forward: emerging roles of bioacoustics in ecology, evolution, and conservation (2026) · cites this paper as Nanni et al. 2017
Cites
Textural Features for Image Classification
Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1973.4309314 · Cited by 22,886 · see citing works
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
librosa: Audio and Music Signal Analysis in Python
McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., & Nieto, O. (2015) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-7b98e3ed-003 · Cited by 3,099 · see citing works
Musical genre classification of audio signals
Tzanetakis, G., & Cook, P. (2002) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2002.800560 · Cited by 2,832 · see citing works
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
Kuncheva, L. I., & Whitaker, C. J. (2003) · Machine Learning · doi:10.1023/a:1022859003006 · Cited by 2,335 · see citing works
Blur Insensitive Texture Classification Using Local Phase Quantization
Ojansivu, V., & Heikkilä, J. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69905-7_27 · Cited by 1,043 · see citing works
Optimizing digital speech coders by exploiting masking properties of the human ear
Schroeder, M. R., Atal, B. S., & Hall, J. L. (1979) · The Journal of the Acoustical Society of America · doi:10.1121/1.383662 · Cited by 422 · see citing works
Rotation-Invariant Image and Video Description With Local Binary Pattern Features
Zhao, G., Ahonen, T., Matas, J., & Pietikäinen, M. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2175739 · Cited by 352 · see citing works
Detecting bird sounds in a complex acoustic environment and application to bioacoustic monitoring
Bardeli, R., Wolff, D., Kurth, F., Koch, M., Tauchert, K.-H., & Frommolt, K. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.09.014 · Cited by 342 · see citing works
Bird species recognition using support vector machines
Fagerlund, S. (2007) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2007/38637 · Cited by 276 · see citing works
An evaluation of Convolutional Neural Networks for music classification using spectrograms
Costa, Y. M. G., Oliveira, L. S., & Junior, C. N. S. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.12.024 · Cited by 247 · see citing works
Self-citationMusic genre classification using LBP textural features
Costa, Y. M. G., Oliveira, L. S., Koerich, A. L., Gouyon, F., & Martins, J. G. (2012) · Signal Processing · doi:10.1016/j.sigpro.2012.04.023 · Cited by 166 · see citing works
Self-citationLETRIST: Locally Encoded Transform Feature Histogram for Rotation-Invariant Texture Classification
Song, T., Li, H., Meng, F., Wu, Q., & Cai, J. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2671899 · Cited by 164 · see citing works
Combining visual and acoustic features for music genre classification
Nanni, L., Costa, Y. M. G., Lumini, A., Kim, M. Y., & Baek, S. R. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.09.018 · Cited by 127 · see citing works
Self-citationAutomatic recognition of animal vocalizations using averaged MFCC and linear discriminant analysis
Lee, C.-H., Chou, C.-H., Han, C., & Huang, R.-Z. (2005) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.07.004 · Cited by 113 · see citing works
Automated bird acoustic event detection and robust species classification
Zhao, Z., Zhang, S.-H., Xu, Z., Bellisario, K., Dai, N., Omrani, H., & Pijanowski, B. C. (2017) · Ecological Informatics · doi:10.1016/j.ecoinf.2017.04.003 · Cited by 110 · see citing works
Self-citationOptimum Detection of Ultrasonic Echoes Applied to the Analysis of the First Layer of a Restored Dome
Vergara, L., Reig, I. B., Castillo, J. G., & Salazar, A. (2007) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2007/26560 · Cited by 94 · see citing works
Classification of dog barks: a machine learning approach
Molnár, C., Kaplan, F., Roy, P., Francois Pachet, B., Pongrácz, P., Dóka, A., & Miklósi, Á. (2008) · Animal Cognition · doi:10.1007/s10071-007-0129-9 · Cited by 94 · see citing works
Music genre recognition using spectrograms
Costa, Y. M. G., Oliveira, L. S., Koericb, A. L., & Gouyon, F. (2011) · Espace ÉTS (ETS) · Cited by 92 · see citing works
Self-citationAn adaptive hybrid pattern for noise-robust texture analysis
Zhu, Z., You, X., Chen, C. L. P., Tao, D., Ou, W., Jiang, X., & Zou, J. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.01.001 · Cited by 88 · see citing works
Deep Image Features in Music Information Retrieval
Gwardys, G., & Grzywczak, D. (2014) · International Journal of Electronics and Telecommunications · doi:10.2478/eletel-2014-0042 · Cited by 85 · see citing works
Combining visual and acoustic features for audio classification tasks
Nanni, L., Costa, Y. M. G., Lucio, D. R., Junior, C. N. S., & Brahnam, S. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.01.013 · Cited by 82 · see citing works
Self-citationUmesh, S., Cohen, L. W., & Nelson, D. J. (1999) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1999.758101 · Cited by 80 · see citing works
Cheng, J., Sun, Y., & Ji, L. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.04.026 · Cited by 74 · see citing works
Chesmore, E. D. (2001) · Applied Acoustics · doi:10.1016/s0003-682x(01)00009-3 · Cited by 73 · see citing works
Rotation Invariant Co-occurrence among Adjacent LBPs
Nosaka, R., Suryanto, C. H., & Fukui, K. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37410-4_2 · Cited by 70 · see citing works
ISMIR 2004 Audio Description Contest
Cano, P., Gómez, E., Gouyon, F., Herrera, P., Koppenberger, M., Ong, B., Serra, X., Streich, S., & Wack, N. (2006) · Cited by 69 · see citing works
Automatic Bird Species Identification for Large Number of Species
Lopes, M. T., Gioppo, L. L., Higushi, T. T., Kaestner, C. A. A., Junior, C. N. S., & Koerich, A. L. (2011) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism.2011.27 · Cited by 65 · see citing works
Urazghildiiev, I. R., Clark, C. W., Krein, T. P., & Parks, S. E. (2009) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2009.2014931 · Cited by 62 · see citing works
A Closer Look On Artist Filters For Musical Genre Classification.
Flexer, A. (2007) · doi:10.5281/zenodo.1415667 · Cited by 61 · see citing works
Data Mining Applied to Acoustic Bird Species Recognition
Vilches, E., Escobar, I., Vallejo, E. E., & TAYLOR, C. E. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.426 · Cited by 58 · see citing works
Panagakis, Y., Kotropoulos, C. L., & Arce, G. R. (2009) · doi:10.5281/zenodo.1416413 · Cited by 58 · see citing works
Panagakis, Y., & Kotropoulos, C. L. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495984 · Cited by 42 · see citing works
San, M., Crocco, M., Cristani, M., Martelli, S., & Murino, V. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.105 · Cited by 42 · see citing works
Automatic classification of audio data
Costa, C. H. L., Valle, J. D., & Koerich, A. L. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398359 · Cited by 42 · see citing works
Automatic track recognition of footprints for identifying cryptic species
Russell, J. C., Hasler, N., Klette, R., & Rosenhahn, B. (2009) · Ecology · doi:10.1890/08-1069.1 · Cited by 41 · see citing works
Music-genre classification system based on spectro-temporal features and feature selection
Lim, S.-C., Lee, J.-S., Jang, S.-J., Lee, S.-P., & Kim, M. Y. (2012) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2012.6414994 · Cited by 40 · see citing works
Music Genre Recognition Using Gabor Filters and LPQ Texture Descriptors
Costa, Y., Oliveira, L., Koerich, A., & Gouyon, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-41827-3_9 · Cited by 35 · see citing works
Self-citationEvolving Automatically High-Level Music Descriptors from Acoustic Signals
Pachet, F., & Zils, A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-39900-1_5 · Cited by 32 · see citing works
Towards effective codebookless model for image classification
Wang, Q., Li, P., Lei Zhang, A., & Zuo, W. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.03.004 · Cited by 30 · see citing works
Combining different local binary pattern variants to boost performance
Nanni, L., Brahnam, S., & Lumini, A. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.11.048 · Cited by 29 · see citing works
Self-citationCombining Visual and Acoustic Features for Bird Species Classification
Nanni, L., Costa, Y. M. G., Lucio, D. R., Junior, C. N. S., & Brahnam, S. (2016) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2016.0067 · Cited by 27 · see citing works
Self-citationDiscovering Time-Constrained Sequential Patterns for Music Genre Classification
Ren, J.-M., & Jang, J. R. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2172426 · Cited by 26 · see citing works
Spaulding, E. L., Robbins, M., Calupca, T., Clark, C. W., Tremblay, T., Waack, A., Warde, A. M., Kemp, J. N., & Newhall, K. (2009) · The Journal of the Acoustical Society of America · doi:10.1121/1.4783964 · Cited by 25 · see citing works
Spaulding, E. L., Robbins, M., Calupca, T. A., Clark, C. W., Tremblay, C. W., Waack, A., Warde, A. M., Kemp, J., & Newhall, K. (2009) · Proceedings of meetings on acoustics · doi:10.1121/1.3340128 · Cited by 25 · see citing works
Comparing textural features for music genre classification
Costa, Y. M. G., Oliveira, L. S., Koerich, A. L., & Gouyon, F. (2012) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252626 · Cited by 23 · see citing works
Self-citationBird species classification using spectrograms
Lucio, D. R., Maldonado, Y., & da Costa, G. (2015) · doi:10.1109/clei.2015.7359990 · Cited by 23 · see citing works
Nanni, L., Paci, M., dos Santos, F. L. C., Skottman, H., Juuti‐Uusitalo, K., & Hyttinen, J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0149399 · Cited by 23 · see citing works
Self-citationLanguage Identification Using Spectrogram Texture
Montalvo, A. R., Costa, Y. M. G., & Calvo, J. R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25751-8_65 · Cited by 23 · see citing works
Self-citationAutomatic Extraction Of Music Descriptors From Acoustic Signals.
Pachet, F., & Zils, A. (2004) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1416105 · Cited by 20 · see citing works
Bird Species Recognition by Wavelet Transformation of a Section of Birdsong
Chou, C.-H., & Liu, P.-H. (2009) · doi:10.1109/uic-atc.2009.85 · Cited by 19 · see citing works
Ensemble of Local Phase Quantization Variants with Ternary Encoding
Nanni, L., Brahnam, S., Lumini, A., & Barrier, T. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-39289-4_8 · Cited by 19 · see citing works
Self-citationOn the classification of underwater acoustic signals. II. Experimental applications involving fish
Deuser, L. M., Middleton, D., Plemons, T. D., & Vaughan, J. K. (1979) · The Journal of the Acoustical Society of America · doi:10.1121/1.382343 · Cited by 13 · see citing works
Bird species classification using visual and acoustic features extracted from audio signal
Lucio, D. R., & Costa, Y. M. G. (2016) · Proceedings/Proceedings - International Conference of the Chilean Computer Science Society · doi:10.1109/sccc.2016.7836063 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Classification of Heart Sound Signal Using Multiple Features
Yaseen, Y., Son, G., & Kwon, S. (2018) · Applied Sciences · doi:10.3390/app8122344 · Cited by 421 · see citing works
Ensemble of convolutional neural networks to improve animal audio classification
Nanni, L., Costa, Y. M. G., Aguiar, R. L., Mangolin, R. B., Brahnam, S., & Junior, C. N. S. (2020) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-020-00175-3 · Cited by 83 · see citing works
Self-citationEnsemble of deep learning, visual and acoustic features for music genre classification
Nanni, L., Costa, Y. M. G., Aguiar, R. L., Junior, C. N. S., & Brahnam, S. (2018) · Journal of New Music Research · doi:10.1080/09298215.2018.1438476 · Cited by 49 · see citing works
Self-citationAMResNet: An automatic recognition model of bird sounds in real environment
Xiao, H., Liu, D., Chen, K., & Zhu, M. (2022) · Applied Acoustics · doi:10.1016/j.apacoust.2022.109121 · Cited by 33 · see citing works
Kobayashi, K., Masuda, K., Haga, C., Matsui, T., Fukui, D., & Machimura, T. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101253 · Cited by 28 · see citing works
Mistry, Y. D., Birajdar, G. K., & Khodke, A. M. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14734-1 · Cited by 24 · see citing works
A novel method for detecting the pantograph–catenary arc based on the arc sound characteristics
Wei, W., Liang, C., Yang, Z., Xu, P., Yan, X., Gao, G., & Wu, G. (2018) · Proceedings of the Institution of Mechanical Engineers Part F Journal of Rail and Rapid Transit · doi:10.1177/0954409718799792 · Cited by 22 · see citing works
Automatic Classification of Monosyllabic and Multisyllabic Birds Using PDHF
Alghamdi, A., Mehtab, T., Iqbal, R., Leeza, M., Islam, N., Hamdi, M., & Shaikh, A. (2021) · Electronics · doi:10.3390/electronics10050624 · Cited by 9 · see citing works
On the Importance of Passive Acoustic Monitoring Filters
Aguiar, R. D. L., Maguolo, G., Nanni, L., Costa, Y. M. G., & Junior, C. N. S. (2021) · Journal of Marine Science and Engineering · doi:10.3390/jmse9070685 · Cited by 7 · see citing works
Self-citationTriplet dissimilarity: a texture classification approach using dissimilarity and siamese networks
Teixeira, L. O., Bertolini, D. S., de Oliveira, L. S., Cavalcanti, G. D. C., & Costa, Y. M. G. (2025) · Soft Computing · doi:10.1007/s00500-025-10677-x · Cited by 4 · see citing works
Self-citationNumata, Y., Sumali, B., Hayashida, K., Tsusaki, H., & Mitsukura, Y. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319897 · Cited by 3 · see citing works
Contrastive Dissimilarity for Writer Identification: a proof of concept
Pignelli, F., Teixeira, L. O., Bertolini, D. S., Nanni, L., Cavalcanti, G. D. C., & Costa, Y. M. G. (2025) · International Conference on Systems, Signals and Image Processing · doi:10.1109/iwssip66997.2025.11151867 · Cited by 2 · see citing works
Self-citationAcoustic Scene Classification by Linear Projections for Dimensionality Reduction
Golom, M. V., Foleis, J. H., Costa, Y. M. G., Goncalves, R. A. P., & Bertolini, D. S. (2023) · International Conference on Systems, Signals and Image Processing · doi:10.1109/iwssip58668.2023.10180239 · Cited by 1 · see citing works
Self-citationSignificance of Acoustic Features in Vertebrate Vocalisations Related Applications
Ramashini, M., Gamage, M. M. M., Murugiah, K., & Perumal, M. (2021) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-030-85773-8_2 · Cited by 1 · see citing works
Listening forward: emerging roles of bioacoustics in ecology, evolution, and conservation
Zsebők, S. (2026) · Biologia Futura · doi:10.1007/s42977-026-00322-5
“Researchers could now measure the fine structure of bird song, frog calls, and whale vocalizations rather than rely solely on the ear (Nanni et al. 2017; Narins et al. 2004; Rogers 1995).”
Whale Species Classifier Using Dorsal Fin And Lateral Body Characteristic
Pamu, K., & Allam, M. (2023) · International Conference on Computer Communication and Informatics (ICCCI) · doi:10.1109/iccci56745.2023.10128494
Automated System for Bird Species Identification Using CNN
Siri, D., Desu, S., Alladi, K., Swathi, D., Singh, S., & Srilakshmi, V. (2023) · E3S Web of Conferences · doi:10.1051/e3sconf/202343001057
Fusion of Visual and Acoustic Signals for Wildlife Recognition
Ong, C. T., Connie, T., Goh, M. K. O., & Choo, K. Y. (2023) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml59573.2023.10348333
Hancrafted Vs. Non-handcrafted Features for Acoustic Scene Classification
Castanho, J. V. S., & Costa, Y. M. G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-84353-2_24
Self-citationSoftware Feature Detection using Source Code Minimaps as Visual Signatures
Gebara, M., Wiese, I., & Costa, Y. M. G. (2025) · International Conference on Systems, Signals and Image Processing · doi:10.1109/iwssip66997.2025.11151888
Self-citationCombining Classifiers for Texture Classification
de Souza, M. G., Costa, Y. M. G., Britto, A. D. S., Foleis, J. H., & Bertolini, D. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-03708-4_25
Self-citationDepression Detection in Palographic Test Images
Polletti, J. L., Constantino, A. A., & Costa, Y. M. G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-43214-8_9
Self-citation
- Authors
- Nanni, L., Aguiar, R. L., Costa, Y. M. G., Brahnam, S., Silla, C. N., Brattin, R. L., & Zhao, Z.
- Affiliations
- University of Padua (IT)
- Universidade Estadual de Maringá (BR)
- Missouri State University (US)
- PPGIA Pontifical Catholic University of Paraná Curitiba Brazil
- CIS Missouri State University Springfield USA
- Purdue University West Lafayette (US)
- Venue
- IET Computer Vision
- Published
- 2017-12-22
- Volume
- 12
- Issue
- 2
- Pages
- 178-184
- ISSN
- 1751-9632 (ISSN-L)
- 1751-9640
- OpenAlex
- W2770554657
- Cited by
- 22
- References
- 55
- Citations by year
- 2026: 2 · 2025: 6 · 2023: 5 · 2022: 1 · 2021: 4 · 2020: 1 · 2018: 3
- FWCI
- 0.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IET Computer Vision (publishedVersion)
- BearWorks (Missouri State University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Padua Research Archive (University of Padova) (submittedVersion)
- Funders
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Fundação Araucária
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Developmental Biology › Animal Vocal Communication and Behavior
- Physical Sciences › Environmental Science › Ecology › Marine animal studies overview
- Keywords
- animal species identification, bird vocalizations, sound image, spectrogram, texture features, feature fusion, audiovisual integration, whale detection, music genre classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
