The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
An End-to-End Neural Network for Polyphonic Piano Music Transcription
Abstract
We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in a frame of audio. The language model is a recurrent neural network that models the correlations between pitch combinations over time. The proposed model is general and can be used to transcribe polyphonic music without imposing any constraints on the polyphony. The acoustic and language model predictions are combined using a probabilistic graphical model. Inference over the output variables is performed using the beam search algorithm. We perform two sets of experiments. We investigate various neural network architectures for the acoustic models and also investigate the effect of combining acoustic and music language model predictions using the proposed architecture. We compare performance of the neural network-based acoustic models with two popular unsupervised acoustic models. Results show that convolutional neural network acoustic models yield the best performance across all evaluation metrics. We also observe improved performance with the application of the music language models. Finally, we present an efficient variant of beam search that improves performance and reduces run-times by an order of magnitude, making the model suitable for real-time applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2014) · Cited by 33,419
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 2×
“The binary pitch outputs are obtained using Viterbi decoding [38], where the scaled likelihoods are used as emission probabilities.”
MethodResultAdhi Harmoko, S., M.Komp, Jacquard, J. M., Zuse, K., & Eniac (2005) · Cited by 18,035
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
(2011) · Cited by 6,993
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · cited 1×
“The RNN parameters θ = W fl , W rl , b l L 0 are calculated using the back propagation through time algo-rithm (BPTT) [32] and SGD.”
MethodLarge Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621
Introduction to algorithms: 4. Turtle graphics
Shyamasundar, R. K. (1996) · Resonance · doi:10.1007/bf02837777 · Cited by 1,467
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
LeCun, Y. A., Bottou, L., Orr, G. B., & Müller, K.-R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_3 · Cited by 1,264
Sequence Transduction with Recurrent Neural Networks
Graves, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.3711 · Cited by 1,170
Calculation of a constant Q spectral transform
Brown, J. C. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.400476 · Cited by 884
Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., & Penn, G. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288864 · Cited by 876
Non-negative matrix factorization for polyphonic music transcription
Smaragdis, P., & Brown, J. C. (2004) · 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684) · doi:10.1109/aspaa.2003.1285860 · Cited by 860
The Neural Autoregressive Distribution Estimator
Larochelle, H., & Murray, I. (2011) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 626
Connectionist learning of belief networks
Neal, R. M. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90065-6 · Cited by 513
Advances in optimizing recurrent networks
Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639349 · Cited by 469
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6392 · Cited by 459
Exploring convolutional neural network structures and optimization techniques for speech recognition
Abdel-Hamid, O., Deng, L., & Yu, D. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-744 · Cited by 384
Multiple fundamental frequency estimation based on harmonicity and spectral smoothness
Klapuri, A. P. (2003) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2003.815516 · Cited by 314
Learning to Transduce with Unbounded MemoryPreprint
Grefenstette, E., Hermann, K. M., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02516 · Cited by 305
Aggregate features and ADABOOST for music classification
Bergstra, J., Casagrande, N., Erhan, D., Eck, D., & Kégl, B. (2006) · Machine Learning · doi:10.1007/s10994-006-9019-7 · Cited by 279
A Discriminative Model for Polyphonic Piano Transcription
Poliner, G. E., & Ellis, D. P. W. (2006) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2007/48317 · Cited by 266
Adaptive Harmonic Spectral Decomposition for Multiple Pitch Estimation
Vincent, E., Bertin, N., & Badeau, R. (2010) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2009.2034186 · Cited by 237
Finding temporal structure in music: blues improvisation with LSTM recurrent networks
Eck, D., & Schmidhuber, J. (2003) · Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing · doi:10.1109/nnsp.2002.1030094 · Cited by 236
Bertin, N., Badeau, R., & Vincent, E. (2010) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2041381 · Cited by 200
Improved musical onset detection with Convolutional Neural Networks
Schluter, J., & Bock, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854953 · Cited by 194
Feature learning and deep architectures: new directions for music informatics
Humphrey, E. J., Bello, J. P., & LeCun, Y. (2013) · Journal of Intelligent Information Systems · doi:10.1007/s10844-013-0248-5 · Cited by 171
Audio Chord Recognition With Recurrent Neural Networks.
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2013) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1418318 · Cited by 152
Rethinking Automatic Chord Recognition with Convolutional Neural Networks
Humphrey, E. J., & Bello, J. P. (2012) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2012.220 · Cited by 151
Polyphonic music transcription by non-negative sparse coding of power spectra
Abdallah, S., & Plumbley, M. D. (2004) · Open Research (University of Surrey) · Cited by 139
Polyphonic piano note transcription with recurrent neural networks
Böck, S., & Schedl, M. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6287832 · Cited by 131
High-dimensional sequence transduction
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638244 · Cited by 100
Music Transcription with ISA and HMM
Emmanuel Vincent, A., & Xavier Rodet, A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30110-3_151 · Cited by 49
Audio Chord Recognition With A Hybrid Recurrent Neural Network.
Sigtia, S., Boulanger-Lewandowski, N., & Dixon, S. (2015) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1416593 · Cited by 48
Self-citationA hybrid recurrent neural network for music transcription
Sigtia, S., Benetos, E., Boulanger-Lewandowski, N., Weyde, T., Garcez, A. D., & Dixon, S. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178333 · Cited by 40
Self-citationAn Rnn-Based Music Language Model For Improving Automatic Music Transcription.
Sigtia, S., Benetos, E., Cherla, S., Weyde, T., Garcez, A. D., & Dixon, S. (2014) · City Research Online (City University London) · doi:10.5281/zenodo.1416791 · Cited by 20
Self-citation
Cited by
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Cakir, E., Parascandolo, G., Heittola, T., Huttunen, H., & Virtanen, T. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2690575 · Cited by 604
Convolutional recurrent neural networks for music classification
Choi, K., Fazekas, G., Sandler, M., & Cho, K. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952585 · Cited by 517
Deng, M., Meng, T., Cao, J., Wang, S., Zhang, J., & Fan, H. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.06.015 · Cited by 332
Metadata
- Authors
- Sigtia, S., Benetos, E., & Dixon, S.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2016-05
- Volume
- 24
- Issue
- 5
- Pages
- 927-939
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2198584637
- Cited by
- 280
- References
- 41
- FWCI
- 43.56
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.