CoChat
Sign Up
PreprintCited by 4Music and Audio ProcessingMusic Technology and Sound StudiesSpeech and Audio Processing

The DOI matches its DataCite record.

BeatNet: CRNN and Particle Filtering for Online Joint Beat, Downbeat and Meter Tracking

Open access
Heydari, Mojtaba, Cwitkowitz, Frank, Duan, Zhiyao·2021·arXiv
DOI 10.48550/arxiv.2108.03576arXiv 2108.03576
Open this paper in CoChat — free accountOpen PDF

Abstract

The online estimation of rhythmic information, such as beat positions, downbeat positions, and meter, is critical for many real-time music applications. Musical rhythm comprises complex hierarchical relationships across time, rendering its analysis intrinsically challenging and at times subjective. Furthermore, systems which attempt to estimate rhythmic information in real-time must be causal and must produce estimates quickly and efficiently. In this work, we introduce an online system for joint beat, downbeat, and meter tracking, which utilizes causal convolutional and recurrent layers, followed by a pair of sequential Monte Carlo particle filters applied during inference. The proposed system does not need to be primed with a time signature in order to perform downbeat tracking, and is instead able to estimate meter and adjust the predictions over time. Additionally, we propose an information gate strategy to significantly decrease the computational cost of particle filtering during the inference step, making the system much faster than previous sampling-based methods. Experiments on the GTZAN dataset, which is unseen during training, show that the system outperforms various online beat and downbeat tracking systems and achieves comparable performance to a baseline offline joint method.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2108.03576

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. [1][1] D. Ellis, 'Beat tracking with dynamic programming,' Journal of New Music Research , vol. 36, no. 1, pp. 5160, 2007. -
  2. [3][3] F. Gouyon, A computational approach to rhythm description Audio features for the computation of rhythm periodicity functions and their use in tempo induction and music content processing. PhD thesis, Universitat Pompeu Fabra, 2005. -
  3. [4][4] S.Böck and S. Schedl, 'Enhanced beat tracking with context-aware neural networks,' in In Proc. ofthe 14th International Conference on Digital Audio Effects (DAFx-11) , 2011, pp. 135-140. -
  4. [5][5] M. E. P. Davies and S.Böck, 'Temporal convolutional networks for musical audio beat tracking,' in In Proc. of the 27th European Signal Processing Conference (EUSIPCO) , 2019. -
  5. [6][6] S.Böck, F. Krebs, and G.Widmer, 'A multi-model approach to beat tracking considering heterogeneous music styles,' in In Proc. of the 15th Intl. Conf. on Music Information Retrieval (ISMIR) , 2014, pp. 603-608. -
  6. [7][7] M. E. P. Davies, P. M. Brossier, and M. D. Plumbley, 'Beat tracking towards automatic musical accompaniment,' Audio Eng. Soc. Conv. Spring Prepr , vol. 2, pp. 751-757, 2005. -
  7. [8][8] A. Gkiokas and V. Katsouros, 'Convolutional neural networks for real-time beat tracking: A dancing robot application,' in In Proc. of the 18th Intl. Conf. on Music Information Retrieval (ISMIR) , 2017, pp. 286-293. -
  8. [9][9] P. M. Brossier, 'Automatic annotation of musical audio for interactive applications,' P. dissertation, Ed., Queen Marry University, London, UK, August 2006, pp. 58102. -
  9. [10][10] A. Mottaghi, K. Behdin, A. Esmaeili, M. Heydari, , and F. Marvasti, 'OBTAIN: Real-time beat tracking in audio signals index terms-onset strength signal, tempo estimation, beat onset, cumulative beat strength signal, peak detection,' International Journal of Signal Processing Systems , pp. 123-129, 2017. -
  10. [11][11] J. L. Oliveira, F. Gouyon, L. G. Martins, , and L. P. Reis, 'IBT: A real-time tempo and beat tracking system,' in In Proc. of the 11th Intl. Conf. on Music Information Retrieval (ISMIR) , 2014, pp. 291-296. -
  11. [12][12] M. Goto., 'AIST annotation for the RWC music database.' in In Proc. of the 7th Intl. Conf. on Music Information Retrieval (ISMIR) , 2006, pp. 359-360. -
  12. [13][13] M. Goto and Y. Muraoka, 'Real-time rhythm tracking for drumless audio signals: Chord change detection for musical decisions,' Speech Communication , vol. 27, no. 3, pp. 311-335, 1999. -
  13. [14][14] M. Goto and Y. Muraok, 'Music understanding at the beat level real-time beat tracking for audio signals,' in In Proceedings of IJCAI- 95 Workshop on Computational Auditory Scene Analysis , 1995, pp. 67-75. -
  14. [15][15] S. Durand, J. P. Bello, B. David, , and G. Richard, 'Robust downbeat tracking using an ensemble of convolutional networks,' Proceedings of the 17th International Society for Music Information Retrieval Conference, ISMIR 2016 , vol. 25, no. 1, pp. 255-261, 2017. -
  15. [16][16] S. Durand, J. Bello, B. D., and G. Richard, 'Downbeat tracking with multiple features and deep neural networks,' in In Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , 2015. -
  16. [17][17] S. Durand, J. P. Bello, B. D., and G. Richard, 'Feature adapted convolutional neural networks for downbeat tracking,' in In Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , 2016. -
  17. [18][18] B. D. Giorgi, M. Mauch, , and M. Levy, 'Downbeat tracking with tempo-invariant convolutional neural networks,' in In Proc. of the 17th Intl. Conf. on Music Information Retrieval (ISMIR) , 2020, pp. 216-222. -
  18. [19][19] G. Peeters and H. Papadopoulos, 'Simultaneous beat and downbeat-tracking using a probabilistic framework: Theory and large-scale evaluation,' IEEE Transactions on audio, speech, and language processing , vol. 19, no. 6, August 2011. -
  19. [20][20] S.Böck, F. Krebs, and G. Widmer, 'Joint beat and downbeat tracking with recurrent neural networks,' in In Proc. of the 7th Intl. Conf. on Music Information Retrieval (ISMIR) , 2016. -
  20. [21][21] F. Krebs, S. Böck, M. Dorfer, and G. Widmer, 'Downbeat tracking using beat-synchronous features and recurrent neural networks,' in In Proc. of the 17th Intl. Conf. on Music Information Retrieval (ISMIR) , 2016. -
  21. [22][22] M. Fuentes, B. Mcfee, H. C. Crayencour, S. Essid, and J. P. Bello, 'Analysis of common design choices in deep learning systems for downbeat tracking,' in Proc. of the 19th Int. Society for Music Information Retrieval Conf. , 2018. -
  22. [23][23] T. Cheng, S. Fukayama, and M. Goto, 'Joint beat and downbeat tracking based on CRNN models and a comparison of using different context ranges in convolutional layers,' in in Proc. of the International Computer Music Conference (ICMC) , 2016. -
  23. [24][24] S. Böck and M. E. P. Davies, 'Deconstruct, analyse, reconstruct: How to improve tempo, beat, and downbeat estimation,' in Proc. of the 21th Int. Society for Music Information Retrieval Conf. , 2020, pp. 574-582. -
  24. [25][25] S. Böck, F. Krebs, A. Durand, S. Pöll, and R. Balsyte, 'ROBOD: a real-time online beat and offbeat drummer sebastian,' in 2017 IEEE signal processing cup , 2017. -
  25. [26][26] C.-Y. Liang, 'Implementing and adapting a downbeat tracking system for real-time applications,' in Master Thesis , Carnegie Mellon University, 2017. -
  26. [27][27] S. Hainsworth and M. Macleod, 'Particle filtering applied to musical tempo tracking.' EURASIP Journal on Applied Signal Processing , vol. 15, pp. 2385-2395, 2004. -
  27. [28][28] S. Hainsworth and M. D. Macleod, 'Beat tracking with particle filtering algorithms,' in Proc. in the IEEE Workshop on Applications of Sienal Processing to Audio and Acoustics , 2003. -
  28. [29][29] A. T. Cemgil and B. Kappen, 'Monte carlo methods for tempo tracking and rhythm quantization,' Journal of Artificial Intelligence Research , vol. 18, pp. 111-222, 2003. -
  29. [30][30] F. Krebs, A. Holzapfel, A. T. Cemgil, and G. Widmer, 'Inferring metrical structure in music using particle filters,' IEEE/ACM Transactions on audio, speech, and language processing , vol. 23, no. 5, pp. 111-222, May 2015. -
  30. [31][31] M. Heydari and Z. Duan, 'Don't look back: An online beat tracking method using RNN and enhanced particle filtering,' in In Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , 2021. -
  31. [32][32] F. Krebs, S. Böck, and G. Widmer, 'An efficient statespace model for joint tempo and meter tracking,' in In Proc. of the 16th Intl. Conf. on Music Information Retrieval (ISMIR) , 2015. -
  32. [33][33] F. Gouyon, A. P. Klapuri, S. Dixon, M. Alonso, G. Tzanetakis, C. Uhle, and P. Cano., 'An experimental comparison of audio tempo induction algorithms,' IEEE Transactions on Audio, Speech, and Language Processing , vol. 14, no. 5, 2006. -
  33. [34][34] F. Krebs, S. Böck, and G. Widmer., 'Rhythmic pattern modeling for beat and downbeat tracking in musical audio,' in Proc. of the 14th Int. Society for Music Information Retrieval Conf. , 2013. -
  34. [35][35] A. Srinivasamurthy and X. Serra, 'A supervised approach to hierarchical metrical cycle tracking from audio music recordings,' in In Proc. of the IEEE Int. Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2014. -
  35. [36][36] U. Marchand and G. Peeters, 'Swing ratio estimation,' in In Proc. of the 18th Int. Conference on Digital Audio Effects (DAFx) , 2015. -
  36. [37][37] G. Tzanetakis and P. Cook, 'Musical genre classification of audio signals,' IEEE Transactions on Speech and Audio Processing, , vol. 10, no. 5, 2002. -
  37. [38][38] T. de Clercq and D. Temperley., 'A corpus analysis of rock harmony,' Popular Music , vol. 30, no. 1, pp. 4770, 2011. -
  38. [39][39] S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, , and G. Widmer, 'Madmom: A new python audio and music signal processing library,' in in Proc. ACM Multimed. Conf, MM 2016, , 2016, pp. 1174-1178.

Cited by

Sorted by citations

  • MOSS-Audio Technical Report

    Yang, C., Yu, C., Chen, H., Zhu, J., Chen, J., Chen, K., Wang, W., Wang, Y., Jiang, Y., Jiang, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.01802

    “A MIR pipeline based on Chordino [39], BeatNet [40], madmom [41], Essentia [42], JukeMIR [43], and related tools estimates chord sequences, beat and tempo statistics, key information, melody-related descriptors, and other low-level musical attributes.”

Metadata

Authors
Heydari, M., Cwitkowitz, F., & Duan, Z.
Affiliations
University of Rochester (US)
ORCID
Duan, Zhiyao
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2108.03576
arXiv
2108.03576
OpenAlex
W4302065815
Cited by
4
References
0
Citations by year
2023: 3 · 2022: 1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (1846184, 1922591)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
  • Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
beat tracking, convolutional recurrent neural network, particle filter, sequential Monte Carlo, causal inference, GTZAN

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review