TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids
Open accessAbstract
Modern speech enhancement algorithms achieve remarkable noise suppression by means of large recurrent neural networks (RNNs). However, large RNNs limit practical deployment in hearing aid hardware (HW) form-factors, which are battery powered and run on resource-constrained microcontroller units (MCUs) with limited memory capacity and compute capability. In this work, we use model compression techniques to bridge this gap. We define the constraints imposed on the RNN by the HW and describe a method to satisfy them. Although model compression techniques are an active area of research, we are the first to demonstrate their efficacy for RNN speech enhancement, using pruning and integer quantization of weights/activations. We also demonstrate state update skipping, which reduces the computational load. Finally, we conduct a perceptual evaluation of the compressed models to verify audio quality on human raters. Results show a reduction in model size and operations of 11.9$\times$ and 2.9$\times$, respectively, over the baseline for compressed models, without a statistical difference in listening preference and only exhibiting a loss of 0.55dB SDR. Our model achieves a computational latency of 2.39ms, well within the 10ms target and 351$\times$ better than previous work.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Authors in [97] developed a cloud-independent TinyML solution that relies only on data obtained from patients.
“The authors developed a cloud independent TinyML solution that merely relies on data obtained from patients.”
From A Comprehensive Survey on TinyML (2023) · cites this paper as [97]
Section 4.2 (p. 4) says the speech-enhancement models can run on a hearing-aid microcontroller, supporting the cloud-independent aspect. But Section 4, Experimental Results (p. 3), says the models were trained and evaluated on the CHiME2 WSJ0 dataset, not solely on data obtained from patients. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 33 · Cited by 116
Research works printed but not linked: 8
- [2][2] H. B. Abrams and J. Kihm, 'An introduction to marketrak ix: A new baseline for the hearing aid market,' Hearing Review , vol. 22, no. 6, p. 16, 2015. -
- [8][8] --, 'Tolerable hearing aid delays. ii. estimation of limits imposed during speech production, ear and hearing,' Ear and Hearing , vol. 23, no. 4, pp. 325-338, 2002. -
- [10][10] I. Fedorov, R. P. Adams, M. Mattina, and P. N. Whatmough, 'SpArSe: Sparse architecture search for CNNs on resourceconstrained microcontrollers,' in Advances in Neural Information Processing Systems (NeurIPS) , 2019, pp. 4978-4990. -
- [17][17] J. Wu, C. Yu, S. Fu, C. Liu, S. Chien, and Y. Tsao, 'Increasing compactness of deep learning based speech enhancement models with parameter pruning and quantization techniques,' IEEE Signal Processing Letters , vol. 26, no. 12, pp. 1887-1891, 2019. -
- [28][28] W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, 'Learning structured sparsity in deep neural networks,' in Advances in neural information processing systems , 2016, pp. 2074-2082. -
- [30][30] V. Campos, B. Jou, X. Gir'o i Nieto, J. Torres, and S. Chang, 'Skip RNN: learning to skip state updates in recurrent neural networks,' in International Conference on Learning Representations , 2018. -
- [33][33] J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, 'Sdrhalf-baked or well done?' in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2019, pp. 626-630. -
- [36][36] J. M. Vaisberg, A. Sabin, and S. Banerjee, 'Speech-in-noise benefits using Bose directional technology,' in American Academy of Audiology Conference , 2020. -
- Authors
- Fedorov, I., Stamenovic, M., Jensen, C., Yang, L.-C., Mandell, A., Gan, Y., Mattina, M., & Whatmough, P. N.
- Affiliations
- Bose (United States) (US)
- University of Rochester (US)
- Venue
- Interspeech 2020
- Publisher
- ISCA
- Published
- 2020-10-25
- Pages
- 4054-4058
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3028019732
- Cited by
- 116
- References
- 33
- Citations by year
- 2026: 8 · 2025: 23 · 2024: 22 · 2023: 19 · 2022: 24 · 2021: 17 · 2020: 3
- FWCI
- 9.85
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- speech enhancement, hearing aids, recurrent neural networks, model compression, weight pruning, integer quantization, resource-constrained deployment, perceptual evaluation, processing delay, noise reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
