Distilled Binary Neural Network for Monaural Speech Separation
Abstract
Monaural speech separation, aiming at solving the cocktail party problem, has many important application scenarios, most of which ask for the real-time response, high energy efficiency and efficient storage. However, the state-of-the-art Deep Neural Network based separation models usually require huge memory and computation for the 32-bit floating point multiply accumulations, hence most of them cannot meet those requirements. Recently, there are many methods proposed to solve the problem, and binary neural networks have drawn many attentions for they compress and speed up its counterparts at the cost of some performance. Hence, in this paper, we binarize Deep Neural Network based separation models, aiming to deploy them on embedded devices for real-time applications. Furthermore, we improve the separation performance by integrating knowledge distillation into the training phase of binary neural network based models, which is referred as Distilled Binary Neural Network (DBNN). To the best of our knowledge, DBNN is the first attempt to integrate two types of model compression. In the experiments, we demonstrate the effectiveness of our proposed method, which successfully binarizes the Deep Neural Network based separation models with a comparable performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The first branch of solutions aims to minimize quantization error, the gap between real and quantized values.
“a) Quantization Error Minimization: The first branch of solutions aims to minimize the quantization error, i.e., the gap between the real values and the quantized values [19,34,62,103,151,158,164,169,178,218,248].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [34]
Binarization of wider networks can achieve a good trade-off between accuracy and model size.
“Inspired by the fact that binarizing the activations reduces their representational capability for the succeeding convolution block, [178] and [34] show that binarization of wider networks (i.e., networks with larger number of filters) can achieve a good trade-off between the accuracy and the model size.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [34]
Changing the width of the neural network architecture could reduce or remove the generalization gap after quantization.
“For instance, the recent work of [34] shows that changing the width of the NN architecture could reduce/remove generalization gap after quantization.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [34]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · see citing works
“…from full-precision teacher models has also been shown as a promising method to recover the accuracy degradation after binarization/ternarization [33, 173, 189, 252]. c) Improved Training Method: Another interesting branch of work aims for better training methods for binary/ternary models [5,…”
- Authors
- Chen, X., Liu, G., Shi, J., Xu, J., & Xu, B.
- Affiliations
- Chinese Academy of Sciences (CN)
- Venue
- 2018 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2018-07
- Pages
- 1-8
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2896555122
- Cited by
- 23
- References
- 2
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 3 · 2023: 4 · 2022: 5 · 2021: 3 · 2020: 5 · 2019: 1
- FWCI
- 1.72
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Funders
- National Natural Science Foundation of China
- Topics
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- monaural speech separation, binary neural networks, knowledge distillation, model compression, cocktail party problem, embedded deployment, energy efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
