The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Abstract
Audio-visual (AV)-automatic speech recognition (ASR) can improve speech recognition accuracy by using lip images, especially in noisy environments.The recently proposed AV Align system integrates speech and image features based on a cross-modal attention mechanism, where attention weights for visual features are estimated by using acoustic features as queries.Although AV Align shows an improvement in recognition accuracy in background noise environments, we have observed that the recognition accuracy degrades significantly in interference speaker environments, where a target speech and an interfering speech overlap each other.In order to improve the speech recognition accuracy of the target speaker in such situations, we propose a method that combines the auxiliary loss function that maximizes the recognition accuracy of the interference speaker and the CTC loss function for training the AV-ASR model.The experimental results using the TCD-TIMIT dataset show that the use of these auxiliary loss functions improves the performance of target-speaker speech recognition in interference speaker environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
Towards End-To-End Speech Recognition with Recurrent Neural Networks
Graves, A., & Jaitly, N. (2014) · Cited by 2,557
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Openface 2.0: facial behavior analysis toolkit
Baltrusaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00019 · Cited by 1,861
(2018) · doi:10.21437/interspeech.2018-1456 · Cited by 1,366
Joint CTC-attention based end-to-end speech recognition using multi-task learning
Kim, S., Hori, T., & Watanabe, S. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953075 · Cited by 844
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801
Purely Sequence-Trained Neural Networks for ASR Based on Lattice-Free MMI
Povey, D., Peddinti, V., Galvez, D., Ghahremani, P., Manohar, V., Na, X., Wang, Y., & Khudanpur, S. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-595 · Cited by 740
Audio-visual speech recognition using deep learning
Noda, K., Yamaguchi, Y., Nakadai, K., Okuno, H. G., & Ogata, T. (2015) · Applied Intelligence · doi:10.1007/s10489-014-0629-7 · Cited by 579
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
Chorowski, J., Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1602 · Cited by 362
End-to-End Audiovisual Speech Recognition
Petridis, S., Stafylakis, T., Ma, P., Cai, F., Tzimiropoulos, G., & Pantic, M. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461326 · Cited by 289
TCD-TIMIT: An Audio-Visual Corpus of Continuous Speech
Harte, N., & Gillen, E. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2407694 · Cited by 250
Hearing Lips and Seeing Voices
Boston, D. (1977) · British Journal of Audiology · doi:10.3109/03005367709078839 · Cited by 190
Audio-Visual Speech Recognition with a Hybrid CTC/Attention Architecture
Petridis, S., Stafylakis, T., Ma, P., Tzimiropoulos, G., & Pantić, M. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639643 · Cited by 180
Attention-based Audio-Visual Fusion for Robust Automatic Speech Recognition
Sterpu, G., Saam, C., & Harte, N. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3243014 · Cited by 90
How to Teach DNNs to Pay Attention to the Visual Modality in Speech Recognition
Sterpu, G., Saam, C., & Harte, N. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2980436 · Cited by 41
Auxiliary Interference Speaker Loss for Target-Speaker Speech Recognition
Kanda, N., Horiguchi, S., Takashima, R., Fujita, Y., Nagamatsu, K., & Watanabe, S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1126 · Cited by 38
Cited by
Practical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization
Abdulaal, A., Liu, Z., & Lancewicki, T. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467174 · Cited by 423
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 410
Jiao, R., Zhang, Y., Ding, L., Xue, B., Zhang, J., Cai, R., & Jin, C. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107840 · Cited by 345
COVID-19 cough classification using machine learning and global smartphone recordings
Pahar, M., Klopper, M., Warren, R., & Niesler, T. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104572 · Cited by 290
A comparative review on multi-modal sensors fusion based on deep learning
Tang, Q., Liang, J., & Zhu, F. (2023) · Signal Processing · doi:10.1016/j.sigpro.2023.109165 · Cited by 280
Metadata
- Publisher
- IEEE
- Published
- 2022
- OpenAlex
- W4288391450
- Cited by
- 926
- References
- 19
- Type
- paratext
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.