Towards Better Decoding and Language Model Integration in Sequence to Sequence Models
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Abstract
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion.In this contribution, we analyse an attention-based seq2seq speech recognition system that directly transcribes recordings into characters.We observe two shortcomings: overconfidence in its predictions and a tendency to produce incomplete transcriptions when language models are used.We propose practical solutions to both problems achieving competitive speaker independent word error rates on the Wall Street Journal dataset: without separate language models we reach 10.6% WER, while together with a trigram language model, we reach 6.7% WER.
Overview
CoChat Summary
TL;DR
This paper analyzes two failure modes of an attention-based seq2seq speech recognizer: overconfident character predictions and incomplete transcripts when using language models. It proposes label smoothing plus decode-time coverage handling, reaching 10.6% WER without a separate language model and 6.7% WER with a trigram language model on WSJ.
Key findings
- The authors report that the baseline seq2seq model on WSJ achieved 14.2% WER on eval92, while unigram label smoothing reduced this to 10.6% and temporal/neighborhood smoothing to 10.7% on eval92.
- With an extended trigram language model and coverage, the best reported result was 6.7% WER on eval92; the corresponding dev93 score was 9.7% for temporal label smoothing + coverage.
- They found that model overconfidence makes beam search less effective and that increasing SoftMax temperature can help beam search, but without changing greedy decoding accuracy.
- For incomplete transcripts, simply constraining EOS was not sufficient at large beam widths; promoting coverage helped, while maximizing transcript length alone caused more word insertions and worse overall WER.
Methods
- Attention-based seq2seq speech recognition model built on Listen, Attend and Spell: a multilayer Bi-LSTM listener, an LSTM speller, and attention.
- Experiments were run on the Wall Street Journal dataset, training on si284, validating on dev93, and evaluating on eval92; inputs were 80-dimensional mel-scale filterbanks with first and second differences and per-speaker normalization.
- Main analysis compared decoding strategies and regularizers: SoftMax temperature changes, label smoothing variants (uniform/unigram and neighborhood/temporal), beam search settings, and a decode-time coverage penalty with an extended trigram language model.
Limitations
- The paper states that adding a language model introduces two challenges: model overconfidence makes balancing the beam-search terms difficult, and incomplete transcripts occur unless a coverage term is added.
- The authors note that simply maximizing transcript length yields more word insertion errors and worse overall WER.
- Evident from the text: results are reported on a single dataset (WSJ) with a particular ASR setup, so generalization beyond this setting is not established in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 7 Not checked · 37 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe two shortcomings: overconfidence in its predictions and a tendency to produce incomplete transcriptions when language models are used.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A width 100 beam search with a trigram language model finds the second transcript, which misses the beginning of the utterance.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we didn't observe consistent gains from this extension of the beam search criterion.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Temporal LS | 6.6M | 14.1 | 10.7 |”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, incomplete transcripts were produced unless a recording coverage term was added.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 46 claims and to verify the 9 references still open.
- It may state two separate claims as one. 16
- Probably not a substantial claim. 5
- The quote only partly states this claim. 2
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Deep learning [1] has led to many breakthroughs including speech and image recognition [2, 3, 4, 5, 6, 7].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“Networks were trained using 8 asynchronous replica workers each employing the ADAM algorithm [21] with default parameters and the learning rate set initially to 10−3, then reduced to 10−4 and 10−5 after 400k and 500k training steps, respectively.”
MethodRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“Especially, Sequence-to-Sequence (seq2seq) neural networks have proved to be very successful on complex transduction tasks, such as machine translation [8, 9, 10], speech recognition [11, 12, 13], or lip-reading [14].”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“Especially, Sequence-to-Sequence (seq2seq) neural networks have proved to be very successful on complex transduction tasks, such as machine translation [8, 9, 10], speech recognition [11, 12, 13], or lip-reading [14].”
BackgroundHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 1×
“Past attentions vectors were used as auxiliary inputs in the emitting RNN either directly [34], or as cumulative coverage information [35].”
MethodAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“Smoothing techniques increase the entropy of a model’s predictions, a technique that was used to promote exploration in reinforcement learning [27, 28, 29].”
MethodGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 2×
“Our solution that employs a coverage penalty at decode time only is most similar to the one used by the Google Translation system [10].”
BackgroundMethod
Research works printed but not linked: 2
- [1][1] Y. LeCun, Y. Bengio, and G. Hinton, 'Deep learning,' Nature , vol. 521, no. 7553, pp. 436-444, 2015. -
- [23][23] A. Graves and N. Jaitly, 'Towards End-To-End Speech Recognition with Recurrent Neural Networks,' in ICML '14 , 2014, pp. 17641772. -
Cited by
Items 301–398 of 398
Skansi, S. (2018) · Undergraduate topics in computer science · doi:10.1007/978-3-319-73004-2_9 · Cited by 2
Ihori, M., Masumura, R., Makishima, N., Tanaka, T., Takashima, A., & Orihashi, S. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.1 · Cited by 4
Graph Neural Network-Based Attribute Auxiliary Structured Grouping for Person Re-Identification
Tang, G., Gao, X., Chen, Z., & Zhong, H. (2021) · IEEE Access · doi:10.1109/access.2021.3069915 · Cited by 2
“And it is also widely utilized in speech recognition [39], and speech recognition [40].”
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.002 · Cited by 2
Guo, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86520-7_12 · Cited by 4
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.005 · Cited by 2
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.008 · Cited by 2
Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead ApproachPreprint
Wang, Y.-S., Kuo, Y., & Katz, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03716 · Cited by 2
KoSpeech: Open-Source Toolkit for End-to-End Korean Speech RecognitionPreprint
Kim, S., Bae, S., & Won, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03092 · Cited by 2
Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue SystemPreprint
Ao, S., & Acharya, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09625 · Cited by 2
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.017 · Cited by 2
Advancing Momentum Pseudo-Labeling with Conformer and Initialization StrategyPreprint
Higuchi, Y., Moritz, N., Roux, J. L., & Hori, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04948 · Cited by 2
“is less prone to the looping and early stopping issues often caused by autoregressive decoder networks [22, 31].”
BackgroundLabel Smoothing is Robustification against Model MisspecificationPreprint
Yamasaki, R., & Tanaka, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.08501 · Cited by 2
“Motivated by these supportive findings, LS has recently been actively adopted, together with a neural network model, in various modern applications such as speech recognition [4], machine translation [5], [6], image classification [7], [8], and visual tracking [9].”
Learning representations of speech from the raw waveform
Zeghidour, N. (2019) · theses.fr (ABES) · doi:10.70675/2d69ced4z8cd6z40f9z996dzacc325ed994f · Cited by 4
“The tendency is that the Hann-fixed setting consistently improves the results in LER and WER of both learnable frontends.”
BackgroundResultInfluentialBilingual Road Text Recognition Based on a Hybrid Model of CTC and Attention
Zaghdoud, R., Boukthir, K., Hamdani, T. M., & Alimi, A. M. (2024) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa63423.2024.10912568 · Cited by 1
“As a result, incorporating a language model has proven to be an effective enhancement method [11].”
LS 2 : Boosting Hidden Separation for Backdoor Defense with Learning Speed-driven Label Smoothing
Peng, J., Yang, H., He, H., Zhao, J., He, H., Dong, H., & Zhang, W. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3616621 · Cited by 1
“Label smoothing is a commonly used technique to improve the generalization of a DNN [36], [37], [38].”
A Vision-Based Subtitle Generator: Text Reconstruction via Subtle Vibrations from Videos
Wang, Y., Wang, Y., Zhang, X., & Ding, X. (2026) · Sensors · doi:10.3390/s26051407 · Cited by 1
“To prevent overfitting, the neighborhood smoothing strategy of [35] was employed, with the probability of the correct label set to 0.8.”
InfluentialImproving Sparse IMU-based Motion Capture with Motion Label Smoothing
Meng, Z., Lu, Y., Hou, Y., Chen, A., Guo, S., & Qin, Y. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i10.37749 · Cited by 1
An Analysis of Decoding for Attention-Based End-to-End Mandarin Speech Recognition
Jiang, D., Zou, W. P., Zhao, S., Yang, G., & Li, X. (2018) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp.2018.8706686 · Cited by 1
“Some previous works in attention decoding have shown that character error rate (CER) of attention-based model will increase if the beam size is too large [24, 13].”
BackgroundMethodInfluentialPark, J.-H., & Sung, W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3163 · Cited by 1
“This is mainly due to looping or skipping problems, which are frequently observed when the encoder of the attention model contains similar outputs at different time-steps [14].”
BackgroundAutomatic Segmented-Syllable and Deep Learning-Based Indonesian Audiovisual Speech Recognition
Suyanto, S., Ramadhani, K. N., Mandala, S., & Kurniawan, A. (2020) · International Conference on Interactive Digital Media (ICIDM) · doi:10.1109/icidm51048.2020.9339650 · Cited by 1
“Three models that are popular today are the Sequence-to-Sequence Model (S2SM) [7], [8]; Listen, Attend and Spell (LAS) [9], and Latent Sequence Decompositions (LSD) [10].”
BackgroundTwo-Stage Pre-Training for Sequence to Sequence Speech Recognition
Fan, Z., Zhou, S., & Xu, B. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534170 · Cited by 1
“A common approach to leverage text-only data is training a language model (LM), and fusing the output of the decoder and the pre-trained LM. Deep fusion [18] and shallow fusion [19] are two ways to integrate LM into the E2E system.”
MethodHybrid End-to-End Architecture for Hindi Speech Recognition System
Kumar, A., Choudhary, T., Dua, M., & Sabharwal, M. (2022) · Algorithms for intelligent systems · doi:10.1007/978-981-16-5747-4_23 · Cited by 1
End-to-End Generation of Written-style Transcript of Speech from Parliamentary Meetings
Mimura, M., & Kawahara, T. (2023) · Journal of Natural Language Processing · doi:10.5715/jnlp.30.88 · Cited by 1
Iterative Shallow Fusion of Backward Language Model for End-To-End Speech Recognition
Ogawa, A., Moriya, T., Kamo, N., Tawara, N., & Delcroix, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094660 · Cited by 3
Toward Developing Attention-Based End-To-End Automatic Speech Recognition
Ahmed, G., Lawaye, A. A., Mir, T. A., & Rana, P. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-4071-4_13 · Cited by 1
Zhang, W., Zhang, T., Luo, C. P., Zhou, H., Yang, C., Qian, X., & Yin, X.-C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887831 · Cited by 1
“A common approach involves generating N-best results from the acoustic model and applying shallow fusion with the language model to determine the highest-scoring sequence [19], [20].”
Lee, S., Chung, W., & Kang, H.-G. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34617 · Cited by 1
Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers
Stooke, A., Prabhavalkar, R., Sim, K. C., & Mengibar, P. M. (2024) · neural information processing systems · doi:10.52202/079017-3184 · Cited by 1
Meng, Q., Ren, P., Li, T., Dai, C., & Liang, H. (2026) · Smart innovation, systems and technologies · doi:10.1007/978-3-032-13048-8_11 · Cited by 1
Integration of TensorFlow based Acoustic Model with Kaldi WFST DecoderPreprint
Lim, M., & Kim, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11018 · Cited by 1
“Therefore, most TensorFlow-based speech recognition studies are related to convolutional neural network (CNN)-based word recognition, language modeling, and language translation [16, 17].”
BackgroundAudio-Visual Decision Fusion for WFST-based and seq2seq ModelsPreprint
Aralikatti, R. C., Roy, S., Thanda, A., Margam, D. K., Kandala, P. A., Sharma, T., & Venkatesan, S. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10832 · Cited by 1
“This is similar to shallow fusion of language models used in machine translation and speech recognition ([25] and [26]).”
Regularization via Adaptive Pairwise Label SmoothingPreprint
Guo, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01559 · Cited by 1
“…(LS) is a commonly used output distribution regularization technique to improve the generalization performance of deep learning models (Szegedy et al. 2016; Chorowski and Jaitly 2016; Vaswani et al. 2017; Zoph et al. 2018; Real et al. 2019; Huang et al. 2019; Li, Dasarathy, and Berisha 2020).”
BackgroundMethodLabel Smoothed Embedding Hypothesis for Out-of-Distribution DetectionPreprint
Bahri, D., Jiang, H., Tay, Y., & Metzler, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05131 · Cited by 2
“…(and therefore the generation quality of auto-regressive sequence models like machine translation) but has been seen to hurt teacher-to-student knowledge distillation (Pereyra et al., 2017; Xie et al., 2016b; Chorowski & Jaitly, 2016; Gao et al., 2020; Lukasik et al., 2020b; M¨uller et al., 2019).”
Bai, N., Luo, R., Nourian, P., & Roders, A. P. (2021) · arXiv (Cornell University) · Cited by 1
Deep Cross-Modal Alignment in Audio-Visual Speech Recognition
Sterpu, G. (2021) · Trinity's Access to Research Output (TARA) (Trinity College Dublin) · Cited by 2
Chi-square Loss for Softmax: an Echo of Neural Network StructurePreprint
Wang, Z., & Wang, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13822 · Cited by 1
“In subsequent work, label smoothing has been applied in many studies (such as Real et al. [6], Chorowski and Jaitly [7], Vaswani et al. [8]).”
MethodKoehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.004 · Cited by 1
Fusing Sentence Embeddings Into LSTM-based Autoregressive Language ModelsPreprint
Zouhar, V., Mosbach, M., & Klakow, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.02402 · Cited by 1
“A commonly used part of these pipelines is the decoding for which an autoregressive LM is essential [Chorowski and Jaitly, 2016].”
MethodAltering Backward Pass Gradients improves ConvergencePreprint
Das, B., Mondal, M., Lall, B., Joshi, S. D., & Roy, S. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12495 · Cited by 1
“Applications of logit suppression include: deep metric learning [42], non-parametric neighborhood component analysis [36], sequence-to-sequence modelling [6], reinforcement learning [14] and face recognition [19, 32].”
BackgroundThe Complexity of Sequential Prediction in Dynamical SystemsPreprint
Raman, V., Subedi, U., & Tewari, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.06614 · Cited by 1
“In particular, F could be a class of language models which use deterministic decoding strategies (e.g. greedy decoding) to output the next token (Chorowski and Jaitly, 2016).”
Regularization of ML models for Earth systems by using longer model timestepsPreprint
Parthipan, R., Anand, M., Christensen, H. M., Vitart, F., Wischik, D., & Jakob Zscheischler, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18023 · Cited by 2
“Label smoothing has improved accuracy across tasks, including image classification (Huang et al., 2019; Li et al., 2020; Real et al., 2019; Szegedy et al., 2016; Zoph et al., 2018), speech recognition (Chorowski and Jaitly, 2016) and in the original Transformer paper (Vaswani et al., 2017).”
Reinforcement learning for dialogue systems optimization with user adaptation
Carrara, N. (2019) · theses.fr (ABES) · doi:10.70675/84c4f883z5773z48d6zba77z4c53329fb3e4 · Cited by 3
Automatic Documentation of ICD Codes with Far-Field Speech RecognitionPreprint
Haque, A., & Fukushima, C. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.11046 · Cited by 2
“When a language model is used during inference but not training [10, 46], we call this shallow fusion .”
MethodMa, M. (2018) · CUNY Academic Works (City University of New York)
Modelling latvian language for automatic speech recognition
Salimbajevs, A. (2019) · E-resource repository of the University of Latvia (University of Latvia)
Character-Aware Attention-Based End-to-End Speech RecognitionPreprint
Meng, Z., Gaur, Y., Li, J., & Gong, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01795
“4 A label-smoothed cross-entropy [30] loss is minimized during training.”
BackgroundNormalized Label Distribution: Towards Learning Calibrated, Adaptable and Efficient Activation MapsPreprint
Uppal, U., & Giddwani, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.06876
“Deep Neural Networks (DNN) are being utilized for wide domain of applications including image classification [1], speech recognition [2], tackling climate change [3], and medical image segmentation [4, 5].”
Cheng, Y., & Xu, B. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428111
“A comprehensive approach to make use of a large amount of text corpus is training a language model (LM) and then performing fusion with the ASR model [11, 12].”
MethodKoehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.015
A Language Model Based Pseudo-Sample Deliberation for Semi-supervised Speech Recognition
Cheng, Y., Wang, J., Cheng, N., Zhou, S., & Xu, B. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533793
“The simplest way is to fuse an external LM, either shallowly [24] or deeply (cold fusion [25]).”
BackgroundSoft-BAC: Soft Bidirectional Alignment Cost for End-to-End Automatic Speech Recognition
Wang, Y., Zhang, H., Bao, F., & Gao, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89363-7_18
A Re-thinking ASR Modeling Framework using Attention Mechanisms
Yang, C.-Y., & Chen, K. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671417
Zhao, P., Tang, B., Liao, C., & Wang, X. (2021) · International Conference on Computational Intelligence and Security (CIS) · doi:10.1109/cis54983.2021.00033
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.016
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.013
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.001
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.007
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.010
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.020
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.011
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.012
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.006
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.014
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.019
Being Greedy Does Not Hurt: Sampling Strategies for End-To-End Speech Recognition
Heymann, J., Lakomkin, E., & Radel, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747344
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.021
Learning Category Distribution for Text Classification
Wang, X., & Zong, C. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3585279
“By focusing on the cross-entropy loss function rather than specific model architecture, label smoothing provides another way to improve the accuracy of the modern neural network models in many downstream tasks such as speech recognition [5], machine translation [22], and confidence calibration [23].”
BackgroundTowards robust natural language and image processing In low-resource scenarios
Liu, L. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/166293
Intermpl: Momentum Pseudo-Labeling With Intermediate CTC Loss
Higuchi, Y., Ogawa, T., Kobayashi, T., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097071 · Cited by 2
“Furthermore, CTC is robust against generating collapsed pseudo-labels, which is frequently caused by autoregressive decoding (e.g., word skipping or repeating [25,31]).”
BackgroundLABO: Towards Learning Optimal Label Regularization via Bi-level Optimization
Lu, P., Rashid, A., Kobyzev, I., Rezagholizadeh, M., & Langlais, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.356
Combining multiple end-to-end speech recognition models based on density ratio approach
Hojo, K., Mori, D., Wakabayashi, Y., Ohta, K., Ogawa, A., & Kitaoka, N. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317388 · Cited by 2
Fashion Label Relation Networks for Attribute Recognition
Wang, T., Huang, Y., & Qian, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-99-8850-1_28
Integration of WFST Language Model in Pre-trained Korean E2E ASR Model
Oh, J., Cho, E., & Kim, J.-H. (2024) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2024.06.015
Li, Y., Wang, Y., Bao, F., Gao, Z., Gao, G., & Zhang, W. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651520
“In addition, the selection, parameter tuning, and integration of external language models make the ASR system more complex [15–19].”
BackgroundBangla Grammatical Error Detection Leveraging Transformer-based Token ClassificationPreprint
Islam, S. B., Tanvir, R. H., & Afnan, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.08344
Automatic Post-editing of Speech Recognition System Output Using Large Language Models
Li, S., Li, J., & Cao, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0914-7_12
Riquelme, A. B. W. (2025) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.5821/dissertation-2117-439777
Beam-Limited k-Step Lookahead for Computationally Efficient HMM Decoding
Kurucan, M. (2025) · Çukurova Üniversitesi Mühendislik Fakültesi Dergisi · doi:10.21605/cukurovaumfd.1708178
Optimality and NP-Hardness of Transformers in Learning Markovian Dynamical Functions
Ding, Y., Lu, S., Lu, Y., Nowicki, T., & Gao, J. (2025) · neural information processing systems · doi:10.52202/085713-2825
Ao, S. (2025) · Open Research Online (The Open University) · doi:10.21954/ou.ro.00102247 · Cited by 2
Harmonic Analysis of Musical Audio using Deep Neural Networks
Korzeniowski, F. (2018) · University Library Linz repository (Johannes Kepler Universitat Linz)
Impact of Soft Segmentation Training on Medical Image Segmentation and Uncertainty Representation
Lemay, A. (2022) · PolyPublie (École Polytechnique de Montréal)
Lanera, C. (2020) · Padua@research (University of Padova)
Sequential decision modeling in uncertain conditions
Kastner, K. (2023) · Papyrus : Institutional Repository (Université de Montréal) · doi:10.71781/10652
Topics in Learning Theory: Prediction, Estimation, and Partial Information
Raman, V. (2025) · Deep Blue (University of Michigan) · doi:10.7302/28165
Essam, N., Ali, W., Wassif, K. T., Abdou, S. M., & Mobarz, H. B. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.01.037
Phoneme Overlapping-Aware Pre-Training with External Text Resources for Multi-Talker ASR
Masumura, R., Tanaka, T., Makishima, N., Ihori, M., Orihashi, S., Kawata, N., Yamane, T., Suzuki, S., & Moriya, T. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434597
“In language model (LM) fusion, LMs trained from external text resources are used to calibrate ASR scores [19], [20].”
Neural machine translation architectures and applications
Berard, A. (2018) · theses.fr (ABES) · doi:10.70675/1292fe5bz9e48z4cf2zb5f0zbba58cf4ac97 · Cited by 1
Boyer, F. (2021) · theses.fr (ABES) · doi:10.70675/bc00512aze508z4608zbf18z148205d22ec4 · Cited by 5
End-to-End Neural Approaches for Speech Translation
Nguyen, M. H. (2022) · theses.fr (ABES) · doi:10.70675/0ff8db61zd49fz46a2z93b1z23c229680847
Architectures et techniques d'entraînement pour la traduction parole-texte multilingue
Le, T. P. H. (2024) · theses.fr (ABES) · doi:10.70675/e8564bf6zaeb9z4a03zb003za2265a3b3370 · Cited by 1
Pelloin, V. (2024) · theses.fr (ABES) · doi:10.70675/7b83ca8ez7b8dz4317za876zc0ca8f18c5a1 · Cited by 2
Ait-Bachir, R., Bagat, R., Illina, I., & Emmanuel Vincent, A. (2026) · Journées d'Études sur la Parole (JEP) · doi:10.21437/jep.2026-41
IBNet: Inverted Bottleneck Network for Lightweight Automatic Speech Recognition
Kadkhodazadeh, N., Leitzell, O., Chand, S., Song, Y., Zuo, F., Wu, C., & Zhang, X. (2026) · IEEE/ACIS International Conference on Software Engineering Research, Management and Applications (SERA) · doi:10.1109/sera69989.2026.11618616
“…years have witnessed rapid progress in end-to-end (E2E) ASR systems, which replace traditional multi-component pipelines—consisting of separate acoustic, pronunciation, and language models—with a single neural network that directly maps acoustic features to character or subword sequences [6], [7].”
Ankeli, O. D., Luqman, H. A., Egbunike, T. N., Gottschalk, J., & Dossou, B. F. P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.05744
Metadata
- Authors
- Chorowski, J., & Jaitly, N.
- Affiliations
- Google (United States) (US)
- ORCID
- Chorowski, Jan
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 523-527
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2577366047
- Cited by
- 358
- References
- 34
- Citations by year
- 2026: 4 · 2025: 14 · 2024: 13 · 2023: 39 · 2022: 26 · 2021: 73 · 2020: 88 · 2019: 49 · 2018: 40 · 2017: 12
- FWCI
- 23.58
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Keywords
- attention-based models, language model integration, trigram language model, end-to-end speech recognition, speaker-independent recognition, Wall Street Journal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.