The DOI matches its Crossref record; 13 of the 14 references it cites were found in a registry or an index.
Recurrent neural network based language model
Abstract
A new recurrent neural network based language model (RNN LM) with applications to speech recognition is presented. Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model. Speech recognition experiments show around 18% reduction of word error rate on the Wall Street Journal task when comparing models trained on the same amount of data, and around 5% on the much harder NIST RT05 task, even when the backoff model is trained on much more data than the RNN LM. We provide ample empirical evidence to suggest that connectionist language models are superior to standard n-gram techniques, except their high computational (training) complexity. Index Terms: language modeling, recurrent neural networks, speech recognition
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“this line of research is an instance of Dynamic Evaluation (Mikolov et al., 2010; Krause et al., 2018; Clark et al., 2022; Rannen-Triki et al., 2024; Karami et al., 2025b;a), where model weights are updated at test time via gradient descent steps on a prediction loss.”
From MesaNet: Sequence Modeling by Locally Optimal Test-Time Training (2025) · cites this paper as Mikolov et al., 2010
In the citing paper
“D. C. Cire¸ san et al. (2010b, 2011) made substantial improvements over the previous state-of-the-art results on MNIST, whilst Mikolov et al. (2010) surpassed all competition at the time on the Wall Street Journal task, an NLP task.”
From Compute Trends Across Three Eras of Machine Learning (2022) · cites this paper as Mikolov et al. (2010)
Citations
Cites
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212 · cited 1×
“In our work, we have used an architecture that is usually called a simple recurrent neural network or Elman network [7].”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
(2005) · Cited by 823
A guide to recurrent neural networks and backpropagation
Mikael Bodén, A. (2001) · Cited by 246
Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model
Bengio, Y., & Senecal, J.-S. (2008) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.912312 · Cited by 223
Training neural network language models on very large corpora
Schwenk, H., & Gauvain, J. (2005) · doi:10.3115/1220575.1220601 · Cited by 123 · cited 3×
“Schwenk [4] describes several possible approaches that can be used for further performance improvements.”
BackgroundA Bit of Progress in Language Modeling Extended VersionPreprint
Goodman, J. (2001) · Cited by 113
The AMI System for the Transcription of Speech in Meetings
Hain, T., Burget, L., Dines, J., Garau, G., Wan, V., Karafiát, M., Vepa, J., & Lincoln, M. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366923 · Cited by 113
Self-citationThe 2005 AMI System for the Transcription of Speech in Meetings
Hain, T., Burget, L., Dines, J., Garau, G., Karafiát, M., Lincoln, M., McCowan, I. A., Moore, D. C., Wan, V., Ordelman, R., et al. (2006) · Lecture notes in computer science · doi:10.1007/11677482_38 · Cited by 70 · cited 2×
“To show that it is possible to obtain meaningful improvements in state of the art system, we experimented with lattices generated by AMI system used for NIST RT05 evaluation [13].”
MethodSelf-citationFast Text Compression with Neural Networks
Mahoney, M. V. (2000) · Cited by 44
Cited by
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
“Recurrent neural network based language model has been proposed to overcome certain limitations of the feedforward NNLM, such as the need to specify the context length (the order of the model N ), and because theoretically RNNs can efficiently represent more complex patterns than the shallow neural networks [14, 2].”
BackgroundSelf-citationDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Matching Networks for One Shot Learning
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04080 · Cited by 854
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
“, 2000), recurrent neural networks (Elman, 1990; Mikolov et al., 2010) and LSTMs (Hochreiter and Schmidhuber, 1997; Graves, 2013).”
BackgroundMethodInfluentialGrandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
“To manage the structured, combinatorial action space, the agent uses an auto-regressive policy 7,10,11 and recurrent pointer network 12 .”
BackgroundMethod
Metadata
- Authors
- Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S.
- Affiliations
- Brno University of Technology (CZ)
- Johns Hopkins University (US)
- Venue
- Interspeech 2010
- Publisher
- ISCA
- Published
- 2010-09-26
- Pages
- 1045-1048
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W179875071
- Cited by
- 5,390
- References
- 14
- Citations by year
- 2026: 66 · 2025: 160 · 2024: 279 · 2023: 404 · 2022: 418 · 2021: 526 · 2020: 741 · 2019: 780 · 2018: 649 · 2017: 504 · 2016: 343 · 2015: 254 · 2014: 147 · 2013: 66 · 2012: 35
- FWCI
- 30.77
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- recurrent neural networks, language models, perplexity reduction, speech recognition, word error rate, n-gram comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Recurrent neural network based language model. https://app.cochat.ai/papers/10.21437/interspeech.2010-343
- The paper
- Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010). Recurrent neural network based language model. Interspeech 2010. https://doi.org/10.21437/interspeech.2010-343
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.