The DOI matches its Crossref record; 143 of the 143 references it cites were found in a registry or an index.
A survey on the application of recurrent neural networks to statistical language modeling
Open accessAbstract
In this paper, we present a survey on the application of recurrent neural networks to the task of statistical language modeling. Although it has been shown that these models obtain good performance on this task, often superior to other state-of-the-art techniques, they suffer from some important drawbacks, including a very long training time and limitations on the number of context words that can be taken into account in practice. Recent extensions to recurrent neural network models have been developed in an attempt to address these drawbacks. This paper gives an overview of the most important extensions. Each technique is described and its performance on statistical language modeling, as described in the existing literature, is discussed. Our structured overview makes it possible to detect the most promising techniques in the field of recurrent neural networks, applied to language modeling, but it also highlights the techniques for which further research is required.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
WordNet: An Electronic Lexical Database
Kilgarriff, A., & Fellbaum, C. (2000) · Language · doi:10.2307/417141 · Cited by 11,464
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
The mathematics of statistical machine translation: parameter estimation
Brown, P. F., Pietra, V. J. D., Pietra, S. A. D., & Mercer, R. L. (1993) · Cited by 3,973
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,672
McClelland, J. L., Rumelhart, D. E., & AU (1987) · doi:10.7551/mitpress/5237.001.0001 · Cited by 3,315
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Popular Ensemble Methods: An Empirical Study
Opitz, D., & Maclin, R. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.614 · Cited by 2,953
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
(1992) · doi:10.5555/176313.176316 · Cited by 2,892
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,701
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Conference of the Association for Machine Translation in the Americas · Cited by 2,372
Boosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,306
Livescu, K., Bezman, A., Borges, N., Yung, L.-Y., Çetin, Ö., Frankel, J., King, S., Magimai.-Doss, M., Chi, X., & Lavoie, L. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,145
Neural Networks: A Systematic Introduction
Rojas, R. (1996) · Medical Entomology and Zoology · Cited by 2,138
A Novel Connectionist System for Unconstrained Handwriting Recognition
Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., & Schmidhuber, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.137 · Cited by 1,983
Statistical Methods for Speech Recognition
Sun, D. X., & Jelinek, F. (1999) · Journal of the American Statistical Association · doi:10.2307/2670189 · Cited by 1,946
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00117832 · Cited by 1,912
(2010) · Cited by 1,906
Reed, R. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248452 · Cited by 1,708
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Isabelle, P. (2002) · Cited by 1,633
Extensions of recurrent neural network language model
Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., & Khudanpur, S. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947611 · Cited by 1,568
The IAM-database: an English sentence database for offline handwriting recognition
Marti, U.-V., & Bunke, H. (2002) · International Journal on Document Analysis and Recognition · doi:10.1007/s100320200071 · Cited by 1,478
Elements of Information Theory
Phoenix, S. J. D. (1992) · Journal of Modern Optics · doi:10.1080/09500349214551641 · Cited by 1,380
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · A Field Guide to Dynamical Recurrent Networks · doi:10.1109/9780470544037.ch14 · Cited by 1,246
Serial Order: A Parallel Distributed Processing Approach
Jordan, M. I. (1997) · Advances in Psychology · doi:10.1016/s0166-4115(97)80111-2 · Cited by 1,242
A Maximum Likelihood Approach to Continuous Speech Recognition
Bahl, L. R., Jelinek, F., & Mercer, R. L. (1983) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1983.4767370 · Cited by 1,212
Discriminative training and maximum entropy models for statistical machine translation
Och, F. J., & Ney, H. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073133 · Cited by 1,034
Proceedings of Interspeech 2013
Lu, L., Ghoshal, A. K., & Renals, S. J. (2013) · Proceedings - International Symposium on Computer Architecture/Proceedings · Cited by 1,003
A stochastic parts program and noun phrase parser for unrestricted text
Church, K. W. (1988) · Proceedings of the second conference on Applied natural language processing - · doi:10.3115/974235.974260 · Cited by 957
Prechelt, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_5 · Cited by 951
Review on Methods to Fix Number of Hidden Neurons in Neural Networks
Sheela, K. G., & Deepa, S. N. (2013) · Mathematical Problems in Engineering · doi:10.1155/2013/425740 · Cited by 941
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Graves, A., Fernández, S., & Schmidhuber, J. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550907_126 · Cited by 925
Polak, E. (1997) · Applied Mathematical Sciences · doi:10.1007/978-1-4612-0663-7 · Cited by 914
(2005) · Cited by 823
Neural Networks: Tricks of the Trade
(1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8 · Cited by 810
Two decades of statistical language modeling: where do we go from here?
Rosenfeld, R. (2000) · Proceedings of the IEEE · doi:10.1109/5.880083 · Cited by 735
LSTM recurrent networks learn simple context-free and context-sensitive languages
Gers, F. A., & Schmidhuber, E. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963769 · Cited by 722
Boosting the margin: A new explanation for the effectiveness of voting methods
Schapire, R. E., Freund, Y., Barlett, P., & Lee, W. S. (1997) · Cited by 711
Vanderwende, L., Daumé, H., & Kirchhoff, K. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 685
Offline Arabic Handwriting Recognition with Multidimensional Recurrent Neural Networks
Graves, A. (2012) · Guide to OCR for Arabic Scripts · doi:10.1007/978-1-4471-4072-6_12 · Cited by 684
Proceedings of the International Conference on Acoustics Speech and Signal Processing
Bilbao, S., Arcas, K., & Chaigne, A. J. (2006) · International Conference on Acoustics, Speech, and Signal Processing · Cited by 635
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing
Lapata, M., & Ng, H. T. (2008) · Cited by 567
An overview of reservoir computing: theory, applications and implementations
Schrauwen, B., Verstraeten, D., & Campenhout, J. V. (2007) · Ghent University Academic Bibliography (Ghent University) · Cited by 553
Strategies for training large scale neural network language models
Mikolov, T., Deoras, A., Povey, D., Burget, L., & Cernocky, J. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163930 · Cited by 509
Optimal Linear Combinations of Neural Networks
Hashem, S. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00098-6 · Cited by 499
Jaeger, H. (2007) · Scholarpedia · doi:10.4249/scholarpedia.2330 · Cited by 459
Optimization. Algorithms and consistent approximations
Terlaky, T. (1998) · European Journal of Operational Research · doi:10.1016/s0377-2217(98)80004-0 · Cited by 457
Learning word embeddings efficiently with noise-contrastive estimation
Mnih, A., & Kavukcuoglu, K. (2013) · neural information processing systems · Cited by 446
An application of recurrent nets to phone probability estimation
Robinson, A. J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279192 · Cited by 441
Proceedings of the Fourteenth International Conference on Machine Learning
Fisher, D. (1997) · Cited by 422
Behaviour Analysis of Multilayer Perceptronswith Multiple Hidden Neurons and Hidden Layers
Panchal, G., Ganatra, A., Kosta, Y. P., & Panchal, D. (2011) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2011.v3.328 · Cited by 398
Graves, A., Fernández, S., & Gomez, F. (2017) · mediaTUM (Technical University of Munich) · Cited by 386
Proceedings of Interspeech 2008
Moore, J. D., Tietze, M. I., & Demberg, V. (2008) · Cited by 356
A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
Mnih, A., & Teh, Y. W. (2012) · arXiv · doi:10.48550/arxiv.1206.6426 · Cited by 315
Offline recognition of unconstrained handwritten texts using HMMs and statistical language models
Horst Bunke, A., Samy Bengio, A., & Vinciarelli, A. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.14 · Cited by 291
Empirical evaluation and combination of advanced language modeling techniques
Mikolov, T., Deoras, A., Kombrink, S., Burget, L., & Černocký, J. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-242 · Cited by 278
Adaptive Statistical Language Modeling: A Maximum Entropy Approach
Rosenfeld, R. (2018) · doi:10.1184/r1/6603032 · Cited by 269
Named entity recognition with long short-term memory
Hammerton, J. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119202 · Cited by 264
RNNLM - Recurrent Neural Network Language Modeling Toolkit
Mikolov, T., Kombrink, S., Deoras, A., Burget, L., & Černocký, J. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · Cited by 256
Lessons in neural network training: overfitting may be harder than expected
Lawrence, S., Giles, C. L., & Tsoi, A. C. (1997) · Cited by 242
Mooney, R. J. (2005) · Cited by 236
Connectionist probability estimators in HMM speech recognition
Renals, S., Morgan, N., Bourlard, H., Cohen, M., & Franco, H. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.260359 · Cited by 228
Proceedings of the 6th International Conference on Spoken Language Processing
Koumpis, K., & Renals, S. J. (2000) · Cited by 225
On-line Learning in Neural Networks
Biehl, M. L., Freking, A., Hölzer, M., Reents, G., & Schlösser, E. (1998) · Cited by 218
Decoding with Large-Scale Neural Language Models Improves Translation
Vaswani, A., Zhao, Y., Fossum, V., & Chiang, D. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1140 · Cited by 213
IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard
Liwicki, M., & Bunke, H. (2005) · Eighth International Conference on Document Analysis and Recognition (ICDAR'05) · doi:10.1109/icdar.2005.132 · Cited by 211
Recurrent neural network based language modeling in meeting recognition
Kombrink, S., Mikolov, T., Karafiát, M., & Burget, L. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-720 · Cited by 198
RECURRENT NEURAL NETWORKS ARE UNIVERSAL APPROXIMATORS
SCHÄFER, A. M., & ZIMMERMANN, H.-G. (2007) · International Journal of Neural Systems · doi:10.1142/s0129065707001111 · Cited by 196
ICDAR 2005 Arabic handwriting recognition competition
Margner, V., Pechwitz, M., & Abed, H. E. (2005) · Eighth International Conference on Document Analysis and Recognition (ICDAR'05) · doi:10.1109/icdar.2005.52 · Cited by 195
Explicit word error minimization in n-best list rescoring
Stolcke, A., Konig, Y., & Weintraub, M. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-68 · Cited by 153
Evaluation Metrics For Language Models
Chen, S. F., Beeferman, D., & Rosenfeld, R. (2018) · doi:10.1184/r1/6605324 · Cited by 148
Comparison of feedforward and recurrent neural network language models
Sundermeyer, M., Oparin, I., Gauvain, J.-L., Freiberg, B., Schlüter, R., & Ney, H. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639310 · Cited by 146
Reservoir-based techniques for speech recognition
Verstraeten, D., Schrauwen, B., & Stroobandt, D. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246804 · Cited by 145
Structured Output Layer neural network language model
Le, H.-S., Oparin, I., Allauzen, A., Gauvain, J., & Yvon, F. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947610 · Cited by 136
ICDAR 2009 Handwriting Recognition Competition
Grosicki, E., & Abed, H. E. (2009) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2009.184 · Cited by 133
Connectionist language modeling for large vocabulary continuous speech recognition
Schwenk, H., & Gauvain, J. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5743830 · Cited by 125
Training neural network language models on very large corpora
Schwenk, H., & Gauvain, J. (2005) · doi:10.3115/1220575.1220601 · Cited by 123
The IBM Attila speech recognition toolkit
Soltau, H., Saon, G., & Kingsbury, B. E. D. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700829 · Cited by 119
Parameter Adaptation in Stochastic Optimization
Almeida, L. B., Langlois, T., Amaral, J. D., & Plakhov, A. Y. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511569920.007 · Cited by 102
Can artificial neural networks learn language models?
Xu, W. H., & Rudnicky, A. I. (2000) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2000-50 · Cited by 95
ICDAR 2009-Arabic handwriting recognition competition
Abed, H. E., & Märgner, V. (2010) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-010-0117-5 · Cited by 92
Hussain, S., Jack, M. A., & McInnes, F. R. (1997) · Cited by 91
Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
Motuk, E., Bilbao, S., & Woods, R. (1996) · Cited by 90
The Expressive Power of Word EmbeddingsPreprint
Chen, Y., Perozzi, B., Al‐Rfou, R., et al. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3226 · Cited by 82
ICDAR 2009 Arabic Handwriting Recognition Competition
Märgner, V., & Abed, H. E. (2009) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2009.256 · Cited by 79
Online Arabic Handwriting Recognition Competition
Kherallah, M., Tagougui, N., Alimi, A. M., Abed, H. E., & Märgner, V. (2011) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2011.289 · Cited by 74
Continuous-Space Language Models for Statistical Machine Translation
Schwenk, H. (2010) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/v10108-010-0014-6 · Cited by 73
ICDAR 2009 Online Arabic Handwriting Recognition Competition
Abed, H. E., Märgner, V., Kherallah, M., & Alimi, A. M. (2009) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2009.284 · Cited by 71
The 2005 AMI System for the Transcription of Speech in Meetings
Hain, T., Burget, L., Dines, J., Garau, G., Karafiát, M., Lincoln, M., McCowan, I. A., Moore, D. C., Wan, V., Ordelman, R., et al. (2006) · Lecture notes in computer science · doi:10.1007/11677482_38 · Cited by 70
Random forests and the data sparseness problem in language modeling
Xu, P., & Jelinek, F. (2006) · Computer Speech & Language · doi:10.1016/j.csl.2006.01.003 · Cited by 63
Proceedings of the 8th European Conference on Speech Communication and Technology
Wrigley, S. N., Brown, G. J., Wan, V., & Renals, S. J. (2003) · Interspeech (USB) · Cited by 62
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Blunsom, P., Cohn, T., Goldwater, S. J., et al. (2009) · The Association for Computational Linguistics · Cited by 60
Liu, Z., Cai, J., & Buse, R. (2003) · Studies in fuzziness and soft computing · doi:10.1007/978-3-540-44850-1 · Cited by 59
Learning Representations for Weakly Supervised Natural Language Processing Tasks
Huang, F., Ahuja, A., Downey, D., Yang, Y., Guo, Y., & Yates, A. (2013) · Computational Linguistics · doi:10.1162/coli_a_00167 · Cited by 55
A study of n-gram and decision tree letter language modeling methods
Potamianos, G., & Jelinek, F. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00018-1 · Cited by 48
The latent words language model
Deschacht, K., Belder, J. D., & Moens, M. (2012) · Computer Speech & Language · doi:10.1016/j.csl.2012.04.001 · Cited by 46
Self-citationStudy on interaction between entropy pruning and kneser-ney smoothing
Chelba, C. I., Brants, T., Neveitt, W., & Xu, P. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-525 · Cited by 39
Arısoy, E., Chen, S. F., Ramabhadran, B., & Sethy, A. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639272 · Cited by 39
Scaling shrinkage-based language models
Chen, S. F., Mangu, L., Ramabhadran, B., Sarikaya, R., & Sethy, A. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5373380 · Cited by 36
Analyzing and predicting language model improvements
Iyer, R., Ostendorf, M., & Meteer, M. W. (2002) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.1997.659013 · Cited by 33
A Fast Re-scoring Strategy to Capture Long-Distance Dependencies
Deoras, A., Mikolov, T., & Church, K. (2011) · Cited by 33
A joint language model with fine-grain syntactic tags
Filimonov, D., & Harper, M. P. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699648.1699657 · Cited by 32
Accelerating recurrent neural network training via two stage classes and parallelization
Huang, Z., Zweig, G., Levit, M., Dumoulin, B., Oğuz, B., & Chang, S. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707751 · Cited by 31
Arısoy, E., Chen, S. F., Ramabhadran, B., & Sethy, A. (2013) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2013.2286919 · Cited by 31
de Vos, N. J. (2013) · Hydrology and earth system sciences · doi:10.5194/hess-17-253-2013 · Cited by 31
Self-supervised discriminative training of statistical language models
Xu, P., Karakos, D., & Khudanpur, S. P. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5373401 · Cited by 30
RNN language model with word clustering and class-based output layer
Shi, Y., Zhang, W.-Q., Liu, J., & Johnson, M. T. (2013) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/1687-4722-2013-22 · Cited by 29
Deoras, A., Mikolov, T., Kombrink, S., & Church, K. (2012) · Speech Communication · doi:10.1016/j.specom.2012.08.004 · Cited by 27
Measuring the Influence of Long Range Dependencies with Neural Network Language Models
Le, H.-S., Allauzen, A., & Yvon, F. (2012) · Cited by 25
Large vocabulary SOUL neural network language models
Le, H.-S., Oparin, I., Messaoudi, A., Allauzen, A., Gauvain, J., & Yvon, F. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-259 · Cited by 24
Proceedings of the 2010 Named Entities Workshop
Li, H., & Kumaran, A. (2010) · Cited by 24
Continuous Space Language Models using Restricted Boltzmann Machines
Niehues, J., & Waibel, A. (2012) · KITopen · doi:10.5445/ir/1000145058 · Cited by 23
On supervised learning from sequential data with applications for speech regognition
Schuster, M. (1999) · Institutional Repositories DataBase (IRDB) · doi:10.11501/3153839 · Cited by 22
POS Tags and Decision Trees for Language Modeling
Heeman, P. A. (1999) · Cited by 21
Random Clusterings for Language Modeling
Emami, A., & Jelinek, F. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415180 · Cited by 21
Phoneme boundary estimation using bidirectional recurrent neural networks and its applications
Fukada, T., Schuster, M., & Sagisaka, Y. (1999) · Systems and Computers in Japan · doi:10.1002/(sici)1520-684x(199904)30:4<20::aid-scj3>3.0.co;2-e · Cited by 20
Improved language modelling through better language model evaluation measures
Clarkson, P., & Robinson, T. (2001) · Computer Speech & Language · doi:10.1006/csla.2000.0156 · Cited by 11
Evaluating Benchmark Problems by Random Guessing
(2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544037.ch13 · Cited by 10
Gil, P. S., Cardoso, A., & Palma, L. B. (2009) · IEEE International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie.2009.5213122 · Cited by 5
Research works printed but not linked: 4
- Ahmed, 2009, Revised N-gram based automatic spelling correction tool to improve retrieval effectiveness, Polibits, 40, 39, 10.17562/PB-40-6 · doi:10.17562/pb-40-6
- Blanco, 2000, A genetic algorithm to obtain the optimal recurrent neural network, Int. J. Approx. Reason., 23, 67, 10.1016/S0888-613X(99)00032-8 · doi:10.1016/s0888-613x(99)00032-8
- Buabin, 2012, Boosted hybrid recurrent neural classifier for text document classification on the Reuters news text corpus, Int. J. Mach. Learn. Comput., 2, 588, 10.7763/IJMLC.2012.V2.195 · doi:10.7763/ijmlc.2012.v2.195
- Wang, 2005, Combining statistical language models via the latent maximum entropy principle, Mach. Learn., 60, 229, 10.1007/s10994-005-0928-7 · doi:10.1007/s10994-005-0928-7
Cited by
A Critical Review of Recurrent Neural Networks for Sequence Learning
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 1,996
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Learning to Diagnose with LSTM Recurrent Neural Networks
Lipton, Z. C., Kale, D. C., Elkan, C., & Wetzel, R. (2015) · arXiv · doi:10.48550/arxiv.1511.03677 · Cited by 879
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
García-Peñalvo, F. J. (2023) · Education in the Knowledge Society (EKS) · doi:10.14201/eks.31279 · Cited by 331
Metadata
- Authors
- De Mulder, W., Bethard, S., & Moens, M.-F.
- Venue
- Computer Speech & Language
- Publisher
- Elsevier BV
- Published
- 2015-03
- Volume
- 30
- Issue
- 1
- Pages
- 61-98
- ISSN
- 0885-2308 (ISSN-L)
- OpenAlex
- W2154137718
- Cited by
- 272
- References
- 143
- FWCI
- 10.23
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- creativecommons.org/licenses/by/3.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.