The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Recurrent neural networks for language understanding
Abstract
Recurrent Neural Network Language Models (RNN-LMs) have recently shown exceptional performance across a variety of applications. In this paper, we modify the architecture to perform Language Understanding, and advance the state-of-the-art for the widely used ATIS dataset. The core of our approach is to take words as input as in a standard RNN-LM, and then to predict slot labels rather than words on the output side. We present several variations that differ in the amount of word context that is used on the input side, and in the use of non-lexical features. Remarkably, our simplest model produces state-of-the-art results, and we advance state-of-the-art through the use of bagof-words, word embedding, named-entity, syntactic, and wordclass features. Analysis indicates that the superior performance is attributable to the task-specific word representations learned by the RNN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
Self-citation(1992) · doi:10.5555/176313.176316 · Cited by 2,892
Extensions of recurrent neural network language model
Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., & Khudanpur, S. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947611 · Cited by 1,568
A Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 838
(2005) · Cited by 823
The ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790
Seide, F., Li, G., Chen, X., & Yu, D. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163899 · Cited by 632
Self-citationContext dependent recurrent neural network language model
Mikolov, T., & Zweig, G. (2012) · 2012 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424228 · Cited by 560
Self-citationChunking with support vector machines
Kudo, T., & Matsumoto, Y. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073361 · Cited by 520
Continuous space language models
Schwenk, H. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2006.09.003 · Cited by 516
Strategies for training large scale neural network language models
Mikolov, T., Deoras, A., Povey, D., Burget, L., & Cernocky, J. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163930 · Cited by 509
Mesnil, G., He, X., Deng, L., & Bengio, Y. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-596 · Cited by 413
Evaluation of spoken language systems: the ATIS domain
Price, P. J. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116612 · Cited by 385
Expanding the scope of the ATIS task: the ATIS-3 corpus
Dahl, D. A., Bates, M., Brown, M., Fisher, W., Hunicke-Smith, K., Pallett, D., Pao, C., Rudnicky, A., & Shriberg, E. (1994) · Proceedings of the workshop on Human Language Technology - HLT '94 · doi:10.3115/1075812.1075823 · Cited by 306
Generative and discriminative algorithms for spoken language understanding
Raymond, C., & Riccardi, G. (2007) · Interspeech 2007 · doi:10.21437/interspeech.2007-448 · Cited by 295
Empirical evaluation and combination of advanced language modeling techniques
Mikolov, T., Deoras, A., Kombrink, S., Burget, L., & Černocký, J. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-242 · Cited by 278
Conversational interfaces: advances and challenges
Zue, V. W., & Glass, J. R. (2000) · Proceedings of the IEEE · doi:10.1109/5.880078 · Cited by 245
Deep Neural Network Language Models
Arısoy, E., Sainath, T. N., Kingsbury, B. E. D., & Ramabhadran, B. (2012) · Cited by 174
Structured Output Layer neural network language model
Le, H.-S., Oparin, I., Allauzen, A., Gauvain, J., & Yvon, F. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947610 · Cited by 136
Continuous space language models for statistical machine translation
Schwenk, H., Dchelotte, D., & Gauvain, J. (2006) · International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) · doi:10.3115/1273073.1273166 · Cited by 131
Connectionist language modeling for large vocabulary continuous speech recognition
Schwenk, H., & Gauvain, J. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5743830 · Cited by 125
Use of kernel deep convex networks and end-to-end learning for spoken language understanding
Deng, L., Tür, G., He, X., & Hakkani-Tür, D. Z. (2012) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424224 · Cited by 115
A data-driven spoken language understanding system
He, Y., & Young, S. J. (2004) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2003.1318505 · Cited by 106
Large, Pruned or Continuous Space Language Models on a GPU for Statistical Machine Translation
Schwenk, H., Rousseau, A., & Attik, M. (2012) · HAL · Cited by 84
Spoken language understanding: a survey
Mori, R. D. (2007) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2007.4430139 · Cited by 67
Sentence simplification for spoken language understanding
Tür, G., Hakkani-Tür, D. Z., Heck, L. P., & Parthasarathy, S. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947636 · Cited by 56
Towards recurrent neural networks language models with linguistic and contextual features
Shi, Y., Wiggers, P., & Jonker, C. M. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-456 · Cited by 45
Self-citationA conversational movie search system based on conditional random fields
Liu, J., Cyphers, S., Pasupat, P., Mcgraw, I. C., & Glass, J. R. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-563 · Cited by 38
Combining statistical and knowledge-based spoken language understanding in conditional models
Wang, Y., Acero, A., Mahajan, M. V., & Lee, J. S. Y. (2006) · International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) · doi:10.3115/1273073.1273186 · Cited by 29
Measuring the Influence of Long Range Dependencies with Neural Network Language Models
Le, H.-S., Allauzen, A., & Yvon, F. (2012) · Cited by 25
Cited by
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
Self-citationDeep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Generative Adversarial Networks for Hyperspectral Image Classification
Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2805286 · Cited by 734
A Survey on Dialogue Systems: Recent Advances and New Frontiers
Chen, H., Liu, X., Yin, D., & Tang, J. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3166054.3166058 · Cited by 579
WiFi CSI Based Passive Human Activity Recognition Using Attention Based BLSTM
Chen, Z., Zhang, L., Jiang, C., Cao, Z., & Cui, W. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2878233 · Cited by 497
Metadata
- Authors
- Yao, K., Zweig, G., Hwang, M.-Y., Shi, Y., & Yu, D.
- Venue
- Interspeech 2013
- Publisher
- ISCA
- Published
- 2013-08-25
- Pages
- 2524-2528
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2400801499
- Cited by
- 315
- References
- 35
- FWCI
- 19.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.