Improved neural network based language modelling and adaptation
Abstract
Neural network language models (NNLM) have become an increasingly popular choice for large vocabulary continuous speech recognition (LVCSR) tasks, due to their inherent gener-alisation and discriminative power. This paper present two tech-niques to improve performance of standard NNLMs. First, the form of NNLM is modelled by introduction an additional out-put layer node to model the probability mass of out-of-shortlist (OOS) words. An associated probability normalisation scheme is explicitly derived. Second, a novel NNLM adaptation method using a cascaded network is proposed. Consistent WER reduc-tions were obtained on a state-of-the-art Arabic LVCSR task over conventional NNLMs. Further performance gains were also observed after NNLM adaptation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It also becomes computationally demanding to normalize the resulting probability distribution correctly using a large n-gram model.
“It also becomes computationally demanding to normalize the resulting probability distribution correctly using a large n-gram model [34].”
From Automatic Speech Recognition With Very Large Conversational Finnish and Estonian Vocabularies (2017) · cites this paper as [34]
Cited by
Sorted by citations
Automatic Speech Recognition With Very Large Conversational Finnish and Estonian Vocabularies
Enarvi, S., Smit, P., Virpioja, S., & Kurimo, M. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2743344 · Cited by 33 · see citing works
“sulting probability distribution correctly using a large n-gram model [34].”
Method
- Authors
- Park, J., Liu, X., Gales, M. J. F., & Woodland, P. C.
- Affiliations
- University of Cambridge (GB)
- Venue
- Interspeech 2010
- Publisher
- ISCA
- Published
- 2010-09-26
- Pages
- 1041-1044
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W101286142
- Cited by
- 81
- References
- 0
- Citations by year
- 2022: 1 · 2021: 3 · 2020: 3 · 2019: 7 · 2018: 11 · 2017: 7 · 2016: 12 · 2015: 13 · 2014: 8 · 2013: 7 · 2012: 6
- FWCI
- 4.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- neural language models, large vocabulary continuous speech recognition, Arabic speech recognition, word error rate reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
