The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
This paper explains how Partially Observable Markov Decision Processes (POMDPs) can provide a principled mathematical framework for modelling the inherent uncertainty in spoken dialogue systems. It briefly summarises the basic mathematics and explains why exact optimisation is intractable. It then describes in some detail a form of approximation called the Hidden Information State model which does scale and which can be used to build practical systems. A prototype HIS system for the tourist information domain is evaluated and compared with a baseline MDP system using both user simulations and a live user trial. The results give strong support to the central contention that the POMDP-based framework is both a tractable and powerful approach to building more robust spoken dialogue systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
SCHEGLOFF, E. A., & SACKS, H. (1973) · Semiotica · doi:10.1515/semi.1973.8.4.289 · Cited by 4,589
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Point-based value iteration: an anytime algorithm for POMDPs
Pineau, J., Gordon, G., & Thrun, S. (2003) · International Joint Conference on Artificial Intelligence · Cited by 1,209
Partially observable Markov decision processes for spoken dialog systems
Williams, J. D., & Young, S. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2006.06.008 · Cited by 853
Self-citationElements of a plan-based theory of speech acts
COHEN, P., & PERRAULT, C. (1979) · Cognitive Science · doi:10.1016/s0364-0213(79)80006-3 · Cited by 724
Mangu, L., Brill, E., & Stolcke, A. (2000) · Computer Speech & Language · doi:10.1006/csla.2000.0152 · Cited by 646
A stochastic model of human-machine interaction for learning dialog strategies
Levin, E., Pieraccini, R., & Eckert, W. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.817450 · Cited by 516
Elements of a Plan‐Based Theory of Speech Acts*
Cohen, P. R., & Perrault, C. R. (1979) · Cognitive Science · doi:10.1207/s15516709cog0303_1 · Cited by 396
Agenda-based user simulation for bootstrapping a POMDP dialogue system
Schatzmann, J., Thomson, B., Weilhammer, K., Ye, H., & Young, S. (2007) · Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers on XX - NAACL '07 · doi:10.3115/1614108.1614146 · Cited by 332
Self-citationSpoken dialogue management using probabilistic reasoning
Roy, N., Pineau, J., & Thrun, S. (2000) · Proceedings of the 38th Annual Meeting on Association for Computational Linguistics - ACL '00 · doi:10.3115/1075218.1075231 · Cited by 285
Posterior probability decoding, confidence estimation and system combination
Evermann, G., & Woodland, P. (2000) · Cited by 233
Using Markov decision process for learning dialogue strategies
Levin, E., Pieraccini, R., & Eckert, W. (2002) · Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181) · doi:10.1109/icassp.1998.674402 · Cited by 177
The Optimal Control of Partially Observable Markov Decision Processes.
Sondik, E. J. (1971) · Medical Entomology and Zoology · Cited by 170
Understanding spontaneous speech: the Phoenix system
Ward, W. (1991) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1991.150352 · Cited by 147
Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems
Schatzmann, J., Georgila, K., & Young, S. (2005) · Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue · doi:10.18653/v1/2005.sigdial-1.6 · Cited by 145
Self-citationProbabilistic methods in spoken–dialogue systems
Young, S. J. (2000) · Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2000.0593 · Cited by 138
Self-citationTalking to machines (statistically speaking)
Young, S. (2002) · 7th International Conference on Spoken Language Processing (ICSLP 2002) · doi:10.21437/icslp.2002-2 · Cited by 136
Self-citationAutomating spoken dialogue management design using machine learning: An industry perspective
Paek, T., & Pieraccini, R. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.010 · Cited by 135
The Hidden Information State Approach to Dialog Management
Young, S. J., Schatzmann, J., Weilhammer, K., & Ye, H. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367185 · Cited by 133
Self-citationFlite: a small fast run-time synthesis engine.
Black, A. W., & Lenzo, K. (2001) · Cited by 122
Factored partially observable Markov decision processes for dialogue management
Williams, J. D., Poupart, P., & Young, S. J. (2005) · Cambridge University Engineering Department Publications Database · Cited by 120
Self-citationScaling POMDPs for Spoken Dialog Management
Williams, J. D., & Young, S. J. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.902050 · Cited by 116
Self-citationThe best of both worlds: unifying conventional dialog systems and POMDPs
Williams, J. D. (2008) · Interspeech (USB) · doi:10.21437/interspeech.2008-355 · Cited by 113
A heuristic variable grid solution method for POMDPs
Brafman, R. I. (1997) · National Conference on Artificial Intelligence · Cited by 105
Computational Models of Grounding in Collaborative Systems
Traum, D. R. (1999) · Cited by 96
USING POMDPS FOR DIALOG MANAGEMENT
Young, S. (2006) · 2006 IEEE Spoken Language Technology Workshop · doi:10.1109/slt.2006.326785 · Cited by 94
Self-citationAn epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes
Bonet, B. (2002) · International Conference on Machine Learning · Cited by 53
Understanding spontaneous speech
Ward, W. (1989) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/100964.100975 · Cited by 50
Training and evaluation of the HIS POMDP dialogue system in noise
Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K., & Young, S. J. (2008) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.3115/1622064.1622087 · Cited by 46
Self-citationBui, T., Poel, M., Nijholt, A., & Zwiers, J. (2009) · Natural Language Engineering · doi:10.1017/s1351324908005032 · Cited by 45
Using particle filters to track dialogue state
Williams, J. D. (2007) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2007.4430163 · Cited by 21
Cited by
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926
Deep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1127 · Cited by 1,204
Recent Trends in Deep Learning Based Natural Language Processing
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02709 · Cited by 1,192
A Persona-Based Neural Conversation Model
Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., & Dolan, B. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1094 · Cited by 898
Metadata
- Authors
- Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., & Yu, K.
- Venue
- Computer Speech & Language
- Publisher
- Elsevier BV
- Published
- 2010-04
- Volume
- 24
- Issue
- 2
- Pages
- 150-174
- ISSN
- 0885-2308 (ISSN-L)
- OpenAlex
- W2115101920
- Cited by
- 481
- References
- 33
- FWCI
- 38.31
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.