The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
POMDP-Based Statistical Spoken Dialog Systems: A Review
Abstract
Statistical dialog systems (SDSs) are motivated by the need for a data-driven framework that reduces the cost of laboriously handcrafting complex dialog managers and that provides robustness against the errors created by speech recognizers operating in noisy environments. By including an explicit Bayesian model of uncertainty and by optimizing the policy via a reward-driven process, partially observable Markov decision processes (POMDPs) provide such a framework. However, exact model representation and optimization is computationally intractable. Hence, the practical application of POMDP-based systems requires efficient algorithms and carefully constructed approximations. This review article provides an overview of the current state of the art in the development of POMDP-based spoken dialog systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
(2004) · doi:10.5555/1036843.1036906 · Cited by 417
Research works printed but not linked: 149
- andre, 2002, State abstraction for programmable reinforcement learning agents, Proc 18th Nat Conf Artif Intell, 119
- 10.1109/TASL.2007.902050 · DOI
- 10.1109/ASRU.2007.4430163 · DOI
- bishop, 2006, Pattern Recognition and Machine Learning
- 10.3115/1075096.1075127 · doi:10.3115/1075096.1075127
- bohus, 2006, A ‘K hypotheses + other’ belief updating model, Proc AAAI Workshop Statist Empirical Approaches Spoken Dialogue Syst, 13
- 10.1109/ASRU.2005.1566498 · DOI
- doshi, 2008, The permutable POMDP: Fast solutions to POMDPs for preference elicitation, Proc 1st Int Conf Autonomous Agents Multiagent Syst, 493
- crook, 2011, Lossless value directed compression of complex user goal states for statistical spoken dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 1029
- 10.1109/SLT.2010.5700896 · doi:10.1109/slt.2010.5700896
- 10.1016/j.csl.2009.07.003 · DOI · Paper page
- 10.1109/ICASSP.2008.4518765 · doi:10.1109/icassp.2008.4518765
- 10.3115/1557690.1557710 · doi:10.3115/1557690.1557710
- ji, 2007, Point-based policy iteration, Proc Nat Conf Artif Intell, 1243
- 10.1017/S1351324908005032 · DOI
- williams, 2005, Factored partially observable Markov decision processes for dialogue management, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 76
- 10.3115/1622064.1622088 · DOI
- 10.1109/ICASSP.2007.367185 · DOI
- 10.1109/ASRU.2005.1566539 · DOI
- gai, 2011, Effective handling of dialogue state in the hidden information state POMDP dialogue manager, ACM Trans Speech Lang Process, 7
- thomson, 2008, Evaluating semantic-level confidence scores with multiple hypotheses, Proc Annu Conf Int Speech Commun Assoc, 1153
- 10.1109/ICASSP.2010.5494939 · DOI
- frampton, 2006, Learning more effective dialogue strategies using limited dialogue move features, Proc Annu Meeting Assoc Comput Linguist, 185
- 10.1162/coli.2008.07-028-R2-05-82 · DOI
- horvitz, 1999, A Computational architecture for conversation, Proc 7th Int Conf User Model, 201
- heckerman, 1998, Inferring informational goals from free-text queries: A Bayesian approach, Proc Conf Uncertainty Artif Intell, 230
- 10.1109/TSA.2003.814380 · DOI
- heeman, 2007, Combining reinforcement learning with information-state update rules, Proc Conf Human Lang Technol North Amer Chapter Assoc Comput Linguist, 268
- pulman, 1996, Conversational games, belief revision and Bayesian networks, Proc CLIN VII 7th Comput Linguist Meeting
- 10.1109/ICASSP.2007.367189 · doi:10.1109/icassp.2007.367189
- 10.1007/978-3-540-30211-7_1 · doi:10.1007/978-3-540-30211-7_1
- 10.1109/ICME.2005.1521447 · DOI
- 10.3115/1220835.1220870 · doi:10.3115/1220835.1220870
- tetreault, 2006, Using reinforcement learning to build a better model of dialogue state, Proc of the Association for Computational Linguistics
- 10.1007/978-3-642-21043-3_11 · doi:10.1007/978-3-642-21043-3_11
- pinault, 2011, Unsupervised clustering of probability distributions of semantic frame graphs for pomdp-based spoken dialogue systems with summary space, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI
- pinault, 2011, Semantic graph clustering for POMDP-based spoken dialog systems, Proc Annu Conf Int Speech Commun Assoc, 1321
- boularias, 2010, Learning the reward model of dialogue POMDPs from data, Proc NIPS Workshop of Mach Learn for Assistive Tech
- pinault, 2009, Feature-based summary space for stochastic dialogue modeling with hierarchical semantic frames, Proc Annu Conf Int Speech Commun Assoc, 284
- bui, 2007, Practical dialogue manager development using POMDPs, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 215
- bui, 2007, A tractable DDN-POMDP approach to affective dialogue modeling for general probabilistic frame-based dialogue systems, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 34
- kurniawati, 2008, SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces, Proc Robot Sci Syst
- williams, 2008, The best of both worlds: Unifying conventional dialog systems and POMDPs, Proc Annu Conf Int Speech Commun Assoc, 1173
- 10.1109/MSP.2010.935874 · DOI
- 10.1109/5.880078 · DOI · Paper page
- 10.1016/j.specom.2008.03.010 · DOI
- oshry, 2009, Voice extensible markup language (VoiceXML) 3 0
- black, 2011, Spoken dialog challenge 2010: Comparison of live and control test results, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2
- henderson, 2005, Hybrid reinforcement/supervised learning for dialogue policies from communicator data, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 68
- 10.1016/S0167-6393(97)00021-6 · DOI · Paper page
- 10.3115/1075218.1075231 · DOI · Paper page
- 10.1109/SLT.2006.326775 · doi:10.1109/slt.2006.326775
- 10.3115/1289189.1289246 · DOI
- 10.1109/TSA.2005.855836 · DOI
- cuayhuitl, 2006, Learning multi-goal dialogue strategies using reinforcement learning with reduced state-action spaces, Proc Annu Conf Int Speech Commun Assoc, 547
- 10.1016/j.csl.2009.04.001 · DOI · Paper page
- lison, 2010, Towards relational POMDPs for adaptive dialogue management, Proc Annu Meeting Assoc Comput Linguist, 7
- 10.1145/1966407.1966412 · DOI
- li, 2009, Reinforcement learning for dialog management using least-squares policy iteration and fast feature selection, Proc Annu Conf Int Speech Commun Assoc, 2475
- 10.3115/1622064.1622098 · DOI
- hirschman, 1997, Overview of Evaluation in Speech and Natural Language Processing, 409
- williams, 2011, An empirical evaluation of a statistical dialog system in public use, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 130
- 10.3115/1556328.1556329 · doi:10.3115/1556328.1556329
- 10.1017/S0269888906000944 · DOI · Paper page
- misu, 2012, Reinforcement learning of question-answering dialogue policies for virtual museum guides, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 84
- 10.1109/ASRU.1997.658989 · DOI
- misu, 2011, Modeling spoken decision making dialogue and optimization of its dialogue strategy, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 221
- 10.3115/976909.979652 · DOI · Paper page
- 10.1145/1966407.1966411 · DOI
- schatzmann, 2005, quantitative evaluation of user simulation techniques for spoken dialogue systems, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 45
- singh, 1999, Reinforcement learning for spoken dialogue systems, Proc Neural Inf Process Syst
- 10.1016/j.specom.2008.05.007 · doi:10.1016/j.specom.2008.05.007
- hastie, 2012, Data-Driven Methods for Adaptive Spoken Dialogue Systems Computational Learning for Conversational Interfaces
- 10.1017/S0269888912000343 · DOI
- schatzmann, 2008, Statistical User and Error Modelling for Spoken Dialogue Systems
- gai, 2009, Back-off action selection in summary space-based POMDP dialogue systems, Proc IEEE Workshop Autom Speech Recog and Understanding, 456
- watambe, 1998, Evaluating dialogue strategies under communication errors using computer-to-computer simulation, IEICE Trans Inf Syst, e81 d, 1025
- 10.1109/ASRU.1997.658991 · DOI · Paper page
- 10.3115/1622064.1622097 · doi:10.3115/1622064.1622097
- georgila, 2005, Learning user simulations for information state update dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 893
- mcgraw, 2010, Collecting voices from the cloud, Proc Int Conf Lang Resources Eval, 1576
- jurek, 2011, Real user evaluation of spoken dialogue systems using amazon mechanical turk, Proc Annu Conf Int Speech Commun Assoc, 3061
- levin, 1997, A stochastic model of computer-human interaction for learning dialogue strategies, Proc EUROSPEECH, 1883, 10.21437/Eurospeech.1997-380 · DOI · Paper page
- zhang, 2001, Spoken dialogue management as planning and acting under uncertainty, Proc EUROSPEECH, 2169, 10.21437/Eurospeech.2001-511 · DOI
- 10.1007/978-3-642-30353-1_24 · DOI
- 10.1109/ICASSP.1998.674402 · DOI · Paper page
- williams, 2008, Integrating expert knowledge into POMDP optimization for spoken dialog systems, Proc AAAI Workshop Adv POMDP Solvers
- 10.1109/ICASSP.2011.5947627 · doi:10.1109/icassp.2011.5947627
- lefevre, 2009, <ref_formula><tex Notation="TeX">$k$</tex></ref_formula>-nearest neighbor Monte-Carlo control algorithm for POMDP-based dialogue systems, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- 10.3115/980691.980788 · DOI
- 10.1162/jmlr.2003.4.6.1107 · DOI
- kearns, 1999, Automatic detection of poor speech recognition at the dialogue level, Proc Annu Meeting Assoc Comput Linguist, 309
- 10.1007/BF00992696 · DOI · Paper page
- goddeau, 2000, Fast reinforcement learning of dialog strategies, Proc Int Conf Acoust Speech Signal Process, 1233
- 10.1162/089976698300017746 · DOI · Paper page
- 10.1098/rsta.2000.0593 · DOI
- singh, 2002, Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system, J Artif Intell Res, 16, 105, 10.1613/jair.859 · DOI · Paper page
- smith, 1994, Spoken Natural Language Dialog Systems A Practical Approach
- 10.1109/SLT.2010.5700878 · DOI
- denecke, 2004, Learning dialogue policies using state aggregation in reinforcement learning, Proc Annu Conf Int Speech Commun Assoc, 325
- de mori, 1998, Spoken Dialogues with Computers
- 10.1109/SLT.2010.5700863 · doi:10.1109/slt.2010.5700863
- 10.1109/ICASSP.2002.1005671 · doi:10.1109/icassp.2002.1005671
- minka, 2001, Expectation propagation for approximate Bayesian inference, Proc Conf Uncertainty Artif Intell, 362
- chandramohan, 2011, User simulation in dialogue systems using inverse reinforcement learning, Proc Annu Conf Int Speech Commun Assoc, 1025
- 10.3115/1557690.1557722 · doi:10.3115/1557690.1557722
- 10.1080/09540090802413145 · DOI
- 10.1016/j.csl.2009.03.002 · doi:10.1016/j.csl.2009.03.002
- 10.1109/SLT.2008.4777855 · doi:10.1109/slt.2008.4777855
- rossignol, 2011, Training a BN-based user model for dialogue simulation with missing data, Proc Int Joint Conf Natural Lang Process, 598
- williams, 2008, Exploiting the ASR N-best by tracking multiple dialog state hypotheses, Proc Annu Conf Int Speech Commun Assoc, 191
- 10.1109/TSA.2005.855836 · DOI
- 10.1109/TASL.2010.2076394 · DOI
- hua ai, 2007, Comparing user simulation models for dialog strategy learning, Proc Conf North Amer Chapt Assoc Comput Linguist, 1
- jurek, 2010, Natural belief-critic: A reinforcement algorithm for parameter estimation in statistical spoken dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 90
- thomson, 2009, Statistical Methods for Spoken Dialogue Management
- 10.1109/ASRU.2007.4430167 · doi:10.1109/asru.2007.4430167
- 10.1007/978-3-540-74628-7_74 · doi:10.1007/978-3-540-74628-7_74
- pietquin, 2005, Comparing ASR modeling methods for spoken dialogue simulation and optimal strategy learning, Proc Annu Conf Int Speech Commun Assoc, 861
- 10.1109/ICASSP.2006.1659954 · DOI
- young, 2002, Talking to machines (statistically speaking), Proc Int l Conf Spoken Language Processing, 9
- 10.1016/j.csl.2006.06.008 · DOI · Paper page
- 10.1016/S0004-3702(98)00023-X · DOI · Paper page
- hansen, 1998, Solving POMDPs by searching in policy space, Proc Conf Uncertainty Artif Intell, 211
- littman, 2002, Advances in Neural Information Processing Systems 14, 1555
- 10.1109/ADPRL.2009.4927543 · DOI
- pineau, 2003, Point-based value iteration: An anytime algorithm for POMDPs, Proc Int Joint Conf Artif Intell, 1025
- rieser, 2006, Cluster-based user simulations for learning dialogue strategies, Proc Annu Conf Int Speech Commun Assoc
- gai, 2011, On-line policy optimisation of spoken dialogue systems via live interaction with human subjects, Proc IEEE Workshop Autom Speech Recog and Understanding, 312
- 10.1109/ICASSP.2000.859185 · DOI
- spaan, 2005, Perseus: Randomized point-based value iteration for POMDPs, J Artif Intell Res, 24, 195, 10.1613/jair.1659 · DOI · Paper page
- 10.3115/1614108.1614146 · DOI · Paper page
- geist, 2011, Managing uncertainty within the KTD framework, Proc Workshop Active Learn Exp Design, 157
- smith, 2005, Point-based POMDP algorithms: Improved analysis and implementation, Proc 13th Annu Conf Uncertainty Artif Intell, 542
- pietquin, 2006, Consistent goal-directed user model for realistic man-machine task-oriented spoken dialogue simulation, Proc IEEE Int Conf Multimedia Expo, 425
- gai, 2011, Statistical Dialogue Modelling
- 10.1145/1102351.1102377 · DOI · Paper page
- gai, 2010, Gaussian processes for fast policy optimisation of a POMDP dialogue manager for a real-world task, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 201
- georgila, 2006, User simulation for spoken dialogue systems: Learning and evaluation, Proc Int Conf Spoken Lang Process, 45
- engel, 2005, Algorithms and Representations for Reinforcement Learning
- pietquin, 2011, Sample efficient on-line learning of optimal dialogue policies with Kalman temporal differences, Proc Int Joint Conf Artif Intell, 1878
- 10.1109/ASRU.2005.1566485 · DOI
- 10.3115/1564144.1564145 · DOI
- janarthanam, 2011, ‘The day after the day after tomorrow?’ A machine learning approach to adaptive temporal expression generation: Training and evaluation with real users, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 142
- kim, 2007, Multimodal dialog system using hidden information state dialog manager, Proc Int Conf Multimodal Interfaces Demonstration Session
- ai, 2007, Knowledge consistent user simulations for dialog systems, Proc Annu Conf Int Speech Commun Assoc, 2697
- 10.1109/TASL.2008.2012071 · DOI
- keizer, 2010, Parameter estimation for agenda-based user simulation, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 116
- pietquin, 2004, A Framework for Unsupervised Learning of Dialogue Strategies
Cited by
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I., Sordoni, A., Bengio, Y., Courville, A., & Pineau, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9883 · Cited by 1,615
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
“Recent advances in machine learning and a renewed interest in neural networks have led to the development of much more complex and efficient conversational agents.(9,40,41) The use of agent-based dialogue management appears to be rare in health applications.”
BackgroundDeep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1127 · Cited by 1,204
Metadata
- Authors
- Young, S., Gasic, M., Thomson, B., & Williams, J. D.
- Venue
- Proceedings of the IEEE
- Published
- 2013-05
- Volume
- 101
- Issue
- 5
- Pages
- 1160-1179
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W1975244201
- Cited by
- 882
- References
- 6
- FWCI
- 48.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: POMDP-Based Statistical Spoken Dialog Systems: A Review. https://app.cochat.ai/papers/10.1109/jproc.2012.2225812
- The paper
- Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013). POMDP-Based Statistical Spoken Dialog Systems: A Review. Proceedings of the IEEE. https://doi.org/10.1109/jproc.2012.2225812
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.