Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 882

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

POMDP-Based Statistical Spoken Dialog Systems: A Review

Young, Steve, Gasic, Milica, Thomson, Blaise, Williams, Jason D.·2013·Proceedings of the IEEE

Abstract

Statistical dialog systems (SDSs) are motivated by the need for a data-driven framework that reduces the cost of laboriously handcrafting complex dialog managers and that provides robustness against the errors created by speech recognizers operating in noisy environments. By including an explicit Bayesian model of uncertainty and by optimizing the policy via a reward-driven process, partially observable Markov decision processes (POMDPs) provide such a framework. However, exact model representation and optimization is computationally intractable. Hence, the practical application of POMDP-based systems requires efficient algorithms and carefully constructed approximations. This review article provides an overview of the current state of the art in the development of POMDP-based spoken dialog systems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/jproc.2012.2225812

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 6 references

Research works printed but not linked: 149

  1. andre, 2002, State abstraction for programmable reinforcement learning agents, Proc 18th Nat Conf Artif Intell, 119
  2. 10.1109/TASL.2007.902050 · DOI
  3. 10.1109/ASRU.2007.4430163 · DOI
  4. bishop, 2006, Pattern Recognition and Machine Learning
  5. 10.3115/1075096.1075127 · doi:10.3115/1075096.1075127
  6. bohus, 2006, A ‘K hypotheses + other’ belief updating model, Proc AAAI Workshop Statist Empirical Approaches Spoken Dialogue Syst, 13
  7. 10.1109/ASRU.2005.1566498 · DOI
  8. doshi, 2008, The permutable POMDP: Fast solutions to POMDPs for preference elicitation, Proc 1st Int Conf Autonomous Agents Multiagent Syst, 493
  9. crook, 2011, Lossless value directed compression of complex user goal states for statistical spoken dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 1029
  10. 10.1109/SLT.2010.5700896 · doi:10.1109/slt.2010.5700896
  11. 10.1016/j.csl.2009.07.003 · DOI · Paper page
  12. 10.1109/ICASSP.2008.4518765 · doi:10.1109/icassp.2008.4518765
  13. 10.3115/1557690.1557710 · doi:10.3115/1557690.1557710
  14. ji, 2007, Point-based policy iteration, Proc Nat Conf Artif Intell, 1243
  15. 10.1017/S1351324908005032 · DOI
  16. williams, 2005, Factored partially observable Markov decision processes for dialogue management, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 76
  17. 10.3115/1622064.1622088 · DOI
  18. 10.1109/ICASSP.2007.367185 · DOI
  19. 10.1109/ASRU.2005.1566539 · DOI
  20. gai, 2011, Effective handling of dialogue state in the hidden information state POMDP dialogue manager, ACM Trans Speech Lang Process, 7
  21. thomson, 2008, Evaluating semantic-level confidence scores with multiple hypotheses, Proc Annu Conf Int Speech Commun Assoc, 1153
  22. 10.1109/ICASSP.2010.5494939 · DOI
  23. frampton, 2006, Learning more effective dialogue strategies using limited dialogue move features, Proc Annu Meeting Assoc Comput Linguist, 185
  24. 10.1162/coli.2008.07-028-R2-05-82 · DOI
  25. horvitz, 1999, A Computational architecture for conversation, Proc 7th Int Conf User Model, 201
  26. heckerman, 1998, Inferring informational goals from free-text queries: A Bayesian approach, Proc Conf Uncertainty Artif Intell, 230
  27. 10.1109/TSA.2003.814380 · DOI
  28. heeman, 2007, Combining reinforcement learning with information-state update rules, Proc Conf Human Lang Technol North Amer Chapter Assoc Comput Linguist, 268
  29. pulman, 1996, Conversational games, belief revision and Bayesian networks, Proc CLIN VII 7th Comput Linguist Meeting
  30. 10.1109/ICASSP.2007.367189 · doi:10.1109/icassp.2007.367189
  31. 10.1007/978-3-540-30211-7_1 · doi:10.1007/978-3-540-30211-7_1
  32. 10.1109/ICME.2005.1521447 · DOI
  33. 10.3115/1220835.1220870 · doi:10.3115/1220835.1220870
  34. tetreault, 2006, Using reinforcement learning to build a better model of dialogue state, Proc of the Association for Computational Linguistics
  35. 10.1007/978-3-642-21043-3_11 · doi:10.1007/978-3-642-21043-3_11
  36. pinault, 2011, Unsupervised clustering of probability distributions of semantic frame graphs for pomdp-based spoken dialogue systems with summary space, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI
  37. pinault, 2011, Semantic graph clustering for POMDP-based spoken dialog systems, Proc Annu Conf Int Speech Commun Assoc, 1321
  38. boularias, 2010, Learning the reward model of dialogue POMDPs from data, Proc NIPS Workshop of Mach Learn for Assistive Tech
  39. pinault, 2009, Feature-based summary space for stochastic dialogue modeling with hierarchical semantic frames, Proc Annu Conf Int Speech Commun Assoc, 284
  40. bui, 2007, Practical dialogue manager development using POMDPs, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 215
  41. bui, 2007, A tractable DDN-POMDP approach to affective dialogue modeling for general probabilistic frame-based dialogue systems, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 34
  42. kurniawati, 2008, SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces, Proc Robot Sci Syst
  43. williams, 2008, The best of both worlds: Unifying conventional dialog systems and POMDPs, Proc Annu Conf Int Speech Commun Assoc, 1173
  44. 10.1109/MSP.2010.935874 · DOI
  45. 10.1109/5.880078 · DOI · Paper page
  46. 10.1016/j.specom.2008.03.010 · DOI
  47. oshry, 2009, Voice extensible markup language (VoiceXML) 3 0
  48. black, 2011, Spoken dialog challenge 2010: Comparison of live and control test results, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2
  49. henderson, 2005, Hybrid reinforcement/supervised learning for dialogue policies from communicator data, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI, 68
  50. 10.1016/S0167-6393(97)00021-6 · DOI · Paper page
  51. 10.3115/1075218.1075231 · DOI · Paper page
  52. 10.1109/SLT.2006.326775 · doi:10.1109/slt.2006.326775
  53. 10.3115/1289189.1289246 · DOI
  54. 10.1109/TSA.2005.855836 · DOI
  55. cuayhuitl, 2006, Learning multi-goal dialogue strategies using reinforcement learning with reduced state-action spaces, Proc Annu Conf Int Speech Commun Assoc, 547
  56. 10.1016/j.csl.2009.04.001 · DOI · Paper page
  57. lison, 2010, Towards relational POMDPs for adaptive dialogue management, Proc Annu Meeting Assoc Comput Linguist, 7
  58. 10.1145/1966407.1966412 · DOI
  59. li, 2009, Reinforcement learning for dialog management using least-squares policy iteration and fast feature selection, Proc Annu Conf Int Speech Commun Assoc, 2475
  60. 10.3115/1622064.1622098 · DOI
  61. hirschman, 1997, Overview of Evaluation in Speech and Natural Language Processing, 409
  62. williams, 2011, An empirical evaluation of a statistical dialog system in public use, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 130
  63. 10.3115/1556328.1556329 · doi:10.3115/1556328.1556329
  64. 10.1017/S0269888906000944 · DOI · Paper page
  65. misu, 2012, Reinforcement learning of question-answering dialogue policies for virtual museum guides, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 84
  66. 10.1109/ASRU.1997.658989 · DOI
  67. misu, 2011, Modeling spoken decision making dialogue and optimization of its dialogue strategy, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 221
  68. 10.3115/976909.979652 · DOI · Paper page
  69. 10.1145/1966407.1966411 · DOI
  70. schatzmann, 2005, quantitative evaluation of user simulation techniques for spoken dialogue systems, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 45
  71. singh, 1999, Reinforcement learning for spoken dialogue systems, Proc Neural Inf Process Syst
  72. 10.1016/j.specom.2008.05.007 · doi:10.1016/j.specom.2008.05.007
  73. hastie, 2012, Data-Driven Methods for Adaptive Spoken Dialogue Systems Computational Learning for Conversational Interfaces
  74. 10.1017/S0269888912000343 · DOI
  75. schatzmann, 2008, Statistical User and Error Modelling for Spoken Dialogue Systems
  76. gai, 2009, Back-off action selection in summary space-based POMDP dialogue systems, Proc IEEE Workshop Autom Speech Recog and Understanding, 456
  77. watambe, 1998, Evaluating dialogue strategies under communication errors using computer-to-computer simulation, IEICE Trans Inf Syst, e81 d, 1025
  78. 10.1109/ASRU.1997.658991 · DOI · Paper page
  79. 10.3115/1622064.1622097 · doi:10.3115/1622064.1622097
  80. georgila, 2005, Learning user simulations for information state update dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 893
  81. mcgraw, 2010, Collecting voices from the cloud, Proc Int Conf Lang Resources Eval, 1576
  82. jurek, 2011, Real user evaluation of spoken dialogue systems using amazon mechanical turk, Proc Annu Conf Int Speech Commun Assoc, 3061
  83. levin, 1997, A stochastic model of computer-human interaction for learning dialogue strategies, Proc EUROSPEECH, 1883, 10.21437/Eurospeech.1997-380 · DOI · Paper page
  84. zhang, 2001, Spoken dialogue management as planning and acting under uncertainty, Proc EUROSPEECH, 2169, 10.21437/Eurospeech.2001-511 · DOI
  85. 10.1007/978-3-642-30353-1_24 · DOI
  86. 10.1109/ICASSP.1998.674402 · DOI · Paper page
  87. williams, 2008, Integrating expert knowledge into POMDP optimization for spoken dialog systems, Proc AAAI Workshop Adv POMDP Solvers
  88. 10.1109/ICASSP.2011.5947627 · doi:10.1109/icassp.2011.5947627
  89. lefevre, 2009, <ref_formula><tex Notation="TeX">$k$</tex></ref_formula>-nearest neighbor Monte-Carlo control algorithm for POMDP-based dialogue systems, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue
  90. 10.3115/980691.980788 · DOI
  91. 10.1162/jmlr.2003.4.6.1107 · DOI
  92. kearns, 1999, Automatic detection of poor speech recognition at the dialogue level, Proc Annu Meeting Assoc Comput Linguist, 309
  93. 10.1007/BF00992696 · DOI · Paper page
  94. goddeau, 2000, Fast reinforcement learning of dialog strategies, Proc Int Conf Acoust Speech Signal Process, 1233
  95. 10.1162/089976698300017746 · DOI · Paper page
  96. 10.1098/rsta.2000.0593 · DOI
  97. singh, 2002, Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system, J Artif Intell Res, 16, 105, 10.1613/jair.859 · DOI · Paper page
  98. smith, 1994, Spoken Natural Language Dialog Systems A Practical Approach
  99. 10.1109/SLT.2010.5700878 · DOI
  100. denecke, 2004, Learning dialogue policies using state aggregation in reinforcement learning, Proc Annu Conf Int Speech Commun Assoc, 325
  101. de mori, 1998, Spoken Dialogues with Computers
  102. 10.1109/SLT.2010.5700863 · doi:10.1109/slt.2010.5700863
  103. 10.1109/ICASSP.2002.1005671 · doi:10.1109/icassp.2002.1005671
  104. minka, 2001, Expectation propagation for approximate Bayesian inference, Proc Conf Uncertainty Artif Intell, 362
  105. chandramohan, 2011, User simulation in dialogue systems using inverse reinforcement learning, Proc Annu Conf Int Speech Commun Assoc, 1025
  106. 10.3115/1557690.1557722 · doi:10.3115/1557690.1557722
  107. 10.1080/09540090802413145 · DOI
  108. 10.1016/j.csl.2009.03.002 · doi:10.1016/j.csl.2009.03.002
  109. 10.1109/SLT.2008.4777855 · doi:10.1109/slt.2008.4777855
  110. rossignol, 2011, Training a BN-based user model for dialogue simulation with missing data, Proc Int Joint Conf Natural Lang Process, 598
  111. williams, 2008, Exploiting the ASR N-best by tracking multiple dialog state hypotheses, Proc Annu Conf Int Speech Commun Assoc, 191
  112. 10.1109/TSA.2005.855836 · DOI
  113. 10.1109/TASL.2010.2076394 · DOI
  114. hua ai, 2007, Comparing user simulation models for dialog strategy learning, Proc Conf North Amer Chapt Assoc Comput Linguist, 1
  115. jurek, 2010, Natural belief-critic: A reinforcement algorithm for parameter estimation in statistical spoken dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 90
  116. thomson, 2009, Statistical Methods for Spoken Dialogue Management
  117. 10.1109/ASRU.2007.4430167 · doi:10.1109/asru.2007.4430167
  118. 10.1007/978-3-540-74628-7_74 · doi:10.1007/978-3-540-74628-7_74
  119. pietquin, 2005, Comparing ASR modeling methods for spoken dialogue simulation and optimal strategy learning, Proc Annu Conf Int Speech Commun Assoc, 861
  120. 10.1109/ICASSP.2006.1659954 · DOI
  121. young, 2002, Talking to machines (statistically speaking), Proc Int l Conf Spoken Language Processing, 9
  122. 10.1016/j.csl.2006.06.008 · DOI · Paper page
  123. 10.1016/S0004-3702(98)00023-X · DOI · Paper page
  124. hansen, 1998, Solving POMDPs by searching in policy space, Proc Conf Uncertainty Artif Intell, 211
  125. littman, 2002, Advances in Neural Information Processing Systems 14, 1555
  126. 10.1109/ADPRL.2009.4927543 · DOI
  127. pineau, 2003, Point-based value iteration: An anytime algorithm for POMDPs, Proc Int Joint Conf Artif Intell, 1025
  128. rieser, 2006, Cluster-based user simulations for learning dialogue strategies, Proc Annu Conf Int Speech Commun Assoc
  129. gai, 2011, On-line policy optimisation of spoken dialogue systems via live interaction with human subjects, Proc IEEE Workshop Autom Speech Recog and Understanding, 312
  130. 10.1109/ICASSP.2000.859185 · DOI
  131. spaan, 2005, Perseus: Randomized point-based value iteration for POMDPs, J Artif Intell Res, 24, 195, 10.1613/jair.1659 · DOI · Paper page
  132. 10.3115/1614108.1614146 · DOI · Paper page
  133. geist, 2011, Managing uncertainty within the KTD framework, Proc Workshop Active Learn Exp Design, 157
  134. smith, 2005, Point-based POMDP algorithms: Improved analysis and implementation, Proc 13th Annu Conf Uncertainty Artif Intell, 542
  135. pietquin, 2006, Consistent goal-directed user model for realistic man-machine task-oriented spoken dialogue simulation, Proc IEEE Int Conf Multimedia Expo, 425
  136. gai, 2011, Statistical Dialogue Modelling
  137. 10.1145/1102351.1102377 · DOI · Paper page
  138. gai, 2010, Gaussian processes for fast policy optimisation of a POMDP dialogue manager for a real-world task, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 201
  139. georgila, 2006, User simulation for spoken dialogue systems: Learning and evaluation, Proc Int Conf Spoken Lang Process, 45
  140. engel, 2005, Algorithms and Representations for Reinforcement Learning
  141. pietquin, 2011, Sample efficient on-line learning of optimal dialogue policies with Kalman temporal differences, Proc Int Joint Conf Artif Intell, 1878
  142. 10.1109/ASRU.2005.1566485 · DOI
  143. 10.3115/1564144.1564145 · DOI
  144. janarthanam, 2011, ‘The day after the day after tomorrow?’ A machine learning approach to adaptive temporal expression generation: Training and evaluation with real users, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 142
  145. kim, 2007, Multimodal dialog system using hidden information state dialog manager, Proc Int Conf Multimodal Interfaces Demonstration Session
  146. ai, 2007, Knowledge consistent user simulations for dialog systems, Proc Annu Conf Int Speech Commun Assoc, 2697
  147. 10.1109/TASL.2008.2012071 · DOI
  148. keizer, 2010, Parameter estimation for agenda-based user simulation, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 116
  149. pietquin, 2004, A Framework for Unsupervised Learning of Dialogue Strategies

Cited by

See all 882

Metadata

Authors
Young, S., Gasic, M., Thomson, B., & Williams, J. D.
Venue
Proceedings of the IEEE
Published
2013-05
Volume
101
Issue
5
Pages
1160-1179
ISSN
  • 0018-9219 (ISSN-L)
  • 1558-2256
OpenAlex
W1975244201
Cited by
882
References
6
FWCI
48.96
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: POMDP-Based Statistical Spoken Dialog Systems: A Review. https://app.cochat.ai/papers/10.1109/jproc.2012.2225812
The paper
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013). POMDP-Based Statistical Spoken Dialog Systems: A Review. Proceedings of the IEEE. https://doi.org/10.1109/jproc.2012.2225812

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.