The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
POMDP-Based Statistical Spoken Dialog Systems: A Review
Abstract
Statistical dialog systems (SDSs) are motivated by the need for a data-driven framework that reduces the cost of laboriously handcrafting complex dialog managers and that provides robustness against the errors created by speech recognizers operating in noisy environments. By including an explicit Bayesian model of uncertainty and by optimizing the policy via a reward-driven process, partially observable Markov decision processes (POMDPs) provide such a framework. However, exact model representation and optimization is computationally intractable. Hence, the practical application of POMDP-based systems requires efficient algorithms and carefully constructed approximations. This review article provides an overview of the current state of the art in the development of POMDP-based spoken dialog systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
Research works printed but not linked: 50
- 10.3115/1075096.1075127 · doi:10.3115/1075096.1075127
- bohus, 2006, A ‘K hypotheses + other’ belief updating model, Proc AAAI Workshop Statist Empirical Approaches Spoken Dialogue Syst, 13
- crook, 2011, Lossless value directed compression of complex user goal states for statistical spoken dialogue systems, Proc Annu Conf Int Speech Commun Assoc, 1029
- 10.1109/SLT.2010.5700896 · doi:10.1109/slt.2010.5700896
- 10.1109/ICASSP.2008.4518765 · doi:10.1109/icassp.2008.4518765
- 10.3115/1557690.1557710 · doi:10.3115/1557690.1557710
- gai, 2011, Effective handling of dialogue state in the hidden information state POMDP dialogue manager, ACM Trans Speech Lang Process, 7
- thomson, 2008, Evaluating semantic-level confidence scores with multiple hypotheses, Proc Annu Conf Int Speech Commun Assoc, 1153
- 10.1109/ICASSP.2007.367189 · doi:10.1109/icassp.2007.367189
- 10.1007/978-3-540-30211-7_1 · doi:10.1007/978-3-540-30211-7_1
- 10.3115/1220835.1220870 · doi:10.3115/1220835.1220870
- tetreault, 2006, Using reinforcement learning to build a better model of dialogue state, Proc of the Association for Computational Linguistics
- 10.1007/978-3-642-21043-3_11 · doi:10.1007/978-3-642-21043-3_11
- pinault, 2011, Unsupervised clustering of probability distributions of semantic frame graphs for pomdp-based spoken dialogue systems with summary space, Proc Workshop Knowl Reasoning Practical Dialog Syst IJCAI
- pinault, 2011, Semantic graph clustering for POMDP-based spoken dialog systems, Proc Annu Conf Int Speech Commun Assoc, 1321
- boularias, 2010, Learning the reward model of dialogue POMDPs from data, Proc NIPS Workshop of Mach Learn for Assistive Tech
- pinault, 2009, Feature-based summary space for stochastic dialogue modeling with hierarchical semantic frames, Proc Annu Conf Int Speech Commun Assoc, 284
- bui, 2007, Practical dialogue manager development using POMDPs, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 215
- 10.1109/SLT.2006.326775 · doi:10.1109/slt.2006.326775
- cuayhuitl, 2006, Learning multi-goal dialogue strategies using reinforcement learning with reduced state-action spaces, Proc Annu Conf Int Speech Commun Assoc, 547
- lison, 2010, Towards relational POMDPs for adaptive dialogue management, Proc Annu Meeting Assoc Comput Linguist, 7
- williams, 2011, An empirical evaluation of a statistical dialog system in public use, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 130
- 10.3115/1556328.1556329 · doi:10.3115/1556328.1556329
- 10.1016/j.specom.2008.05.007 · doi:10.1016/j.specom.2008.05.007
- schatzmann, 2008, Statistical User and Error Modelling for Spoken Dialogue Systems
- gai, 2009, Back-off action selection in summary space-based POMDP dialogue systems, Proc IEEE Workshop Autom Speech Recog and Understanding, 456
- 10.3115/1622064.1622097 · doi:10.3115/1622064.1622097
- mcgraw, 2010, Collecting voices from the cloud, Proc Int Conf Lang Resources Eval, 1576
- williams, 2008, Integrating expert knowledge into POMDP optimization for spoken dialog systems, Proc AAAI Workshop Adv POMDP Solvers
- 10.1109/ICASSP.2011.5947627 · doi:10.1109/icassp.2011.5947627
- lefevre, 2009, <ref_formula><tex Notation="TeX">$k$</tex></ref_formula>-nearest neighbor Monte-Carlo control algorithm for POMDP-based dialogue systems, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- 10.1109/SLT.2010.5700863 · doi:10.1109/slt.2010.5700863
- 10.1109/ICASSP.2002.1005671 · doi:10.1109/icassp.2002.1005671
- chandramohan, 2011, User simulation in dialogue systems using inverse reinforcement learning, Proc Annu Conf Int Speech Commun Assoc, 1025
- 10.3115/1557690.1557722 · doi:10.3115/1557690.1557722
- 10.1016/j.csl.2009.03.002 · doi:10.1016/j.csl.2009.03.002
- 10.1109/SLT.2008.4777855 · doi:10.1109/slt.2008.4777855
- rossignol, 2011, Training a BN-based user model for dialogue simulation with missing data, Proc Int Joint Conf Natural Lang Process, 598
- 10.1109/TSA.2005.855836 · DOI
- hua ai, 2007, Comparing user simulation models for dialog strategy learning, Proc Conf North Amer Chapt Assoc Comput Linguist, 1
- 10.1109/ASRU.2007.4430167 · DOI
- 10.1007/978-3-540-74628-7_74 · doi:10.1007/978-3-540-74628-7_74
- littman, 2002, Advances in Neural Information Processing Systems 14, 1555
- smith, 2005, Point-based POMDP algorithms: Improved analysis and implementation, Proc 13th Annu Conf Uncertainty Artif Intell, 542
- gai, 2011, Statistical Dialogue Modelling
- gai, 2010, Gaussian processes for fast policy optimisation of a POMDP dialogue manager for a real-world task, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 201
- pietquin, 2011, Sample efficient on-line learning of optimal dialogue policies with Kalman temporal differences, Proc Int Joint Conf Artif Intell, 1878
- janarthanam, 2011, ‘The day after the day after tomorrow?’ A machine learning approach to adaptive temporal expression generation: Training and evaluation with real users, 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 142
- kim, 2007, Multimodal dialog system using hidden information state dialog manager, Proc Int Conf Multimodal Interfaces Demonstration Session
- pietquin, 2004, A Framework for Unsupervised Learning of Dialogue Strategies
Cited by
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I., Sordoni, A., Bengio, Y., Courville, A., & Pineau, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9883 · Cited by 1,615
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
“Recent advances in machine learning and a renewed interest in neural networks have led to the development of much more complex and efficient conversational agents.(9,40,41) The use of agent-based dialogue management appears to be rare in health applications.”
BackgroundDeep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1127 · Cited by 1,204
Recent Trends in Deep Learning Based Natural Language Processing
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02709 · Cited by 1,192
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gašić, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1547 · Cited by 1,028
Self-citationSemantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874
Self-citationA Network-based End-to-End Trainable Task-oriented Dialogue System
Wen, T.-H., Vandyke, D., Mrkšić, N., Gasic, M., Rojas Barahona, L. M., Su, P.-H., Ultes, S., & Young, S. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1042 · Cited by 778
Self-citationDeep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Serban, I., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., & Bengio, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10983 · Cited by 697
A Survey of Data Augmentation Approaches for NLP
Feng, S., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., & Hovy, E. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.84 · Cited by 670
A Theoretical Framework for Conversational Search
Radlinski, F., & Craswell, N. (2017) · Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval · doi:10.1145/3020165.3020183 · Cited by 439
Topic Aware Neural Response Generation
Xing, C., Wu, W., Wu, Y., Liu, J., Huang, Y., Zhou, M., & Ma, W.-Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10981 · Cited by 438
Conversational Recommender System
Sun, Y., & Zhang, Y. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210002 · Cited by 378
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
De Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.475 · Cited by 369
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210183 · Cited by 361
Williams, J. D., Asadi, K., & Zweig, G. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1062 · Cited by 324
Self-citationSequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures
Lei, W., Jin, X., Kan, M.-Y., Ren, Z., He, X., & Yin, D. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1133 · Cited by 320
Towards Conversational Search and Recommendation: System Ask, User Respond
Zhang, Y., Chen, X., Ai, Q., Yang, L., & Croft, W. B. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271776 · Cited by 319
Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems
Madotto, A., Wu, C.-S., & Fung, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1136 · Cited by 302
Budzianowski, P., & Vulić, I. (2019) · Proceedings of the 3rd Workshop on Neural Generation and Translation · doi:10.18653/v1/d19-5602 · Cited by 279
Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
Dhingra, B., Li, L., Li, X., Gao, J., Chen, Y.-N., Ahmed, F., & Deng, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1045 · Cited by 272
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773 · Cited by 254
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
Survey on reinforcement learning for language processing
Uc-Cetina, V., Navarro-Guerrero, N., Martin-Gonzalez, A., Weber, C., & Wermter, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10205-5 · Cited by 177
Liu, B., Tür, G., Hakkani-Tür, D., Shah, P., & Heck, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1187 · Cited by 148
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131
Task-Oriented Dialogue as Dataflow Synthesis
Andreas, J., Bufe, J., Burkett, D., Chen, C., Clausman, J., Crawford, J., Crim, K., DeLoach, J., Dorner, L., Eisner, J., et al. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00333 · Cited by 98
End-to-end joint learning of natural language understanding and dialogue manager
Yang, X., Chen, Y.-N., Hakkani-Tur, D., Crook, P., Li, X., Gao, J., & Deng, L. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953246 · Cited by 80
Approaches for Dialog Management in Conversational Agents
Harms, J.-G., Kucherbaev, P., Bozzon, A., & Houben, G.-J. (2019) · IEEE Internet Computing · doi:10.1109/mic.2018.2881519 · Cited by 78
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2018) · Proceedings of ACL 2018, Tutorial Abstracts · doi:10.18653/v1/p18-5002 · Cited by 73
Dialogue Systems and Conversational Agents for Patients with Dementia: The Human–Robot Interaction
Russo, A., D'Onofrio, G., Gangemi, A., Giuliani, F., Mongiovi, M., Ricciardi, F., Greco, F., Cavallo, F., Dario, P., Sancarlo, D., et al. (2019) · Rejuvenation Research · doi:10.1089/rej.2018.2075 · Cited by 47
Challenges in Building Highly Interactive Dialogue Systems
Ward, N. G., & DeVault, D. (2016) · AI Magazine · doi:10.1609/aimag.v37i4.2687 · Cited by 46
Personalized chatbot trustworthiness ratings
Srivastava, B., Rossi, F., Usmani, S., & Bernagozzi, M. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3023919 · Cited by 40
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
Roller, S., Boureau, Y.-L., Weston, J., Bordes, A., Dinan, E., Fan, A., Gunning, D., Ju, D., Li, M., Poff, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.12442 · Cited by 38
Developing enhanced conversational agents for social virtual worlds
Griol, D., Sanchis, A., Molina, J. M., & Callejas, Z. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.09.099 · Cited by 32
Let’s talk! speaking virtual counselor offers you a brief intervention
Yasavur, U., Lisetti, C., & Rishe, N. (2014) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-014-0169-9 · Cited by 32
Identifying and Avoiding Confusion in Dialogue with People with Alzheimer's Disease
Chinaei, H., Currie, L. C., Danks, A., Lin, H., Mehta, T., & Rudzicz, F. (2017) · Computational Linguistics · doi:10.1162/coli_a_00290 · Cited by 30
What's to Know? Uncertainty as a Guide to Asking Goal-Oriented Questions
Abbasnejad, E., Wu, Q., Shi, Q., & van den Hengel, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00428 · Cited by 16
Metadata
- Authors
- Young, S., Gasic, M., Thomson, B., & Williams, J. D.
- Venue
- Proceedings of the IEEE
- Published
- 2013-05
- Volume
- 101
- Issue
- 5
- Pages
- 1160-1179
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W1975244201
- Cited by
- 882
- References
- 111
- FWCI
- 48.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.