MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Open accessThe DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Abstract
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The majority of evaluation protocols largely simplify experimental assumptions and design static, pre-defined rules, with relatively short and isolated task settings (Cobbe et al., 2021; Chen et al., 2021; Budzianowski et al., 2018).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some tasks have clear ground truth answers and can be scored with exact correctness (Du et al., 2024; Dunn et al., 2017), while others are open-ended (dialogue (Budzianowski et al., 2018), summarization (Maharana et al., 2024), preference following (Zhao et al., 2025c)) where multiple outputs are acceptable and reference-based scoring is not reliable.”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The former types of benchmarks are static and only test the conversational agent on pre-collected conversation trajectories[4, 3, 2].”
From $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 4×
“Among them, the first two metrics relate to the dialogue task completion - whether the system has provided an appropriate entity (In-form rate) and then answered all the requested attributes (Success rate); while fluency is measured via BLEU score (Papineni et al., 2002).”
BackgroundMethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“(2017) which frames the dialogue as a context to response mapping problem, a sequence-to-sequence model (Sutskever et al., 2014) is augmented with a belief tracker and a discrete database accessing component as additional features to inform the word decisions in the decoder.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“In the case of MultiWOZ dataset, the LSTM cell serving as a decoder and an encoder achieved the highest score with the global type of attention (Bahdanau et al., 2014).”
MethodMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“As this requires a multi-annotator metric over a multi-label task, we used Fleiss’ kappa metric (Fleiss, 1971) per single dialogue act.”
MethodA Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926 · cited 2×
“Due to the lack of an explicit goal in the conversation, recent studies have shown that systems trained with this type of corpus not only struggle in generating consistent and diverse responses (Li et al., 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · cited 2×
“Although previous work (Vinyals and Le, 2015) has shown that a large learning system can learn to generate interesting responses from these corpora, the lack of grounding conversations onto an existing knowledge base or APIs limits the usability of developed systems.”
BackgroundLiu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1230 · Cited by 1,228 · cited 3×
“, 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundPOMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882 · cited 4×
“These problems can either be addressed separately (Young et al., 2013), or jointly in an end-to-end fashion (Bordes et al., 2017; Wen et al., 2017; Li et al., 2017).”
BackgroundInfluentialSelf-citationSemantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874 · cited 5×
“To make a fair comparison, we constrained our dataset to only the restaurant sub-domain which contains around 25 k dia-Cam676 Table 5 presents the results on two metrics: BLEU score (Papineni et al., 2002) and slot error rate (SER) (Wen et al., 2015).”
BackgroundMethodInfluentialSelf-citationThe ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790 · cited 4×
“One of the earliest dataset collected in this fashion is the ATIS corpus (Hemphill et al., 1990), where conversations between a client and an airline help-desk operator were recorded.”
BackgroundMethodResultInfluential
Research works printed but not linked: 7
- Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. Proceedings of ICLR .
- Paweł Budzianowski, I˜ nigo Casanueva, Bo-Hsiang Tseng, and Milica Gaˇ si´ c. 2018. Towards endto-end multi-domain dialogue modelling. Tech. Rep. CUED/F-INFENG/TR.706, University of Cambridge, Engineering Department .
- Seokhwan Kim, Luis Fernando D'Haro, Rafael E Banchs, Jason D Williams, Matthew Henderson, and Koichiro Yoshino. 2016. The fifth dialog state tracking challenge. In Spoken Language Technology Workshop (SLT), 2016 IEEE , pages 511-517. IEEE.
- Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end taskcompletion neural dialogue systems. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , volume 1, pages 733-743.
- David R. Traum. 1999. Foundations of Rational Agency , chapter Speech Acts for Dialogue Agents. Springer.
- Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gaˇ si´ c, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A networkbased end-to-end trainable task-oriented dialogue system. EACL .
- Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference , pages 404-413.
Cited by
A Survey on Deep Semi-Supervised Learning
Yang, X., Song, Z., King, I., & Xu, Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3220219 · Cited by 923
“Answers [106], IMDB [107], Yelp review [104], Snippets [108], Ohsumed [109], TagMyNews [110], MR [111], Elec [112], Rotten Tomatoes [111], RCV1 [113] [114], [115], [116], [117], [118], [119] Dialogue policy learning MultiWOZ [120] [121] Dialogue generation Cornell Movie Dialogs Corpus [122], Ubuntu Dialogue Corpus [12…”
BackgroundLongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2024) · arXiv · doi:10.48550/arxiv.2410.10813 · Cited by 493
“Long-Term Dialogue Benchmarks As the ability of dialogue systems improve, research starts to focus on long-term dialogue understanding beyond traditional dialogue modeling benchmarks (Budzianowski et al., 2018; Wei et al., 2018).”
Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset
Rastogi, A., Zang, X., Sunkara, S., Gupta, R., & Khaitan, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6394 · Cited by 450
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., & Fung, P. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1078 · Cited by 415
“One of the motivations to perform unseen domain DST is because collecting a large-scale task-oriented dataset for a new domain is expensive and time-consuming (Budzianowski et al., 2018), and there are a large amount of domains in realistic scenarios.”
BackgroundTowards conversational diagnostic artificial intelligence
Tu, T., Schaekermann, M., Palepu, A., Saab, K., Freyberg, J., Tanno, R., Wang, A., Li, B., Amin, M., Cheng, Y., et al. (2025) · Nature · doi:10.1038/s41586-025-08866-7 · Cited by 366
“Conversational AI systems for goal-oriented dialogue and task completion have a rich history 73–75 .”
Advances and challenges in conversational recommender systems: A survey
Gao, C., Lei, W., He, X., de Rijke, M., & Chua, T.-S. (2021) · AI Open · doi:10.1016/j.aiopen.2021.06.002 · Cited by 337
“2002b) is used to measure the fluency of these models mimicking human conversations (Budzianowski et al., 2018; Zhang and OuZhou, 2020).”
MethodEric, M., Goel, R., Paul, S., Kumar, A., Sethi, A. R., Ku, P., Goyal, A. K., Agarwal, S., Gao, S., & Hakkani-Tür, D. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01669 · Cited by 337
“One of the largest resources of its kind is the MultiWOZ dataset, which spans 7 distinct taskoriented domains including hotel, taxi, and restaurant booking among others (Budzianowski et al., 2018).”
BackgroundEfficient Intent Detection with Dual Sentence Encoders
Casanueva, I., Temčinas, T., Gerz, D., Henderson, M., & Vulić, I. (2020) · Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI · doi:10.18653/v1/2020.nlp4convai-1.5 · Cited by 336
Self-citationS oloist : BuildingTask Bots at Scale with Transfer Learning and Machine Teaching
Peng, B., Li, C., Li, J., Shayandeh, S., Lidén, L., & Gao, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00399 · Cited by 371
“To ensure there is no overlap between pre-training and fine-tuning datasets, we exclude the data akin to MultiWOZ (Budzianowski et al., 2018), CamRest676 (Wen et al.”
BackgroundMethodInfluentialBudzianowski, P., & Vulić, I. (2019) · Proceedings of the 3rd Workshop on Neural Generation and Translation · doi:10.18653/v1/d19-5602 · Cited by 279
“Text task (see Figure 1) on the multi-domain MultiWOZ dataset (Budzianowski et al., 2018b) suggest that our GPT-based task-oriented dialogue model learns to generate and understand domain-specific tokens, which in turn leads to a seamless adapta-”
BackgroundMethodResultInfluentialSelf-citationGraph Neural Networks for Natural Language Processing: A Survey
Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., & Long, B. (2023) · Foundations and Trends in Machine Learning · doi:10.1561/2200000096 · Cited by 278
Domain Adaptation: Challenges, Methods, Datasets, and Applications
Singhal, P., Walambe, R., Ramanna, S., & Kotecha, K. (2023) · IEEE Access · doi:10.1109/access.2023.3237025 · Cited by 234
Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback
Wu, H., Gao, Y., Guo, X., Al-Halah, Z., Rennie, S., Grauman, K., & Feris, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01115 · Cited by 264
“Furthermore, a challenge with implementing dialog-interface image search systems is that currently conversational systems typically require cumbersome hand-engineering and/or large-scale dialog data [35, 6].”
BackgroundRecent advances in deep learning based dialogue systems: a systematic survey
Ni, J., Young, T., Pandelea, V., Xue, F., & Cambria, E. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10248-8 · Cited by 264
A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems
Yi, Z., Ouyang, J., Xu, Z., Liu, Y., Liao, T., Luo, H., & Shen, Y. (2025) · ACM Computing Surveys · doi:10.1145/3771090 · Cited by 306
“Approaches for Open-domain Dialogue Systems.”
DialogSum: A Real-Life Scenario Dialogue Summarization Dataset
Chen, Y., Liu, Y., Chen, L., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.449 · Cited by 150
“We follow previous work (Li et al., 2017; Zhang et al., 2018; Budzianowski et al., 2018; Di-nan et al., 2019)”
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue
Wu, C.-S., Hoi, S. C. H., Socher, R., & Xiong, C. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.66 · Cited by 237
Recent advances and challenges in task-oriented dialog systems
Zhang, Z., Takanobu, R., Zhu, Q., Huang, M., & Zhu, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1692-3 · Cited by 175
Towards Topic-Guided Conversational Recommender System
Zhou, K., Zhou, Y., Zhao, W. X., Wang, X., & Wen, J.-R. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.365 · Cited by 158
Survey on evaluation methods for dialogue systems
Deriu, J., Rodrigo, A., Otegi, A., Echegoyen, G., Rosset, S., Agirre, E., & Cieliebak, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09866-x · Cited by 215
“MultiWOZ Tourist information 10,438 (Budzianowski et al. 2018)”
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2023) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 156
“There is some data for conversations with travel agents [131, 148], conducting meetings [8], navigation, scheduling, and weather queries to an in-car personal assistant [90], and other resources [15, 260], as well as multi-domain resources [39, 224].”
BackgroundTaskmaster-1: Toward a Realistic and Diverse Dialog Dataset
Byrne, B., Krishnamoorthi, K., Sankar, C., Neelakantan, A., Goodrich, B., Duckworth, D., Yavuz, S., Dubey, A., Kim, K.-Y., & Cedilnik, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1459 · Cited by 139
“Size: The total of 13,215 dialogs in this corpus is on par with similar, recently released datasets such as MultiWOZ (Budzianowski et al., 2018).”
BackgroundMethodInfluentialConveRT: Efficient and Accurate Conversational Representations from Transformers
Henderson, M., Casanueva, I., Mrkšić, N., Su, P.-H., Wen, T.-H., & Vulić, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.196 · Cited by 176
Self-citationEfficient Dialogue State Tracking by Selectively Overwriting Memory
Kim, S., Yang, S., Kim, G., & Lee, S.-W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.53 · Cited by 186
Yu, T., Zhang, R., Er, H., Li, S., Xue, E., Pang, B., Lin, X. V., Tan, Y. C., Shi, T., Li, Z., et al. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1204 · Cited by 158
KLUE: Korean Language Understanding Evaluation
Park, S., Moon, J., Kim, S., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09680 · Cited by 75
“On the other hand, for the taxi domain, we do not define instances in advance, but dynamically generate the instances during the dialogue collection as in MultiWOZ [10].”
BackgroundMethodInfluentialZang, X., Rastogi, A., & Chen, J. (2020) · Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI · doi:10.18653/v1/2020.nlp4convai-1.13 · Cited by 188
“Many public datasets like DSTC2 (Hen-derson et al., 2014), WOZ (Wen et al., 2017a), SimulatedDialogue (Shah et al., 2018), Multi-WOZ (Budzianowski et al., 2018), TaskMas-ter (Byrne et al., 2019), SGD (Rastogi et al., 2019), etc. have been very useful to facilitate research in this area.”
BackgroundTripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking
Heck, M., van Niekerk, C., Lubis, N., Geishauser, C., Lin, H.-C., Moresi, M., & Gasic, M. (2020) · Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2020.sigdial-1.4 · Cited by 181
Self-citationConversational Information Seeking
Zamani, H., Trippas, J. R., Dalton, J., & Radlinski, F. (2023) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000081 · Cited by 79
UBAR: Towards Fully End-to-End Task-Oriented Dialog System with GPT-2
Yang, Y., Li, Y., & Quan, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17674 · Cited by 243
Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System
Su, Y., Shu, L., Mansimov, E., Gupta, A., Cai, D., Lai, Y.-A., & Zhang, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.319 · Cited by 248
SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking
Lee, H., Lee, J., & Kim, T.-Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1546 · Cited by 172
“Note that MultiWOZ has more domains and slots to be learned than WOZ 2.0 corpus.”
MethodInfluentialMinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
Lin, Z., Madotto, A., Winata, G. I., & Fung, P. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.273 · Cited by 224
Large Language Models are few(1)-shot Table Reasoners
Chen, W. (2023) · Findings of the Association for Computational Linguistics: EACL 2023 · doi:10.18653/v1/2023.findings-eacl.83 · Cited by 97
Conversational Agents: Goals, Technologies, Vision and Challenges
Allouch, M., Azaria, A., & Azoulay, R. (2021) · Sensors · doi:10.3390/s21248448 · Cited by 168
“MultiWOZ [233] is a tourist-dialogue dataset, annotated with dialogue belief states and dialogue actions.”
BackgroundTowards Conversational Recommendation over Multi-Type Dialogs
Liu, Z., Wang, H., Niu, Z.-Y., Wu, H., Che, W., & Liu, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.98 · Cited by 168
Few-shot Natural Language Generation for Task-Oriented Dialog
Peng, B., Zhu, C., Li, C., Li, X., Li, J., Zeng, M., & Gao, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.17 · Cited by 164
Dialog State Tracking: A Neural Reading Comprehension Approach
Gao, S., Sethi, A., Agarwal, S., Chung, T., & Hakkani-Tur, D. (2019) · Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w19-5932 · Cited by 164
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2
Ham, D., Lee, J.-G., Jang, Y., & Kim, K.-E. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.54 · Cited by 161
“…al., 2019; Lee et al., 2019a) which maps the dialogue history to the dialogue state (the composite function of NLU and DST), and the Word-level POL (Budzianowski et al., 2018; Pei et al., 2019; Chen et al., 2019; Mehri et al., 2019; Zhao et al., 2019) which maps the previous utterance and dialogue…”
BackgroundMethodInfluentialSLURP: A Spoken Language Understanding Resource Package
Bastianelli, E., Vanzo, A., Świętojański, P., & Rieser, V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.588 · Cited by 306
Task-Oriented Dialog Systems That Consider Multiple Appropriate Responses under the Same Context
Zhang, Y., Ou, Z., & Yu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6507 · Cited by 155
Conversational question answering: a survey
Zaib, M., Zhang, W. E., Sheng, Q. Z., Mahmood, A., & Zhang, Y. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01744-y · Cited by 189
“Questions asked by users are either open-domain [33, 108] in which questions are domainfree and in a broad range, or restricted to specific application domains (i.e., closed-domain) such as Travel [3], Restaurants [8], Movies [9], and Hospitals [8].”
BackgroundText-to-Text Pre-Training for Data-to-Text Tasks
Kale, M., & Rastogi, A. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.14 · Cited by 138
Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions FollowingPreprint
He, Y., Jin, D., Wang, C., Bi, C., Mandyam, K., Zhang, H., Zhu, C., Li, N., Xu, T., Lv, H., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15553 · Cited by 162
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented DialoguePreprint
Mehri, S., Eric, M., & Hakkani-Tür, D. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13570 · Cited by 164
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Chen, W., Chen, J., Qin, P., Yan, X., & Wang, W. Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1360 · Cited by 140
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks
Chen, L., Lv, B., Wang, C., Zhu, S., Tan, B., & Yu, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6250 · Cited by 135
“Data We choose MultiWOZ 2.0 (Budzianowski et al. 2018) and MultiWOZ 2.1 (Eric et al. 2019) as our training and evaluation datasets, both of which are task-oriented corpora comprised of dialogues between human and human.”
MethodInfluentialMediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
Zhu, C., Liu, Y., Mei, J., & Zeng, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.474 · Cited by 133
A survey on deep reinforcement learning for audio-based applications
Latif, S., Cuayáhuitl, H., Pervez, F., Shamshad, F., Ali, H. S., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10224-2 · Cited by 149
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123
Training Neural Response Selection for Task-Oriented Dialogue Systems
Henderson, M., Vulić, I., Gerz, D., Casanueva, I., Budzianowski, P., Coope, S., Spithourakis, G., Wen, T.-H., Mrkšić, N., & Su, P.-H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1536 · Cited by 145
“…a recent rise of interest in neural architectures for modelling response selection (Wu et al., 2017; Chaudhuri et al., 2018; Zhou et al., 2018; Tao et al., 2019), but the progress is still hindered by insufficient domain-specific training data (El Asri et al., 2017; Budzianowski et al., 2018).”
BackgroundSelf-citationBarikeri, S., Lauscher, A., Vulić, I., & Glavaš, G. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.151 · Cited by 121
“, 2020), and two dialog tasks: 2) dialog state tracking on MultiWoZ (Budzianowski et al., 2018), and 3) conversational response generation on DSTC-7 (Yoshino et al.”
BackgroundMethodInfluentialHe, W., Dai, Y., Zheng, Y., Wu, Y., Cao, Z., Liu, D., Jiang, P., Yang, M., Huang, F., Si, L., et al. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21320 · Cited by 194
Zhao, T., Xie, K., & Eskénazi, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1123 · Cited by 116
“Table 3 shows that LiteAttnCat is on (Budzianowski et al., 2018).”
BackgroundMethodInfluentialFew-Shot Named Entity Recognition: An Empirical Baseline Study
Huang, J., Li, C., Subudhi, K., Jose, D., Balakrishnan, S., Chen, W., Peng, B., Gao, J., & Han, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.813 · Cited by 146
“…2017) on social domain, MIT Moive (Liu et al., 2013b) and MIT Restaurant 2 (Liu et al., 2013a) on review domain, SNIPS 3 (Coucke et al., 2018), ATIS 4 (Hakkani-T¨ur et al., 2016) and Multiwoz 5 (Budzianowski et al., 2018) on dialogue domain, and I2B2 6 (Stubbs and Uzuner, 2015) on medical domain.”
BackgroundContinual Learning in Task-Oriented Dialogue Systems
Madotto, A., Lin, Z., Zhou, Z., Moon, S., Crook, P., Liu, B., Zhou, Y., Cho, E., Fung, P., & Wang, Z. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.590 · Cited by 161
“Consequently, in this paper we propose a Continual Learning benchmark by jointly pre-processing four task-oriented dataset such as Task-Master 2019 (TM19) (Byrne et al., 2019), Task-Master 2020 (TM20) (Byrne et al., 2019), Schema Guided Dialogue (SGD) (Rastogi et al., 2019) and MultiWoZ (Budzianowski et al., 2018).”
MethodIn-Context Learning for Few-Shot Dialogue State Tracking
Hu, Y., Lee, C.-H., Xie, T., Yu, T., Smith, N. A., & Ostendorf, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.193 · Cited by 56
Dialogue State Tracking with a Language Model using Schema-Driven Prompting
Lee, C., Cheng, H., & Mari Ostendorf, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.404 · Cited by 165
“The backbone module of a typical system is dialogue state tracking (DST), where the user goal is inferred from the dialogue history (Henderson et al., 2014; Shah et al., 2018; Budzianowski et al., 2018).”
BackgroundDeepStruct: Pretraining of Language Models for Structure Prediction
Wang, C., Liu, X., Chen, Z., Hong, H., Tang, J., & Song, D. X. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.67 · Cited by 171
A Repository of Conversational Datasets
Henderson, M., Budzianowski, P., Casanueva, I., Coope, S., Gerz, D., Kumar, G., Mrkšić, N., Spithourakis, G., Su, P.-H., Vulić, I., et al. (2019) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/w19-4101 · Cited by 108
“, 2017), or to provide tourist information (Henderson et al., 2014c; Budzianowski et al., 2018).”
BackgroundMethodInfluentialSelf-citationWebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Yao, S., Chen, H., Yang, J., & Narasimhan, K. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1508 · Cited by 107
Louvan, S., & Magnini, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.42 · Cited by 107
doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset
Feng, S., Wan, H., Gunasekara, C., Patel, S., Joshi, S., & Lastras, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.652 · Cited by 159
ConvLab: Multi-Domain End-to-End Dialog System Platform
Lee, S., Zhu, Q., Takanobu, R., Zhang, Z., Zhang, Y., Li, X., Li, J., Peng, B., Li, X., Huang, M., et al. (2019) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/p19-3011 · Cited by 104
“For the former, Budzianowski et al. (2018) collected the MultiWOZ dataset, a dialog corpus with dialogs ranging over multiple domains for the trip information setting, whereas there is no open-platform yet that is designed to handle multi-domain, multi-intent phenomena.”
BackgroundAre Large Language Models All You Need for Task-Oriented Dialogue?
Hudeček, V., & Dušek, O. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.21 · Cited by 193
“To our knowledge, our zero-shot experiments establish a state-of-the-art result in unsupervised TOD modeling on the MultiWOZ and Schema-guided datasets (Budzianowski et al., 2018; Rastogi et al., 2020).”
MethodInfluentialYe, F., Manotumruksa, J., & Yılmaz, E. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.34 · Cited by 102
Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog
Qin, L., Xiao, X., Che, W., Zhang, Y., & Liu, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.565 · Cited by 117
Few-Shot Bot: Prompt-Based Learning for Dialogue SystemsPreprint
Madotto, A., Lin, Z., Winata, G. I., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08118 · Cited by 113
“For instance, it would be interesting to explore more datasets and tasks such as conversational semantic parsing, e.g., CoSQL (Yu et al., 2019), data-flow (Andreas et al., 2020), treeDST (Cheng et al., 2020), multi-domain DST, e.g., SGD (Rastogi et al., 2020), MWoZ (since in this paper we only use a single domain), Bi…”
MethodInfluentialStructured Fusion Networks for Dialog
Mehri, S., Srinivasan, T., & Eskénazi, M. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5921 · Cited by 99
“When using re-inforcement learning, SFNs match the performance of LiteAttnCat (Zhao et al., 2019) on the combined Model BLEU Inform Success Combined Score Supervised Learning Seq2Seq (Budzianowski et al., 2018) 18.80 71.29% 60.29% 84.59 Seq2Seq w/ Attention (Budzianowski et al., 2018) score.”
BackgroundMethodInfluentialMeta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems
Mi, F., Huang, M., Zhang, J., & Faltings, B. V. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/437 · Cited by 99
CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset
Zhu, Q., Huang, K., Zhang, Z., Zhu, X., & Huang, M. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00314 · Cited by 99
Task-Oriented Dialogue as Dataflow Synthesis
Andreas, J., Bufe, J., Burkett, D., Chen, C., Clausman, J., Crawford, J., Crim, K., DeLoach, J., Dorner, L., Eisner, J., et al. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00333 · Cited by 98
“However, Joint Goal does not fully characterize the effec-Error tiveness of a state tracking system in real-world interactions, as it allows the model to recover from an error at an earlier turn by conditioning on gold agent utterances after the error.”
BackgroundMethodInfluentialGuided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog
Takanobu, R., Zhu, H., & Huang, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1010 · Cited by 97
“We use MultiWOZ (Budzianowski et al., 2018), a multi-domain, multi-intent task-oriented dialog corpus that contains 7 domains, 13 intents, 25 slot types, 10,483 dialog sessions, and 71,544 dialog turns in our experiments.”
BackgroundMethodMulti-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question AnsweringPreprint
Zhou, L., & Small, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06192 · Cited by 96
“For example, the MultiWOZ 2.0/2.1 datasets (Budzianowski et al., 2018; Eric et al., 2019) have 30 (domain, slot) pairs and more than 4, 500 values (Wu et al., 2019).”
BackgroundMethodInfluentialHyST: A Hybrid Approach for Flexible and Accurate Dialogue State Tracking
Goel, R., Paul, S., & Hakkani-Tür, D. Z. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1863 · Cited by 93
Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking
Campagna, G., Foryciarz, A., Moradshahi, M., & Lam, M. S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.12 · Cited by 90
Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation
Ren, L., Ni, J., & McAuley, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1196 · Cited by 96
Toward Scalable Neural Dialogue State Tracking ModelPreprint
Nouri, E., & Hosseini-Asl, E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00899 · Cited by 89
“Moreover, we also evaluate on recen;t proposed multi-domain dataset, Multi-WoZ (Budzianowski et al., 2018), which consists of seven domains, i.e. restaurant, hotel, train, attraction, hospital, taxi, and police.”
BackgroundParaphrase Augmented Task-Oriented Dialog Generation
Gao, S., Zhang, Y., Ou, Z., & Zhou, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.60 · Cited by 88
Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
Tseng, Y.-M., Huang, Y.-C., Hsiao, T.-Y., Chen, W.-L., Huang, C.-W., Meng, Y., & Chen, Y.-N. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.969 · Cited by 87
Guiding Large Language Models via Directional Stimulus Prompting
Li, Z., Peng, B., He, P., Galley, M., Gao, J., & Yan, X. (2023) · neural information processing systems · doi:10.52202/075280-2735 · Cited by 171
SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations
Kottur, S., Moon, S., Geramifard, A., & Damavandi, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.401 · Cited by 137
“…have been developed in the past to support various assistant scenarios ( e.g . booking hotels, reserving hotels) (Henderson et al., 2014; Rastogi et al., 2019; Budzianowski et al., 2018; Eric et al., 2019), defin-ing many challenges in handling user requests under the unimodal dialog setting.”
BackgroundMirages. On Anthropomorphism in Dialogue Systems
Abercrombie, G., Curry, A., Dinkar, T., Rieser, V., & Talat, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.290 · Cited by 83
“They found that only one strictly task-based system (MultiWoz, Budzianowski et al., 2018) did not appear as anthropomorphic to participants.”
BackgroundA Short Survey of Pre-trained Language Models for Conversational AI-A New Age in NLP
Zaib, M., Sheng, Q. Z., & Zhang, W. E. (2020) · Australasian Computer Science Week Multiconference (ACSW) · doi:10.1145/3373017.3373028 · Cited by 83
“They used multi-domain dataset MultiWoz [2] to learn and understand the domain-specific tokens which make it easier to adapt to unseen domains.”
MethodOPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.12017 · Cited by 81
“…2020), which is a held-out datato-text task for transforming data triples to text, Spider (Yu et al., 2018), a SQL query generation task given a database and an input query, and fully supervised in our framework, and MultiWoZ (Budzianowski et al., 2018), is a held-out dialogue state tracking task.”
BackgroundOverview of the Ninth Dialog System Technology Challenge: DSTC9
Gunasekara, C., Kim, S., D’Haro, L. F., Rastogi, A., Chen, Y.-N., Eric, M., Hedayatnia, B., Gopalakrishnan, K., Liu, Y., Huang, C.-W., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3426331 · Cited by 83
“Compared to MultiWOZ 2.0 [23], MultiWOZ2.”
InfluentialLi, H., Xu, M., & Song, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.881 · Cited by 160
A pragmatic guide to geoparsing evaluation
Gritta, M., Pilehvar, M. T., & Collier, N. (2018) · Language Resources and Evaluation · doi:10.1007/s10579-019-09475-3 · Cited by 95
“The authors made changes to over 32% of state annotations across 40% of dialogue turns, which is a significant correction to the original dataset [9].”
BackgroundConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems
Zhu, Q., Zhang, Z., Fang, Y. M., Li, X. R., Takanobu, R., Li, J., Peng, B., Gao, J., Zhu, X., & Huang, M. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.19 · Cited by 78
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog
Ekstedt, E., & Skantze, G. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.268 · Cited by 139
Han, T., Liu, X., Takanabu, R., Lian, Y., Huang, C., Wan, D., Peng, W., & Huang, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88483-3_16 · Cited by 76
“Task-oriented dialogue systems have made unprecedented progress with multiple state-of-the-art (SOTA) models underpinned by a number of publicly available datasets (Zhu et al., 2020a; Hender-son et al., 2014; Williams et al., 2013; Wen et al., 2017; Rastogi et al., 2019; Budzianowski et al., 2018).”
BackgroundMethodDialogue TransformersPreprint
Vlasov, V., Mosig, J. E. M., & Nichol, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00486 · Cited by 76
“[22] provide validation and test datasets that should contain only dialogues in which the given tasks were completed.”
BackgroundDialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances
Gu, X., Yoo, K. M., & Ha, J.-W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17527 · Cited by 102
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Gupta, P., Jiao, C., Yeh, Y.-T., Mehri, S., Eskénazi, M., & Bigham, J. P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.33 · Cited by 124
“We adopt the experimental setup from PPTOD (Su et al., 2022a), and conduct few-shot experiments on MultiWOZ 2.0 (Budzianowski et al., 2018).”
MethodInfluentialIntroducing MANtIS: a novel Multi-Domain Information Seeking Dialogues DatasetPreprint
Penha, G., Bălan, A. O., & Hauff, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04639 · Cited by 75
“MultiWOZ [4] EMNLP NLP 8,438 3 3 3 3 QuAC [8] EMNLP NLP 13,594 3 3 a 3 3 WoW [11] ICLR ML 22,311 3 3 3 3”
Few-Shot Named Entity Recognition: A Comprehensive StudyPreprint
Huang, J., Li, C., Subudhi, K., Jose, D., Balakrishnan, S., Chen, W., Peng, B., Gao, J., & Han, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.14978 · Cited by 87
“…2017) on social domain, MIT Moive (Liu et al., 2013b) and MIT Restaurant 2 (Liu et al., 2013a) on review domain, SNIPS 3 (Coucke et al., 2018), ATIS 4 (Hakkani-Tür et al., 2016) and Multi-woz 5 (Budzianowski et al., 2018) on dialogue domain, and I2B2 6 (Stubbs and Uzuner, 2015) on medical domain.”
BackgroundTopic-Aware Multi-turn Dialogue Modeling
Xu, Y., Zhao, H., & Zhang, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17668 · Cited by 74
Learning an Unreferenced Metric for Online Dialogue Evaluation
Sinha, K., Parthasarathi, P., Wang, J., Lowe, R., Hamilton, W. L., & Pineau, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.220 · Cited by 64
Slot Self-Attentive Dialogue State Tracking
Ye, F., Manotumruksa, J., Zhang, Q., Li, S., & Yılmaz, E. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449939 · Cited by 73
“We conduct extensive experiments on both MultiWOZ 2.0 [3] and MultiWOZ 2.1 [11] and show that STAR achieves better performance than existing methods that have taken slot correlations into consideration.”
MethodInfluentialLeveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking
Lin, Z., Liu, B., Moon, S., Crook, P., Zhou, Z., Wang, Z., Yu, Z., Madotto, A., Cho, E., & Subba, R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.448 · Cited by 73
Metadata
- Authors
- Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gašić, M.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 5016-5026
- OpenAlex
- W2891732163
- Cited by
- 1,028
- References
- 48
- Citations by year
- 2026: 39 · 2025: 66 · 2024: 86 · 2023: 209 · 2022: 186 · 2021: 196 · 2020: 181 · 2019: 64 · 2018: 1
- FWCI
- 45.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M018946/1, RG80792)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- MultiWOZ, task-oriented dialogue, multi-domain dialogue, dialogue modeling, dialogue dataset, chatbots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.