MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Open accessThe DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Abstract
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The majority of evaluation protocols largely simplify experimental assumptions and design static, pre-defined rules, with relatively short and isolated task settings (Cobbe et al., 2021; Chen et al., 2021; Budzianowski et al., 2018).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some tasks have clear ground truth answers and can be scored with exact correctness (Du et al., 2024; Dunn et al., 2017), while others are open-ended (dialogue (Budzianowski et al., 2018), summarization (Maharana et al., 2024), preference following (Zhao et al., 2025c)) where multiple outputs are acceptable and reference-based scoring is not reliable.”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The former types of benchmarks are static and only test the conversational agent on pre-collected conversation trajectories[4, 3, 2].”
From $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 4×
“Among them, the first two metrics relate to the dialogue task completion - whether the system has provided an appropriate entity (In-form rate) and then answered all the requested attributes (Success rate); while fluency is measured via BLEU score (Papineni et al., 2002).”
BackgroundMethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“(2017) which frames the dialogue as a context to response mapping problem, a sequence-to-sequence model (Sutskever et al., 2014) is augmented with a belief tracker and a discrete database accessing component as additional features to inform the word decisions in the decoder.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“In the case of MultiWOZ dataset, the LSTM cell serving as a decoder and an encoder achieved the highest score with the global type of attention (Bahdanau et al., 2014).”
MethodMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“As this requires a multi-annotator metric over a multi-label task, we used Fleiss’ kappa metric (Fleiss, 1971) per single dialogue act.”
MethodA Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926 · cited 2×
“Due to the lack of an explicit goal in the conversation, recent studies have shown that systems trained with this type of corpus not only struggle in generating consistent and diverse responses (Li et al., 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · cited 2×
“Although previous work (Vinyals and Le, 2015) has shown that a large learning system can learn to generate interesting responses from these corpora, the lack of grounding conversations onto an existing knowledge base or APIs limits the usability of developed systems.”
BackgroundLiu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1230 · Cited by 1,228 · cited 3×
“, 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundPOMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882 · cited 4×
“These problems can either be addressed separately (Young et al., 2013), or jointly in an end-to-end fashion (Bordes et al., 2017; Wen et al., 2017; Li et al., 2017).”
BackgroundInfluentialSelf-citationSemantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874 · cited 5×
“To make a fair comparison, we constrained our dataset to only the restaurant sub-domain which contains around 25 k dia-Cam676 Table 5 presents the results on two metrics: BLEU score (Papineni et al., 2002) and slot error rate (SER) (Wen et al., 2015).”
BackgroundMethodInfluentialSelf-citationThe ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790 · cited 4×
“One of the earliest dataset collected in this fashion is the ATIS corpus (Hemphill et al., 1990), where conversations between a client and an airline help-desk operator were recorded.”
BackgroundMethodResultInfluential
Research works printed but not linked: 7
- Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. Proceedings of ICLR .
- Paweł Budzianowski, I˜ nigo Casanueva, Bo-Hsiang Tseng, and Milica Gaˇ si´ c. 2018. Towards endto-end multi-domain dialogue modelling. Tech. Rep. CUED/F-INFENG/TR.706, University of Cambridge, Engineering Department .
- Seokhwan Kim, Luis Fernando D'Haro, Rafael E Banchs, Jason D Williams, Matthew Henderson, and Koichiro Yoshino. 2016. The fifth dialog state tracking challenge. In Spoken Language Technology Workshop (SLT), 2016 IEEE , pages 511-517. IEEE.
- Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end taskcompletion neural dialogue systems. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , volume 1, pages 733-743.
- David R. Traum. 1999. Foundations of Rational Agency , chapter Speech Acts for Dialogue Agents. Springer.
- Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gaˇ si´ c, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A networkbased end-to-end trainable task-oriented dialogue system. EACL .
- Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference , pages 404-413.
Cited by
Items 601–700 of 1,004
A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction
van Niekerk, C., Geishauser, C., Heck, M. C., Feng, S., Lin, H.-C., Lubis, N., Ruppik, B., Vukovic, R., & Gašić, M. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00734 · Cited by 10
Self-citationMasson, M., Agerri, R., Sallaberry, C., Bessagnet, M.-N., Parc, A. L., & Roose, P. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114001 · Cited by 9
“Other corpora include the MultiWOZ dialogue dataset [38], the Stanford NLI dataset [39] for text inference, and the Heldugazte corpus [20], which assists in categorizing tweets as formal or informal.”
BackgroundMethodMulti-Granularity Representations of Dialog
Mehri, S., & Eskénazi, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1184 · Cited by 6
“These hidden sizes match the best performing hyperparameters iden-tified by Budzianowski et al. (2018).”
BackgroundMethodInfluentialSemi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling
Tseng, B.-H., Rei, M., Budzianowski, P., Turner, R. E., Byrne, B., & Korhonen, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1125 · Cited by 6
“Our analysis on the MultiWOZ corpus (Budzianowski et al., 2018), covering a range of domains and topics, finds that these meth-”
MethodInfluentialSelf-citationBack to the Future for Dialogue Research
Cohen, P. R. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7073 · Cited by 6
“Extensions to I+S to allow goals from multiple domains (Budzianowski et al. 2018) should consider constraints (such as temporal ones) across those goals, e.”
BackgroundAutomatic Labeled Dialogue Generation for Nursing Record Systems
Mairittha, T., Mairittha, N., & Inoue, S. (2020) · Journal of Personalized Medicine · doi:10.3390/jpm10030062 · Cited by 7
Investigating Proactivity in Task-Oriented Dialogues
Balaraman, V., & Magnini, B. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.8243 · Cited by 7
“The MultiWoZ2.”
BackgroundInfluentialA Student-Teacher Architecture for Dialog Domain Adaptation Under the Meta-Learning Setting
Qian, K., Wei, W., & Zhou, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17614 · Cited by 6
Hi-DST: A Hierarchical Approach for Scalable and Extensible Dialogue State Tracking
Dey, S., & Desarkar, M. S. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.23 · Cited by 6
“(cid:129) (cid:222) : That is all I need. n (cid:222) : { { (attraction, area, east), (attraction, name, Funky Fun Figure 1: A sample conversation from the Multi-WOZ (Budzianowski et al., 2018) dataset (dialogue id PMUL3336). become a real problem for task-based chatbots.”
BackgroundMethodInfluentialA Therapeutic Dialogue Agent for Polish Language
Zygadło, A. (2021) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw52867.2021.9666281 · Cited by 6
Zhao, L., Zheng, F., Zeng, W., He, K., Geng, R., Jiang, H., Wu, W., & Xu, W. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531933 · Cited by 14
“TODSum [34] is a 5-domain task-oriented dialogue summarization dataset based on MUltiWOZ [4], which contains dialogue, summary and unique dialogue state.”
BackgroundIntent Classification and Slot Filling Model for In-Vehicle Services in Korean
Lim, J., Son, S., Lee, S., Chun, C., Park, S. S., Hur, Y., & Lim, H. (2022) · Applied Sciences · doi:10.3390/app122312438 · Cited by 7
Kraus, M., Riekenbrauck, R., & Minker, W. (2023) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3563359.3597403 · Cited by 10
“Although there are various data corpora that exist for conventional dialog modeling [2, 38], none of them are sufficient for modeling proactive dialog.”
BackgroundMethodBREAK: Breaking the Dialogue State Tracking Barrier with Beam Search and Re-ranking
Won, S., Kwak, H., Shin, J., Han, J., & Jung, K. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.159 · Cited by 6
“We conduct our experiments on MultiWOZ 2.1-2.4 (Eric et al., 2020; Zang et al., 2020; Han et al., 2021; Ye et al., 2022b), the improved versions made by continuously refining annotation errors from Multi-WOZ 2.0 (Budzianowski et al., 2018).”
BackgroundMethodInfluentialAugmenting Multi-Turn Text-to-SQL Datasets with Self-Play
Liu, Q., Ye, Z., Yu, C., Song, L., & Blunsom, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.411 · Cited by 15
Multi-Label Intent Detection via Contrastive Task Specialization of Sentence Encoders
Vulić, I., Casanueva, I., Spithourakis, G., Mondal, A., Wen, T.-H., & Budzianowski, P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.512 · Cited by 13
“T O D supports a multitude of applications such as automating different customer service tasks, facilitating bookings in hospitality and travel industries, or providing assistance in health-care or finance (Raux et al., 2003; El Asri et al., 2017; Xu et al., 2017; Budzianowski et al., 2018).”
BackgroundSelf-citationControllable User Dialogue Act Augmentation for Dialogue State Tracking
Lai, C.-M., Hsu, M.-H., Huang, C., & Chen, Y.-N. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.5 · Cited by 6
Dialogue Term Extraction using Transfer Learning and Topological Data Analysis
Vukovic, R., Heck, M. C., Ruppik, B., van Niekerk, C., Zibrowius, M., & Gašić, M. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.53 · Cited by 13
Self-citationTemplate-based Approach to Zero-shot Intent Recognition
Lamanov, D., Burnyshev, P., Artemova, K., Malykh, V., Bout, A., & Piontkovskaya, I. (2022) · International Conference on Natural Language Generation · doi:10.18653/v1/2022.inlg-main.2 · Cited by 13
SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation
Jiang, J., Dong, C., Kurohashi, S., & Aizawa, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.249 · Cited by 18
Iizuka, S., Mochizuki, S., Ohashi, A., Yamashita, S., Guo, A., & Higashinaka, R. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27668 · Cited by 12
Travel Agency Task Dialogue Corpus: A Multimodal Dataset with Age-Diverse Speakers
Inaba, M., Chiba, Y., Qi, Z., Higashinaka, R., Komatani, K., Miyao, Y., & Nagai, T. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3675166 · Cited by 11
“Task-oriented dialogue systems have long been features in the study of dialogue [3, 4, 23, 36].”
BackgroundLeveraging Intent Entity Enhancement for Task-Oriented Dialogue
Chen, J., Li, S., Kan, B., Wang, F. L., & Hao, T. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649940 · Cited by 6
“Three publicly available standard datasets CamRest [22], SMD [21], and MWOZ [23] are utilized to validate the effectiveness of the proposed model.”
MethodTiwari, A., Saha, S., Sengupta, S., Maitra, A., Ramnani, R., & Bhattacharyya, P. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.76 · Cited by 11
“MultiWoz (Budzianowski et al., 2018) Service booking × × × × × intent, slot, dialogue act”
An, J., Cho, S., Bang, J., & Kim, M. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3181350 · Cited by 6
Vision Powered Conversational AI for Easy Human Dialogue Systems
Basnyat, B., Singh, N., Roy, N., & Gangopadhyay, A. (2020) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass50613.2020.00088 · Cited by 6
“Some examples of the Question-Answer Datasets for an academic research purpose are Question-Answer Wikipedia articles, WikiQA Corpus, Yahoo Language Data, TREC QA Collection, Multi-Domain Wizard-of-Oz dataset (MultiWOZ) [17, 18, 16, 8].”
MethodSOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain
Zhang, R., & Eickhoff, C. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.342 · Cited by 6
Variance-reduced First-order Meta-learning for Natural Language Processing Tasks
Wang, L., Huang, K., Ma, T., Gu, Q., & Huang, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.206 · Cited by 10
“Multi-domain dialog state tracking in MultiWOZ is a challenging task for meta-learning, due to the differences in dialogues between each domain.”
MethodQiu, Y., Zhang, J., & Zhou, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.178 · Cited by 6
“4 Experimental Setup et al., 2017), MSR-E2E (Li et al., 2018), MWOZ (Budzianowski et al., 2018), Schema (Ras-togi et al., 2020), SMD (Eric et al., 2017) and Taskmaster (Byrne et al., 2019)).”
BackgroundUnderstanding Dialogue for Human Communication
Magnini, B., & Louvan, S. (2022) · Springer eBooks · doi:10.1007/978-3-031-03945-4_20 · Cited by 8
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue
Zeng, W., He, K., Wang, Y., Zeng, C., Wang, J., Xian, Y., & Xu, W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.360 · Cited by 11
Context-Dependent Embedding Utterance Representations for Emotion Recognition in Conversations
Pereira, P., Moniz, H., Dias, I., & Carvalho, J. P. (2023) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2023.wassa-1.21 · Cited by 11
Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response GenerationPreprint
Pei, J., Ren, P., Monz, C., & de Rijke, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08151 · Cited by 6
Parthasarathi, P., Pineau, J., & Chandar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.10427 · Cited by 6
“0 (Budzianowski et al., 2018) – with goal-oriented dialogues and PersonaChat (Zhang et al.”
BackgroundMethodInfluentialRetrieval-based Goal-Oriented Dialogue GenerationPreprint
González, A. V., Augenstein, I., & Søgaard, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.13717 · Cited by 6
“In addition to these metrics, we assess our performance using the dialogue success metrics typically used in belief tracking (Budzianowski et al., 2018).”
BackgroundMethodInfluentialKumar, A., Eugenio, B. D., Bhattacharya, A., Aurisano, J., & Johnson, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.02215 · Cited by 6
“We believe work like ours complements work conducted on large datasets such as Multiwoz (Budzianowski et al., 2018), which are somewhat artificially generated.”
MethodUsing Textual Interface to Align External Knowledge for End-to-End Task-Oriented Dialogue SystemsPreprint
Wu, Q., Alnuhait, D., Chen, D., & Zhou, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13710 · Cited by 6
Alexpaca: Learning Factual Clarification Question Generation Without ExamplesPreprint
Toles, M., Huang, Y., Yu, Z., & Gravano, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.11571 · Cited by 6
“What information is missing, such as the desired price range, is usually easily defined by which slots are empty (Budzianowski et al. 2018).”
BackgroundA Multi-Task BERT Model for Schema-Guided Dialogue State Tracking
Kapelonis, E., Georgiou, E., & Potamianos, A. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10852 · Cited by 5
Learning With an Open Horizon in Ever-Changing Dialogue Circumstances
Geishauser, C., van Niekerk, C., Lubis, N., Lin, H.-C., Heck, M. C., Feng, S., Ruppik, B., Vukovic, R., & Gašić, M. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3385289 · Cited by 10
Self-citationGenerative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions
Finch, J. D., Josyula, Y., & Choi, J. D. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.66 · Cited by 5
AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
Kim, T., Singh, J., Mehri, S., Acikgoz, E. C., Mukherjee, S., Bozdag, N. B., Shashidhar, S., Tür, G., & Hakkani-Tur, D. Z. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3695295 · Cited by 5
“One of the most impactful planning applications is task planning , where agents generate and execute domain-specific plans (e.g., itineraries) to help users achieve goals while adhering to contextual constraints [4], [20], [33].”
Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention
Yang, L., Li, J., Li, S., & Shinozaki, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.304 · Cited by 5
Cai, Y., Liu, H., Ou, Z., Huang, Y., & Feng, J. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.44 · Cited by 7
Data-to-text generation using conditional generative adversarial with enhanced transformer
Seifossadat, E., & Sameti, H. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000487 · Cited by 6
“2020) is the GPT-2 model pretrained on a large D2T corpus and are fine-tuned on the MultiWoz dataset.”
InfluentialTurn-Level Active Learning for Dialogue State Tracking
Zhang, Z., Fang, M., Ye, F., Chen, L., & Namazi‐Rad, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.478 · Cited by 5
Characterizing the Complexity and Its Impact on Testing in ML-Enabled Systems : A Case Sutdy on Rasa
Cao, J., Chen, B., Hu, L., Gao, J., Huang, K., Song, X., & Peng, X. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00034 · Cited by 5
“A task-oriented dialogue system (TDS) aims to assist users in performing specific tasks, such as restaurant booking and flight booking [15].”
Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling
Heck, M. C., Geishauser, C., Lubis, N., van Niekerk, C., Feng, S., Lin, H.-C., Ruppik, B., Vukovic, R., & Gašić, M. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34580 · Cited by 5
Self-citationStyle Variation as a Vantage Point for Code-Switching
Chandu, K. R., & Black, A. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2574 · Cited by 5
Self-supervised attention flow for dialogue state tracking
Pan, B., Yang, Y., Li, B., & Cai, D. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.118 · Cited by 5
Multi-Modal Dialogue Policy Learning for Dynamic and Co-operative Goal Setting
Tiwari, A., Saha, T., Saha, S., Sengupta, S., Maitra, A., Ramnani, R., & Bhattacharyya, P. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533878 · Cited by 5
Addressing Slot-Value Changes in Task-oriented Dialogue Systems Through Dialogue Domain Adaptation
Fondazione Bruno Kessler, T., Labruna, T., Magnini, B., & Fondazione Bruno Kessler, T. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_089 · Cited by 5
Recognising Flexible Intents and Multiple Domains in Extended Human-Robot Dialogues
Wilcock, G. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-96451-1_13 · Cited by 5
Exploiting more associations between slots for multi-domain dialog state tracking
Bai, H., Yang, Y. K., & Wang, J. (2022) · Big Data Mining and Analytics · doi:10.26599/bdma.2021.9020013 · Cited by 5
“Think Before You Speak”: Improving Multi-Action Dialog Policy by Planning Single-Action Dialogs
Zhang, S., Zhao, J., Wang, P., Yu, L., Huang, Y., & Feng, J. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/626 · Cited by 5
Explaining Dialogue Evaluation Metrics using Adversarial Behavioral Analysis
Khalid, B., & Lee, S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.430 · Cited by 13
“We specifically focus on four conversational datasets Daily Dialogue (DDial) (Li et al., 2017b), Persona Chat (PerCh) (Zhang et al., 2018), Reddit (Gao et al., 2020) and MultiWOz (Budzianowski et al., 2018) to test our heuristics.”
BackgroundDeepPress: guided press release topic-aware text generation using ensemble transformers
Rahali, A., & Akhloufi, M. A. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08393-4 · Cited by 6
“Peng et al. [43] proposed SC-GPT2, a GPT2 model with 345M parameters that is pre-trained on a large data-to-text dialog corpus consisting of 400,000 examples and finally fine-tuned on MultiWoz [37].”
BackgroundMethodA dimensional model of interaction style variation in spoken dialog
Ward, N., & Avila, J. E. (2023) · Speech Communication · doi:10.1016/j.specom.2023.03.002 · Cited by 8
“For example, several recent large efforts have aimed to collect dialogs to exemplify a sweet-spot style that is simultaneously natural for humans and implementable with current technology [11, 12].”
BackgroundAlaswad, S., Kalganova, T., & Awad, W. (2023) · 2023 International Conference on IT Innovation and Knowledge Discovery (ITIKD) · doi:10.1109/itikd56332.2023.10099518 · Cited by 2
“One more thing, a report of benchmark results among various aspects is stated that enhance the usability of data and encourage for future work [20].”
BackgroundMosharrof, A., Maqbool, M. H., & Siddique, A. B. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36.133072 · Cited by 5
Liu, Y., Ultes, S., Minker, W., & Maier, W. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3597125 · Cited by 6
“Because most of taskoriented dialogues in MultiWOZ are multi-domain, the continuous prompt model learns the possibility, domain transitions, from latent high-dimensional continuous prompts.”
MethodSelf-citationComplex Reasoning in Natural Languag
Zhao, W., Geva, M., Lin, B. Y., Yasunaga, M., Madaan, A., & Yu, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-tutorials.2 · Cited by 12
Huang, Y., Lei, W., Fu, J., & Lv, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.129 · Cited by 5
Ohashi, A., & Higashinaka, R. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.1 · Cited by 9
TOD-Flow: Modeling the Structure of Task-Oriented Dialogues
Sohn, S., Lyu, Y., Liu, A., Logeswaran, L., Kim, D.-K., Shim, D., & Lee, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.204 · Cited by 9
“We show 2 such graphs in Figure 6 for RideSharing_1 schema and Figure 7 for RentalCars_1 schema respectively. of MultiWOZ (bottom).”
MethodInfluentialScalable-DSC: A Structural Template Prompt Approach to Scalable Dialogue State Correction
Su, H., Xie, H., Huang, H., Song, S., Fang, R., Huang, X., & Feng, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.490 · Cited by 5
“We adopt MultiWOZ 2.0-2.4(Budzianowski et al., 2018; Eric et al., 2020; Zang et al., 2020; Han et al., 2021; Ye et al., 2021a) as the datasets in our experiments.”
MethodInfluentialDSTEA: Improving Dialogue State Tracking via Entity Adaptive pre-training
Lee, Y., Kim, T., Yoon, H., Kang, P., Bang, J., & Kim, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111542 · Cited by 5
“2 (Budzianowski et al., 2018; Eric et al., 2019; Zang et al., 2020), the most widely used datasets for DST.”
BackgroundModel discrepancy policy optimization for task-oriented dialogue
Zhou, Z., Liu, Z., Dong, Z., & Liu, Y. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101636 · Cited by 7
Developing Conversational Intelligent Tutoring for Speaking Skills in Second Language Learning
Lee, J., Huang, J., Cho, M., Roh, Y., Kwon, O., & Lee, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63028-6_11 · Cited by 13
Table-to-Text Generation With Pretrained Diffusion Models
Krylov, A. S., & Somov, O. (2024) · IEEE Access · doi:10.1109/access.2024.3440006 · Cited by 6
“Mul-tiWoz [3] consists of 10K human-human dialogues for developing task-oriented dialogue systems with approximately 56K training samples.”
BackgroundMethodKumar, S., Bhatia, S., Aggarwal, M., & Chakraborty, T. (2024) · Natural language processing. · doi:10.1017/nlp.2024.42 · Cited by 5
Tseng, B.-H., Budzianowski, P., Wu, Y.-C., & Gašić, M. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5920 · Cited by 5
“We perform our experiments with the Multi-Domain Wizard-of-Oz (MultiWOZ) dataset (Budzianowski et al., 2018) that is a rich dialogue dataset spanning over 7 domains.”
BackgroundMethodSelf-citationGated Attentive Convolutional Network Dialogue State Tracker
Liu, S., Liu, S., & Xu, W. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054225 · Cited by 5
“Besides, on MultiWoZ [2], we achieve 49.2% goal accuracy.”
BackgroundMethodInfluentialTask-Oriented Dialog Systems for Dravidian Languages
Kanakagiri, T., & Radhakrishnan, K. (2021) · Cited by 6
Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory
Li, Y., Yang, Y., Quan, X., & Yu, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.39 · Cited by 5
Projection of Turn Completion in Incremental Spoken Dialogue Systems
Ekstedt, E., & Skantze, G. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.45 · Cited by 11
Informed Sampling for Diversity in Concept-to-Text NLG
Zhou, G., & Λάμπουρας, Γ. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.213 · Cited by 5
“Specifically, the MultiWOZ dataset (Budzianowski et al., 2018) provides 1872 distinct references for the MR [INFORM(WELCOME); INFORM(BYE)], e.”
MethodInfluentialIntention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue
Ma, Z., Li, J., Zhang, Z., Li, G., & Cheng, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.174 · Cited by 7
“We follow Eric et al. (2017), Madotto et al. (2018) and Wu et al. (2019) to partition SMD, and follow Budzianowski et al. (2018) and Qin et al. (2020) to partition Multi-WOZ 2.1.”
MethodInfluentialA Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation
Campagna, G., Semnani, S. J., Kearns, R., Sato, L. J. K., Xu, S., & Lam, M. S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.317 · Cited by 5
Enhanced Multi-Domain Dialogue State Tracker With Second-Order Slot Interactions
Jiao, F., Guo, Y., Huang, M., & Nie, L. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3221044 · Cited by 6
“[1] have extended the dialogue information into multiple domains, posing new challenges to the community.”
BackgroundX-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents
Moradshahi, M., Shen, T., Bali, K., Choudhury, M., de Chalendar, G., Goel, A., Kim, S., Kodali, P., Kumaraguru, P., Semmar, N., et al. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.174 · Cited by 15
End-to-End Task-Oriented Dialogue Systems Based on Schema
Imrattanatrai, W., & Fukuda, K. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.645 · Cited by 5
“As opposed to other datasets such as MultiWOZ (Budzianowski et al., 2018) which only defines a single service per domain, the SGD dataset was specifically designed to cover multi-turn dialogue conversations concerning the presence of multiple services per domain.”
MethodResultLiu, H., Cai, Y., Ou, Z., Huang, Y., & Feng, J. (2022) · doi:10.18653/v1/2022.seretod-1.10 · Cited by 9
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows
Zhao, J., Li, Y., Du, W., Ji, Y., Yu, D., Lyu, M. R.-T., & Wang, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.715 · Cited by 8
Ontology-Based Dialogue System for Domain-Specific Knowledge Acquisition
Iga, V. I. R., & Silaghi, G. C. (2023) · Proceedings of the International Conference on Information Systems Development · doi:10.62036/isd.2023.46 · Cited by 5
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging
Yavuz, S., Hashimoto, K., Liu, W., Keskar, N. S., Socher, R., & Xiong, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.412 · Cited by 6
“However, unlabeled dialogue corpora in target domain can easily be curated from past conversation logs or collected via crowd-sourcing (Byrne et al., 2019; Budzianowski et al., 2018) at a more reasonable cost.”
BackgroundAdjusting Image Attributes of Localized Regions with Low-level DialoguePreprint
Lin, T.-H., Rudnicky, A. I., Bui, T., Kim, D. S., & Oh, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04678 · Cited by 5
“It is easy to imagine a dialogue for information-seeking domains (Raux et al., 2005; Bohus and Rudnicky, 2009; Dhingra et al., 2017; Wen et al., 2017; El Asri et al., 2017; Budzianowski et al., 2018) such as restaurant booking or movie booking, as these conversations exist in everyday life.”
BackgroundImproving Longer-range Dialogue State TrackingPreprint
Zhang, Y., Cao, Y. J., Mahdieh, M., Zhao, J. W., & Wu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.00109 · Cited by 5
Towards a Neural Era in Dialogue Management for Collaboration: A Literature SurveyPreprint
Mannekote, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.09021 · Cited by 8
“The lack of collaborative analogues of large benchmark corpora such as MultiWOZ (Budzianowski et al., 2018) and SchemaGuided Dialog (Rastogi et al., 2020; Mehri and Eskenazi, 2021) as well as shared community challenges such as DSTC (Williams et al., 2014b), which were instrumental in catapulting…”
BackgroundLiu, J., Tan, Y. H., Fu, B., & Lim, K. H. (2024) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.48550/arxiv.2411.12307 · Cited by 7
“Early methods used bidirectional contextual LSTMs (Ghosal et al. 2021) to capture context-aware utterance representations for tasks such as MultiWOZ intent classification (Budzianowski et al. 2018).”
Prior Lessons of Incremental Dialogue and Robot Action Management for the Age of Language Models
Kennington, C., Lison, P., & Schlangen, D. (2025) · Dialogue & Discourse · doi:10.5210/dad.2025.305 · Cited by 4
“Modular SDS s are often task-based in that they help the user achieve a goal such as booking a flight; they do not usually focus on social aspects beyond what helps to accomplish the task (Budzianowski et al., 2018; Zhang et al., 2020b).”
Beyond More Context: Retrieval Diversity Boosts Multi-Turn Intent Understanding
Lin, Z., Xiao, C., & Zhao, K. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792567 · Cited by 7
“𝑆 Φ is a log-odds style verifier, 𝜏 𝑐 > 0 is a calibration temperature.”
Learning-to-Explain through 20Q Gaming: An Explainable Recommender for Cybersecurity Education
Nusrat, M., Hossain, G., Kinshuk, & Bhuiyan, S. (2025) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon67450.2025.11381070 · Cited by 4
Priya, P., Firdaus, M., & Ekbal, A. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101661 · Cited by 5
OmniDialog: An Omnipotent Pre-Training Model for Task-Oriented Dialogue System
Yang, M., Ng, S.-K., & Fu, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3542301 · Cited by 7
“Here, we consider the MultiWOZ 2.0 benchmark [49].”
InfluentialThe Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
Baidya, A., Das, K., & Gao, X. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1205 · Cited by 7
Migratable AI: Personalizing Dialog Conversations with Migration Context
Tejwani, R., Katz, B., & Breazeal, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-24667-8_8 · Cited by 5
Natural Language Interfaces to Data
Quamar, A., Efthymiou, V., Lei, C., & Özcan, F. (2022) · doi:10.1561/9781638280293 · Cited by 4
Metadata
- Authors
- Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gašić, M.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 5016-5026
- OpenAlex
- W2891732163
- Cited by
- 1,028
- References
- 48
- Citations by year
- 2026: 39 · 2025: 66 · 2024: 86 · 2023: 209 · 2022: 186 · 2021: 196 · 2020: 181 · 2019: 64 · 2018: 1
- FWCI
- 45.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M018946/1, RG80792)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- MultiWOZ, task-oriented dialogue, multi-domain dialogue, dialogue modeling, dialogue dataset, chatbots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.