MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Open accessThe DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Abstract
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The majority of evaluation protocols largely simplify experimental assumptions and design static, pre-defined rules, with relatively short and isolated task settings (Cobbe et al., 2021; Chen et al., 2021; Budzianowski et al., 2018).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some tasks have clear ground truth answers and can be scored with exact correctness (Du et al., 2024; Dunn et al., 2017), while others are open-ended (dialogue (Budzianowski et al., 2018), summarization (Maharana et al., 2024), preference following (Zhao et al., 2025c)) where multiple outputs are acceptable and reference-based scoring is not reliable.”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The former types of benchmarks are static and only test the conversational agent on pre-collected conversation trajectories[4, 3, 2].”
From $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 4×
“Among them, the first two metrics relate to the dialogue task completion - whether the system has provided an appropriate entity (In-form rate) and then answered all the requested attributes (Success rate); while fluency is measured via BLEU score (Papineni et al., 2002).”
BackgroundMethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“(2017) which frames the dialogue as a context to response mapping problem, a sequence-to-sequence model (Sutskever et al., 2014) is augmented with a belief tracker and a discrete database accessing component as additional features to inform the word decisions in the decoder.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“In the case of MultiWOZ dataset, the LSTM cell serving as a decoder and an encoder achieved the highest score with the global type of attention (Bahdanau et al., 2014).”
MethodMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“As this requires a multi-annotator metric over a multi-label task, we used Fleiss’ kappa metric (Fleiss, 1971) per single dialogue act.”
MethodA Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926 · cited 2×
“Due to the lack of an explicit goal in the conversation, recent studies have shown that systems trained with this type of corpus not only struggle in generating consistent and diverse responses (Li et al., 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · cited 2×
“Although previous work (Vinyals and Le, 2015) has shown that a large learning system can learn to generate interesting responses from these corpora, the lack of grounding conversations onto an existing knowledge base or APIs limits the usability of developed systems.”
BackgroundLiu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1230 · Cited by 1,228 · cited 3×
“, 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundPOMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882 · cited 4×
“These problems can either be addressed separately (Young et al., 2013), or jointly in an end-to-end fashion (Bordes et al., 2017; Wen et al., 2017; Li et al., 2017).”
BackgroundInfluentialSelf-citationSemantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874 · cited 5×
“To make a fair comparison, we constrained our dataset to only the restaurant sub-domain which contains around 25 k dia-Cam676 Table 5 presents the results on two metrics: BLEU score (Papineni et al., 2002) and slot error rate (SER) (Wen et al., 2015).”
BackgroundMethodInfluentialSelf-citationThe ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790 · cited 4×
“One of the earliest dataset collected in this fashion is the ATIS corpus (Hemphill et al., 1990), where conversations between a client and an airline help-desk operator were recorded.”
BackgroundMethodResultInfluential
Research works printed but not linked: 7
- Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. Proceedings of ICLR .
- Paweł Budzianowski, I˜ nigo Casanueva, Bo-Hsiang Tseng, and Milica Gaˇ si´ c. 2018. Towards endto-end multi-domain dialogue modelling. Tech. Rep. CUED/F-INFENG/TR.706, University of Cambridge, Engineering Department .
- Seokhwan Kim, Luis Fernando D'Haro, Rafael E Banchs, Jason D Williams, Matthew Henderson, and Koichiro Yoshino. 2016. The fifth dialog state tracking challenge. In Spoken Language Technology Workshop (SLT), 2016 IEEE , pages 511-517. IEEE.
- Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end taskcompletion neural dialogue systems. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , volume 1, pages 733-743.
- David R. Traum. 1999. Foundations of Rational Agency , chapter Speech Acts for Dialogue Agents. Springer.
- Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gaˇ si´ c, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A networkbased end-to-end trainable task-oriented dialogue system. EACL .
- Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference , pages 404-413.
Cited by
Items 801–900 of 1,004
Unsupervised Flow Discovery from Task-Oriented Dialogues
Ferreira, P., Martins, D., Alves, A., Silva, C., & Oliveira, H. G. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-78934-2_32 · Cited by 3
Dual Learning for Dialogue State Tracking
Chen, Z., Chen, L., Zhao, Y., Zhu, S., & Yu, K. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2401-1_26 · Cited by 3
Pretrain-Finetune Based Training of Task-Oriented Dialogue Systems in a Real-World Setting
Srivastava, M., Lu, Y., Peschon, R., & Li, C. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-industry.5 · Cited by 3
“A few synthetic datasets (Weston et al., 2015; Asri et al., 2017; Budzianowski et al., 2018) have been proposed but they do not capture the real-world variations and subtleties of the task-oriented dialogues.”
BackgroundModelling Personalized Dialogue Generation in Multi-Party Settings
Kumar, R. S., Chauhan, D. S., Dias, G. H., & Ekbal, A. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534278 · Cited by 3
Understanding Dialogue for Human Communication
Magnini, B., & Louvan, S. (2021) · Springer eBooks · doi:10.1007/978-3-030-44982-7_20-1 · Cited by 3
Towards a Progression-Aware Autonomous Dialogue Agent
Sanders, A., Strzalkowski, T., Si, M., Chang, A., Dey, D., Braasch, J., & Wang, D. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.87 · Cited by 6
Spithourakis, G., Vulić, I., Lis, M., Casanueva, I., & Budzianowski, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.124 · Cited by 3
Self-citationTextual Datasets For Portuguese-Brazilian Language Models
Sanches, M. F., de Sá, J. M. C., Foerste, H. T. S., de Souza, R. R., dos Reis, J. C., & Villas, L. A. (2022) · Data Science Workshop · doi:10.5753/dsw.2022.224294 · Cited by 3
“The state of the art in NLP is moving fast, with new architectures [Vaswani et al. 2017], models [Devlin et al. 2018, Radford et al. 2019] and datasets [Budzianowski et al. 2018, Wagner et al. 2018] being created and shared.”
BackgroundMethodTopic Shift Detection in Chinese Dialogues: Corpus and Benchmark
Lin, J.-Y., Fan, Y., Jiang, F. X., Chu, X., & Li, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41682-8_11 · Cited by 3
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues
Cordier, T., Urvoy, T., Lefèvre, F., & Rojas-Barahona, L. M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.32 · Cited by 3
Olabiyi, O., Bhattarai, P., Bruss, C. B., & Kulis, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01693 · Cited by 3
“A combined score (Combined) is also reported using Combined = (Inform + Success) x 0.5 + BLEU as an overall quality measure, as suggested in Budzianowski et al. (2018).”
BackgroundMethodInfluentialData-Efficient Methods for Dialogue SystemsPreprint
Shalyminov, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02929 · Cited by 3
“…(Eric et al., 2017) with 3,000 dialogues in 3 goal-oriented domains, namely in-car navigation, weather information, and appointment scheduling, — MultiWOZ (Budzianowski et al., 2018, Eric et al., 2019, Zang et al., 2020) — a multi- domain, multi-task goal-oriented dataset with 10,000…”
BackgroundDual Slot Selector via Local Reliability Verification for Dialogue State TrackingPreprint
Guo, J., Shuang, K., Li, J., & Wang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12578 · Cited by 3
“Our proposed DSS-DST achieves state-of-theart joint accuracy on three of the most actively studied datasets: MultiWOZ 2.0 (Budzianowski et al., 2018), MultiWOZ 2.1 (Eric et al., 2019), and MultiWOZ 2.2 (Zang et al., 2020) with joint accuracy of 56.93%, 60.73%, and 58.04%.”
MethodInfluentialZero-shot Multi-Domain Dialog State Tracking Using Descriptive RulesPreprint
Altszyler, E., Brusco, P., Bassiou, N., BYRNES, J. J., & Vergyri, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13275 · Cited by 3
“0 dataset [17], following Ramadan et al.”
Multi-Step Dialogue Workflow Action PredictionPreprint
Ramakrishnan, R., Elenberg, E. R., Narangodage, H., & McDonald, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.09593 · Cited by 3
“Instead, MultiWoz is better suited for Dialogue State Tracking, where the system needs to identify user intents and slot values.”
MethodInfluentialEfficient Encoder-Decoder Transformer Decoding for Decomposable TasksPreprint
Lu, B.-R., Haduong, N., Lin, C.-Y., Cheng, H., Smith, N. A., & Ostendorf, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.13112 · Cited by 3
“Multi-domain Wizard-of-Oz dataset (MultiWoZ)[2] is a task-oriented dialogue dataset.”
BackgroundMaronikolakis, A., Ramallo, A. P., Cheng, W., & Kober, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.08907 · Cited by 3
A simulated dataset for proactive robot task inference from streaming natural language dialogues
Xu, H., Li, C., Yuan, X., Tao, Z., & Liu, H. (2025) · Scientific Data · doi:10.1038/s41597-025-05727-w · Cited by 7
“MultiWOZ 28 represents a widely studied case of multi-domain, task-oriented dialogue systems, covering scenarios such as hotel booking and restaurant reservations.”
Agent WARPP: Workflow Adherence via Runtime Parallel PersonalizationPreprint
Mazzolenis, M. E., & Zhang, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.19543 · Cited by 5
“Existing TOD datasets like MultiWOZ (Budzianowski et al., 2020), SGD (Rastogi et al., 2020), STAR (Mosig et al.,”
AWARE-US: Preference-Aware Infeasibility Resolution in Tool-Calling AgentsPreprint
Kurmaz, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.02643 · Cited by 3
Mbaye, D., Mbengue, T., Seye, M. R., Diallo, M., Ndiaye, M. L., Adjanohoun, D. S., Wade, C. S., Sow, D., Munyaka, J.-C. B., & Chenal, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.09716 · Cited by 3
“The corpus consists of 1,500 turns each, which has been translated from a portion of the English Multi-Domain MultiWOZ dataset (Budzianowski et al., 2020) to enable the creation of dialogue agents for African languages.”
Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented DialoguePreprint
Gao, N., Zhang, W., Dai, Y., Shi, L., Wang, Z., Wang, Y., He, W., Wang, J., & Wang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.22697 · Cited by 3
Proactive Memory for Ad-Hoc Recall over Streaming DialoguesPreprint
Wang, B., Li, J., & Xu, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.04885 · Cited by 3
“This line of research reflects a progressive shift from traditional single-turn modeling (Budzianowski et al., 2018; Wei et al., 2018) toward long-term dialogue understanding.”
Somasekharan, N., Hassan, Y., Lin, S., Panapitiya, G., Emami, P., Acharya, A., Horawalavithana, S., & Pan, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.18630 · Cited by 5
“…M IRROR B ENCH , reliable-simulator work, S IMULA - TOR A RENA , and non-collaborative simulators analyze when simulated users preserve or distort measured assistant quality [28, 54, 20, 56]; and broader task-oriented and social-simulation work provides context for this protocol [15, 49, 4, 11].”
Lu, J., Holleis, T., Zhang, Y., Aumayer, B., Nan, F., Bai, F., Ma, S., Ma, S., Li, M., Yin, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.04682 · Cited by 2
“Contrary to prior approaches like dialog state tracking (Henderson et al., 2014; Budzianowski et al., 2018; Rastogi et al., 2020), which require the model to explicitly generate dialog states and actions under a predefined ontology, and derive a tool call from those structured outputs, tool-use…”
On the Complexity in Task-oriented Spoken Dialogue Systems
Ultes, S., & Maier, W. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406138 · Cited by 2
Self-citationIvCDS: An End-to-End Driver Simulator for Personal In-Vehicle Conversational Assistant
Ji, T., Yin, X., Cheng, P., Zhou, L., Liu, S., Bao, W., & Lyu, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315493 · Cited by 2
“However, such success heavily relies on the availability of relevant datasets such as MultiWOZ, KVRET and BANKING77 [24,30,31], which usually needs rich human resources to construct because the annotation of user intent and response requires interaction with human annotators.”
BackgroundMethodOn the Underspecification of Situations in Open-domain Conversational Datasets
Otani, N., Araki, J., Kim, H., & Hovy, E. H. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.2 · Cited by 2
“…generally allocate dedicated representations for essential situational information such as physical signals (Haber et al., 2019; Moon et al., 2020) and task-specific information or domain knowledge (Budzianowski et al., 2018), but their coverage is limited to one or a few specialized domains.”
BackgroundTask-based dialogue policy learning based on diffusion models
Liu, Z., Pang, R., & Dong, Z. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05810-6 · Cited by 2
“The dataset used for the experiments is MultiWOZ [49].”
Ohashi, A., & Higashinaka, R. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34681 · Cited by 2
AgentTOD: A Task-Oriented Dialogue Agent with a Flexible and Adaptive API Calling Paradigm
Xu, H.-D., Mao, X.-L., Sun, F., Che, T.-Y., Xu, C., & Huang, H. (2025) · ACM Transactions on Information Systems · doi:10.1145/3745021 · Cited by 5
Zhao, Y., Niu, B., Qin, L., & Wang, S. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.171 · Cited by 2
Adaptive-TOD: An LLM-driven and adaptive agent for diverse interaction modes
Li, P., Yang, Q., Xu, S., Li, X., Li, Z., Chen, W., Liu, Y., Guo, T., Tang, J., & Wen, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130991 · Cited by 4
Unbiased multimodal intent recognition with auxiliary rationale generation
Dong, Q., Dai, R., Duan, G., Qin, K., Zhang, Y., & He, T. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131197 · Cited by 7
Discourse-Aware Language Representation
Zhang, Z., Ouyang, S., & Zhao, H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3621229 · Cited by 2
“According to answer styles, conversational QA tasks can be roughly divided into generative [48] (to generate answer texts to given questions), extractive [44], [45] (to extract spans from given contexts to answer questions) and multi-choice [5] (to select the most appropriate answer among given…”
A Hybrid Pipeline and Large Language Model System for Task-Oriented Dialogue
Zheng, Y., Tan, W., Li, Q., Liu, L., Yan, Z., & Dai, H. (2025) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc67675.2025.00069 · Cited by 2
“We evaluate the proposed hybrid dialogue system framework on the standard MultiWOZ 2.2 dataset [26], [27].”
InfluentialSrinivas, V., Englhardt, Z., Iyer, V., & Patel, S. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1371 · Cited by 2
From answering to discussing: Advancing human-AI cognitive collaboration in dialogue agents
Li, P., Wang, J., Yang, Q., Guo, T., Liu, Y., & Wen, Y. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104711 · Cited by 3
Liu, J., Tan, Y. K., Fu, B., & Lim, K. H. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i47.41442 · Cited by 4
Khanal, A., Ranade, K., Agrawal, R., Basu, K. S., & Narayan, A. (2026) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/ggjl7344 · Cited by 2
Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents
Lin, X., Deng, Z., Wang, S., Yang, G. H., & Deng, Y. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.536 · Cited by 2
“Datasets such as MultiWOZ (Budzianowski et al., 2020), Schema-Guided Dialogue (Rastogi et al., 2020), Taskmaster (Byrne et al., 2019), etc., canonize that slice by framing the agent as a benevolent assistant whose sole duty is to satisfy explicit user requests.”
Comfrey: Mitigating Integration Failures in LLM-enabled Software at Run-Time
Shao, Y., Huang, Y., Zou, J., Shi, Y., Yang, L., Ma, L., Su, T., & Wan, C. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787847 · Cited by 2
“The conversation data are constructed from a multi-domain dialogue dataset MultiWOZ [11] its scenario, GPT-4o is adopted with a task-specific template designed by the authors ( e.g. , “generate 10 variations of laws and business domain”).”
Xuan, Z., Wu, Y., Yan, Z., Namboodiri, V., & Yang, Y. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831999 · Cited by 2
“Preference-centered datasets such as PrefEval and MultiWOZ [7, 59] focus on isolated choices or preference queries without longitudinal structure, so they cannot describe how preferences change as users live with an assistant.”
Goal-Embedded Dual Hierarchical Model for Task-Oriented Dialogue Generation
Lai, Y.-A., Gupta, A., & Zhang, Y. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1075 · Cited by 2
LS-DST: Long and Sparse Dialogue State Tracking with Smart History Collector in Insurance Marketing
Song, L., Yao, M., Bi, Y., Wu, Z., Wang, J., Xiao, J., Juan, W., & Yu, X. W. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463058 · Cited by 2
“MultiWOZ [2, 5]), they could not be applied to customer-to-agent dialogues directly for its specific characters.”
BackgroundGraph-based Dynamic Word Embeddings
Lu, Y., Cheng, X., Liang, Z., & Rao, Y. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/594 · Cited by 2
On Controlling Fallback Responses for Grounded Dialogue Generation
Lu, H., Lam, W. P., Cheng, H., & Meng, H. M. L. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.204 · Cited by 6
“While most of the dialogue datasets focus only on either chitchat (Zhang et al., 2018) or task-oriented dialogue (Budzianowski et al., 2018), CMU D O G in-terleaves chitchat and task-oriented dialogue (Zhou et al., 2018).”
BackgroundReflective action selection based on positive-unlabeled learning and causality detection model
Tanaka, S., Yoshino, K., Sudoh, K., & Nakamura, S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101463 · Cited by 4
DGAC: Dialogue Graph Auto Construction Based on Data with a Regular Structure
Nagovitsin, M. S., & Kuznetsov, D. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-19032-2_52 · Cited by 3
Sakai, Y., & Matsumoto, M. (2023) · IEEJ Journal of Industry Applications · doi:10.1541/ieejjia.22006784 · Cited by 2
Mosharrof, A., Maqbool, M. H., & Siddique, A. B. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36 · Cited by 2
Mitigating Negative Style Transfer in Hybrid Dialogue System
Li, S., Cheng, Q., Li, L., & Qiu, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26539 · Cited by 2
Syntactic and Semantic Uniformity for Semantic Parsing and Task-Oriented Dialogue Systems
Chen, B., & Miyao, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.60 · Cited by 2
“This is possibly because MultiWOZ2.”
BackgroundMethodInfluentialCan Sequence-to-Sequence Transformers Naturally Understand Sequential Instructions?
Zhou, X. S., Gupta, A., Upadhyay, S., Bansal, M., & Faruqui, M. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.45 · Cited by 3
“…models (Vaswani et al., 2017; Raffel et al., 2020) have shown remarkable performance on many natural language understanding tasks including semantic parsing (Yu et al., 2018), dialog state tracking (Budzianowski et al., 2018), procedure text understanding (Dalvi et al., 2018) etc.”
BackgroundAddressing Domain Changes in Task-oriented Conversational Agents through Dialogue Adaptation
Labruna, T., & Magnini, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2023.eacl-srw.16 · Cited by 2
“…aspects: first, assessing the impact of domain changes in current dialogue state tracking models; second taking advantage of the generative capacity of large pre-trained language models (e.g., (Chen et al., 2019) (Raffel et al., 2020) (Li et al., 2022)) to provide appropriate dialogue adaptations.”
BackgroundSummarization of Dialogues and Conversations At Scale
Yang, D., & Zhu, C. (2023) · doi:10.18653/v1/2023.eacl-tutorials.3 · Cited by 5
“The past decade also saw the emergence of many dialogue and conversation datasets such as MultiWoz (Budzianowski et al., 2018) and Ubuntu (Lowe et al., 2015).”
BackgroundAutomatic Dialog Flow Extraction and Guidance
Ferreira, P. (2023) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2023.eacl-srw.12 · Cited by 4
“Interaction can be done through any chat application, such as Slack or Microsoft Teams; Transcripts of existing dialogs in Portuguese, such as CORAA (Junior et al., 2021); Customer support services on social networks, such as conversations with telecom operators on Twitter 7 ; Movie sub-titles (Lison and Tiedemann, 20…”
BackgroundLiu, Q., He, S., Liu, C., Liu, K., & Zhao, J. (2023) · Journal of Computer Science and Technology · doi:10.1007/s11390-021-1064-y · Cited by 2
Combining Structured and Unstructured Knowledge in an Interactive Search Dialogue System
Stoyanchev, S., Pandey, S., Keizer, S., Braunschweiler, N., & Doddipatla, R. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.50 · Cited by 2
“…search can be modelled as task-oriented dialogue using structured knowledge, symbolic dialogue state representation, and a statistical policy that addresses both task and conversational phenomena, such as clarifications and social dialogue acts (Budzianowski et al., 2018; Yan et al., 2017).”
BackgroundYe, L., Ultes, S., Minker, W., & Maier, W. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.20 · Cited by 6
Self-citationEnhancing Task Bot Engagement with Synthesized Open-Domain Dialog
Li, M., Peng, B., Galley, M., Gao, J., & Zhang, Z. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.46 · Cited by 2
Sources of Noise in Dialogue and How to Deal with Them
Chen, D., & Zhou, Y. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.1 · Cited by 2
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation
Qin, C., Chen, C., & Joty, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.414 · Cited by 12
RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training
Tang, Y.-C., Wang, W., Yen, A.-Z., & Peng, W.-C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.698 · Cited by 5
Learning From Free-Text Human Feedback – Collect New Datasets Or Extend Existing Ones?
Petrak, D., Moosavi, N. S., Tian, Y., Rozanov, N., & Gurevych, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1011 · Cited by 10
Stricker, A., & Paroubek, P. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389695 · Cited by 2
“FusedChat [11] is developed using the well-known Mul-tiWOZ corpus [8] as its foundation.”
MethodPetouo, F., Arafat, Y., Mushabbir, M. A., Hasan, K., & Mahmud, H. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441070 · Cited by 2
“In general, the Multi-Domain Wizard-of-Oz [12] MultiWoz dataset proves to be a valuable resource for developing and evaluating task-oriented dialogue systems.”
BackgroundMethodInfluentialDialogue Segmentation based on Dynamic Context Coherence
Pu, H., & Wang, L. (2023) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3639233.3639340 · Cited by 2
“…[22]: A manually annotated English dataset, including 711 dialogues sampled from two existing multi-turn dialogue datasets: MultiWOZ Corpus [30] and Stanford Dialog Dataset [31], where each dialogue is about one topic; (2) Doc2Dial: A synthetic English dataset, consisting of 4,130…”
MethodSimulated Task Oriented Dialogues for Developing Versatile Conversational Agents
Wang, X., Sen, P., Li, R., & Yılmaz, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_10 · Cited by 3
Indonesian generative chatbot model for student services using GPT
Priccilia, S., & Girsang, A. S. (2024) · International Journal of Informatics and Communication Technology (IJ-ICT) · doi:10.11591/ijict.v13i1.pp50-56 · Cited by 2
“To explore the factors influencing the model's suboptimal performance, we conducted additional experiment using the publicly available MultiWOZ dataset [29].”
MethodSTN4DST: A Scalable Dialogue State Tracking Based on Slot Tagging Navigation
Yang, P., Huang, H., Shi, S., & Mao, X.-L. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3393733 · Cited by 2
“2 is the latest revision of MultiWOZ [40], a well-known task-oriented dialogue dataset containing over 10,000 annotated dialogues spanning 8 domains.”
BackgroundPriya, P., Firdaus, M., & Ekbal, A. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3421672 · Cited by 2
“EmoWOZ consists of Wizard-of-OZ [69] conversations from MultiWOZ [70] as well as human–machine conversations covering seven domains— Attraction, Hospital, Hotel, Police, Restaurant, Taxi , and Train .”
MethodPrompt-Based End-to-End Cross-Domain Dialogue State Tracking
Lu, H., Zhong, L., Jiang, H., Chen, W., Yuan, C., & Wang, X. (2024) · Electronics · doi:10.3390/electronics13183587 · Cited by 2
Su, H., Du, W., Wang, X., & Nguyen, C.-T. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681217 · Cited by 2
“For example, to train a conversational recommendation system, it is needed to collect diverse dialogue samples annotated with recommendations and various user requests [1, 19, 20].”
Improving Multi-Domain Task-Oriented Dialogue System with Offline Reinforcement Learning
Prajapat, D., & Toshniwal, D. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825582 · Cited by 2
Cooking with context: Leveraging context for procedural question answering
Frummet, A., Elsweiler, D., & Kruschwitz, U. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104212 · Cited by 3
Kim, S., Park, S., Jung, J., Kim, J., & Cho, S. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.523 · Cited by 4
All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue
Feng, S., Lin, H.-C., Lubis, N., van Niekerk, C., Heck, M. C., Ruppik, B., Vukovic, R., & Gašić, M. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.812 · Cited by 2
“All systems and sub-modules are trained on EmoWOZ, an extension of MultiWOZ (Budzianowski et al., 2018) annotated with user emotion and system emotional-conduct labels.”
Self-citationTARIC-SLU: A Tunisian Benchmark Dataset for Spoken Language Understanding
Mdhaffar, S., Bougares, F., Mori, R. D., Zaiem, S., Ravanelli, M., & Estève, Y. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4evgpxxgwme5 · Cited by 18
“The availability of English SLU systems has been driven, among other factors, by the availability of annotated resources such as ATIS (Hemphill et al., 1990), SNIPS (Coucke et al., 2018), Spoken SQuAD (Li et al., 2018), MultiWOZ (Budzianowski et al., 2018), the Fluent Speech Command (FSC) corpus (Lugosch et al., 2019)…”
BackgroundPoelitz, C., & McKenna, N. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2qropbkccqtp · Cited by 2
“Prior work on generating conversational data and dialogue acts with clarifications (Deng et al., 2023; Rahmani et al., 2023; Aliannejadi et al., 2021b; Budzianowski et al., 2020) often leveraged human annotation with Crowdsourcing making data generation expensive.”
Crowd Knowledge Enhanced Multimodal Conversational Assistant in Travel Domain
Liao, L., Kennedy, L. S., Wilcox, L. D., & Chua, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-37731-1_33 · Cited by 2
Interactive Learning of Dialog Scenarios from Examples
Deksne, D., & Skadiņš, R. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200607 · Cited by 2
“MultiWOZ Corpus [12] contains 3,406 single domain and 7,032 multi-domain task-oriented dialogs collected through the crowd work using the Wizard of Oz method.”
MethodInfluentialResource Constrained Dialog Policy Learning Via Differentiable Inductive Logic Programming
Zhou, Z., Beirami, A., Crook, P., Shah, P., Subba, R., & Geramifard, A. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.597 · Cited by 2
MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator
Cucurnia, D., Rozanov, N., Sucameli, I., Ciuffoletti, A., & Simi, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-demos.5 · Cited by 2
“This complexity is comparable with those of public dialogue datasets, like Multiwoz or Taskmaster-1 (Budzianowski et al., 2018; Byrne et al., 2019).”
BackgroundResultInfluentialARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions
Tanaka, S., Yoshino, K., Sudoh, K., & Nakamura, S. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.9 · Cited by 6
Slot Transferability for Cross-domain Slot Filling
Lu, H., Han, Z., Yuan, C., Wang, X., Lei, S., Jiang, H., & Wei, W. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.440 · Cited by 5
“In order to further evaluate the generalization ability of our approach, we also construct a cross-task slot filling dataset called MultiWoz-Slot (MWS) based on the multi-domain task-oriented dialog dataset MultiWoz (Budzianowski et al., 2018; Eric et al., 2020).”
MethodCrowdsourcing Ecologically-Valid Dialogue Data for German
Frommherz, Y., & Zarcone, A. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.686050 · Cited by 2
“Lately, however, the focus has shifted to large-scale collections, both due to demands of huge data-driven models and thanks to the availability of online data collection platforms (e.g., Wang et al., 2012; Eric et al., 2017; Wen et al., 2017; Budzianowski et al., 2018; Wei et al., 2020).”
BackgroundMethodInfluentialDialogStitch: Synthetic Deeper and Multi-Context Task-Oriented Dialogs
Kottur, S., Sankar, C., Zhou, Y., & Geramifard, A. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.3 · Cited by 2
Towards Handling Unconstrained User Preferences in Dialogue
Pandey, S. K., Stoyanchev, S., & Doddipatla, R. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-5538-9_6 · Cited by 2
Improving Dialogue State Tracking by Joint Slot Modeling
Chiang, T.-R., & Yeh, Y.-T. (2021) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2021.nlp4convai-1.15 · Cited by 2
Counterfactual Matters: Intrinsic Probing For Dialogue State Tracking
Huang, Y., Feng, J., Wu, X., & Du, X. (2021) · doi:10.18653/v1/2021.eancs-1.1 · Cited by 2
On selection of topics that users are interested in but are not familiar with
Sakai, Y., & Matsumoto, M. (2022) · International Power Electronics Conference (IPEC) · doi:10.23919/ipec-himeji2022-ecce53331.2022.9807252 · Cited by 2
“Examples include tourist information [2], virtual coach [3], restaurant search [4] and taxi booking [5].”
BackgroundA Novel Transformer-Based Model for Dialog State Tracking
Yu, M., Liu, K., Yang, W., & Yang, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06050-2_11 · Cited by 2
A Versatile Adaptive Curriculum Learning Framework for Task-oriented Dialogue Policy Learning
Zhao, Y., Qin, H., Wang, Z., Zhu, C., & Wang, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.54 · Cited by 2
Lightweight Transformers for Conversational AI
Pressel, D., Liu, W., Johnston, M. O., & Chen, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-industry.25 · Cited by 2
“This model is particularly strong on MultiWoz (Budzianowski et al., 2018), a common task-oriented dialogue benchmark, but the design is ill-suited for real-world system deployment – API calls and NLG are integrated directly into the LM, making the model difficult to adapt, control, and maintain over time.”
BackgroundBuilding Dialogue Understanding Models for Low-resource Language Indonesian from Scratch
Di, D., Song, X., Zhang, W., Zhang, Y., & Wang, F. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3575803 · Cited by 2
“To conduct extensive experiments for Indonesian, we follow the method of MultiWOZ [6], which is a large-scale task-oriented English dialogue dataset, to collect and annotate a counterpart and richer-domain dataset in ID, named ID-WOZ.”
BackgroundMethodInfluentialDu, M., Cheng, L., Xu, B., Wang, Z., Wang, S., Yuan, J., & Pan, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892082 · Cited by 2
“By contrast, available public datasets like MultiWOZ [4] cover a limited number of domains.”
BackgroundMethodReward estimation with scheduled knowledge distillation for dialogue policy learning
Qiu, J., Zhang, H., & Yang, Y. (2023) · Connection Science · doi:10.1080/09540091.2023.2174078 · Cited by 2
“The performances of agents at different epochs on the Microsoft Dialogue Challenge and MultiWOZ datasets are presented in Tables 4 and 5, respectively.”
MethodInfluentialIntroducing Multi-modality in Persuasive Task Oriented Virtual Sales Agent
Raut, A., Das, S., Tiwari, A., Saha, S., Maitra, A., Ramnani, R., & Sengupta, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30111-7_46 · Cited by 2
Metadata
- Authors
- Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gašić, M.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 5016-5026
- OpenAlex
- W2891732163
- Cited by
- 1,028
- References
- 48
- Citations by year
- 2026: 39 · 2025: 66 · 2024: 86 · 2023: 209 · 2022: 186 · 2021: 196 · 2020: 181 · 2019: 64 · 2018: 1
- FWCI
- 45.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M018946/1, RG80792)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- MultiWOZ, task-oriented dialogue, multi-domain dialogue, dialogue modeling, dialogue dataset, chatbots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.