MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Open accessThe DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Abstract
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The majority of evaluation protocols largely simplify experimental assumptions and design static, pre-defined rules, with relatively short and isolated task settings (Cobbe et al., 2021; Chen et al., 2021; Budzianowski et al., 2018).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some tasks have clear ground truth answers and can be scored with exact correctness (Du et al., 2024; Dunn et al., 2017), while others are open-ended (dialogue (Budzianowski et al., 2018), summarization (Maharana et al., 2024), preference following (Zhao et al., 2025c)) where multiple outputs are acceptable and reference-based scoring is not reliable.”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Budzianowski et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The former types of benchmarks are static and only test the conversational agent on pre-collected conversation trajectories[4, 3, 2].”
From $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 4×
“Among them, the first two metrics relate to the dialogue task completion - whether the system has provided an appropriate entity (In-form rate) and then answered all the requested attributes (Success rate); while fluency is measured via BLEU score (Papineni et al., 2002).”
BackgroundMethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“(2017) which frames the dialogue as a context to response mapping problem, a sequence-to-sequence model (Sutskever et al., 2014) is augmented with a belief tracker and a discrete database accessing component as additional features to inform the word decisions in the decoder.”
BackgroundMethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“In the case of MultiWOZ dataset, the LSTM cell serving as a decoder and an encoder achieved the highest score with the global type of attention (Bahdanau et al., 2014).”
MethodMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“As this requires a multi-annotator metric over a multi-label task, we used Fleiss’ kappa metric (Fleiss, 1971) per single dialogue act.”
MethodA Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926 · cited 2×
“Due to the lack of an explicit goal in the conversation, recent studies have shown that systems trained with this type of corpus not only struggle in generating consistent and diverse responses (Li et al., 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · cited 2×
“Although previous work (Vinyals and Le, 2015) has shown that a large learning system can learn to generate interesting responses from these corpora, the lack of grounding conversations onto an existing knowledge base or APIs limits the usability of developed systems.”
BackgroundLiu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1230 · Cited by 1,228 · cited 3×
“, 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundPOMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882 · cited 4×
“These problems can either be addressed separately (Young et al., 2013), or jointly in an end-to-end fashion (Bordes et al., 2017; Wen et al., 2017; Li et al., 2017).”
BackgroundInfluentialSelf-citationSemantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874 · cited 5×
“To make a fair comparison, we constrained our dataset to only the restaurant sub-domain which contains around 25 k dia-Cam676 Table 5 presents the results on two metrics: BLEU score (Papineni et al., 2002) and slot error rate (SER) (Wen et al., 2015).”
BackgroundMethodInfluentialSelf-citationThe ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790 · cited 4×
“One of the earliest dataset collected in this fashion is the ATIS corpus (Hemphill et al., 1990), where conversations between a client and an airline help-desk operator were recorded.”
BackgroundMethodResultInfluentialThe Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I., & Pineau, J. (2015) · Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4640 · Cited by 748 · cited 4×
“…two categories: corpora with structured semantic labels (Hemphill et al., 1990; Williams et al., 2013; Asri et al., 2017; Wen et al., 2017; Eric et al., 2017; Shah et al., 2018); and corpora without semantic labels but with an implicit user goal in mind (Ritter et al., 2010; Lowe et al., 2015).”
BackgroundInfluentialAn iterative design methodology for user-friendly natural language office information applications
Kelley, J. F. (1984) · ACM Transactions on Information Systems · doi:10.1145/357417.357420 · Cited by 702 · cited 3×
“The Wizard-of-Oz framework (WOZ) (Kelley, 1984) was first proposed as an iterative approach to improve user experiences when designing a conversational sys-tem.”
BackgroundInfluentialLiu, C., Lowe, R., Serban, I. V., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08023 · Cited by 603 · cited 3×
“, 2016) but are also extremely hard to evaluate (Liu et al., 2016).”
BackgroundThe Second Dialog State Tracking Challenge
Henderson, M., Thomson, B., & Williams, J. D. (2014) · Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) · doi:10.18653/v1/w14-4337 · Cited by 538 · cited 4×
“To illustrate the contribution of the new corpus, we compare it on several important statistics with the DSTC2 corpus (Henderson et al., 2014a), the SFX corpus (Gašić et al.”
MethodResultInfluentialUnsupervised Modeling of Twitter Conversations
Ritter, A., Cherry, C., & Dolan, B. (2010) · NPARC · Cited by 528
Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
Mesnil, G., Dauphin, Y., Yao, K., Bengio, Y., Deng, L., Hakkani-Tur, D., He, X., Heck, L., Tur, G., Yu, D., et al. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2383614 · Cited by 461 · cited 2×
“…of using statistical framework and machine learning for various system components, such as natural language understanding (Henderson et al., 2013; Mesnil et al., 2015; Mrkšić et al., 2017a), dialogue management (Gašić and Young, 2014; Tegho et al., 2018), language generation (Wen et al.,…”
BackgroundMethodNeural Belief Tracker: Data-Driven Dialogue State Tracking
Mrkšić, N., Ó Séaghdha, D., Wen, T.-H., Thomson, B., & Young, S. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1163 · Cited by 459 · cited 3×
“The corpus was later extended to additional two languages for cross-lingual research (Mrkˇsi´c et al., 2017b).”
BackgroundSelf-citationKey-Value Retrieval Networks for Task-Oriented Dialogue
Eric, M., Krishnan, L., Charette, F., & Manning, C. D. (2017) · Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w17-5506 · Cited by 388 · cited 7×
“, 2016), and even end-to-end dialogue modelling (Zhao and Eskenazi, 2016; Wen et al., 2017; Eric et al., 2017).”
BackgroundMethodInfluentialThe Information State Approach to Dialogue Management
Traum, D. R., & Larsson, S. (2003) · Text, Speech and Language Technology · doi:10.1007/978-94-010-0019-2_15 · Cited by 342 · cited 1×
“, 2013); the need to integrate third-party services and dialogue context in the decision-making (Traum and Larsson, 2003; Paek and Pieraccini, 2008); and finally, the ability to generate natural and engaging responses (Stent et al.”
BackgroundWord-Based Dialog State Tracking with Recurrent Neural Networks
Henderson, M., Thomson, B., & Young, S. (2014) · Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) · doi:10.18653/v1/w14-4340 · Cited by 337 · cited 2×
“This is in contrast to previous approaches (Henderson et al., 2014a) and such WOZ set-up has been successfully validated by Wen et al. (2017) and Asri et al. (2017).”
MethodResultThe Dialog State Tracking Challenge
Williams, J. D., Raux, A., Ramachandran, D., & Black, A. W. (2013) · Cited by 317
Zhao, T., & Eskenazi, M. (2016) · Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3601 · Cited by 255 · cited 2×
“, 2016), and even end-to-end dialogue modelling (Zhao and Eskenazi, 2016; Wen et al., 2017; Eric et al., 2017).”
BackgroundCONVERSATION ACTS IN TASK‐ORIENTED SPOKEN DIALOGUE
Traum, D. R., & Hinkelman, E. A. (1992) · Computational Intelligence · doi:10.1111/j.1467-8640.1992.tb00380.x · Cited by 229 · cited 2×
“One of the major challenges of this task is the definition of a set and structure of dialogue acts (Traum and Hinkelman, 1992; Bunt, 2006).”
BackgroundLet's go public! taking a spoken dialog system to the real world
Raux, A., Langner, B., Bohus, D., Black, A. W., & Eskenazi, M. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-399 · Cited by 226 · cited 4×
“One famous example of this kind is the Let’s Go Bus Information System which offers live bus schedule information over the phone (Raux et al., 2005) leading to the first Di-alogue State Tracking Challenge (Williams et al., 2013).”
BackgroundInfluentialGlobally Coherent Text Generation with Neural Checklist Models
Kiddon, C., Zettlemoyer, L., & Choi, Y. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1032 · Cited by 219 · cited 2×
“, 2018), language generation (Wen et al., 2015; Kiddon et al., 2016), and even end-to-end dialogue modelling (Zhao and Eskenazi, 2016; Wen et al.”
BackgroundFrames: a corpus for adding memory to goal-oriented dialogue systems
El Asri, L., Schulz, H., Sharma, S., Zumer, J., Harris, J., Fine, E., Mehrotra, R., & Suleman, K. (2017) · Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w17-5526 · Cited by 212 · cited 6×
“Subsequently, this approach is followed by Asri et al. (2017) to collect the Frame corpus in a more complex travel booking domain, and Eric et al. (2017) to collect a corpus of conversations for in-car navigation.”
BackgroundMethodResultInfluentialConversational AI: The Science Behind the Alexa Prize
Ram, A., Prasad, R., Khatri, C., Venkatesh, A., Gabriel, R., Liu, Q., Nunn, J., Hedayatnia, B., Cheng, M., Nagar, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.03604 · Cited by 185 · cited 1×
“As the need of hands-free use cases continues to grow, building a conversational agent that can handle tasks across different application domains has become more and more prominent (Ram et al., 2018).”
BackgroundMrkšić, N., Vulić, I., Séaghdha, D. Ó., Leviant, I., Reichart, R., Gašić, M., Korhonen, A., & Young, S. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00063 · Cited by 170
Self-citationMulti-domain Neural Network Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gašić, M., Mrkšić, N., Rojas-Barahona, L. M., Su, P.-H., Vandyke, D., & Young, S. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1015 · Cited by 167 · cited 3×
“What is even worse is that the capability of the initial system introduces additional biases to the collected data, which may result in a mismatch between the training and testing sets (Wen et al., 2016).”
BackgroundMethodInfluentialSelf-citationDialogue management in the Mercury flight reservation system
Seneff, S., & Polifroni, J. (2000) · ANLP/NAACL 2000 Workshop on Conversational systems - · doi:10.3115/1117562.1117565 · Cited by 162
The third Dialog State Tracking Challenge
Henderson, M., Thomson, B., & Williams, J. D. (2014) · 2014 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078595 · Cited by 158 · cited 2×
“This is in contrast to previous approaches (Henderson et al., 2014a) and such WOZ set-up has been successfully validated by Wen et al. (2017) and Asri et al. (2017).”
MethodResultLarge-Scale Multi-Domain Belief Tracking with Knowledge Sharing
Ramadan, O., Budzianowski, P., & Gašić, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2069 · Cited by 158 · cited 1×
“The proposed method (Ramadan et al., 2018) exploits the semantic similarity between dialogue utterances and the ontology terms which allows the information to be shared across domains.”
MethodSelf-citationStochastic language generation for spoken dialogue systems
Oh, A. H., & Rudnicky, A. I. (2000) · ANLP/NAACL 2000 Workshop on Conversational systems - · doi:10.3115/1117562.1117568 · Cited by 151 · cited 2×
“Natural Language Generation from a structured meaning representation (Oh and Rudnicky, 2000; Bohus and Rudnicky, 2005) has been a very popular research topic in the community, and the lack of data has been a long standing block for the field to adopt more machine learning methods.”
BackgroundThe Fourth Dialog State Tracking Challenge
Kim, S., D’Haro, L. F., Banchs, R. E., Williams, J. D., & Henderson, M. (2016) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-2585-3_36 · Cited by 151
Building a Conversational Agent Overnight with Dialogue Self-Play
Shah, P., Hakkani-Tür, D. Z., Tür, G., Rastogi, A., Bapna, A., Nayak, N., & Heck, L. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04871 · Cited by 147 · cited 4×
“…two categories: corpora with structured semantic labels (Hemphill et al., 1990; Williams et al., 2013; Asri et al., 2017; Wen et al., 2017; Eric et al., 2017; Shah et al., 2018); and corpora without semantic labels but with an implicit user goal in mind (Ritter et al., 2010; Lowe et al., 2015).”
BackgroundMethodInfluentialAutomating spoken dialogue management design using machine learning: An industry perspective
Paek, T., & Pieraccini, R. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.010 · Cited by 135 · cited 2×
“…ambiguity when understanding human language (Williams et al., 2013); the need to integrate third-party services and dialogue context in the decision-making (Traum and Lars-son, 2003; Paek and Pieraccini, 2008); and finally, the ability to generate natural and engaging responses (Stent et al., 2005).”
BackgroundDeep Neural Network Approach for the Dialog State Tracking Challenge
Henderson, M., Thomson, B., & Young, S. J. (2013) · Cited by 133
Evaluating Evaluation Methods for Generation in the Presence of Variation
Stent, A., Marge, M., & Singhai, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30586-6_38 · Cited by 126 · cited 2×
“…ambiguity when understanding human language (Williams et al., 2013); the need to integrate third-party services and dialogue context in the decision-making (Traum and Lars-son, 2003; Paek and Pieraccini, 2008); and finally, the ability to generate natural and engaging responses (Stent et al., 2005).”
BackgroundGaussian Processes for POMDP-Based Dialogue Manager Optimization
Gasic, M., & Young, S. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2282190 · Cited by 123
Self-citationScalable multi-domain dialogue state tracking
Rastogi, A., Hakkani-Tur, D., & Heck, L. (2017) · 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru.2017.8268986 · Cited by 110 · cited 2×
“Since multi-domain dialogue state tracking is still in its infancy and there are not many comparable approaches available (Rastogi et al., 2017), we instead report our state-of-the-art result on the restaurant subset of the MultiWOZ corpus as the reference baseline. proposed method (Ramadan et al.,…”
BackgroundSorry, I Didn’t Catch That! An Investigation of Non-understanding Errors and Recovery Strategies
Bohus, D., & Rudnicky, A. I. (2018) · doi:10.1184/r1/6609683 · Cited by 109
An Analysis of Domestic Abuse Discourse on Reddit
Schrading, N., Ovesdotter Alm, C., Ptucha, R., & Homan, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1309 · Cited by 86 · cited 2×
“, 2010) dataset, the Reddit conversations (Schrading et al., 2015), and the Ubuntu technical support corpus (Lowe et al.”
BackgroundIncremental on-line adaptation of POMDP-based dialogue managers to extended domains
Gašić, M., Kim, D., Tsiakoulis, P., Breslin, C. J., Henderson, M., Szummer, M., Thomson, B., & Young, S. J. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-40 · Cited by 76 · cited 2×
“To illustrate the contribution of the new corpus, we compare it on several important statistics with the DSTC2 corpus (Henderson et al., 2014a), the SFX corpus (Gaˇsi´c et al., 2014), the WOZ 2 .”
MethodSelf-citationSpoken Dialog Challenge 2010: Comparison of Live and Control Test Results
Black, A. W., Burger, S., Conkie, A. D., Hastie, H., Keizer, S., Lemon, O., Merigaud, N., Parent, G., Schubiner, G., Thomson, B., et al. (2011) · Cited by 64
Dimensions in Dialogue Act Annotation
Bunt, H. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jchd238vsa7 · Cited by 62 · cited 2×
“One of the major challenges of this task is the definition of a set and structure of dialogue acts (Traum and Hinkelman, 1992; Bunt, 2006).”
BackgroundBenchmarking Uncertainty Estimates with Deep Reinforcement Learning for Dialogue Policy Optimisation
Tegho, C., Budzianowski, P., & Gašić, M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462348 · Cited by 4 · cited 2×
“, 2017a), dialogue management (Gašić and Young, 2014; Tegho et al., 2018), language generation (Wen et al.”
BackgroundSelf-citation
Research works printed but not linked: 7
- Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. Proceedings of ICLR .
- Paweł Budzianowski, I˜ nigo Casanueva, Bo-Hsiang Tseng, and Milica Gaˇ si´ c. 2018. Towards endto-end multi-domain dialogue modelling. Tech. Rep. CUED/F-INFENG/TR.706, University of Cambridge, Engineering Department .
- Seokhwan Kim, Luis Fernando D'Haro, Rafael E Banchs, Jason D Williams, Matthew Henderson, and Koichiro Yoshino. 2016. The fifth dialog state tracking challenge. In Spoken Language Technology Workshop (SLT), 2016 IEEE , pages 511-517. IEEE.
- Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end taskcompletion neural dialogue systems. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , volume 1, pages 733-743.
- David R. Traum. 1999. Foundations of Rational Agency , chapter Speech Acts for Dialogue Agents. Springer.
- Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gaˇ si´ c, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A networkbased end-to-end trainable task-oriented dialogue system. EACL .
- Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference , pages 404-413.
Cited by
A Survey on Deep Semi-Supervised Learning
Yang, X., Song, Z., King, I., & Xu, Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3220219 · Cited by 923
“Answers [106], IMDB [107], Yelp review [104], Snippets [108], Ohsumed [109], TagMyNews [110], MR [111], Elec [112], Rotten Tomatoes [111], RCV1 [113] [114], [115], [116], [117], [118], [119] Dialogue policy learning MultiWOZ [120] [121] Dialogue generation Cornell Movie Dialogs Corpus [122], Ubuntu Dialogue Corpus [12…”
BackgroundLongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2024) · arXiv · doi:10.48550/arxiv.2410.10813 · Cited by 493
“Long-Term Dialogue Benchmarks As the ability of dialogue systems improve, research starts to focus on long-term dialogue understanding beyond traditional dialogue modeling benchmarks (Budzianowski et al., 2018; Wei et al., 2018).”
Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset
Rastogi, A., Zang, X., Sunkara, S., Gupta, R., & Khaitan, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6394 · Cited by 450
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., & Fung, P. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1078 · Cited by 415
“One of the motivations to perform unseen domain DST is because collecting a large-scale task-oriented dataset for a new domain is expensive and time-consuming (Budzianowski et al., 2018), and there are a large amount of domains in realistic scenarios.”
BackgroundTowards conversational diagnostic artificial intelligence
Tu, T., Schaekermann, M., Palepu, A., Saab, K., Freyberg, J., Tanno, R., Wang, A., Li, B., Amin, M., Cheng, Y., et al. (2025) · Nature · doi:10.1038/s41586-025-08866-7 · Cited by 366
“Conversational AI systems for goal-oriented dialogue and task completion have a rich history 73–75 .”
Metadata
- Authors
- Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gašić, M.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 5016-5026
- OpenAlex
- W2891732163
- Cited by
- 1,028
- References
- 48
- Citations by year
- 2026: 39 · 2025: 66 · 2024: 86 · 2023: 209 · 2022: 186 · 2021: 196 · 2020: 181 · 2019: 64 · 2018: 1
- FWCI
- 45.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M018946/1, RG80792)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- MultiWOZ, task-oriented dialogue, multi-domain dialogue, dialogue modeling, dialogue dataset, chatbots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.