CoChat
Sign Up
Cited by 98Speech and dialogue systemsNatural Language Processing TechniquesAI in Service Interactions

Task-Oriented Dialogue as Dataflow Synthesis

Open access
Andreas, Jacob, Bufe, John, Burkett, David, Chen, Charles, Clausman, Josh, Crawford, Jean, Crim, Kate, DeLoach, Jordan +37 more·2020·Transactions of the Association for Computational Linguistics
DOI 10.1162/tacl_a_00333
Open this paper in CoChat — free account

Abstract

We describe an approach to task-oriented dialogue in which dialogue state is represented as a dataflow graph. A dialogue agent maps each user utterance to a program that extends this graph. Programs include metacomputation operators for reference and revision that reuse dataflow fragments from previous turns. Our graph-based state enables the expression and manipulation of complex user intents, and explicit metacomputation makes these intents easier for learned models to predict. We introduce a new dataset, SMCalFlow, featuring complex dialogues about events, weather, places, and people. Experiments show that dataflow graphs and metacomputation substantially improve representability and predictability in these natural dialogues. Additional experiments on the MultiWOZ dataset show that our dataflow representation enables an otherwise off-the-shelf sequence-to-sequence model to match the best existing task-specific state tracking model. The SMCalFlow dataset, code for replicating experiments, and a public leaderboard are available at https://www.microsoft.com/en-us/research/project/dataflow-based-dialogue-semantic-machines .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/tacl_a_00333

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 8 · Cited by 98

Research works printed but not linked: 39

  1. James F. Allen, Bradford W. Miller, Eric K. Ringger, and Teresa Sikorski. 1996. A robust system for natural spoken dialogue. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics , pages 62-70. Association for Computational Linguistics.
  2. John Langshaw Austin. 1962. How to Do Things with Words . William James Lectures. Oxford University Press. Edited by James O. Urmson. A second edition appeared in 1975.
  3. Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2016. Learning end-to-end goal-oriented dialog. In Proceedings of the International Conference on Learning Representations .
  4. Pawel Budzianowski, Tsung-Hsien Wen, BoHsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. MultiWOZ -A large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing .
  5. Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019. Taskmaster-1: Toward a realistic and diverse dialog dataset. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing , pages 4516-4525, Hong Kong, China.
  6. Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyag Gao, and Dilek Hakkani-Tur. 2019. MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. arXiv:1907.01669 [cs.CL] . · arXiv
  7. Jianfeng Gao, Michel Galley, Lihong Li, et al. 2019. Neural approaches to conversational AI. Foundations and Trends® in Information Retrieval , 13(2-3):127-298.
  8. Rahul Goel, Shachi Paul, and Dilek Hakkani-Tür. 2019. HyST: A hybrid approach for flexible and accurate dialogue state tracking. In Proceedings of the Conference of the International Speech Communication Association .
  9. Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics , 12(3):175204.
  10. Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. Semantic parsing for task oriented dialog using hierarchical representations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing .
  11. Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley .
  12. R. Hindley. 1969. The principal type-scheme of an object in Combinatory Logic. Transactions of the American Mathematical Society , 146:2960.
  13. Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. 2017. Search-based neural structured learning for sequential question answering. In Proceedings of the Annual Meeting of the Association for Computational Linguistics .
  14. John B. Kam and Jeffrey D. Ullman. 1976. Global data flow analysis and iterative algorithms. Journal of the ACM (JACM) , 23(1):158-171.
  15. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations .
  16. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics .
  17. Shalom Lappin and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics , 20(4):535-561.
  18. Robin Milner. 1978. A theory of type polymorphism in programming. Journal of Computer and System Sciences , 17:348-375.
  19. Ruslan Mitkov. 2014. Anaphora Resolution . Routledge.
  20. Kenton Murray and David Chiang. 2018. Correcting length bias in neural machine translation. In Proceedings of the Conference on Machine Translation .
  21. Arvind Neelakantan, Semih Yavuz, Sharan Narang, Vishaal Prasad, Ben Goodrich, Daniel Duckworth, Chinnadhurai Sankar, and Xifeng Yan. 2019. Neural Assistant: Joint action prediction, response generation, and latent knowledge reasoning. arXiv:1910.14613 [cs.LG] . · arXiv
  22. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages 1532-1543.
  23. Roberto Pieraccini, Evelyne Tzoukermann, Zakhar Gorelov, J.-L. Gauvain, Esther Levin, C.H. Lee, and Jay G. Wilpon. 1992. A speech understanding system based on statistical representation of semantics. In Proceedings of 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , volume 1, pages 193-196. IEEE.
  24. Charles Rich, Candace L. Sidner, and Neal Lesh. 2001. Collagen: Applying collaborative discourse theory to human-computer interaction. AI Magazine , 22(4):15-15.
  25. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics , pages 1073-1083.
  26. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van EssDykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics , 26(3):339-373.
  27. Alane Suhr, Srinivasan Iyer, and Yoav Artzi. 2018. Learning to map context-dependent sentences to executable formal queries. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics .
  28. Andreas Vlachos and Stephen Clark. 2014. A new corpus and imitation learning framework for context-dependent semantic parsing. Transactions of the Association for Computational Linguistics , 2:547-560.
  29. Tsung-Hsien Wen, David Vandyke, Nikola Mrkši´ c, Milica Gasic, Lina M. Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In Proceedings of the European Association for Computational Linguistics , pages 438-449.
  30. Jason D. Williams, Kavosh Asadi, and Geoffrey Zweig. 2017. Hybrid code networks: Practical and efficient end-to-end dialog control with supervised and reinforcement learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics .
  31. Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multidomain state generator for task-oriented dialogue systems. In Proceedings of the Annual Meeting of the Association for Computational Linguistics .
  32. Steve Young, Milica Ga˘ si` c, Blaise Thomson, and Jason D. Williams. 2013. POMDP-based statistical spoken dialog systems: A review. Proc. IEEE , 101(5):1160-1179.
  33. Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, and Dragomir Radev. 2019a. CoSQL: A conversational textto-SQL challenge towards cross-domain natural language interfaces to databases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 1962-1979, Hong Kong, China.
  34. Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, et al. 2019b. SParC: Cross-domain semantic parsing in context. In Proceedings of the Annual Meeting of the Association for Computational Linguistics .
  35. John Zelle. 1995. Using Inductive Logic Programming to Automate the Construction of Natural Language Parsers . Ph.D. thesis, Department of Computer Sciences, The University of Texas at Austin.
  36. Luke Zettlemoyer and Michael Collins. 2009. Learning context-dependent mappings from sentences to logical form. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP , pages 976-984. Association for Computational Linguistics.
  37. Rui Zhang, Tao Yu, He Yang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019. Editing-based SQL query generation for cross-domain context-dependent questions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing .
  38. Tiancheng Zhao and Maxine Eskenazi. 2016. Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages 1-10, Los Angeles. Association for Computational Linguistics.
  39. Victor Zue, Stephanie Seneff, Joseph Polifroni, Michael Phillips, Christine Pao, David Goodine, David Goddeau, and James Glass. 1994. PEGASUS: A spoken dialogue interface for online air travel planning. Speech Communication , 15(3-4):331-340.

Authors
Andreas, J., Bufe, J., Burkett, D., Chen, C., Clausman, J., Crawford, J., Crim, K., DeLoach, J., Dorner, L., Eisner, J., Fang, H., Guo, A., Hall, D., Hayes, K., Hill, K., Ho, D., Iwaszuk, W., Jha, S., Klein, D., Krishnamurthy, J., Lanman, T., Liang, P., Lin, C. H., Lintsbakh, I., McGovern, A., Nisnevich, A., Pauls, A., Petters, D., Read, B., Roth, D., Roy, S., Rusak, J., Short, B., Slomin, D., Snyder, B., Striplin, S., Su, Y., Tellman, Z., Thomson, S., Vorobev, A., Witoszko, I., Wolfe, J., Wray, A., Zhang, Y., & Zotov, A.
Affiliations
Microsoft Semantic Machines
ORCID
Venue
Transactions of the Association for Computational Linguistics
Publisher
MIT Press - Journals
Published
2020-12
Volume
8
Pages
556-571
ISSN
2307-387X (ISSN-L)
DOI
10.1162/tacl_a_00333
OpenAlex
W3087300239
Cited by
98
References
8
Citations by year
2026: 2 · 2025: 5 · 2024: 7 · 2023: 31 · 2022: 26 · 2021: 26 · 2020: 1
FWCI
8.25
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
task-oriented dialogue, dialogue state tracking, data flow graphs, user intent modeling, MultiWOZ, sequence-to-sequence models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review