CoChat
Sign Up
Cited by 70Speech and dialogue systemsNatural Language Processing TechniquesTopic Modeling

Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

Open access
Balakrishnan, Anusha, Rao, Jinfeng, Upasani, Kartikeya, White, Michael, Subba, Rajen·2019·Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
DOI 10.18653/v1/p19-1080
Open this paper in CoChat — free accountOpen PDF

Abstract

Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems.Avenues like the E2E NLG Challenge have encouraged the development of neural approaches, particularly sequence-tosequence (Seq2Seq) models for this problem.The semantic representations used, however, are often underspecified, which places a higher burden on the generation model for sentence planning, and also limits the extent to which generated responses can be controlled in a live system.In this paper, we (1) propose using tree-structured semantic representations, like those used in traditional rule-based NLG systems, for better discourse-level structuring and sentence-level planning; (2) introduce a challenging dataset using this representation for the weather domain; (3) introduce a constrained decoding approach for Seq2Seq models that leverages this representation to improve semantic correctness; and (4) demonstrate promising results on our dataset and the E2E dataset.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/p19-1080

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Intrinsic hallucination can occur between the dialogue manager and natural language generation, where a dialogue act is transformed into a natural language representation.

    “Intrinsic hallucination can occur between the DM and NLG, where a dialogue act such as recommend(NAME= peninsula hotel , AREA= tsim sha tsui ) is transformed into a natural language representation 'the hotel named peninsula hotel is located in tsim sha tsui area.' [10, 152].”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [10]

    Section 4.2 (page unavailable) says the NLG model can hallucinate content while generating text from an input meaning representation, and the response-generation section describes turning representations containing dialogue acts into natural-language responses. Our copy does not identify this as “intrinsic hallucination,” locate it specifically at a dialogue-manager–NLG handoff, or contain the Peninsula Hotel example. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 51 · Cited by 70

Research works printed but not linked: 40

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. · arXiv
  2. John Carroll, Ann Copestake, Dan Flickinger, and Vic- tor Poznański. 1999. An efficient chart generator for (semi-) lexicalist grammars. In Proc. EWNLG-99.
  3. John Carroll and Stefan Oepen. 2005. High efficiency realization for a wide-coverage unification grammar. In Proc. IJCNLP-05.
  4. Ondřej Dušek and Filip Jurcicek. 2016. Sequence-to- sequence generation for spoken dialogue via deep syntax trees and strings. In Proceedings of the 54th Annual Meeting of the Association for Computa- tional Linguistics (Volume 2: Short Papers), pages 45-51. Association for Computational Linguistics. · DOI
  5. Ondřej Dušek, Jekaterina Novikova, and Verena Rieser. 2018. Findings of the E2E NLG challenge. In Pro- ceedings of the 11th International Conference on Natural Language Generation, pages 322-328. As- sociation for Computational Linguistics.
  6. Ondřej Dušek, Jekaterina Novikova, and Verena Rieser. 2019. Evaluating the state-of-the-art of end-to-end natural language generation: The E2E NLG Chal- lenge. arXiv preprint arXiv:1901.11528. · arXiv
  7. Henry Elder, Sebastian Gehrmann, Alexander O'Connor, and Qun Liu. 2018. E2E NLG challenge submission: Towards controllable generation of diverse natural language. In Proceedings of the 11th International Conference on Natural Language Generation, pages 457-462.
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
  9. Martin Kay. 1996. Chart generation. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, pages 200-204, Santa Cruz, California, USA. Association for Computa- tional Linguistics. · DOI
  10. Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. Globally coherent text generation with neu- ral checklist models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Lan- guage Processing, pages 329-339. Association for Computational Linguistics. · DOI
  11. Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), Melbourne, Australia. Association for Com- putational Linguistics.
  12. Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. 2017. Neural AMR: Sequence-to-sequence models for parsing and gen- eration. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguis- tics (Volume 1: Long Papers), pages 146-157, Van- couver, Canada. Association for Computational Lin- guistics.
  13. Oliver Lemon, Johanna Moore, Mary Ellen Foster, and Michael White. 2004. Generating tailored, compar- ative descriptions in spoken dialogue. In Proc. of FLAIRS. AAAI.
  14. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning semantic correspondences with less super- vision. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th Interna- tional Joint Conference on Natural Language Pro- cessing of the AFNLP: Volume 1-Volume 1, pages 91-99. Association for Computational Linguistics.
  15. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory: Towards a functional theory of text organization. TEXT, 8(3):243-281.
  16. Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Pro- cessing. MIT Press, Cambridge, MA, USA.
  17. Hongyuan Mei, Mohit Bansal, and R. Matthew Walter. 2016. What to talk about and how? selective gener- ation using lstms with coarse-to-fine alignment. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 720-730. Association for Computational Lin- guistics. · DOI
  18. Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. Step-by-step: Separating planning from realization in neural data-to-text generation. arXiv preprint arXiv:1904.03396. · arXiv
  19. Hiroko Nakanishi, Yusuke Miyao, and Jun'ichi Tsujii. 2005. Probabilistic methods for disambiguation of an HPSG-based chart generator. In Proc. IWPT-05.
  20. Jekaterina Novikova, Ondrej Dusek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In EMNLP.
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proc. ACL-02.
  22. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 confer- ence on empirical methods in natural language pro- cessing (EMNLP), pages 1532-1543.
  23. Matt Post and David Vilar. 2018. Fast lexically con- strained decoding with dynamic beam allocation for neural machine translation. In NAACL-HLT.
  24. Ratish Puduppully, Li Dong, and Mirella Lapata. 2018. Data-to-text generation with content selection and planning. arXiv preprint arXiv:1809.00582. · arXiv
  25. Owen Rambow, Srinivas Bangalore, and Marilyn Walker. 2001. Natural language generation in dia- log systems. In Proceedings of the First Interna- tional Conference on Human Language Technology Research, HLT '01, pages 1-4, Stroudsburg, PA, USA. Association for Computational Linguistics. · DOI
  26. Lena Reed, Shereen Oraby, and Marilyn Walker. 2018. Can neural generators for dialogue learn sentence planning and discourse structuring? In Proceed- ings of the 11th International Conference on Natural Language Generation, pages 284-295. Association for Computational Linguistics.
  27. Ehud Reiter. 2018. A structured review of the validity of BLEU. Computational Linguistics, 44:393-401.
  28. Ehud Reiter and Robert Dale. 2000. Building Natural- Language Generation Systems. Cambridge Univer- sity Press.
  29. Stuart Shieber. 1988. A uniform architecture for pars- ing and generation. In Proc. COLING-88.
  30. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-1958.
  31. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural net- works. In Advances in neural information process- ing systems, pages 3104-3112.
  32. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Advances in Neural In- formation Processing Systems, pages 2692-2700.
  33. Marilyn Walker, Amanda Stent, Francois Mairesse, and Rashmi Prasad. 2007. Individual and domain adap- tation in sentence planning for dialogue. Journal of Artificial Intelligence Research (JAIR), 30:413-456.
  34. Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, M. Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, and Steve Young. 2016. Multi-domain neural network language generation for spoken di- alogue systems. In Proceedings of the 2016 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 120-129. Association for Computational Linguistics. · DOI
  35. Tsung-Hsien Wen, Milica Gasic, Nikola Mrkšić, Pei- Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural lan- guage generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1711-1721. Association for Computational Linguis- tics. · DOI
  36. Michael White. 2006. Efficient realization of coordi- nate structures in combinatory categorial grammar. Research on Language and Computation, 4(1):39- 75. · DOI
  37. Michael White, Robert A. J. Clark, and Johanna D. Moore. 2010. Generating tailored, comparative de- scriptions with contextually appropriate intonation. Computational Linguistics, 36(2):159-201.
  38. Michael White and Rajakrishnan Rajkumar. 2009. Per- ceptron reranking for CCG realization. In Proceed- ings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 410-419, Singapore. Association for Computational Linguis- tics.
  39. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2253-2263. Association for Computational Linguis- tics. · DOI · Paper page
  40. Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2018. Learning neural templates for text gen- eration. In EMNLP.

Authors
Balakrishnan, A., Rao, J., Upasani, K., White, M., & Subba, R.
Affiliations
Meta (United States) (US)
ORCID
Venue
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Publisher
Association for Computational Linguistics
Published
2019
Pages
831-844
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/p19-1080
OpenAlex
W2949413855
Cited by
70
References
51
Citations by year
2026: 1 · 2025: 2 · 2024: 3 · 2023: 9 · 2022: 10 · 2021: 17 · 2020: 22 · 2019: 6
FWCI
5.26
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
constrained decoding, compositional representations, task-oriented dialogue, sequence-to-sequence models, semantic correctness, sentence planning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review