Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network
Open accessAbstract
Data-driven approaches using neural networks have achieved promising performances in natural language generation (NLG).However, neural generators are prone to make mistakes, e.g., neglecting an input slot value and generating a redundant slot value.Prior works refer this to hallucination phenomenon.In this paper, we study slot consistency for building reliable NLG systems with all slot values of input dialogue act (DA) properly generated in output sentences.We propose Iterative Rectification Network (IRN) for improving general NLG systems to produce both correct and fluent responses.It applies a bootstrapping algorithm to sample training candidates and uses reinforcement learning to incorporate discrete reward related to slot inconsistency into training.Comprehensive studies have been conducted on multiple benchmark datasets, showing that the proposed methods have significantly reduced the slot error rate (ERR) for all strong baselines.Human evaluations also have confirmed its effectiveness.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Intrinsic hallucination can occur between the dialogue manager and natural language generation, where a dialogue act is transformed into a natural language representation.
“Intrinsic hallucination can occur between the DM and NLG, where a dialogue act such as recommend(NAME= peninsula hotel , AREA= tsim sha tsui ) is transformed into a natural language representation 'the hotel named peninsula hotel is located in tsim sha tsui area.' [10, 152].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [152]
The Introduction (p. not available) says NLG converts dialogue acts into natural-language sentences and that neural generators can hallucinate, including by missing or misplacing slots. It does not locate the error specifically between a dialogue manager and NLG, use the term “intrinsic hallucination,” or give the peninsula-hotel example. (LLM, conf=0.82)
Experimental — accuracy not yet measuredImproving slot consistency can help reduce the hallucination phenomenon of missing or misplacing slot values in generated templates.
“Improving the slot consistency can help reduce the hallucination phenomenon of missing or misplacing slot values in generated templates.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as Li et al. [152]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_2 · Cited by 941 · see citing works
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gašić, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. J. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1199 · Cited by 876 · see citing works
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
Xia, Y., Tian, F., Wu, L., Lin, J., Qin, T., Yu, N., & Liu, T. (2017) · neural information processing systems · Cited by 177 · see citing works
Self-citationMulti-domain Neural Network Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gašić, M., Mrkšić, N., Rojas-Barahona, L. M., Su, P.-H., Vandyke, D., & Young, S. J. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1015 · Cited by 168 · see citing works
The Optimal Reward Baseline for Gradient-Based Reinforcement LearningPreprint
Weaver, L., & Tao, N. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2315 · Cited by 158 · see citing works
Stochastic language generation for spoken dialogue systems
Oh, A. H., & Rudnicky, A. I. (2000) · doi:10.3115/1117562.1117568 · Cited by 157 · see citing works
Trainable sentence planning for complex information presentation in spoken dialog systems
Stent, A. J., Prasad, R., & Walker, M. A. (2004) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/1218955.1218966 · Cited by 154 · see citing works
Wen, T.-H., Gašić, M., Kim, D., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. J. (2015) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4639 · Cited by 152 · see citing works
Generation by Inverting a Semantic Parser that Uses Statistical Machine Translation
Wong, Y. W., & Mooney, R. J. (2007) · Cited by 99 · see citing works
Training a sentence planner for spoken dialogue using boosting
Walker, M. A., Rambow, O., & Rogati, M. (2002) · Computer Speech & Language · doi:10.1016/s0885-2308(02)00027-x · Cited by 91 · see citing works
A Global Model for Concept-to-Text Generation
Konstas, I., & Lapata, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4025 · Cited by 78 · see citing works
Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue
Balakrishnan, A., Rao, J., Upasani, K., White, M., & Subba, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1080 · Cited by 70 · see citing works
A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation
Nie, F., Yao, J.-G., Wang, J., Pan, R., & Lin, C.-Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1256 · Cited by 68 · see citing works
Stochastic Language Generation in Dialogue using Factored Language Models
Mairesse, F., & Young, S. J. (2014) · Computational Linguistics · doi:10.1162/coli_a_00199 · Cited by 64 · see citing works
Natural language generation for spoken dialogue system using RNN encoder-decoder networks
Tran, V. K., & Nguyen, L.-M. (2017) · Deakin Research Online (Deakin University) · Cited by 48 · see citing works
Ratnaparkhi, A. (2002) · Computer Speech & Language · doi:10.1016/s0885-2308(02)00025-6 · Cited by 45 · see citing works
Global-to-local Memory Pointer Networks for Task-Oriented DialoguePreprint
Wu, C.-S., Socher, R., & Xiong, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04713 · Cited by 34 · see citing works
Tran, V.-K., Nguyen, L.-M., & Tojo, S. (2017) · Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL) · doi:10.18653/v1/w17-5528 · Cited by 31 · see citing works
Conditional Random Fields for Responsive Surface Realisation using Global Features
Dethlefs, N., Hastie, H., Cuayáhuitl, H., & Lemon, O. (2013) · Cited by 26 · see citing works
Probabilistic Natural Language Generation with Wasserstein Autoencoders.Preprint
Bahuleyan, H., Mou, L., Vamaraju, K., Zhou, H., & Vechtomova, O. (2018) · arXiv (Cornell University) · Cited by 5 · see citing works
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732
Flagged first, then by citations
Research works printed but not linked: 3
- Ondřej Dušek and Filip Jurčíček. 2016. Sequence- to-sequence generation for spoken dialogue via deep syntax trees and strings. arXiv preprint arXiv:1606.05491. · arXiv
- Juraj Juraska, Panagiotis Karagiannis, Kevin K. Bow- den, and Marilyn A. Walker. 2018. A deep en- semble model with slot alignment for sequence-to- sequence natural language generation. In Proceed- ings of NAACL-HLT, pages 152-162.
- Franc ¸ois Mairesse and Steve Young. 2014. Stochas- tic language generation in dialogue using fac- tored language models. Computational Linguistics, 40(4):763-799.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
“Moreover, there are some broad metrics, such as Correctness [6, 11, 99, 186], Accuracy [98, 208], and Informativeness [103] considering both missing and additional contents (extrinsic hallucinations) compared to the input source.”
BackgroundMethodInfluentialRecent advances in deep learning based dialogue systems: a systematic survey
Ni, J., Young, T., Pandelea, V., Xue, F., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10248-8 · Cited by 279 · see citing works
Context-Controlled Topic-Aware Neural Response Generation for Open-Domain Dialog Systems
Ling, Y., Cai, F., Hu, X., Liu, J., Chen, W., & Chen, H. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102392 · Cited by 33 · see citing works
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions
Qin, L., Pan, W., Chen, Q., Liao, L., Zhou, Y., Zhang, Y., Che, W., & Li, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.363 · Cited by 32 · see citing works
Self-citationModularized Pre-Training for End-to-End Task-Oriented Dialogue
Qin, L., Xiao, X., Wang, L., Zhang, Y., & Che, W. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3244503 · Cited by 17 · see citing works
Self-citationAdapting language generation to dialogue environments and users for task-oriented dialogue systems
Ohashi, A., & Higashinaka, R. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100153 · Cited by 13 · see citing works
Qin, L., Xie, T., Huang, S., Chen, Q., Xiao, X., & Che, W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.182 · Cited by 10 · see citing works
Self-citationAre Current Task-Oriented Dialogue Systems Able to Satisfy Impolite Users?
Hu, Z., Chen, N. F., & Lee, R. K.-W. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3521020 · Cited by 7 · see citing works
Wang, H., Kwan, W.-C., Li, M., Zhou, Z., & Wong, K. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101637 · Cited by 6 · see citing works
Data-to-text generation using conditional generative adversarial with enhanced transformer
Seifossadat, E., & Sameti, H. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000487 · Cited by 5 · see citing works
Neural Sequence Segmentation as Determining the Leftmost Segments
Li, Y., Liu, L., & Yao, K. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.116 · Cited by 4 · see citing works
Self-citationA Congolese Swahili Task-Oriented Dialogue System for Addressing Humanitarian Crises
Kimanuka, U., Maina, C. W., Büyük, O., & Godelive, M. K. (2025) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.12403 · Cited by 3 · see citing works
Interpretable NLG for Task-oriented Dialogue Systems with Heterogeneous Rendering Machines
Li, Y., & Yao, K. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17571 · Cited by 3 · see citing works
Self-citationXia, S.-J., Zhang, H., Jiang, Y., Chen, X., Chen, Y., Li, Z., Wan, Z., & Sun, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5731427 · Cited by 3 · see citing works
IvCDS: An End-to-End Driver Simulator for Personal In-Vehicle Conversational Assistant
Ji, T., Yin, X., Cheng, P., Zhou, L., Liu, S., Bao, W., & Lyu, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315493 · Cited by 2 · see citing works
Retrieval Enhanced Segment Generation Neural Network for Task-Oriented Dialogue Systems
Chen, M., Lin, Z., Sun, R., Ouyang, K., Zheng, H.-T., Xie, R., & Wu, W. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747600 · Cited by 2 · see citing works
Gao, J. Y., Bi, W., Xu, R., & Shi, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.220 · Cited by 2 · see citing works
Arslan, D., & Eryiğit, G. (2021) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista52262.2021.9548636
Continuous-Speech Spoken Dialogue System Based on Joint Decision-Making of Audio and Semantics
Deng, Q., Peng, Q., Ma, Z., Zhao, P., & Wang, D. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220553
EntiTractor: entity extraction for coherent hallucination correction in paragraph-level text
Yao, J., Yao, J., Zang, W., Zhao, K., Zhang, H. M., & Wang, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133359
- Authors
- Li, Y., Yao, K., Qin, L., Che, W., Li, X., & Liu, T.
- Affiliations
- Harbin Institute of Technology (CN)
- Ant Group (China) (CN)
- Corresponding author
- Liu, Ting
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 97-106
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3034643843
- Cited by
- 19
- References
- 21
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 1 · 2023: 3 · 2022: 4 · 2021: 5 · 2020: 1
- FWCI
- 1.24
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (61976072, 61632011, 61772153)
- Ant Financial Services Group
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- natural language generation, task-oriented dialogue systems, reinforcement learning, hallucinations, response fluency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
