MuTual: A Dataset for Multi-Turn Dialogue Reasoning
Open accessAbstract
Non-task oriented dialogue systems have achieved great success in recent years due to largely accessible conversation data and the development of deep learning techniques.Given a context, current systems are able to yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities.To facilitate the conversation reasoning research, we introduce Mu-Tual, a novel dataset for Multi-Turn dialogue Reasoning, consisting of 8,860 manually annotated dialogues based on Chinese student English listening comprehension exams.Compared to previous benchmarks for non-task oriented dialogue systems, MuTual is much more challenging since it requires a model that can handle various reasoning problems.Empirical results show that state-of-the-art methods only reach 71%, which is far behind the human performance of 94%, indicating that there is ample room for improving reasoning ability.MuTual is available at https://github. com/Nealcly/MuTual. * Contribution during internship at MSRA.M: Ma'am
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There are some reasoning works in the area of dialogue, but few in reducing hallucinations.
“There are some reasoning works in the area of dialogue [39, 83, 272], but few in reducing hallucinations.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503 · see citing works
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1421 · Cited by 2,805 · see citing works
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312 · see citing works
What Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,514 · see citing works
(2019) · doi:10.18653/v1/p19-1356 · Cited by 1,291 · see citing works
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. Z. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1152 · Cited by 1,186 · see citing works
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177 · see citing works
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Lai, G., Xie, Q., Liu, H., Yang, Y., & Hovy, E. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1082 · Cited by 1,044 · see citing works
Neural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · doi:10.3115/v1/p15-1152 · Cited by 1,009 · see citing works
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · see citing works
Levesque, H. J., Davis, E., & Morgenstern, L. (2011) · Cited by 879 · see citing works
The TREC-8 Question Answering Track Report
Voorhees, E. M. (1999) · Cited by 843 · see citing works
ConceptNet 5.5: An Open Multilingual Graph of General KnowledgePreprint
Speer, R., Chin, J., & Havasi, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03975 · Cited by 832 · see citing works
From Eliza to XiaoIce: Challenges and opportunities with social chatbots
Shum, H.-Y., He, X.-D., & Li, D. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700826 · Cited by 749 · see citing works
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1009 · Cited by 712 · see citing works
Wu, Y., Wu, W., Chen, X., Zhou, M., & Li, Z. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1046 · Cited by 511 · see citing works
Self-citationThink you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462 · see citing works
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., & Gardner, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1246 · Cited by 393 · cited 2× · see citing works
“DROP (Dua et al., 2019) and COSMOS (Huang et al., 2019) focus on factual understanding and commonsense comprehension, respectively.”
BackgroundLooking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1023 · Cited by 372 · see citing works
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Zhou, X., Lü, L., Dong, D., Liu, Y., Chen, Y., Zhao, W. X., Yu, D., & Wu, H. R. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1103 · Cited by 339 · see citing works
Neural Text Summarization: A Critical Evaluation
Kryscinski, W., Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1051 · Cited by 335 · see citing works
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1242 · Cited by 308 · see citing works
Augmenting End-to-End Dialogue Systems With Commonsense Knowledge
Young, T., Cambria, E., Chaturvedi, I., Zhou, H., Biswas, S., & Huang, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11923 · Cited by 305 · see citing works
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I. V., & Pineau, J. (2015) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4640 · Cited by 282 · see citing works
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
Huang, L., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1243 · Cited by 278 · see citing works
Dialogue Natural Language Inference
Welleck, S., Weston, J., Szlam, A., & Cho, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1363 · Cited by 249 · see citing works
DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., & Cardie, C. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00264 · Cited by 245 · see citing works
Modeling Multi-turn Conversation with Deep Utterance AggregationPreprint
Zhang, Z., Li, J., Zhu, P., Zhao, H., & Liu, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09102 · Cited by 218 · see citing works
Smart Reply: Automated Response Suggestion for EmailPreprint
Kannan, A., Kurach, K., Ravi, S., Kaufmann, T., Tomkins, A., Miklos, B., Corrado, G. S., Lukacs, L., Ganea, M., Young, P., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04870 · Cited by 141 · see citing works
Tao, C., Wu, W., Xu, C., Hu, W., Zhao, D., & Yan, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1001 · Cited by 128 · see citing works
A Sequential Matching Framework for Multi-Turn Response Selection in Retrieval-Based Chatbots
Wu, Y., Wu, W., Chen, X., Xu, C., Li, Z., & Zhou, M. (2018) · Computational Linguistics · doi:10.1162/coli_a_00345 · Cited by 71 · see citing works
Self-citationImproving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration
Pan, Z., Bai, K., Wang, Y., Zhou, L., & Liu, X. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1191 · Cited by 48 · see citing works
Domain adaptive training BERT for response selectionPreprint
Whang, T., Lee, D., Lee, C., Yang, K., Oh, D., & Lim, H. S. (2019) · arXiv (Cornell University) · Cited by 21 · see citing works
Flagged first, then by citations
2 of the 38 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, Advances in Neural Information Pro- cessing Systems 30, pages 5998-6008.
- Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. In 7th International Conference on Learning Represen- tations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
- Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang. 2018. Aug- menting end-to-end dialogue systems with common- sense knowledge. In Proceedings of the Thirty- Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Arti- ficial Intelligence (IAAI-18), and the 8th AAAI Sym- posium on Educational Advances in Artificial Intel- ligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 4970-4977.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Graph Neural Networks for Natural Language Processing: A Survey
Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., & Long, B. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000096 · Cited by 387 · see citing works
Recent advances in deep learning based dialogue systems: a systematic survey
Ni, J., Young, T., Pandelea, V., Xue, F., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10248-8 · Cited by 279 · see citing works
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
Jian, L., Cui, L., Liu, H., Huang, D., Wang, Y., & Zhang, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/501 · Cited by 209 · see citing works
Self-citationDialogSum: A Real-Life Scenario Dialogue Summarization Dataset
Chen, Y., Liu, Y., Liang, C., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.449 · Cited by 183 · see citing works
Self-citationAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136 · see citing works
Self-citationAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135 · see citing works
Self-citationSemEval-2020 Task 4: Commonsense Validation and Explanation
Wang, C., Liang, S., Jin, Y., Wang, Y., Zhu, X., & Zhang, Y. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.39 · Cited by 84 · see citing works
Self-citationWavLLM: Towards Robust and Adaptive Speech Large Language Model
Hu, S., Zhou, L., Liu, S., Chen, S., Meng, L., Hao, H., Pan, J. S., Liu, X., Li, J., Sivasankaran, S., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.263 · Cited by 57 · see citing works
Self-citationFilling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
Liu, L., Zhang, Z., Zhao, H., Zhou, X., & Zhou, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17582 · Cited by 50 · see citing works
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Gupta, P., Jiao, C., Yeh, Y.-T., Mehri, S., Eskénazi, M., & Bigham, J. P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.33 · Cited by 46 · see citing works
Commonsense-Focused Dialogues for Response Generation: An Empirical Study
Zhou, P., Gopalakrishnan, K., Hedayatnia, B., Kim, S., Pujara, J., Ren, X., Liu, Y., & Hakkani-Tür, D. Z. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.13 · Cited by 38 · see citing works
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues
Ghosal, D., Shen, S., Majumder, N., Mihalcea, R. F., & Poria, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.344 · Cited by 37 · see citing works
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms
Zhou, P., Khanna, R., Lee, S., Lin, B. Y., Ho, D., Pujara, J., & Ren, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.598 · Cited by 35 · see citing works
Natural Language Inference in Context - Investigating Contextual Reasoning over Long Texts
Liu, H., Cui, L., Liu, J., & Zhang, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17580 · Cited by 35 · see citing works
Self-citationBotChat: Evaluating LLMs’ Capabilities of Having Multi-Turn Dialogues
Duan, H., Wei, J., Wang, C., Liu, H., Fang, Y., Zhang, S., Lin, D., & Chen, K. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.201 · Cited by 30 · see citing works
Dialogue Graph Modeling for Conversational Machine Reading
Ouyang, S., Zhang, Z., & Zhao, H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.279 · Cited by 30 · see citing works
Enhanced Speaker-Aware Multi-Party Multi-Turn Dialogue Comprehension
Ma, X., Zhang, Z., & Zhao, H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3284516 · Cited by 29 · see citing works
Structural Pre-training for Dialogue Comprehension
Zhang, Z., & Zhao, H. (2021) · doi:10.18653/v1/2021.acl-long.399 · Cited by 28 · see citing works
Li, Y., & Zhao, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.176 · Cited by 27 · see citing works
Logiqa 2.0—An improved dataset for logical reasoning in natural language understanding
Liu, H., Liu, J., Cui, L., Teng, Z., Duan, N., Zhou, M., & Zhang, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3293046 · Cited by 26 · see citing works
Self-citationChen, S., Jin, S., & Xie, X. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468569 · Cited by 26 · see citing works
Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation
Zhou, P., Gopalakrishnan, K., Hedayatnia, B., Kim, S., Pujara, J., Ren, X., Liu, Y., & Hakkani-Tür, D. Z. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.88 · Cited by 26 · see citing works
Fact-Driven Logical Reasoning for Machine Reading Comprehension
Ouyang, S., Zhang, Z., & Zhao, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29850 · Cited by 24 · see citing works
A Survey on Response Selection for Retrieval-based Dialogues
Tao, C., Feng, J., Yan, R., Wu, W., & Jiang, D. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/627 · Cited by 23 · see citing works
Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents
Chae, H., Song, Y., Ong, K., Kwon, T., Kim, M., Yu, Y., Lee, D., Kang, D., & Yeo, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.342 · Cited by 23 · see citing works
Multi-Turn Dialogue Reading Comprehension With Pivot Turns and Knowledge
Zhang, Z., Li, J., & Zhao, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3058616 · Cited by 23 · see citing works
Dialogue-adaptive language model pre-training from quality estimation☆
Li, J., Zhang, Z., & Zhao, H. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.10.036 · Cited by 21 · see citing works
A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses
Zhang, Y., Ren, P., & de Rijke, M. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24496 · Cited by 20 · see citing works
Self-citationCIDER: Commonsense Inference for Dialogue Explanation and Reasoning
Ghosal, D., Hong, P., Shen, S., Majumder, N., Mihalcea, R., & Poria, S. (2021) · Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.33 · Cited by 19 · see citing works
A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training
Liu, Y., Feng, S., Wang, D., Song, K., Ren, F., & Zhang, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17585 · Cited by 19 · see citing works
Self-citationTask-specific Objectives of Pre-trained Language Models for Dialogue AdaptationPreprint
Li, J., Zhang, Z., Zhao, H., Zhou, X., & Zhou, X. (2020) · arXiv (Cornell University) · Cited by 19 · see citing works
Comparing Test Sets with Item Response Theory
Vania, C., Htut, P. M., Huang, W., Mungra, D., Pang, R. Y., Phang, J., Liu, H., Cho, K., & Bowman, S. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.92 · Cited by 18 · see citing works
Does BERT Solve Commonsense Task via Commonsense KnowledgePreprint
Cui, L., Cheng, S., Wu, Y., & Zhang, Y. (2020) · arXiv (Cornell University) · Cited by 18 · see citing works
Self-citationTaxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches, and Future Directions
Jia, Q., Liu, Y., Ren, S., & Zhu, K. Q. (2023) · ACM Computing Surveys · doi:10.1145/3622933 · Cited by 17 · see citing works
On Commonsense Cues in BERT for Solving Commonsense Tasks
Cui, L., Cheng, S., Wu, Y., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.61 · Cited by 17 · see citing works
Self-citationGRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning
Zheng, Z., Qiu, S., Fan, L., Zhu, Y., & Zhu, S. C. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.182 · Cited by 15 · see citing works
Advances in Multi-turn Dialogue Comprehension: A SurveyPreprint
Zhang, Z., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04984 · Cited by 15 · see citing works
DialogSum Challenge: Summarizing Real-Life Scenario Dialogues
Chen, Y., Liu, Y., & Zhang, Y. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.33 · Cited by 14 · see citing works
Self-citationReflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality
Zhou, P., Cho, H., Jandaghi, P., Lee, D., Lin, B. Y., Pujara, J., & Ren, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.714 · Cited by 14 · see citing works
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge
Gao, S., Hwang, J. D., Kanno, S., Wakaki, H., Mitsufuji, Y., & Bosselut, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.120 · Cited by 13 · see citing works
Evidence, my Dear Watson: Abstractive dialogue summarization on learnable relevant utterances
Italiani, P., Frisoni, G., Moro, G., Carbonaro, A., & Sartori, C. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127132 · Cited by 13 · see citing works
Robust Prompt Optimization for Large Language Models Against Distribution Shifts
Li, M., Wang, W., Feng, F., Cao, Y., Zhang, J., & Chua, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.95 · Cited by 12 · see citing works
RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners
Sanyal, S., Liao, Z., & Ren, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.653 · Cited by 11 · see citing works
Do Large Language Models Understand Conversational Implicature – A Case Study with a Chinese Sitcom
Yue, S., Song, S., Cheng, X., & Hu, H. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8367-0_24 · Cited by 11 · see citing works
Fact-driven Logical Reasoning.Preprint
Ouyang, S., Zhang, Z., & Zhao, H. (2021) · arXiv (Cornell University) · Cited by 11 · see citing works
Socratic Question Generation: A Novel Dataset, Models, and Evaluation
Ang, B. H., Gollapalli, S. D., & Ng, S.-K. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.12 · Cited by 10 · see citing works
Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling
Li, Y., Zhao, H. Y., & Zhang, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.177 · Cited by 10 · see citing works
Exploring Dense Retrieval for Dialogue Response Selection
Lan, T., Cai, D., Wang, Y., Su, Y., Huang, H., & Mao, X.-L. (2023) · ACM Transactions on Information Systems · doi:10.1145/3632750 · Cited by 10 · see citing works
SAPBERT: Speaker-Aware Pretrained BERT for Emotion Recognition in Conversation
Lim, S., & Kim, J. (2022) · Algorithms · doi:10.3390/a16010008 · Cited by 9 · see citing works
Cao, T., Zeng, S., Zhao, S., Mansur, M., & Chang, B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86365-4_6 · Cited by 9 · see citing works
Probing Commonsense Explanation in Dialogue Response Generation
Zhou, P., Jandaghi, P., Cho, H., Lin, B. Y., Pujara, J., & Ren, X. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.349 · Cited by 9 · see citing works
Think Before You Speak: Learning to Generate Implicit Knowledge for Response Generation by Self-Talk
Zhou, P., Hedayatnia, B., Gopalakrishnan, K., Kim, S., Pujara, J., Ren, X., Liu, Y., & Hakkani-Tür, D. Z. (2021) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2021.nlp4convai-1.23 · Cited by 8 · see citing works
Leveraging speaker-aware structure and factual knowledge for faithful dialogue summarization
Zhao, L., Xu, W., Zhang, C., & Guo, J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108550 · Cited by 7 · see citing works
Discourse-Aware Graph Networks for Textual Logical Reasoning
Huang, Y., Liu, L., Xu, K., Fang, M., Lin, L., & Liang, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3280178 · Cited by 7 · see citing works
Commonsense-Focused Dialogues for Response Generation: An Empirical StudyPreprint
Zhou, P., Gopalakrishnan, K., Hedayatnia, B., Kim, S., Pujara, J., Ren, X., Liu, Y., & Hakkani-Tür, D. Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06427 · Cited by 7 · see citing works
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization
Li, Y., Peng, B., He, P., Galley, M., Zhou, Y., & Gao, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.76 · Cited by 6 · see citing works
Improving conversational search with query reformulation using selective contextual history
Al-Thani, H., Elsayed, T., & Jansen, B. J. J. (2022) · Data and Information Management · doi:10.1016/j.dim.2022.100025 · Cited by 6 · see citing works
HiBERT: Detecting the illogical patterns with hierarchical BERT for multi-turn dialogue reasoning
Wang, X., Zhang, H., Zhao, S., Chen, H., Cheng, B. H., Ding, Z., Xu, S., Yan, W., & Lan, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.038 · Cited by 6 · see citing works
IMAD: Image-Augmented Multi-Modal Dialogue
Moskvoretskii, V., Frolov, A., & Kuznetsov, D. (2024) · Journal of Mathematical Sciences · doi:10.1007/s10958-024-07434-0 · Cited by 6 · see citing works
He, J., Peng, B., Liao, Y., Liu, Q., & Xiong, D. (2021) · doi:10.18653/v1/2021.acl-long.469 · Cited by 6 · see citing works
Representation Learning for Conversational Data using Discourse Mutual Information Maximization
Santra, B., Roychowdhury, S., Mandal, A., Gurram, V., Naik, A., Gupta, M., & Goyal, P. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.124 · Cited by 5 · see citing works
Channel-Aware Decoupling Network for Multiturn Dialog Comprehension
Zhang, Z., Zhao, H., & Liu, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3220047 · Cited by 5 · see citing works
Who Needs External References?—Text Summarization Evaluation Using Original Documents
Foysal, A. A., & Böck, R. (2023) · AI · doi:10.3390/ai4040049 · Cited by 5 · see citing works
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing
Wang, Y., Weng, F., Yang, S., Qin, Z., Huang, M., & Wang, W. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.598 · Cited by 5 · see citing works
An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation
Li, Z., Maimaiti, M., Sheng, J., Ke, Z., Silamu, W., Wang, Q., & Li, X. (2020) · Symmetry · doi:10.3390/sym12111756 · Cited by 5 · see citing works
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Zhao, C., Yao, W., Yu, D., Song, K., Yu, D., & Chen, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.23 · Cited by 5 · see citing works
Coreference-aware Double-channel Attention Network for Multi-party Dialogue Reading Comprehension
Li, Y., Zou, B., Fan, Y., Dong, M., & Hong, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191414 · Cited by 5 · see citing works
MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure
Huang, Y., Zhang, H., Hong, R., Liang, X., Zhang, C., & Yu, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.310 · Cited by 5 · see citing works
OmniDialog: An Omnipotent Pre-Training Model for Task-Oriented Dialogue System
Yang, M., Ng, S.-K., & Fu, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3542301 · Cited by 4 · see citing works
ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA
Li, J., Wang, Q., Wang, Z., Zhang, Y., & Mao, Z. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34622 · Cited by 4 · see citing works
Self-citationDebiasing Counterfactual Context With Causal Inference for Multi-Turn Dialogue Reasoning
Wang, X., Zhang, H., Zhao, S., Chen, H., Ding, Z., Wan, Z., Cheng, B. H., & Lan, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3343608 · Cited by 4 · see citing works
Kumar, S., Bhatia, S., Aggarwal, M., & Chakraborty, T. (2024) · Natural language processing. · doi:10.1017/nlp.2024.42 · Cited by 4 · see citing works
Smoothing Dialogue States for Open Conversational Machine Reading
Zhang, Z., Ouyang, S., Zhao, H., Utiyama, M., & Sumita, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.299 · Cited by 4 · see citing works
FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning
Wang, X., Zhang, H., Zhao, S., Zou, Y., Chen, H., Ding, Z., Cheng, B., & Lan, Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.362 · Cited by 4 · see citing works
Zhang, X., Huang, H., Chi, Z., & Mao, X.-L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.857 · Cited by 4 · see citing works
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection
Liu, Y., Shi, F., Gao, W., Wang, D., & Zhang, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.828 · Cited by 4 · see citing works
Self-citationAn Evaluation Dataset and Strategy for Building Robust Multi-turn Response Selection Model
Han, K., Lee, S., & Lee, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.180 · Cited by 4 · see citing works
Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers
Zhong, W., Ma, T., Wang, J., Yin, J., Zhao, T., Lin, C.-Y., & Duan, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.480 · Cited by 4 · see citing works
Zhou, P., Gopalakrishnan, K., Hedayatnia, B., Kim, S., Pujara, J., Ren, X., Liu, Y., & Hakkani-Tür, D. Z. (2021) · arXiv (Cornell University) · Cited by 4 · see citing works
Explainable Hallucination Mitigation in Large Language Models: A Survey
Deng, W., Li, J., Zhang, H., Li, J., Deng, Z., Cheng, D., Feng, Z., & Zeng, H. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70110 · Cited by 3 · see citing works
Yang, X., Ba, K., Xie, J., Wang, Y., Kong, L. J., & Liu, Y. (2025) · Cyberpsychology Behavior and Social Networking · doi:10.1177/21522715251403844 · Cited by 3 · see citing works
Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection
Zhang, L., Ma, D., Li, S., & Wang, H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.430 · Cited by 3 · see citing works
STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension
Wang, B., Feng, C., Nair, A., Mao, M., Desai, J., Çelikyılmaz, A., Li, H., Mehdad, Y., & Radev, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.329 · Cited by 3 · see citing works
Cao, B., Wang, K., Ma, K., Fan, J., Yan, R., & Xu, Y. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3223924 · Cited by 3 · see citing works
Saliency Guided Debiasing: Detecting and mitigating biases in LMs using feature attribution
Joshi, R. K., Chatterjee, A., & Ekbal, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126851 · Cited by 3 · see citing works
EDeR: Towards Understanding Dependency Relations Between Events
Li, R., Haslum, P., & Cui, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.926 · Cited by 3 · see citing works
Self-citationCTRD: A Chinese Theme-Rheme Discourse Dataset
Fu, B., Tong, Y., Dawei, T., Chen, Y., Shi, X., & Zhu, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88480-2_6 · Cited by 3 · see citing works
Multi-Stage Pre-training Enhanced by ChatGPT for Multi-Scenario Multi-Domain Dialogue Summarization
Zhou, W., Li, G., Cheng, X., Liang, X., Zhu, J., Zhai, F., & Li, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.460 · Cited by 3 · see citing works
R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
Yuan, T., He, Z., Dong, L., Wang, Y., Zhao, R., Xia, T., Xu, L., Zhou, B., Li, F., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.10019 · Cited by 2 · see citing works
“For human-annotated datasets, there is always a trade-off between the scale of the instances that are annotated and the quality of the annotations (Cui et al., 2020).”
BackgroundDiscourse-Aware Language Representation
Zhang, Z., Ouyang, S., & Zhao, H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3621229 · Cited by 2 · see citing works
CausalDialogue: Modeling Utterance-level Causality in Conversations
Tuan, Y.-L., Albalak, A., Xu, W., Saxon, M., Pryor, C., Getoor, L. C., & Wang, W. Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.792 · Cited by 2 · see citing works
Syndicom: Improving Conversational Commonsense with Error-Injection and Natural Language Feedback
Richardson, C., & Heck, L. P. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.27 · Cited by 2 · see citing works
Towards End-to-End Open Conversational Machine Reading
Zhou, S., Ouyang, S., Zhang, Z., & Zhao, H. Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.154 · Cited by 2 · see citing works
Logical Reasoning for Task Oriented Dialogue Systems
Beygi, S., Fazel-Zarandi, M., Cervone, A., Krishnan, P., & Jonnalagadda, S. (2022) · Workshop on e-Commerce and NLP (ECNLP) · doi:10.18653/v1/2022.ecnlp-1.10 · Cited by 2 · see citing works
Problems and Countermeasures in Natural Language Processing EvaluationPreprint
Dong, Q., Sui, Z., Zhan, W., & Chang, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09712 · Cited by 2 · see citing works
Si, P., Qiu, Y., Zhang, J., & Yang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05406 · Cited by 2 · see citing works
On the Underspecification of Situations in Open-domain Conversational Datasets
Otani, N., Araki, J., Kim, H., & Hovy, E. H. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.2 · Cited by 1 · see citing works
Misbah, A. M., Farouk, M., & AbdulAzim, M. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341905 · Cited by 1 · see citing works
Reducing Sensitivity on Speaker Names for Text Generation from Dialogues
Jia, Q. F., Tang, H., & Zhu, K. Q. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.129 · Cited by 1 · see citing works
- Authors
- Cui, L., Wu, Y., Liu, S., Zhang, Y., & Zhou, M.
- Affiliations
- Westlake University (CN)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 1406-1416
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3034446185
- Cited by
- 110
- References
- 38
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 4 · 2023: 28 · 2022: 22 · 2021: 37 · 2020: 6
- FWCI
- 8.88
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Westlake University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- dialogue dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
