The DOI matches its Crossref record, with a small difference in title or year; 36 of the 38 references it cites were found in a registry or an index.
MuTual: A Dataset for Multi-Turn Dialogue Reasoning
Open accessAbstract
Non-task oriented dialogue systems have achieved great success in recent years due to largely accessible conversation data and the development of deep learning techniques.Given a context, current systems are able to yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities.To facilitate the conversation reasoning research, we introduce Mu-Tual, a novel dataset for Multi-Turn dialogue Reasoning, consisting of 8,860 manually annotated dialogues based on Chinese student English listening comprehension exams.Compared to previous benchmarks for non-task oriented dialogue systems, MuTual is much more challenging since it requires a model that can handle various reasoning problems.Empirical results show that state-of-the-art methods only reach 71%, which is far behind the human performance of 94%, indicating that there is ample room for improving reasoning ability.MuTual is available at https://github. com/Nealcly/MuTual. * Contribution during internship at MSRA.M: Ma'am
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“There are some reasoning works in the area of dialogue [39, 83, 272], but few in reducing hallucinations.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312
What Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,953
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. Z. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1152 · Cited by 1,315
(2019) · doi:10.18653/v1/p19-1356 · Cited by 1,291
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
Neural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · doi:10.3115/v1/p15-1152 · Cited by 1,146
Flagged first, then by citations
2 of the 38 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, Advances in Neural Information Pro- cessing Systems 30, pages 5998-6008.
- Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. In 7th International Conference on Learning Represen- tations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
- Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang. 2018. Aug- menting end-to-end dialogue systems with common- sense knowledge. In Proceedings of the Thirty- Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Arti- ficial Intelligence (IAAI-18), and the 8th AAAI Sym- posium on Educational Advances in Artificial Intel- ligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 4970-4977.
Cited by
Sorted by citations
Items 101–120 of 120
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection
Deng, J., Dai, H., Guo, X., Ju, Y., & Peng, W. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.584 · Cited by 1
Contrastive Learning for Inference in Dialogue
Ishii, E., Xu, Y., Wilie, B., Ji, Z., Lovenia, H., Chung, W., & Fung, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.631 · Cited by 1
Exploring Dialog Act Recognition in Open Domain Conversational Agents
Sultana, M., & Zai͏̈ane, O. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-39831-5_22 · Cited by 1
Generating Math Word Problems from Equations with Topic Controlling and Commonsense EnforcementPreprint
Cao, T., Zeng, S., Zhao, S., Mansur, M., & Chang, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.07379 · Cited by 1
Chen, Y., Zhang, C., Luo, D., D'Haro, L. F., Tan, R. T., & Li, H. (2024) · arXiv · doi:10.48550/arxiv.2405.14646
Multi -Turn Response Selection with Temporal Gated Graph Convolutional Networks
Tao, S., Zhao, Q., Wang, L., & He, L. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534300
Meta-learning for natural language processing
Zou, L. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-89931-4.00005-5
On the Compositional Generalization in Versatile Open-domain Dialogue
Fu, T., Zhao, X., Liu, L., & Yan, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.760
CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs
Aksu, T., Hazarika, D., Mehri, S., Kim, S., Hakkani-Tür, D. Z., Liu, Y., & Namazifar, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.717 · Cited by 1
Common Sense Reasoning on Fully Localized AI; A Novel systematic Survey
Karunakaran, T. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.66589
Agent-Based ChatGPT for Multi-Party Conversation Task Processing
万, 静. (2025) · Software Engineering and Applications · doi:10.12677/sea.2025.142022
Roy, A., Sengupta, A., & Pani, S. (2025) · doi:10.31224/4666
Cao, W., Liu, H., & Jing, P. (2025) · International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA) · doi:10.1109/caibda65784.2025.11182636
Towards Context-Based Violence Detection: A Korean Crime Dialogue Dataset
Kim, M., Yeen, H., & Koo, M. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.42
2T-FT: Two-Token Fine-tuning improves zero-shot performance with minimal training
Neto, P. S., Dyonisio, J. D. S., Lemos, J. F. S. S., Kühne, F., Guerra, R. S., & Drews, P. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12052-9
EVA: Editing for Versatile Alignment Against Jailbreaks
Wang, Y., Qiu, H., Xu, Y., Yang, S., Qin, Z., Huang, M., & Wang, W. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3694189 · Cited by 1
Commonsense Knowledge Acquisition in Dialogue via Multi-Result Difference Comparison
Wang, K., Ma, Z., Lv, K., & Liu, J. (2025) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait67930.2025.11522104
Abstractive Dialogue Summarization using Fine-Tuning Pre-Trained Language Model BART
Rodiah, D., & Ramadhani, H. S. (2026) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v10i3.7488
A topic semantic relational knowledge-enhanced conversational model for improving response accuracy
Li, K., Guo, X., Zhang, S., & Liu, J. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-026-02912-0
Liu, Y., Qian, Z., Li, P., & Zhu, Q. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729293
Metadata
- Authors
- Cui, L., Wu, Y., Liu, S., Zhang, Y., & Zhou, M.
- Affiliations
- Westlake University (CN)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 1406-1416
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3034446185
- Cited by
- 110
- References
- 38
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 4 · 2023: 28 · 2022: 22 · 2021: 37 · 2020: 6
- FWCI
- 8.88
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Westlake University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- dialogue dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
