Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Open accessAbstract
Multi-hop Reading Comprehension (RC) requires reasoning and aggregation across several paragraphs.We propose a system for multi-hop RC that decomposes a compositional question into simpler sub-questions that can be answered by off-the-shelf single-hop RC models.Since annotations for such decomposition are expensive, we recast subquestion generation as a span prediction problem and show that our method, trained using only 400 labeled examples, generates sub-questions that are as effective as humanauthored sub-questions.We also introduce a new global rescoring approach that considers each decomposition (i.e. the sub-questions and their answers) to select the best final answer, greatly improving overall performance.Our experiments on HOTPOTQA show that this approach achieves the state-of-the-art results, while providing explainable evidence for its decision making in the form of sub-questions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
Self-citationReading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520 · see citing works
Flagged first, then by citations
Research works printed but not linked: 19
- Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In EMNLP.
- Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In NAACL.
- Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehen- sion. In ACL.
- Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. 2018. Neural models for reasoning over multiple mentions using coreference. In NAACL.
- Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Su- leyman, and Phil Blunsom. 2015. Teaching ma- chines to read and comprehend. In NIPS.
- Percy Liang, Michael Jordan, and Dan Klein. 2011. Learning dependency-based compositional seman- tics. In ACL.
- Sewon Min, Victor Zhong, Richard Socher, and Caim- ing Xiong. 2018. Efficient and robust question an- swering from minimal context over documents. In ACL.
- Jekaterina Novikova, Ondej Duek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In EMNLP.
- Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In EMNLP.
- Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP.
- Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In ICLR.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS.
- Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2017. Constructing Datasets for Multi-hop Reading Comprehension Across Documents. In TACL.
- Caiming Xiong, Victor Zhong, and Richard Socher. 2018. DCN+: Mixed objective and deep residual coattention for question answering. In ICLR.
- Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In EMNLP.
- Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. 2018. Fast and accurate reading comprehension by combining self-attention and convolution. In ICLR.
- John M. Zelle and Raymond J. Mooney. 1996. Learn- ing to parse database queries using inductive logic programming. In AAAI/IAAI.
- Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Struc- tured classification with probabilistic categorial grammars. In UAI.
- Victor Zhong, Caiming Xiong, Nitish Shirish Keskar, and Richard Socher. 2019. Coarse-grain fine-grain coattention network for multi-evidence question an- swering. In ICLR.
Cited by
Sorted by citations
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2210.03350 · Cited by 294 · see citing works
Self-citation♫ MuSiQue: Multihop Questions via Single-hop Question Composition
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00475 · Cited by 283 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N., & Lewis, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.378 · Cited by 303 · see citing works
Self-citation<i>Did Aristotle Use a Laptop?</i> A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242 · see citing works
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1911.10470 · Cited by 152 · see citing works
Self-citationTurpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
“Decomposition-based approaches (Min et al., 2019; Perez et al., 2020; Chen et al., 2022; Creswell and Shanahan, 2022; Tafjord et al., 2022; Eisenstein et al., 2022; Reppert et al., 2023) improve faithfulness by limiting contextual cues that may bias CoT reasoning, with Radhakrishnan et al. (2023)…”
BackgroundLEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration
Cao, Y., Gao, Z., Li, Z., Xie, X., Zhou, S. K., & Xu, J. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3748191.3748194 · Cited by 11 · see citing works
Beamaggr: Beam aggregation reasoning over multi-source knowledge for multi-hop question answering
Chu, Z., Chen, J., Chen, Q., Wang, H., Zhu, K., Du, X., Yu, W., Liu, M., & Qin, B. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.67 · Cited by 11 · see citing works
- Authors
- Min, S., Zhong, V., Zettlemoyer, L., & Hajishirzi, H.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 6097-6109
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2949134692
- Cited by
- 197
- References
- 5
- Citations by year
- 2026: 13 · 2025: 20 · 2024: 13 · 2023: 35 · 2022: 35 · 2021: 44 · 2020: 28 · 2019: 9
- FWCI
- 11.09
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1616112, 1252835, 1562364)
- Allen Institute for Artificial Intelligence
- Samsung
- Office of Naval Research (N00014-17-S-B001, N00014-18, N00014-18-1-2826, N00014, N00014-17)
- Army Research Office (W911NF-16-1-, W911NF)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- question decomposition, span prediction, HotpotQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
