The DOI matches its DataCite record; 37 of the 37 references it cites were found in a registry or an index.
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Open accessAbstract
Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question. This paper introduces a new graph-based recurrent retrieval approach that learns to retrieve reasoning paths over the Wikipedia graph to answer multi-hop open-domain questions. Our retriever model trains a recurrent neural network that learns to sequentially retrieve evidence paragraphs in the reasoning path by conditioning on the previously retrieved documents. Our reader model ranks the reasoning paths and extracts the answer span included in the best reasoning path. Experimental results show state-of-the-art results in three open-domain QA datasets, showcasing the effectiveness and robustness of our method. Notably, our method achieves significant improvement in HotpotQA, outperforming the previous best model by more than 14 points.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698
Self-citationWeight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural NetworksPreprint
Salimans, T., & Kingma, D. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07868 · Cited by 1,551
Latent retrieval for weakly supervised open domain question answering
Lee, K., Chang, M.-W., & Toutanova, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1612 · Cited by 771
Using the Output Embedding to Improve Language Models
Press, O., & Wolf, L. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2025 · Cited by 729
Scaling Neural Machine Translation
Ott, M., Edunov, S., Grangier, D., & Auli, M. (2018) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w18-6301 · Cited by 631
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615
End-to-End Open-Domain Question Answering with BERTserini
Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., & Lin, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-4013 · Cited by 535
Dynamic Coattention Networks For Question AnsweringPreprint
Xiong, C., Zhong, V., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01604 · Cited by 504
Self-citationSimple and Effective Multi-Paragraph Reading Comprehension
Clark, C., & Gardner, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1078 · Cited by 434
Tying Word Vectors and Word Classifiers: A Loss Framework for Language ModelingPreprint
Inan, H., Khosravi, K., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01462 · Cited by 368
Self-citationR3: Reinforced Ranker-Reader for Open-Domain Question Answering
Wang, S., Yu, M., Guo, X., Wang, Z., Klinger, T., Zhang, W., Chang, S., Tesauro, G., Zhou, B., & Jiang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12053 · Cited by 351
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
Wang, Z., Ng, P., Ma, X., Nallapati, R. M., & Xiang, B. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1599 · Cited by 277
Fast and Accurate Annotation of Short Texts with Wikipedia Pages
Ferragina, P., & Scaiella, U. (2011) · IEEE Software · doi:10.1109/ms.2011.122 · Cited by 242
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
Ding, M., Zhou, C., Chen, Q., Yang, H., & Tang, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1259 · Cited by 222
Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Min, S., Zhong, V., Zettlemoyer, L., & Hajishirzi, H. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1613 · Cited by 197
Self-citationReal-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
Seo, M., Lee, J., Kwiatkowski, T., Parikh, A. P., Farhadi, A., & Hajishirzi, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1436 · Cited by 160
Self-citationEvidence Aggregation for Answer Re-Ranking in Open-Domain Question AnsweringPreprint
Wang, S., Yu, M., Jiang, J., Zhang, W., Guo, X., Chang, S., Wang, Z., Klinger, T., Tesauro, G. J., & Campbell, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05116 · Cited by 154
Efficient and Robust Question Answering from Minimal Context over Documents
Min, S., Zhong, V., Socher, R., & Xiong, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1160 · Cited by 150
Self-citationA Discrete Hard EM Approach for Weakly Supervised Question Answering
Min, S., Chen, D., Hajishirzi, H., & Zettlemoyer, L. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1284 · Cited by 144
Self-citationCompositional Questions Do Not Necessitate Multi-hop Reasoning
Min, S., Wallace, E., Singh, S., Gardner, M., Hajishirzi, H., & Zettlemoyer, L. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1416 · Cited by 138
Self-citationRevealing the Importance of Semantic Retrieval for Machine Reading at Scale
Nie, Y., Wang, S., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1258 · Cited by 132
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering
Feldman, Y., & El‐Yaniv, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1222 · Cited by 103
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., & Manning, C. D. (2018) · arXiv · doi:10.48550/arxiv.1809.09600 · Cited by 97
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering
Lee, J., Yun, S., Kim, H., Ko, M., & Kang, J. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1053 · Cited by 97
Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
Hu, M., Peng, Y., Huang, Z., & Li, D. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1221 · Cited by 59
Adaptive Document Retrieval for Deep Question Answering
Kratzwald, B., & Feuerriegel, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1055 · Cited by 56
Dynamically Fused Graph Network for Multi-hop ReasoningPreprint
Xiao, Y., Qu, Y., Qiu, L., Zhou, H., Li, L., Zhang, W., & Yu, Y. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06933 · Cited by 42
PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and TextPreprint
Sun, H., Bedrax-Weiss, T., & Cohen, W. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09537 · Cited by 33
Answering Complex Open-domain Questions Through Iterative Query GenerationPreprint
Qi, P., Lin, X., Mehr, L., Wang, Z., & Manning, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07000 · Cited by 28
Do Multi-hop Readers Dream of Reasoning Chains?
Wang, H., Yu, M., Guo, X., Das, R., Xiong, W., & Gao, T. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5813 · Cited by 26
Weaver: Deep Co-Encoding of Questions and Documents for Machine ReadingPreprint
Raison, M., Mazaré, P.-E., Das, R., & Bordes, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.10490 · Cited by 15
Cited by
Sorted by citations
Self-supervised Learning: Generative or Contrastive
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3090866 · Cited by 2,008
A Metaverse: Taxonomy, Components, Applications, and Open Challenges
Park, S., & Kim, Y. (2022) · IEEE Access · doi:10.1109/access.2021.3140175 · Cited by 2,046
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829
“…reasoning paths, such as constructing semantic graphs (Xu et al., 2021a), learning an RNN to retrieve reasoning paths over the Wikipedia graph (Asai et al., 2020), fine-tuning with human annotated reasoning paths on math problems (Cobbe et al., 2021), or training an extractor with…”
BackgroundHow Much Knowledge Can You Pack into the Parameters of a Language Model?
Roberts, A., Raffel, C., & Shazeer, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.437 · Cited by 638
Retrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575
Metadata
- Authors
- Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., & Xiong, C.
- Affiliations
- University of Washington (US)
- Salesforce (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1911.10470
- OpenAlex
- W2990928880
- Cited by
- 152
- References
- 37
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 23 · 2022: 37 · 2021: 54 · 2020: 27 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multi-hop question answering, open-domain question answering, graph-based retrieval, HotpotQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
