Unsupervised Question Decomposition for Question Answering
Open accessAbstract
We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering.Since labeling questions with decompositions is cumbersome, we take an unsupervised approach to produce sub-questions, also enabling us to leverage millions of questions from the internet.Specifically, we propose an algorithm for One-to-N Unsupervised Sequence transduction (ONUS) that learns to map one hard, multi-hop question to many simpler, singlehop sub-questions.We answer sub-questions with an off-the-shelf QA model and give the resulting answers to a recomposition model that combines them into a final answer.We show large QA improvements on HOTPOTQA over a strong baseline on the original, out-ofdomain, and multi-hop dev sets.ONUS automatically learns to decompose different kinds of questions, while matching the utility of supervised and heuristic decomposition methods for QA and exceeding those methods in fluency.Qualitatively, we find that using subquestions is promising for shedding light on why a QA system makes a prediction. 1 * KC was a part-time research scientist at Facebook AI Research while working on this paper.1 Our code, data, and pretrained models are available at https://github.com/facebookresearch/ UnsupervisedDecomposition. What profession do H. L. Mencken and Albert Camus have in common?
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545 · see citing works
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228 · see citing works
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · see citing works
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520 · see citing works
Generating Sentences by Editing Prototypes
Guu, K., Hashimoto, T. B., Oren, Y., & Liang, P. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00030 · Cited by 300 · see citing works
Compositional Questions Do Not Necessitate Multi-hop Reasoning
Min, S., Wallace, E., Singh, S., Gardner, M., Hajishirzi, H., & Zettlemoyer, L. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1416 · Cited by 138 · see citing works
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale
Nie, Y., Wang, S., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1258 · Cited by 132 · see citing works
Flagged first, then by citations
Research works printed but not linked: 39
- Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2015. Neural module networks. CVPR, pages 39-48.
- Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016. Learning to compose neural net- works for question answering. In NAACL, pages 1545-1554, San Diego, California. Association for Computational Linguistics. · DOI
- Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural ma- chine translation. In ICLR.
- Mikel Artetxe and Holger Schwenk. 2019. Margin- based parallel corpus mining with multilingual sen- tence embeddings. In ACL, pages 3197-3203, Flo- rence, Italy. Association for Computational Linguis- tics. · DOI
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open- domain questions. In ACL, pages 1870-1879, Van- couver, Canada. Association for Computational Lin- guistics. · DOI · Paper page
- Paul Francis Christiano, Buck Shlegeris, and Dario Amodei. 2018. Supervising strong learners by am- plifying weak experts. CoRR, abs/1810.08575. · arXiv
- Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehen- sion. In ACL, pages 845-855, Melbourne, Australia. Association for Computational Linguistics. · DOI
- Abhishek Das, Satwik Kottur, José M.F. Moura, Stefan Lee, and Dhruv Batra. 2017. Learning cooperative visual dialog agents with deep reinforcement learn- ing. In ICCV.
- Li Dong and Mirella Lapata. 2016. Language to log- ical form with neural attention. In ACL, pages 33- 43, Berlin, Germany. Association for Computational Linguistics. · DOI · Paper page
- Manaal Faruqui and Dipanjan Das. 2018. Identifying well-formed natural language questions. In EMNLP, pages 798-803, Brussels, Belgium. Association for Computational Linguistics. · DOI
- David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris Welty. 2010. Building watson: An overview of the deepqa project. AI Magazine, 31(3):59-79. · DOI
- Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Sil- ver, and Koray Kavukcuoglu. 2017. Reinforcement learning with unsupervised auxiliary tasks. In ICLR.
- Yichen Jiang and Mohit Bansal. 2019a. Avoiding rea- soning shortcuts: Adversarial evaluation, training, and model development for multi-hop QA. In ACL, pages 2726-2736, Florence, Italy. Association for Computational Linguistics. · DOI
- Yichen Jiang and Mohit Bansal. 2019b. Self- assembling modular networks for interpretable multi-hop reasoning. In EMNLP, Hong Kong, China. Association for Computational Linguistics.
- Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. 2017b. CLEVR: A diagnostic dataset for compositional language and elementary visual rea- soning. In CVPR.
- Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C Lawrence Zit- nick, and Ross Girshick. 2017c. Inferring and exe- cuting programs for visual reasoning. In ICCV.
- Guillaume Lample and Alexis Conneau. 2019. Cross- lingual language model pretraining. In NeurIPS.
- Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Unsupervised ma- chine translation using monolingual corpora only. In ICLR.
- Patrick Lewis, Ludovic Denoyer, and Sebastian Riedel. 2019. Unsupervised question answering by cloze translation. In ACL, pages 4896-4910, Florence, Italy. Association for Computational Linguistics. · DOI
- Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc V Le, and Ni Lao. 2018. Memory aug- mented policy optimization for program synthesis and semantic parsing. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, NeurIPS, pages 9994-10006.
- Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018. Mixed pre- cision training. In ICLR.
- Sewon Min, Victor Zhong, Luke Zettlemoyer, and Han- naneh Hajishirzi. 2019b. Multi-hop reading compre- hension through question decomposition and rescor- ing. In ACL, pages 6097-6109, Florence, Italy. As- sociation for Computational Linguistics. · DOI · Paper page
- Rodrigo Nogueira and Kyunghyun Cho. 2017. Task- oriented query reformulation with reinforcement learning. In EMNLP, pages 574-583, Copenhagen, Denmark. Association for Computational Linguis- tics. · DOI · Paper page
- Michael Petrochuk and Luke Zettlemoyer. 2018. Sim- pleQuestions nearly solved: A new upperbound and baseline approach. In EMNLP, pages 554-558, Brussels, Belgium. Association for Computational Linguistics. · DOI
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In ACL, pages 86-96, Berlin, Germany. Association for Computational Linguis- tics. · DOI · Paper page
- Florian Strub, Harm de Vries, Jérémie Mary, Bilal Piot, Aaron Courville, and Olivier Pietquin. 2017. End-to-end optimization of goal-driven and visually grounded dialogue systems. In IJCAI, pages 2765- 2771. · DOI
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. CoRR, abs/1803.05355. · arXiv
- James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2019. The FEVER2.0 shared task. In Proceedings of the Second Workshop on Fact Extraction and VERifica- tion (FEVER), pages 1-6, Hong Kong, China. Asso- ciation for Computational Linguistics. · DOI
- Ming Tu, Kevin Huang, Guangtao Wang, Jing Huang, Xiaodong He, and Bowen Zhou. 2020. Select, an- swer and explain: Interpretable multi-hop reading comprehension over multiple documents. In AAAI.
- Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, NeurIPS, pages 5998-6008.
- Ziyun Wang and Brenden M. Lake. 2019. Model- ing question asking using neural program generation. CoRR, abs/1907.09899. · arXiv
- Terry Winograd. 1972. Understanding Natural Lan- guage. Academic Press, Inc., USA.
- Terry Winograd. 1991. Thinking Machines: Can There Be? Are We? University of California Press, Berke- ley.
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, R'emi Louf, Morgan Funtow- icz, and Jamie Brew. 2019. Huggingface's trans- formers: State-of-the-art natural language process- ing. CoRR, abs/1910.03771. · arXiv
- Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gard- ner, Yoav Goldberg, Daniel Deutch, and Jonathan Berant. 2020. Break it down: A question under- standing benchmark. TACL.
- W. Woods, R. Kaplan, and B. Nash-Webber. 1974. The lunar sciences natural language information system. Final Report 2378, Bolt, Beranek and Newman, Inc., Cambridge, MA. · DOI
- Hainan Xu and Philipp Koehn. 2017. Zipporah: a fast and scalable data cleaning system for noisy web- crawled parallel corpora. In EMNLP, pages 2945- 2950, Copenhagen, Denmark. Association for Com- putational Linguistics. · DOI
- Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering. In EMNLP, pages 2369-2380, Brussels, Belgium. As- sociation for Computational Linguistics. · DOI · Paper page
- Guangyou Zhou, Tingting He, Jun Zhao, and Po Hu. 2015. Learning continuous word embedding with metadata for question retrieval in community ques- tion answering. In ACL, pages 250-259, Beijing, China. Association for Computational Linguistics. · DOI
Cited by
Sorted by citations
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2210.03350 · Cited by 294 · see citing works
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N., & Lewis, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.378 · Cited by 303 · see citing works
<i>Did Aristotle Use a Laptop?</i> A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242 · see citing works
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
“Decomposition-based approaches (Min et al., 2019; Perez et al., 2020; Chen et al., 2022; Creswell and Shanahan, 2022; Tafjord et al., 2022; Eisenstein et al., 2022; Reppert et al., 2023) improve faithfulness by limiting contextual cues that may bias CoT reasoning, with Radhakrishnan et al. (2023)…”
BackgroundSelf-citationDiscovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · see citing works
“, 2021), generating subquestions for a question (Perez et al., 2020), and question-answering and summarization (Joshi et al.”
BackgroundSelf-citationLeveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
Li, R., & Du, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.452 · Cited by 12 · see citing works
Chang, Y., Li, Z., Zhang, H., Kong, Y., Wu, Y., So, H. K.-H., Guo, Z., Zhu, L., & Wong, N. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.790 · Cited by 2 · see citing works
- Authors
- Perez, E., Lewis, P., Yih, W.-T., Cho, K., & Kiela, D.
- Affiliations
- Meta (United States) (US)
- Meta (Israel) (IL)
- New York University (US)
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 8864-8880
- OpenAlex
- W3007129127
- Cited by
- 123
- References
- 10
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 9 · 2023: 28 · 2022: 24 · 2021: 35 · 2020: 7
- FWCI
- 8.97
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1922658)
- Nvidia
- Samsung Advanced Institute of Technology
- Open Philanthropy Project
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- question decomposition, multi-hop question answering, HotpotQA, out-of-domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
