Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass traditional term-based techniques, which are constructed from raw corpora.In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap.The question generation system is trained on general domain data, but is applied to documents in the targeted domain.This allows us to create arbitrarily large, yet noisy, question-passage relevance pairs that are domain specific.Furthermore, when this is coupled with a simple hybrid termneural model, first-stage retrieval performance can be improved further.Empirically, we show that this is an effective strategy for building neural passage retrieval models in the absence of large training corpora.Depending on the domain, this technique can even approach the accuracy of supervised models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 15×
“First, we see that both QA and QGen consistently outperform neural baselines such as ICT and Ngram that are based on sub-string masking or matching.”
BackgroundMethodResultInfluentialBillion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 8×
“Another technique is representation-based models that embed ques-Encoder tions and passages into a common dense subspace (Palangi et al., 2016) and use nearest neighbour search for retrieval (Liu et al., 2011; Johnson et al., 2017).”
BackgroundMethodInfluentialDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
Learning to rank: from pairwise approach to listwise approach
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273513 · Cited by 1,960 · cited 2×
“The objective is to minimize the negative log-likelihood: This loss function is a special case of ListNet loss (Cao et al., 2007) where all relevance judgements are binary, and only one passage is marked relevant for each training example.”
Background
Cited by
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.99 · Cited by 362
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2022) · arXiv · doi:10.48550/arxiv.2212.10496 · Cited by 362
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021) · arXiv · doi:10.48550/arxiv.2104.08663 · Cited by 45
Promptagator: Few-shot Dense Retrieval From 8 Examples
Dai, Z., Zhao, V. Y., Ma, J., Luan, Y., Ni, J., Lu, J., Bakalov, A., Guu, K., Hall, K. B., & Chang, M.-W. (2022) · arXiv · doi:10.48550/arxiv.2209.11755 · Cited by 43
“Data augmentation via synthetic query generation has previously been explored (Ma et al., 2021; Shakeri et al., 2020), but these question generators are learned from high-resource QA datasets, and often cannot generalize well to new retrieval tasks.”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Ma, J., Korotkov, I., Yang, Y., Hall, K., & McDonald, R.
- Venue
- Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
- Published
- 2021
- Pages
- 1075-1088
- OpenAlex
- W3156836409
- Cited by
- 80
- References
- 18
- FWCI
- 12.47
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation. https://app.cochat.ai/papers/10.18653/v1/2021.eacl-main.92
- The paper
- Ma, J., Korotkov, I., Yang, Y., Hall, K., & McDonald, R. (2021). Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. https://doi.org/10.18653/v1/2021.eacl-main.92
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.