CoChat
Sign Up
PreprintCited by 14Topic ModelingNatural Language Processing TechniquesSemantic Web and Ontologies

The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.

Reasoning Over Virtual Knowledge Bases With Open Predicate Relations

Open access
Sun, Haitian, Verga, Pat, Dhingra, Bhuwan, Salakhutdinov, Ruslan, Cohen, William W.·2021·arXiv
DOI 10.48550/arxiv.2102.07043arXiv 2102.07043
Open this paper in CoChat — free accountOpen PDF

Abstract

We present the Open Predicate Query Language (OPQL); a method for constructing a virtual KB (VKB) trained entirely from text. Large Knowledge Bases (KBs) are indispensable for a wide-range of industry applications such as question answering and recommendation. Typically, KBs encode world knowledge in a structured, readily accessible form derived from laborious human annotation efforts. Unfortunately, while they are extremely high precision, KBs are inevitably highly incomplete and automated methods for enriching them are far too inaccurate. Instead, OPQL constructs a VKB by encoding and indexing a set of relation mentions in a way that naturally enables reasoning and can be trained without any structured supervision. We demonstrate that OPQL outperforms prior VKB methods on two different KB reasoning tasks and, additionally, can be used as an external memory integrated into a language model (OPQL-LM) leading to improvements on two open-domain question answering tasks.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2102.07043

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • OPQL uses a dual-encoder pre-training process to encode relation mentions, which can be integrated into a language model (OPQL-LM) to improve performance on open-domain question answering.

    “Open Predicate Query Language (OPQL) (Sun et al., 2021) uses a dual-encoder pre-training process to encode relation mentions, which can be integrated into a language model (OPQL-LM) to improve performance on open-domain question answering.”

    From The AI Hippocampus: How Far are We From Human Memory? (2026) · cites this paper as Sun et al., 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 35

  1. Ahn, S., Choi, H., P¨ arnamaa, T., and Bengio, Y. A neural knowledge language model. arXiv preprint arXiv:1608.00318 , 2016. · arXiv
  2. Angeli, G., Premkumar, M. J. J., and Manning, C. D. Leveraging linguistic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pp. 344-354, 2015.
  3. Baldini Soares, L., FitzGerald, N., Ling, J., and Kwiatkowski, T. Matching the blanks: Distributional similarity for relation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 2895-2905, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.
  4. Chen, D., Fisch, A., Weston, J., and Bordes, A. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 1870-1879, 2017.
  5. Chen, Q., Zhu, X., Ling, Z.-H., Inkpen, D., and Wei, S. Neural natural language inference models enhanced with external knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 2406-2417, 2018.
  6. Cohen, W. W., Sun, H., Hofer, R. A., and Siegler, M. Scalable neural methods for reasoning with a symbolic knowledge base. International Conference on Learning Representations , 2020.
  7. Dhingra, B., Zaheer, M., Balachandran, V., Neubig, G., Salakhutdinov, R., and Cohen, W. W. Differentiable reasoning over a virtual knowledge base. In International Conference on Learning Representations , 2020.
  8. Etzioni, O., Banko, M., Soderland, S., and Weld, D. S. Open information extraction from the web. Communications of the ACM , 51(12):68-74, 2008.
  9. Fader, A., Soderland, S., and Etzioni, O. Identifying relations for open information extraction. In Proceedings of the 2011 conference on empirical methods in natural language processing , pp. 1535-1545, 2011.
  10. F´ evry, T., Soares, L. B., FitzGerald, N., Choi, E., and Kwiatkowski, T. Entities as experts: Sparse memory access with entity supervision. Conference on Empirical Methods in Natural Language Processing , 2020.
  11. Gupta, S., Kenkre, S., and Talukdar, P. CaRe: Open knowledge graph embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pp. 378-388, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/ v1/D19-1036. URL https://www.aclweb.org/ anthology/D19-1036 .
  12. Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M.W. Realm: Retrieval-augmented language model pretraining. arXiv preprint arXiv:2002.08909 , 2020. · arXiv
  13. Henaff, M., Weston, J., Szlam, A., Bordes, A., and LeCun, Y. Tracking the world state with recurrent entity networks. arXiv preprint arXiv:1612.03969 , 2016. · arXiv
  14. Karpukhin, V., O˘ guz, B., Min, S., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 , 2020. · arXiv
  15. Ling, J., FitzGerald, N., Shan, Z., Soares, L. B., F´ evry, T., Weiss, D., and Kwiatkowski, T. Learning crosscontext entity representations from text. arXiv preprint arXiv:2001.03765 , 2020. · arXiv
  16. Logan, R., Liu, N. F., Peters, M. E., Gardner, M., and Singh, S. Barack's wife hillary: Using knowledge graphs for fact-aware language modeling. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019a. doi: 10.18653/v1/p19-1598. URL http: //dx.doi.org/10.18653/v1/P19-1598 . · DOI · Paper page
  17. Logan, R., Liu, N. F., Peters, M. E., Gardner, M., and Singh, S. Barack's wife hillary: Using knowledge graphs for factaware language modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 5962-5971, 2019b.
  18. Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18-1076. URL https: //www.aclweb.org/anthology/P18-1076 . · DOI
  19. Miller, A., Fisch, A., Dodge, J., Karimi, A.-H., Bordes, A., and Weston, J. Key-value memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pp. 1400-1409, 2016.
  20. Min, B., Grishman, R., Wan, L., Wang, C., and Gondek, D. Distant supervision for relation extraction with an incomplete knowledge base. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pp. 777-782, 2013.
  21. Mitchell, T., Cohen, W., Hruschka, E., Talukdar, P., Betteridge, J., Carlson, A., Dalvi, B., Gardner, M., Kisiel, B., Krishnamurthy, J., Lao, N., Mazaitis, K., Mohamed, T., Nakashole, N., Platanios, E., Ritter, A., Samadi, M., Settles, B., Wang, R., Wijaya, D., Gupta, A., Chen, X., Saparov, A., Greaves, M., and Welling, J. Never-ending learning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15) , 2015.
  22. Newell, A., Shaw, J. C., and Simon, H. A. Report on a general problem-solving program. In Proceedings of the International Conference on Information Processing , 1959.
  23. Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., and Smith, N. A. Knowledge enhanced contextual word representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pp. 43-54, 2019.
  24. Qi, P., Lee, H., Sido, O., Manning, C. D., et al. Retrieve, rerank, read, then iterate: Answering open-domain questions of arbitrary complexity from text. arXiv preprint arXiv:2010.12527 , 2020. · arXiv
  25. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 , 2019. · arXiv
  26. Seo, M., Kwiatkowski, T., Parikh, A., Farhadi, A., and Hajishirzi, H. Phrase-indexed question answering: A new challenge for scalable document comprehension. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pp. 559-564,
  27. Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/ v1/D18-1052. URL https://www.aclweb.org/ anthology/D18-1052 .
  28. Seo, M., Lee, J., Kwiatkowski, T., Parikh, A., Farhadi, A., and Hajishirzi, H. Real-time open-domain question answering with dense-sparse phrase index. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 4430-4441, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1436. URL https: //www.aclweb.org/anthology/P19-1436 . · DOI
  29. Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., and Cohen, W. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pp. 4231-4242, 2018.
  30. Sun, H., Bedrax-Weiss, T., and Cohen, W. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pp. 2380-2390, 2019.
  31. Sun, H., Arnold, A., Bedrax Weiss, T., Pereira, F., and Cohen, W. W. Faithful embeddings for knowledge base queries. Advances in Neural Information Processing Systems , 33, 2020.
  32. Talmor, A. and Berant, J. The web as a knowledge-base for answering complex questions. In North American Association for Computational Linguistics (NAACL) , 2018.
  33. Verga, P., Sun, H., Soares, L. B., and Cohen, W. W. Facts as experts: Adaptable and interpretable neural memory over symbolic knowledge. arXiv preprint arXiv:2007.00849 , 2020. · arXiv
  34. Yih, W.-t., Chang, M.-W., He, X., and Gao, J. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pp. 1321-1331, Beijing, China, July 2015. Association for Computational Linguistics. URL http: //www.aclweb.org/anthology/P15-1128 .
  35. Zhang, Y., Dai, H., Kozareva, Z., Smola, A. J., and Song, L. Variational reasoning for question answering with knowledge graph. In AAAI , 2018.

Cited by

Sorted by citations

Metadata

Authors
Sun, H., Verga, P., Dhingra, B., Salakhutdinov, R., & Cohen, W. W.
Affiliations
  • Carnegie Mellon University (US)
  • Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
Pages
9966-9977
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2102.07043
arXiv
2102.07043
OpenAlex
W3173027903
Cited by
14
References
5
Citations by year
2025: 2 · 2023: 4 · 2022: 3 · 2021: 5
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
Keywords
knowledge base reasoning, knowledge base completion, external memory, language models, open-domain question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review