The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Question Answering on Freebase via Relation Extraction and Textual Evidence
Open accessAbstract
Existing knowledge-based question answering systems often rely on small annotated training data.While shallow methods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at answering questions involving multiple constraints.Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia.We first present a neural network based relation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers.Experiments on the WebQuestions question answering dataset show that our method achieves an F 1 of 53.3%, a substantial improvement over the state-of-the-art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
(2010) · Cited by 8,261
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
Freebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852
Distant supervision for relation extraction without labeled data
Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690287 · Cited by 2,752
Teaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
Yago: a core of semantic knowledge
Suchanek, F. M., Kasneci, G., & Weikum, G. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242667 · Cited by 2,512
DBpedia: A Nucleus for a Web of Open Data
Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-76298-0_52 · Cited by 2,454
Toward an architecture for never-ending language learning
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., & Mitchell, T. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7519 · Cited by 2,000 · cited 1×
“…to large scale structured KBs like Freebase, DBPedia (Unger et al., 2012; Cai and Yates, 2013; Berant et al., 2013; Kwiatkowski et al., 2013; Xu et al., 2014), and on noisy KBs (Banko et al., 2007; Carlson et al., 2010; Krishnamurthy and Mitchell, 2012; Fader et al., 2013; Parikh et al., 2015).”
BackgroundTraining linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
(2010) · Cited by 1,906
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 1,517
Open information extraction from the web
Etzioni, O., Banko, M., Soderland, S., & Weld, D. S. (2008) · Communications of the ACM · doi:10.1145/1409360.1409378 · Cited by 1,012
Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 874
The TREC-8 Question Answering Track Report
Voorhees, E. M. (1999) · Cited by 831
WikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829 · cited 1×
“TREC QA evaluations (Voorhees and Tice, 1999) were a major boost to unstructured QA leading to richer datasets and sophisticated methods (Wang et al., 2007; Heilman and Smith, 2010; Yao et al., 2013; Yih et al., 2013; Yu et al., 2014; Yang et al., 2015; Hermann et al., 2015).”
BackgroundSemantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
Yih, W.-T., Chang, M.-W., He, X., & Gao, J. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1128 · Cited by 756
Relation Extraction with Matrix Factorization and Universal Schemas
Riedel, S., Yao, L., McCallum, A., & Marlin, B. M. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 723
Learning to parse database queries using inductive logic programming
Zelle, J. M., & Mooney, R. J. (1996) · Cited by 642
Question Answering with Subgraph Embeddings
Bordes, A., Chopra, S., & Weston, J. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1067 · Cited by 635
Large-scale Simple Question Answering with Memory Networks
Bordes, A., Usunier, N., Chopra, S., & Weston, J. (2015) · arXiv · doi:10.48550/arxiv.1506.02075 · Cited by 528 · cited 1×
“Designing large training datasets for these methods is relatively easy (Yao and Van Durme, 2014; Bordes et al., 2015; Serban et al., 2016).”
BackgroundTemplate-based question answering over RDF data
Unger, C., Bühmann, L., Lehmann, J., Ngonga Ngomo, A.-C., Gerber, D., & Cimiano, P. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187923 · Cited by 498
Semantic Parsing via Paraphrasing
Berant, J., & Liang, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1133 · Cited by 495
Information Extraction over Structured Data: Question Answering with Freebase
Yao, X., & Van Durme, B. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1090 · Cited by 490
Question Answering over Freebase with Multi-Column Convolutional Neural Networks
Dong, L., Wei, F., Zhou, M., & Xu, K. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1026 · Cited by 445
Self-citationSemantic Parsing for Single-Relation Question Answering
Yih, W.-T., He, X., & Meek, C. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2105 · Cited by 410
Open question answering over curated and extracted knowledge bases
Fader, A., Zettlemoyer, L., & Etzioni, O. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623677 · Cited by 407
What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA
Wang, M., Smith, N. A., & Mitamura, T. (2007) · Cited by 396
Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
Cai, Q., & Yates, A. (2013) · Cited by 352
Deep Learning for Answer Sentence Selection
Yu, L., Hermann, K. M., Blunsom, P., & Pulman, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1632 · Cited by 340
Paraphrase-Driven Learning for Open Question Answering
Fader, A., Zettlemoyer, L., & Etzioni, O. (2013) · Cited by 329
Building a Semantic Parser Overnight
Wang, Y., Berant, J., & Liang, P. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1129 · Cited by 318
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Serban, I. V., García-Durán, A., Gulcehre, C., Ahn, S., Chandar, S., Courville, A., & Bengio, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1056 · Cited by 280
Question Answering Using Enhanced Lexical Semantic Models
Yih, W.-T., Chang, M., Meek, C., & Pastusiak, A. (2013) · Cited by 270
Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling
Xu, K., Feng, Y., Huang, S., & Zhao, D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1062 · Cited by 268
Self-citationMore Accurate Question Answering on Freebase
Bast, H., & Haussmann, E. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806472 · Cited by 257
Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions
Heilman, M., & Smith, N. A. (2018) · doi:10.1184/r1/6473795 · Cited by 234
Scaling Semantic Parsers with On-the-Fly Ontology Matching
Kwiatkowski, T., Choi, E., Artzi, Y., & Zettlemoyer, L. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1161 · Cited by 225
Natural Language Questions for the Web of Data
Yahya, M., Berberich, K., Elbassuoni, S., Ramanath, M., Tresp, V., & Weikum, G. (2012) · MPG.PuRe (Max Planck Society) · Cited by 223
Answer Extraction as Sequence Tagging with Tree Edit Distance
Yao, X., Durme, B. V., Callison-Burch, C., & Clark, P. E. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 213
Large-scale Semantic Parsing without Question-Answer Pairs
Reddy, S., Lapata, M., & Steedman, M. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00190 · Cited by 211
Self-citationRelational Inference for Wikification
Cheng, X., & Roth, D. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1184 · Cited by 200
A Dependency-Based Neural Network for Relation Classification
Liu, Y., Wei, F., Li, S., Ji, H., Zhou, M., & WANG, H. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.3115/v1/p15-2047 · Cited by 193
Transforming Dependency Structures to Logical Forms for Semantic Parsing
Reddy, S., Täckström, O., Collins, M., Kwiatkowski, T., Das, D., Steedman, M. J., & Lapata, M. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00088 · Cited by 184
Self-citationOpen information extraction for the web
Etzioni, O., & Banko, M. (2009) · Cited by 152
Weakly Supervised Training of Semantic Parsers
Krishnamurthy, J., & Mitchell, T. M. (2012) · KiltHub Repository · doi:10.1184/r1/6476447 · Cited by 145
Knowledge-Based Question Answering as Machine Translation
Bao, J., Duan, N., Zhou, M., & Zhao, T. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1091 · Cited by 132
S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking
Yang, Y., & Chang, M.-W. (2015) · doi:10.3115/v1/p15-1049 · Cited by 119
Open Domain Question Answering via Semantic Enrichment
Sun, H., Ma, H., Yih, W.-T., Tsai, C.-T., Liu, J., & Chang, M.-W. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741651 · Cited by 114
Answering Natural Language Questions via Phrasal Semantic Parsing
Xu, K., Zhang, S., Feng, Y., & Zhao, D. (2014) · Communications in computer and information science · doi:10.1007/978-3-662-45924-9_30 · Cited by 83
Self-citationLean Question Answering over Freebase from Scratch
Yao, X. (2015) · doi:10.3115/v1/n15-3014 · Cited by 60
Joshi, M., Sawant, U., & Chakrabarti, S. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1117 · Cited by 42
Grounded Semantic Parsing for Complex Knowledge Extraction
Parikh, A. P., Poon, H., & Toutanova, K. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1077 · Cited by 27
Cited by
Hao, Y., Zhang, Y., Liu, K., He, S., Liu, Z., Wu, H., & Zhao, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1021 · Cited by 359
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02232 · Cited by 354
Improved Neural Relation Detection for Knowledge Base Question Answering
Yu, M., Yin, W., Hasan, K. S., dos Santos, C., Xiang, B., & Zhou, B. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1053 · Cited by 314
Answering Natural Language Questions by Subgraph Matching over Knowledge Graphs
Hu, S., Zou, L., Yu, J. X., Wang, H., & Zhao, D. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2766634 · Cited by 271
Self-citationA Survey on Knowledge Graph Embedding: Approaches, Applications and Benchmarks
Dai, Y., Wang, S., Xiong, N. N., & Guo, W. (2020) · Electronics · doi:10.3390/electronics9050750 · Cited by 263
Metadata
- Authors
- Xu, K., Reddy, S., Feng, Y., Huang, S., & Zhao, D.
- Venue
- Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2016
- Pages
- 2326-2336
- OpenAlex
- W2964212344
- Cited by
- 279
- References
- 54
- FWCI
- 21.42
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.