A survey on question answering systems with classification
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Question answering systems (QASs) generate answers of questions asked in natural languages. Early QASs were developed for restricted domains and have limited capabilities. Current QASs focus on types of questions generally asked by users, characteristics of data sources consulted, and forms of correct answers generated. Research in the area of QASs began in 1960s and since then, a large number of QASs have been developed. To identify the future scope of research in this area, the need of a comprehensive survey on QASs arises naturally. This paper surveys QASs and classifies them based on different criteria. We identify the current status of the research in the each category of QASs, and suggest future scope of the research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 1×
“Different types of QASs use different techniques such as snippet tolerant property, keyword matching, and rules for making matching of the answers through WordNet (Miller, 1995; Carbonell et al., 2000).”
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Prud’hommeaux, E. (2008) · Medical Entomology and Zoology · Cited by 3,097
Li, X., & Roth, D. (2002) · Proceedings of the 19th international conference on Computational linguistics - · doi:10.3115/1072228.1072378 · Cited by 1,234 · cited 5×
“Li and Roth (2002) classify questions into a fine grained content based categorization but they deal with a very limited class of real world questions.”
InfluentialJumping NLP Curves: A Review of Natural Language Processing Research [Review Article]
Cambria, E., & White, B. (2014) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2014.2307227 · Cited by 1,102
Learning surface text patterns for a Question Answering system
Ravichandran, D., & Hovy, E. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073092 · Cited by 809 · cited 2×
“These patterns could be lexico-syntactic or lexico-semantic patterns (Ravichandran and Hovy, 2002; Zheng, 2002).”
Natural language interfaces to databases – an introduction
Androutsopoulos, I., Ritchie, G. D., & Thanisch, P. (1995) · Natural Language Engineering · doi:10.1017/s135132490000005x · Cited by 768 · cited 3×
“One such system, MASQUE (Androutsopoulos et al., 1993) represents natural language questions in a logic representation, and then it translates the logic query into a database query for retrieving intended information from database (Androutsopoulos et al., 1995; Lopeza and Uren, 2011).”
Template-based question answering over RDF data
Unger, C., Bühmann, L., Lehmann, J., Ngonga Ngomo, A.-C., Gerber, D., & Cimiano, P. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187923 · Cited by 498 · cited 1×
“Unger et al. (2012) use a template based pattern matching approach on Resource Description Framework (RDF) data by using SPARQL (Prudhommeaux and Seaborne, 2007).”
Jumping NLP Curves: A Review of Natural Language Processing Research
Cambria, E., & White, B. (2014) · Cited by 455
Baseball: an automatic question-answerer
Green, B. F., Wolf, A. K., Chomsky, C., & Laughery, K. (1961) · Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference on - IRE-AIEE-ACM '61 (Western) · doi:10.1145/1460690.1460714 · Cited by 383 · cited 1×
“Green et al. (1961) propose BASEBALL, a QAS that provides information associated with a baseball league played in America during a particular season.”
Conceptual Graphs for a Data Base Interface
Sowa, J. F. (1976) · IBM Journal of Research and Development · doi:10.1147/rd.204.0336 · Cited by 367 · cited 1×
“These conceptual graph based models represent the sentence of the text with a structure formed by vertices and edges in graph (Sowa, 1976).”
Progress in natural language understanding
Woods, W. A. (1973) · AFIPS National Computer Conference (NCC) · doi:10.1145/1499586.1499695 · Cited by 351 · cited 1×
“This system provides answers to questions related to dates, location, etc. Woods (1973) propose LUNAR, a QAS that provides information about soil samples taken from Apollo lunar exploration.”
Sentic patterns: Dependency-based rules for concept-level sentiment analysis
Poria, S., Cambria, E., Winterstein, G., & Huang, G.-B. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.05.005 · Cited by 345 · cited 1×
“One of the prominent opinion mining systems, SenticNet detects sentiment polarity of a single sentence by using machine-learning and knowledge-based techniques (Poria et al., 2012, 2013, 2014).”
Question Answering from Frequently Asked Question Files: Experiences with the FAQ Finder System
Burke, R., Hammond, K. J., Kulyukin, V., Lytinen, S. L., Tomuro, N., & Schoenberg, S. (1997) · doi:10.1609/aimag.v18i2.1294 · Cited by 333 · cited 1×
“FAQFinder (Burke et al., 1997) does matching of the questions with the question list compiled in a knowledge base through statistical similarity and semantic similarity.”
A PDTB-styled end-to-end discourse parser
LIN, Z., NG, H. T., & KAN, M.-Y. (2014) · Natural Language Engineering · doi:10.1017/s1351324912000307 · Cited by 286 · cited 1×
“Hence, the technology is still far from ideal discourse analysis of text (Ziheng et al., 2014).”
The TREC question answering track
VOORHEES, E. M. (2001) · Natural Language Engineering · doi:10.1017/s1351324901002789 · Cited by 285 · cited 1×
“The research in open domain question answering from unstructured data sources was instantiated by the TREC Evaluation campaign which is taking place regularly every year since 1999 (Voorhees, 2001, 2004; Voorhees and Weishedel, 2000).”
A survey on question answering technology from an information retrieval perspective
Kolomiyets, O., & Moens, M.-F. (2011) · Information Sciences · doi:10.1016/j.ins.2011.07.047 · Cited by 252 · cited 6×
“The pros of factoid type questions asked in QASs are as follows: The expected answer types for most factoid type questions are generally named entities which could be traced in documents through named entity tagging softwares (Kolomiyets, 2011; Vanessa, 2011).”
InfluentialScaling question answering to the web
Kwok, C., Etzioni, O., & Weld, D. S. (2001) · ACM Transactions on Information Systems · doi:10.1145/502115.502117 · Cited by 230
Performance issues and error analysis in an open-domain question answering system
Moldovan, D., Paşca, M., Harabagiu, S., & Surdeanu, M. (2003) · ACM Transactions on Information Systems · doi:10.1145/763693.763694 · Cited by 229 · cited 4×
“The task of generating answers to the users’ questions is directly related to type of questions asked (Moldovan et al., 2003).”
InfluentialThe Handbook of Computational Linguistics and Natural Language Processing
(2010) · Wiley eBooks · doi:10.1002/9781444324044 · Cited by 228 · cited 1×
“Current QASs operate at lexical and sentence level for deducing the meaning of questions (Kolomiyets, 2011; Alexander Clark, 2010; Saeedeh Momtazi, 2011; Suresh kumar and Zayaraz, 2014).”
Progress in natural language understanding-An application to lunar geology
Woods, W. A. (1973) · Managing Requirements Knowledge, International Workshop on · doi:10.1109/afips.1973.115 · Cited by 214
Question Answering in Restricted Domains: An Overview
Mollá, D., & Vicedo, J. L. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.1.41 · Cited by 197 · cited 2×
“Restricted domain QASs answer domain specific questions (Molla and Vicedo, 2007).”
The structure and performance of an open-domain question answering system
Moldovan, D., Harabagiu, S., Pasca, M., Mihalcea, R., Girju, R., Goodrum, R., & Rus, V. (2000) · Proceedings of the 38th Annual Meeting on Association for Computational Linguistics - ACL '00 · doi:10.3115/1075218.1075289 · Cited by 182 · cited 2×
“QASs require advance natural language processing techniques to analyze the text at pragmatic and discourse level for generating answers (Higashinaka and Isozaki, 2008; Verberne et al., 2007, 2008, 2010; Moldovan et al., 2000).”
Automatic labeling of semantic roles
Gildea, D., & Jurafsky, D. S. (2000) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075218.1075283 · Cited by 176 · cited 2×
“The Semantic role labeling [shallow semantic parsing] aims at identification and labeling of arguments in the text (Daniel and Daniel, 2000).”
A rule-based question answering system for reading comprehension tests
Riloff, E., & Thelen, M. (2000) · ANLP/NAACL 2000 Workshop on Reading comprehension tests as evaluation for computer-based language understanding sytems - · doi:10.3115/1117595.1117598 · Cited by 168 · cited 1×
“Another QAS, QUARC developed by Riloff and Thelen (2000) classifies questions into different wh-types and derives their expected answer types through the use of lexical and semantic clues.”
Mining opinion components from unstructured reviews: A review
Khan, K., Baharudin, B., Khan, A., & Ullah, A. (2014) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2014.03.009 · Cited by 153 · cited 5×
“Opinion detection-to classify text as subjective or objective is still a research problem (Khan, 2014).”
InfluentialIs Question Answering fit for the Semantic Web?: A survey
López, V., Uren, V. S., Sabou, M., & Motta, E. (2011) · Semantic Web · doi:10.3233/sw-2011-0041 · Cited by 149 · cited 6×
“The pros of factoid type questions asked in QASs are as follows: The expected answer types for most factoid type questions are generally named entities which could be traced in documents through named entity tagging softwares (Kolomiyets, 2011; Vanessa, 2011).”
InfluentialOmnibase: Uniform Access to Heterogeneous Data for Question Answering
Katz, B., Felshin, S., Yüret, D., Ibrahim, A., Lin, J., Marton, G. A., McFarland, A. J., & Temelkuran, B. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36271-1_23 · Cited by 128 · cited 5×
“source in QAS Naluri Wong (2004), Start Katz et al. (2002), Webcoop Benamara (2004)”
InfluentialAutomatic question answering using the web: Beyond the Factoid
Soricut, R., & Brill, E. (2006) · Information Retrieval · doi:10.1007/s10791-006-7149-y · Cited by 115 · cited 1×
“Such a large collection can be utilized as a knowledge base for answering users’ questions (Soricut and Brill, 2006).”
Merging SenticNet and WordNet-Affect emotion lists for sentiment analysis
Poria, S., Gelbukh, A. F., Cambria, E., Yang, P., Hussain, A., & Durrani, T. S. (2012) · International Conference on Signal Processing · doi:10.1109/icosp.2012.6491803 · Cited by 115 · cited 1×
“One of the prominent opinion mining systems, SenticNet detects sentiment polarity of a single sentence by using machine-learning and knowledge-based techniques (Poria et al., 2012, 2013, 2014).”
Research and Reviews in Question Answering System
Dwivedi, S. K., & Singh, V. (2013) · Procedia Technology · doi:10.1016/j.protcy.2013.12.378 · Cited by 105 · cited 4×
“Based on the literature review (Manning and Schu¨tze, 1999; Frederik, 2010; Dwivedi, 2013), there are broadly three approaches for making analysis of natural language questions and source documents.”
InfluentialSoft pattern matching models for definitional question answering
Cui, H., Kan, M., & Chua, T. (2007) · ACM Transactions on Information Systems · doi:10.1145/1229179.1229182 · Cited by 82 · cited 1×
“Descriptive type questions: the questions which require finding the definition or description of the term [event or entity] in the question (Cui et al., 2007; Vanessa, 2011).”
MASQUE/SQL: an efficient and portable natural language query interface for relational databases
Androutsopoulos, I., Ritchie, G., & Thanisch, P. (1993) · Cited by 77
Corpus-based Question Answering for why-Questions
Higashinaka, R., & Isozaki, H. (2008) · Cited by 65
Is question answering an acquired skill?
Ramakrishnan, G., Chakrabarti, S., Paranjpe, D., & Bhattacharya, P. (2004) · International World Wide Web Conference (WWW) · doi:10.1145/988672.988688 · Cited by 54 · cited 1×
“Chakrabarti et al. develop a QAS (Chakrabarti et al., 2004) which makes use of a linguistic and pattern based approach for answer finding.”
What Is Not in the Bag of Words for Why-QA?
Verberne, S., Boves, L., Oostdijk, N. H. J., & Coppen, P. A. J. M. (2010) · Computational Linguistics · doi:10.1162/coli.09-032-r1-08-034 · Cited by 50 · cited 4×
“QASs require advance natural language processing techniques to analyze the text at pragmatic and discourse level for generating answers (Higashinaka and Isozaki, 2008; Verberne et al., 2007, 2008, 2010; Moldovan et al., 2000).”
InfluentialFunction-Based Question Classification for General QA
Bu, F., Zhu, X., Hao, Y., & Zhu, X. (2010) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 46
kumar, G. S., & Zayaraz, G. (2014) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2014.03.001 · Cited by 36 · cited 4×
“Current QASs have got a satisfactory performance in answering factoid type questions (Kolomiyets, 2011; Vanessa, 2011; Indurkhya and Damereau, 2010; Dwivedi, 2013; Suresh kumar and Zayaraz, 2014).”
InfluentialFuzzy Clustering for Semi-supervised Learning – Case Study: Construction of an Emotion Lexicon
Poria, S., Gelbukh, A. F., Das, D., & Bandyopadhyay, S. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37807-2_7 · Cited by 35 · cited 1×
“One of the prominent opinion mining systems, SenticNet detects sentiment polarity of a single sentence by using machine-learning and knowledge-based techniques (Poria et al., 2012, 2013, 2014).”
Using syntactic information for improvingwhy-question answering
Verberne, S., Boves, L., Oostdijk, N. H. J., & Coppen, P. A. J. M. (2008) · International Conference on Computational Linguistics · doi:10.3115/1599081.1599201 · Cited by 32 · cited 2×
“QASs require advance natural language processing techniques to analyze the text at pragmatic and discourse level for generating answers (Higashinaka and Isozaki, 2008; Verberne et al., 2007, 2008, 2010; Moldovan et al., 2000).”
Singh, M., & Jain, S. (2011) · Communications in computer and information science · doi:10.1007/978-3-642-22540-6_59 · Cited by 27 · cited 2×
“Data space systems could get benefit out of heterogeneity by providing pay-as-you-go integration of data sources which are populated as a set of participants (Singh and Jain, 2011).”
Self-citationMishra, A. K., & Jain, S. K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-18117-2_19 · Cited by 7 · cited 1×
“In this type of analysis, systems identify the discourse structure of the connected text i.e. types of discourse relationship existing in between sentences (elaboration, explanation, contrast)in text (Verberne et al., 2010; Mishra and Jain, 2014, 2015).”
Self-citationIs question answering fit for the semantic web
LopezVanessa, UrenVictoria, SabouMarta, & MottaEnrico (2011) · Semantic Web · doi:10.5555/2336450.2336455 · Cited by 5
Cited by
Survey on evaluation methods for dialogue systems
Deriu, J., Rodrigo, A., Otegi, A., Echegoyen, G., Rosset, S., Agirre, E., & Cieliebak, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09866-x · Cited by 215
Metadata
- Authors
- Mishra, A., & Jain, S. K.
- Venue
- Journal of King Saud University - Computer and Information Sciences
- Published
- 2016-07
- Volume
- 28
- Issue
- 3
- Pages
- 345-361
- ISSN
- 1319-1578 (ISSN-L)
- OpenAlex
- W2183366530
- Cited by
- 210
- References
- 43
- FWCI
- 10.32
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.