The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Introduction to “This is Watson”
Abstract
In 2007, IBM Research took on the grand challenge of building a computer system that could compete with champions at the game of Jeopardy!™. In 2011, the open-domain question-answering (QA) system, dubbed Watson, beat the two highest ranked players in a nationally televised two-game Jeopardy! match. This paper provides a brief history of the events and ideas that positioned our team to take on the Jeopardy! challenge, build Watson, IBM Watson™, and ultimately triumph. It describes both the nature of the QA challenge represented by Jeopardy! and our overarching technical approach. The main body of this paper provides a narrative of the DeepQA processing pipeline to introduce the articles in this special issue and put them in context of the overall system. Finally, this paper summarizes our main results, describing how the system, as a holistic combination of many diverse algorithmic techniques, performed at champion levels, and it briefly discusses the team's future research plans.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
WordNet: An Electronic Lexical Database
Kilgarriff, A., & Fellbaum, C. (2000) · Language · doi:10.2307/417141 · Cited by 11,464
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
WordNet: An Electronic Lexical Database
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.001.0001 · Cited by 4,857
Freebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852
Jurafsky, D. S., & Martin, J. (2000) · Cited by 4,132
Yago: a core of semantic knowledge
Suchanek, F. M., Kasneci, G., & Weikum, G. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242667 · Cited by 2,512
DBpedia - A crystallization point for the Web of Data
Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., & Hellmann, S. (2009) · Journal of Web Semantics · doi:10.1016/j.websem.2009.07.002 · Cited by 2,137
Building Watson: An Overview of the DeepQA Project
Ferrucci, D., Brown, E., Chu‐Carroll, J., Fan, J., Gondek, D., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J., et al. (2010) · AI Magazine · doi:10.1609/aimag.v31i3.2303 · Cited by 1,497
Self-citationRobust Disambiguation of Named Entities in Text
Hoffart, J., Yosef, M. A., Bordino, I., Fürstenau, H., Pinkal, M., Spaniol, M., Taneva, B., Thater, S., & Weikum, G. (2011) · Cited by 925
Fish Oil, Raynaud's Syndrome, and Undiscovered Public Knowledge
Swanson, D. R. (1986) · Perspectives in Biology and Medicine · doi:10.1353/pbm.1986.0087 · Cited by 866
Ferrucci, D., Levas, A., Bagchi, S., Gondek, D., & Mueller, E. T. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2012.06.009 · Cited by 351
Self-citationA composite kernel to extract relations between entities with both flat and structured features
Zhang, M., Zhang, J., Su, J., & Zhou, G. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220279 · Cited by 242
A Statistical Approach to Anaphora Resolution
Ge, N., Hale, J., & Eugene Charniak, D. (1998) · Cited by 221
Question-answering by predictive annotation
Prager, J., Brown, E., Coden, A., & Radev, D. (2000) · Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/345508.345574 · Cited by 209
Question analysis: How Watson reads a clue
Lally, A., Prager, J. M., McCord, M. C., Boguraev, B. K., Patwardhan, S., Fan, J., Fodor, P., & Chu-Carroll, J. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2184637 · Cited by 166
New Directions In Question Answering
Maybury, M. (2008) · Text, Speech and Language Technology · doi:10.1007/978-1-4020-4746-6_18 · Cited by 161
Natural language question-answering systems: 1969
Simmons, R. F. (1970) · Communications of the ACM · doi:10.1145/361953.361963 · Cited by 148
Automatic knowledge extraction from documents
Fan, J., Kalyanpur, A. A., Gondek, D. C., & Ferrucci, D. A. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2186519 · Cited by 135
Self-citationThe Internet Movie Database (IMDb)
(1999) · Electronic Resources Review · doi:10.1108/err.1999.3.5.56.52 · Cited by 133
Natural language question answering systems
Bolc, L. (1980) · Medical Entomology and Zoology · Cited by 103
McCord, M. C., Murdock, J. W., & Boguraev, B. K. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2185409 · Cited by 101
A framework for merging and ranking of answers in DeepQA
Gondek, D. C., Lally, A., Kalyanpur, A. A., Murdock, J. W., Duboue, P. A., Zhang, L., Pan, Y., Qiu, Z. M., & Welty, C. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2188760 · Cited by 99
Open-Domain Question–Answering
Prager, J. M. (2006) · now publishers, Inc. eBooks · doi:10.1561/9781601980533 · Cited by 86
Structured data and inference in DeepQA
Kalyanpur, A. A., Boguraev, B. K., Patwardhan, S., Murdock, J. W., Lally, A., Welty, C., Prager, J. M., Coppola, B., Fokoue, A., Zhang, L., et al. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2188737 · Cited by 67
Relation extraction and scoring in DeepQA
Wang, C., Kalyanpur, A. A., Fan, J., Boguraev, B. K., & Gondek, D. C. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2187239 · Cited by 64
Typing candidate answers using type coercion
Murdock, J. W., Kalyanpur, A. A., Welty, C., Fan, J., Ferrucci, D. A., Gondek, D. C., Zhang, L., & Kanayama, H. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2187036 · Cited by 47
Self-citationFact-based question decomposition in DeepQA
Kalyanpur, A. A., Patwardhan, S., Boguraev, B. K., Lally, A., & Chu‐Carroll, J. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2188934 · Cited by 40
Simulation, learning, and optimization techniques in Watson's game strategies
Tesauro, G., Gondek, D. C., Lenchner, J., et al. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2188931 · Cited by 17
Cited by
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
DaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492
DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., & Bowling, M. (2017) · Science · doi:10.1126/science.aam6960 · Cited by 786
A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering
Wang, D., & Nyberg, E. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.3115/v1/p15-2116 · Cited by 353
Blockchain-Powered Parallel Healthcare Systems Based on the ACP Approach
Wang, S., Wang, J., Wang, X., Qiu, T., Yuan, Y., Ouyang, L., Guo, Y., & Wang, F.-Y. (2018) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2018.2865526 · Cited by 316
Metadata
- Authors
- Ferrucci, D. A.
- Venue
- IBM Journal of Research and Development
- Publisher
- IBM
- Published
- 2012-05
- Volume
- 56
- Issue
- 3.4
- Pages
- 1:1-1:15
- ISSN
- 0018-8646 (ISSN-L)
- OpenAlex
- W160318044
- Cited by
- 484
- References
- 29
- FWCI
- 32.93
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.