The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Discovering key concepts in verbose queries
Abstract
Current search engines do not, in general, perform well with longer, more verbose queries. One of the main issues in processing these queries is identifying the key concepts that will have the most impact on effectiveness. In this paper, we develop and evaluate a technique that uses query-dependent, corpus-dependent, and corpus-independent features for automatic extraction of key concepts from verbose queries. We show that our method achieves higher accuracy in the identification of key concepts than standard weighting methods such as inverse document frequency. Finally, we propose a probabilistic model for integrating the weighted key concepts identified by our method into a query, and demonstrate that this integration significantly improves retrieval effectiveness for a large set of natural language description queries derived from TREC topics on several newswire and web collections.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Witten, I. H. (2008) · Cited by 9,143
A Language Modeling Approach to Information Retrieval
Ponte, J. M., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130368 · Cited by 2,503
Self-citationA language modeling approach to information retrieval
Ponte, J. M., & Croft, W. B. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291008 · Cited by 1,821 · cited 2×
“where TM (ci) is a set of top M documents 3 retrieved in response to ci from collection C, and p(ci|·) is calculated using a maximum likelihood estimate [25].”
MethodSelf-citationQuary Expansion Using Local and Global Document Analysis
Xu, J., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130364 · Cited by 1,252
Self-citationLDA-based document models for ad-hoc retrieval
Wei, X., & Croft, W. B. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148204 · Cited by 1,075
Self-citationImproved automatic keyword extraction given more linguistic knowledge
Hulth, A. (2003) · Proceedings of the 2003 conference on Empirical methods in natural language processing - · doi:10.3115/1119355.1119383 · Cited by 951
Learning Algorithms for Keyphrase Extraction
Turney, P. D. (2000) · Information Retrieval · doi:10.1023/a:1009976227802 · Cited by 889
A Markov random field model for term dependencies
Metzler, D., & Croft, W. B. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076115 · Cited by 843
Self-citationQuery expansion using local and global document analysis
Xu, J., & Croft, W. B. (1996) · Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '96 · doi:10.1145/243199.243202 · Cited by 831
Self-citationAn Algorithm that Learns What's in a Name
Bikel, D. M., Schwartz, R., & Weischedel, R. M. (1999) · Machine Learning · doi:10.1023/a:1007558221122 · Cited by 776
Domain-specific keyphrase extraction
Frank, E., Paynter, G. W., Witten, I. H., Gutwin, C. A., & Nevill-Manning, C. G. (1999) · Research Commons (University of Waikato) · Cited by 647
Cluster-based retrieval using language models
Liu, X., & Croft, W. B. (2004) · Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1008992.1009026 · Cited by 448
Self-citationStatistics-Based Summarization - Step One: Sentence Compression
Knight, K. K., & Marcu, D. (2000) · Cited by 392
Language Modeling for Information Retrieval
(2003) · Springer eBooks · doi:10.1007/978-94-017-0171-6 · Cited by 371
Finding advertising keywords on web pages
Yih, W.-T., Goodman, J., & Carvalho, V. R. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135813 · Cited by 336
Query performance prediction in web search environments
Zhou, Y., & Croft, W. B. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277835 · Cited by 267
Self-citationLatent concept expansion using markov random fields
Metzler, D., & Croft, W. B. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277796 · Cited by 229
Self-citationMari Ostendorf, E., Collins, M., Narayanan, S. R., & Oard, D. W. (2009) · Cited by 224
Church, K. W., & Gale, W. A. (1995) · Natural Language Engineering · doi:10.1017/s1351324900000139 · Cited by 215
Query expansion using random walk models
Collins-Thompson, K., & Callan, J. (2005) · Proceedings of the 14th ACM international conference on Information and knowledge management · doi:10.1145/1099554.1099727 · Cited by 177
Corpus structure, language models, and ad hoc information retrieval
Kurland, O., & Lee, L. (2004) · Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1008992.1009027 · Cited by 169
Term-specific smoothing for the language modeling approach to information retrieval
Hiemstra, D. (2002) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/564376.564385 · Cited by 85
A study of Poisson query generation model for information retrieval
Mei, Q., Fang, H., & Zhai, C. (2007) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1277741.1277797 · Cited by 39
(1996)
Cited by
Effects of LLM-based Search on Decision Making: Speed, Accuracy, and Overreliance
Spatharioti, S. E., Rothschild, D., Goldstein, D. G., & Hofman, J. M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714082 · Cited by 36
Aji, A., Wang, Y., Agichtein, E., & Gabrilovich, E. (2010) · Proceedings of the 19th ACM international conference on Information and knowledge management · doi:10.1145/1871437.1871519 · Cited by 24
Metadata
- Authors
- Bendersky, M., & Croft, W. B.
- Venue
- Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
- Publisher
- ACM
- Published
- 2008-07-20
- Pages
- 491-498
- OpenAlex
- W2025356973
- Cited by
- 271
- References
- 26
- FWCI
- 18.36
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.