The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
An information-theoretic approach to automatic query expansion
Abstract
Techniques for automatic query expansion from top retrieved documents have shown promise for improving retrieval effectiveness on large collections; however, they often rely on an empirical ground, and there is a shortage of cross-system comparisons. Using ideas from Information Theory, we present a computationally simple and theoretically justified method for assigning scores to candidate expansion terms. Such scores are used to select and weight expansion terms within Rocchio's framework for query reweigthing. We compare ranking with information-theoretic query expansion versus ranking with other query expansion techniques, showing that the former achieves better retrieval effectiveness on several performance measures. We also discuss the effect on retrieval effectiveness of the main parameters involved in automatic query expansion, such as data sparseness, query difficulty, number of selected documents, and number of selected terms, pointing out interesting relationships.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Relevance feedback in information retrieval
Rocchio, J. J. (1971) · Medical Entomology and Zoology · Cited by 2,680
A Language Modeling Approach to Information Retrieval
Ponte, J. M., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130368 · Cited by 2,503
Robertson, S. E., Walker, S., Jones, S., Hancock‐Beaulieu, M., & Gatford, M. (1994) · Text REtrieval Conference · Cited by 2,212
Relevance weighting of search terms
Robertson, S. E., & Jones, K. S. (1976) · Journal of the American Society for Information Science · doi:10.1002/asi.4630270302 · Cited by 2,081
A language modeling approach to information retrieval
Ponte, J. M., & Croft, W. B. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291008 · Cited by 1,821
Estimation of probabilities from sparse data for the language model component of a speech recognizer
Katz, S. (1987) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1987.1165125 · Cited by 1,637
The vocabulary problem in human-system communication
Furnas, G. W., Landauer, T. K., Gomez, L. M., & Dumais, S. T. (1987) · Communications of the ACM · doi:10.1145/32206.32212 · Cited by 1,504
Improving retrieval performance by relevance feedback
Salton, G., & Buckley, C. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199006)41:4<288::aid-asi8>3.0.co;2-h · Cited by 1,495
Quary Expansion Using Local and Global Document Analysis
Xu, J., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130364 · Cited by 1,252
Query Expansion using Lexical-Semantic Relations
Voorhees, E. M. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_7 · Cited by 947
Probabilistic models of information retrieval based on measuring the divergence from randomness
Amati, G., & Van Rijsbergen, C. J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582416 · Cited by 883
Reexamining the cluster hypothesis: scatter/gather on retrieval results
Hearst, M. A., & Pedersen, J. O. (1996) · Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '96 · doi:10.1145/243199.243216 · Cited by 822
Explorations in Automatic Thesaurus Discovery
Grefenstette, G. (1994) · Springer eBooks · doi:10.1007/978-1-4615-2710-7 · Cited by 749
Elements of Information Theory, Wiley Series in Telecommunications
Cover, T. M., & Allen, T. (1991) · Cited by 623
Improving automatic query expansion
Mitra, M., Singhal, A., & Buckley, C. (1998) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/290941.290995 · Cited by 557
Improving the effectiveness of information retrieval with local context analysis
Xu, J., & Croft, W. B. (2000) · ACM Transactions on Information Systems · doi:10.1145/333135.333138 · Cited by 533
A THEORETICAL BASIS FOR THE USE OF CO‐OCCURRENCE DATA IN INFORMATION RETRIEVAL
van Rijsbergen, C. J. (1977) · Journal of Documentation · doi:10.1108/eb026637 · Cited by 474
Automatic Query expansion using SMART : TREC 3
Buckley, C., Salton, G., Allan, J., & Singhal, A. (1994) · Cited by 464
USING PROBABILISTIC MODELS OF DOCUMENT RETRIEVAL WITHOUT RELEVANCE INFORMATION
CROFT, W. B., & HARPER, D. J. (1979) · Journal of Documentation · doi:10.1108/eb026683 · Cited by 433
Combining classifiers in text categorization
Larkey, L. S., & Croft, W. B. (1996) · Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '96 · doi:10.1145/243199.243276 · Cited by 410
Similarity-Based Models of Word Cooccurrence Probabilities
Dagan, I., Lee, L., & Pereira, F. C. N. (1999) · Machine Learning · doi:10.1023/a:1007537716579 · Cited by 375
Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive.
Robertson, S. E., Walker, S., & Hancock‐Beaulieu, M. (1998) · Text REtrieval Conference · Cited by 341
Overview of the Sixth Text REtrieval Conference (TREC-6)
Voorhees, E. M., & Harman, D. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(99)00043-6 · Cited by 317
ON TERM SELECTION FOR QUERY EXPANSION
ROBERTSON, S. E. (1990) · Journal of Documentation · doi:10.1108/eb026866 · Cited by 312
Boosting and Rocchio applied to text filtering
Schapire, R. E., Singer, Y., & Singhal, A. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.290996 · Cited by 301
Cluster-based language models for distributed retrieval
Xu, J., & Croft, W. B. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312687 · Cited by 293
Information retrieval: the role of equipment and an assessment of the technology
Berul, L. H. (2024) · doi:10.17918/00005341 · Cited by 289
Local Feedback in Full-Text Retrieval Systems
Attar, R., & Fraenkel, A. S. (1977) · Journal of the ACM · doi:10.1145/322017.322021 · Cited by 230
The seventh text REtrieval conference (TREC-7)
(1999) · doi:10.6028/nist.sp.500-242 · Cited by 189
Overview of the Seventh Text REtrieval Conference
Voorhees, E. M. (1998) · Text REtrieval Conference · Cited by 187
Models for retrieval with probabilistic indexing
Fuhr, N. (1989) · Information Processing & Management · doi:10.1016/0306-4573(89)90091-5 · Cited by 177
On modeling of information retrieval concepts in vector spaces
Wong, S. K. M., Ziarko, W., Raghavan, V. V., & Wong, P. C. N. (1987) · ACM Transactions on Database Systems · doi:10.1145/22952.22957 · Cited by 172
Relevance feedback and other query modification techniques
Harman, D. K. (1992) · Information Retrieval · Cited by 155
AN EVALUATION OF FEEDBACK IN DOCUMENT RETRIEVAL USING CO‐OCCURRENCE DATA
Harper, D. J., & van Rijsbergen, C. J. (1978) · Journal of Documentation · doi:10.1108/eb026659 · Cited by 154
Automatic feedback using past queries
Fitzpatrick, L., & Dent, M. (1997) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/258525.258597 · Cited by 148
Spoken Dialogues with Computers
Mori, R. D. (1998) · Cited by 136
The selection of good search terms
van Rijsbergen, C. J., Harper, D. J., & Porter, M. (1981) · Information Processing & Management · doi:10.1016/0306-4573(81)90029-7 · Cited by 123
The sixth text Retrieval conference (TREC-6)
(1998) · doi:10.6028/nist.sp.500-240 · Cited by 113
Srinivasan, P. (1996) · Information Processing & Management · doi:10.1016/0306-4573(95)00076-3 · Cited by 112
Evaluating user interfaces to information retrieval systems: a case study on user support
Brajnik, G., Mizzaro, S., & Tasso, C. (1996) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/243199.243249 · Cited by 98
A freely available wide coverage morphological analyzer for English
Karp, D., Schabes, Y., Zaidel, M., & Egedi, D. (1992) · International Conference on Computational Linguistics · doi:10.3115/992383.992409 · Cited by 92
A user-centred evaluation of ranking algorithms for interactive query expansion
Efthimiadis, E. N. (1993) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/160688.160710 · Cited by 74
AID, an Associative Interactive Dictionary for online searching
Doszkocs, T. E. (1978) · Online Review · doi:10.1108/eb023972 · Cited by 42
Modelling documents with multiple poisson distributions
Margulis, E. L. (1993) · Information Processing & Management · doi:10.1016/0306-4573(93)90005-x · Cited by 20
Yang, K., Maglaughlin, K. L., Meho, L. I., & Sumner, R. G. (1998) · Text REtrieval Conference · Cited by 10
Combined models for topic spotting and topic-dependent language modeling
Bigi, B., Mori, R. D., El-Bèze, M., & Spriet, T. (2002) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.1997.659133 · Cited by 4
Self-citation
Cited by
A Survey of Automatic Query Expansion in Information Retrieval
Carpineto, C., & Romano, G. (2012) · ACM Computing Surveys · doi:10.1145/2071389.2071390 · Cited by 1,109
Self-citationNovelty and diversity in information retrieval evaluation
Clarke, C. L. A., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., & MacKinnon, I. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390446 · Cited by 962
Cronen-Townsend, S., Zhou, Y., & Croft, W. B. (2002) · Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/564376.564429 · Cited by 663
Query expansion techniques for information retrieval: a survey
Azad, H. K., & Deepak, A. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.05.009 · Cited by 313
Query expansion by mining user logs
Cui, H., Wen, J.-R., Nie, J.-Y., & Ma, W.-Y. (2003) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2003.1209002 · Cited by 298
Metadata
- Authors
- Carpineto, C., de Mori, R., Romano, G., & Bigi, B.
- Venue
- ACM Transactions on Information Systems
- Published
- 2001-01
- Volume
- 19
- Issue
- 1
- Pages
- 1-27
- ISSN
- 1046-8188 (ISSN-L)
- 1558-2868
- OpenAlex
- W2034117506
- Cited by
- 365
- References
- 50
- FWCI
- 39.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.