The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Analyzing and evaluating query reformulation strategies in web search logs
Abstract
Users frequently modify a previous search query in hope of retrieving better results. These modifications are called query reformulations or query refinements. Existing research has studied how web search engines can propose reformulations, but has given less attention to how people perform query reformulations. In this paper, we aim to better understand how web searchers refine queries and form a theoretical foundation for query reformulation. We study users' reformulation strategies in the context of the AOL query logs. We create a taxonomy of query refinement strategies and build a high precision rule-based classifier to detect each type of reformulation. Effectiveness of reformulations is measured using user click behavior. Most reformulation strategies result in some benefit to the user. Certain strategies like add/remove words, word substitution, acronym expansion, and spelling correction are more likely to cause clicks, especially on higher ranked results. In contrast, users often click the same result as their previous query or select no results when forming acronyms and reordering words. Perhaps the most surprising finding is that some reformulations are better suited to helping users when the current results are already fruitful, while other reformulations are more effective when the results are lacking. Our findings inform the design of applications that can assist searchers; examples are described in this paper.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 14,807
WordNet: An Electronic Lexical Database
Kilgarriff, A., & Fellbaum, C. (2000) · Language · doi:10.2307/417141 · Cited by 11,464
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096
WordNet: An Electronic Lexical Database
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.001.0001 · Cited by 4,857
Optimizing search engines using clickthrough data
Joachims, T. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775067 · Cited by 3,873
Analysis of a very large web search engine query log
Silverstein, C., Marais, H., Henzinger, M., & Moricz, M. (1999) · ACM SIGIR Forum · doi:10.1145/331403.331405 · Cited by 1,153
Improving web search ranking by incorporating user behavior information
Agichtein, E., Brill, E., & Dumais, S. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148177 · Cited by 1,041
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., & Gay, G. (2007) · ACM Transactions on Information Systems · doi:10.1145/1229179.1229181 · Cited by 677
Pass, G., Chowdhury, A., & Torgeson, C. (2006) · Proceedings of the 1st international conference on Scalable information systems - InfoScale '06 · doi:10.1145/1146847.1146848 · Cited by 666
Generating query substitutions
Jones, R., Rey, B., Madani, O., & Greiner, W. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135835 · Cited by 618
Improving automatic query expansion
Mitra, M., Singhal, A., & Buckley, C. (1998) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/290941.290995 · Cited by 557
Query Recommendation Using Query Logs in Search Engines
Baeza-Yates, R., Hurtado, C., & Mendoza, M. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30192-9_58 · Cited by 537
Query chains: learning to rank from implicit feedback
Radlinski, F., & Joachims, T. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081899 · Cited by 476
Learning user interaction models for predicting web search result preferences
Agichtein, E., Brill, E., Dumais, S., & Ragno, R. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148175 · Cited by 462
Evaluating implicit measures to improve web search
Fox, S., Karnawat, K., Mydland, M., Dumais, S., & White, T. (2005) · ACM Transactions on Information Systems · doi:10.1145/1059981.1059982 · Cited by 412
Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs
Jones, R., & Klinkner, K. L. (2008) · Proceedings of the 17th ACM conference on Information and knowledge management · doi:10.1145/1458082.1458176 · Cited by 385
Information re-retrieval: repeat queries in Yahoo's logs
Teevan, J., Adar, E., Jones, R., & Potts, M. A. S. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277770 · Cited by 293
Patterns of Search: Analyzing and Modeling Web Query Refinement
Lau, T., & Horvitz, E. (1999) · CISM International Centre for Mechanical Sciences · doi:10.1007/978-3-7091-2490-1_12 · Cited by 253
Using terminological feedback for web search refinement: a log-based study
Anick, P. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860453 · Cited by 249
Analysis of multiple query reformulations on the web: The interactive information retrieval context
Rieh, S. Y., & Xie, H. (. (2006) · Information Processing & Management · doi:10.1016/j.ipm.2005.05.005 · Cited by 219
Combining evidence for automatic Web session identification
He, D., Göker, A., & Harper, D. J. (2002) · Information Processing & Management · doi:10.1016/s0306-4573(01)00060-7 · Cited by 192
The Intention Behind Web Queries
Baeza-Yates, R., Calderón-Benavides, L., & González-Caro, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11880561_9 · Cited by 183
A temporal comparison of AltaVista Web searching
Jansen, B. J., Spink, A., & Pedersen, J. (2005) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20145 · Cited by 176
Mining anchor text for query refinement
Kraft, R., & Zien, J. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988763 · Cited by 169
Defining a session on Web search engines
Jansen, B. J., Spink, A., Blakely, C., & Koshman, S. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20564 · Cited by 166
A unified and discriminative model for query refinement
Guo, J., Xu, G., Li, H., & Cheng, X. (2008) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1390334.1390400 · Cited by 115
Query logs alone are not enough
Grimes, C., Tang, D., & Russell, D. M. (2007) · Cited by 96
Characterizing Web user sessions
Arlitt, M. F. (2000) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/362883.362920 · Cited by 94
Are click-through data adequate for learning web search rankings?
Dou, Z., Song, R., Yuan, X., & Wen, J.-R. (2008) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1458082.1458095 · Cited by 82
The re:search engine: simultaneous support for finding and re-finding
Teevan, J. B. (2007) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1294211.1294217 · Cited by 52
Patterns and transitions of query reformulation during web searching
Jansen, B. J. J., Zhang, M., & Spink, A. H. (2007) · International Journal of Web Information Systems · doi:10.1108/17440080710848116 · Cited by 37
Identification of user sessions with hierarchical agglomerative clustering
Murray, G. C., Lin, J., & Chowdhury, A. (2006) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504301312 · Cited by 32
Cited by
A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion
Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., & Nie, J.-Y. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806493 · Cited by 472
Query expansion techniques for information retrieval: a survey
Azad, H. K., & Deepak, A. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.05.009 · Cited by 313
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225
“As an upstream technique, query reformulation is designed to refine user queries so that they are more effective at retrieving relevant documents [10, 11].”
Task-Oriented Query Reformulation with Reinforcement Learning
Nogueira, R., & Cho, K. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1061 · Cited by 120
“For instance, Huang and Efthimiadis (2009) estimate that 28-52% of all the web queries are modifications of previous ones.”
BackgroundThe Effects of Generative AI on Computing Students’ Help-Seeking Preferences
Hou, I., Mettille, S., Man, O., Li, Z., Zastudil, C., & MacNeil, S. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636248 · Cited by 97
Metadata
- Authors
- Huang, J., & Efthimiadis, E. N.
- Venue
- Proceedings of the 18th ACM conference on Information and knowledge management
- Publisher
- ACM
- Published
- 2009-11-02
- Pages
- 77-86
- OpenAlex
- W2163987313
- Cited by
- 277
- References
- 32
- FWCI
- 14.94
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.