CoChat

Building simulated queries for known-item topics: an analysis using six european languages

Azzopardi, Leif, de Rijke, Maarten, Balog, Krisztian · 2007 · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.

Abstract

There has been increased interest in the use of simulated queries for evaluation and estimation purposes in Information Retrieval. However, there are still many unaddressed issues regarding their usage and impact on evaluation because their quality, in terms of retrieval performance, is unlike real queries. In this paper, wefocus on methods for building simulated known-item topics and explore their quality against real known-item topics. Using existing generation models as our starting point, we explore factors which may influence the generation of the known-item topic. Informed by this detailed analysis (on six European languages) we propose a model with improved document and term selection properties, showing that simulated known-item topics can be generated that are comparable to real known-item topics. This is a significant step towards validating the potential usefulness of simulated queries: for evaluation purposes, and becausebuilding models of querying behavior provides a deeper insight into the querying process so that better retrieval mechanisms can be developed to support the user.

Overview

Integrity signals

Registry

Crossref · 10.1145/1277741.1277820

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 20 references

Research works printed but not linked: 4

  1. Balog K., 2006, Working Notes CLEF 2006
  2. Inoue M., 2006, Proc. SIGIR 2006 Workshop on New Directions in Multilingual Information Access
  3. Sigurbjörnsson B., 2006, Springer Verlag
  4. Tague J., 1981, Proc. 3rd annual ACM conference on Research and development in information retrieval, 236

Cited by

See all 166

Metadata

Authors
Azzopardi, L., de Rijke, M., & Balog, K.
Affiliations
  • University of Glasgow (GB)
  • University of Amsterdam, Amsterdam, Netherlands
  • University of Amsterdam (NL)
Venue
Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Publisher
ACM
Published
2007-07-23
Pages
455-462
OpenAlex
W2159665776
Cited by
135
References
20
Citations by year
2026: 4 · 2025: 3 · 2024: 7 · 2023: 8 · 2022: 7 · 2021: 8 · 2020: 4 · 2019: 6 · 2018: 9 · 2017: 10 · 2016: 10 · 2015: 10 · 2014: 8 · 2013: 6 · 2012: 8
FWCI
4.36
Type
conference paper
Language
en
Open access
closed
Locations
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Mathematics › Theoretical Computer Science › Algorithms and Data Compression
Keywords
query generation, information retrieval evaluation, document selection, term selection, multilingual information retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Building simulated queries for known-item topics: an analysis using six european languages. https://app.cochat.ai/papers/10.1145/1277741.1277820
The paper
Azzopardi, L., de Rijke, M., & Balog, K. (2007). Building simulated queries for known-item topics: an analysis using six european languages. Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. https://doi.org/10.1145/1277741.1277820

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.