The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Evaluating Stochastic Rankings with Expected Exposure
Open accessAbstract
We introduce the concept of expected exposure as the average attention ranked items receive from users over repeated samples of the same query. Furthermore, we advocate for the adoption of the principle of equal expected exposure: given a fixed information need, no item should receive more or less expected exposure than any other item of the same relevance grade. We argue that this principle is desirable for many retrieval objectives and scenarios, including topical diversity and fair ranking. %Leveraging user models from existing retrieval metrics, we propose a general evaluation methodology based on expected exposure and draw connections to related metrics in information retrieval evaluation. Importantly, this methodology relaxes classic information retrieval assumptions, allowing a system, in response to a query, to produce a distribution over rankings instead of a single fixed ranking. We study the behavior of the expected exposure metric and stochastic rankers across a variety of information access conditions, including ad hoc retrieval and recommendation. %We believe that measuring and optimizing expected exposure metrics using randomization opens a new area for retrieval algorithm development and progress.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Plackett, R. L. (1975) · Applied Statistics · doi:10.2307/2346567 · Cited by 731
Unbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531
Equity of Attention: Amortizing Individual Fairness in Rankings
Biega, A. J., Gummadi, K. P., & Weikum, G. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210063 · Cited by 337
Self-citationQuantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92
Cited by
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
A Survey on the Fairness of Recommender Systems
Wang, Y., Ma, W., Zhang, M., Liu, Y., & Ma, S. (2023) · ACM Transactions on Information Systems · doi:10.1145/3547333 · Cited by 356
Evaluating Recommender Systems: Survey and Framework
Zangerle, E., & Bauer, C. (2023) · ACM Computing Surveys · doi:10.1145/3556536 · Cited by 283
Provider fairness across continents in collaborative recommender systems
Gómez, E., Boratto, L., & Salamó, M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2021.102719 · Cited by 39
FAIR : Fairness‐aware information retrieval evaluation
Gao, R., Ge, Y., & Shah, C. (2022) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24648 · Cited by 19
Metadata
- Authors
- Diaz, F., Mitra, B., Ekstrand, M. D., Biega, A. J., & Carterette, B.
- Venue
- Proceedings of the 29th ACM International Conference on Information & Knowledge Management
- Publisher
- ACM
- Published
- 2020-10-19
- Pages
- 275-284
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W3023202929
- Cited by
- 171
- References
- 6
- FWCI
- 18.26
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating Stochastic Rankings with Expected Exposure. https://app.cochat.ai/papers/10.1145/3340531.3411962
- The paper
- Diaz, F., Mitra, B., Ekstrand, M. D., Biega, A. J., & Carterette, B. (2020). Evaluating Stochastic Rankings with Expected Exposure. Proceedings of the 29th ACM International Conference on Information & Knowledge Management. https://doi.org/10.1145/3340531.3411962
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.