The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Estimating Position Bias without Intrusive Interventions
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To counteract this effect, we computed propensity scores using a swap experiment (Agarwal et al., 2019).”
From SPECTER: Document-level Representation Learning using Citation-informed Transformers (2020) · cites this paper as Agarwal et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Accurately interpreting clickthrough data as implicit feedback
Joachims, T., Granka, L., Pan, B., Hembrooke, H., & Gay, G. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076063 · Cited by 1,408
Self-citationPredicting clicks: estimating the click-through rate for new ads
Richardson, M., Dominowska, E., & Ragno, R. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242643 · Cited by 1,049
Unbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531 · cited 9×
“It was recently shown that counterfactual inference methods provide a provably unbiased and consistent approach to LTR despite biased data [18].”
BackgroundMethodInfluentialSelf-citationChuklin, A., Markov, I., & de Rijke, M. (2015) · Synthesis Lectures on Information Concepts, Retrieval, and Services · doi:10.1007/978-3-031-02294-4 · Cited by 398
Research works printed but not linked: 25
- 10.1145/3097983.3098155 · DOI
- Aman Agarwal , Ivan Zaitsev , and Thorsten Joachims . 2018 . Counterfactual Learning-to-Rank for Additive Metrics and Deep Models. In ICML Workshop on Machine Learning for Causal Inference, Counterfactual Prediction, and Autonomous Action (CausalML). Aman Agarwal, Ivan Zaitsev, and Thorsten Joachims. 2018. Counterfactual Learning-to-Rank for Additive Metrics and Deep Models. In ICML Workshop on Machine Learning for Causal Inference, Counterfactual Prediction, and Autonomous Action (CausalML).
- 10.1145/3209978.3209986 · DOI · Paper page
- 10.1145/3209978.3210050 · DOI
- 10.1145/2094072.2094078 · DOI · Paper page
- 10.1145/1526709.1526711 · DOI · Paper page
- 10.1145/1341531.1341545 · DOI · Paper page
- 10.1111/j.2517-6161.1977.tb01600.x · DOI · Paper page
- Miroslav Dud'i k, John Langford , and Lihong Li . 2011 . Doubly Robust Policy Evaluation and Learning . In Proc. of the 28th International Conference on Machine Learning (ICML). 1097--1104 . Miroslav Dud'i k, John Langford, and Lihong Li. 2011. Doubly Robust Policy Evaluation and Learning. In Proc. of the 28th International Conference on Machine Learning (ICML). 1097--1104.
- 10.1145/1390334.1390392 · DOI · Paper page
- 10.1145/1526709.1526712 · DOI · Paper page
- 10.1145/775047.775067 · DOI · Paper page
- 10.1145/1229179.1229181 · DOI · Paper page
- Thorsten Joachims , Adith Swaminathan , and Maarten de Rijke . 2018 . Deep Learning with Logged Bandit Feedback. In 6th International Conference on Learning Representations (ICLR). Thorsten Joachims, Adith Swaminathan, and Maarten de Rijke. 2018. Deep Learning with Logged Bandit Feedback. In 6th International Conference on Learning Representations (ICLR).
- 10.1145/1390156.1390223 · DOI
- Lihong Li , Shunbao Chen , Jim Kleban , and Ankur Gupta . 2014. Counterfactual Estimation and Optimization of Click Metrics for Search Engines. CoRR , Vol. abs/ 1403 . 1891 (2014). Lihong Li, Shunbao Chen, Jim Kleban, and Ankur Gupta. 2014. Counterfactual Estimation and Optimization of Click Metrics for Search Engines. CoRR, Vol. abs/1403.1891 (2014). · arXiv
- 10.1145/1935826.1935878 · DOI · Paper page
- Maeve O'Brien and Mark T Keane . 2006 . Modeling result--list searching in the World Wide Web: The role of relevance topologies and trust bias . In Proc. of the 28th Annual Conference of the Cognitive Science Society (CogSci). 1--881 . Maeve O'Brien and Mark T Keane. 2006. Modeling result--list searching in the World Wide Web: The role of relevance topologies and trust bias. In Proc. of the 28th Annual Conference of the Cognitive Science Society (CogSci). 1--881.
- 10.1093/biomet/70.1.41 · DOI · Paper page
- Tobias Schnabel , Adith Swaminathan , Ashudeep Singh , Navin Chandak , and Thorsten Joachims . 2016 . Recommendations as Treatments: Debiasing Learning and Evaluation . In Proc. of the 33rd International Conference on Machine Learning (ICML). 1670--1679 . Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. In Proc. of the 33rd International Conference on Machine Learning (ICML). 1670--1679.
- Adith Swaminathan and Thorsten Joachims . 2015 a. Batch Learning from Logged Bandit Feedback through Counterfactual Risk Minimization . Journal of Machine Learning Research (JMLR) , Vol. 16 (Sep 2015), 1731--1755. Adith Swaminathan and Thorsten Joachims. 2015a. Batch Learning from Logged Bandit Feedback through Counterfactual Risk Minimization. Journal of Machine Learning Research (JMLR), Vol. 16 (Sep 2015), 1731--1755.
- Adith Swaminathan and Thorsten Joachims . 2015 b. The Self-Normalized Estimator for Counterfactual Learning . In Proc. of the 28th International Conference on Neural Information Processing Systems (NIPS). 3231--3239 . Adith Swaminathan and Thorsten Joachims. 2015b. The Self-Normalized Estimator for Counterfactual Learning. In Proc. of the 28th International Conference on Neural Information Processing Systems (NIPS). 3231--3239.
- 10.1145/2911451.2911537 · DOI · Paper page
- 10.1145/3159652.3159732 · DOI · Paper page
- 10.1145/1772690.1772793 · DOI · Paper page
Cited by
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.207 · Cited by 1,108
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
“[100] adopt intervention harvesting, to learn the propensity.”
BackgroundSelf-citationCausal Intervention for Leveraging Popularity Bias in Recommendation
Zhang, Y., Feng, F., He, X., Wei, T., Song, C., Ling, G., & Zhang, Y. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462875 · Cited by 443
Fairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E. H., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330745 · Cited by 362
Recommending what video to watch next: a multitask ranking system
Zhao, Z., Hong, L., Wei, L., Chen, J., Nath, A., Andrews, S., Kumthekar, A., Sathiamoorthy, M., Yi, X., & Chi, E. (2019) · Proceedings of the 13th ACM Conference on Recommender Systems · doi:10.1145/3298689.3346997 · Cited by 311
Metadata
- Authors
- Agarwal, A., Zaitsev, I., Wang, X., Li, C., Najork, M., & Joachims, T.
- Affiliations
- Cornell University, Ithaca, NY, USA
- Google Inc., Mountain View, CA, USA
- Venue
- Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining
- Publisher
- ACM
- Published
- 2019-01-30
- Pages
- 474-482
- Cited by
- 107
- References
- 5
- Funders
- National Science Foundation (IIS-1615706 and IIS-1513692)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Estimating Position Bias without Intrusive Interventions. https://app.cochat.ai/papers/10.1145/3289600.3291017
- The paper
- Agarwal, A., Zaitsev, I., Wang, X., Li, C., Najork, M., & Joachims, T. (2019). Estimating Position Bias without Intrusive Interventions. Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. https://doi.org/10.1145/3289600.3291017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.