The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Fairness of Exposure in Rankings
Open accessAbstract
Rankings are ubiquitous in the online world today. As we have transitioned from finding books in libraries to ranking products, jobs, job applicants, opinions and potential romantic partners, there is a substantial precedent that ranking systems have a responsibility not only to their users but also to the items being ranked. To address these often conflicting responsibilities, we propose a conceptual and computational framework that allows the formulation of fairness constraints on rankings in terms of exposure allocation. As part of this framework, we develop efficient algorithms for finding rankings that maximize the utility for the user while provably satisfying a specifiable notion of fairness. Since fairness goals can be application specific, we show how a broad range of fairness constraints can be implemented using our framework, including forms of demographic parity, disparate treatment, and disparate impact constraints. We illustrate the effect of these constraints by providing empirical results on two ranking problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“t in understanding its societal impacts. While algorithmic decisions can counteract existing biases, algorithmic and data-driven decision making affords new mechanisms for introducing unintended bias [2]. There have been numerous attempts to define notions of fairness in the supervised learning setting. The individual…”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“lized odds represents the equal opportunity principle for supervised learning and defines the constraint that the false positive and true positive rates should be equal for different protected groups [15]. Several recent works have focused on learning algorithms compatible with these definitions of fair classification…”
BackgroundThe use of MMR, diversity-based reranking for reordering documents and producing summaries
Carbonell, J., & Goldstein, J. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291025 · Cited by 2,329
The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries
Carbonell, J., & Goldstein, J. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130369 · Cited by 2,156
(2013) · Cited by 1,126
THE PROBABILITY RANKING PRINCIPLE IN IR
ROBERTSON, S. E. (1977) · Journal of Documentation · doi:10.1108/eb026647 · Cited by 1,103
Predicting clicks: estimating the click-through rate for new ads
Richardson, M., Dominowska, E., & Ragno, R. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242643 · Cited by 1,049
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995
Novelty and diversity in information retrieval evaluation
Clarke, C. L. A., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., & MacKinnon, I. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390446 · Cited by 962
Recommendations as Treatments: Debiasing Learning and EvaluationPreprint
Schnabel, T., Swaminathan, A., Singh, A., Chandak, N., & Joachims, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.05352 · Cited by 950
Self-citationShaping the Web: Why the Politics of Search Engines Matters
Introna, L. D., & Nissenbaum, H. (2017) · Computer Ethics · doi:10.4324/9781315259697-19 · Cited by 708
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · cited 1×
“positive and true positive rates should be equal for different protected groups [15]. Several recent works have focused on learning algorithms compatible with these definitions of fair classification [32, 34, 36], including causal approaches to fairness [20, 21, 23]. In this paper, we draw on many of the concepts intr…”
BackgroundUnbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531 · cited 2×
“ion bias to vj = 1 log(1+j) just like in the standard definition of DCG. More generally, one can also plug in the actual position-bias value, which can be estimated through an intervention experiment [18]. Job-seeker example. We come back to the job-seeker example from the introduction, and as illustrated in Figure 1.…”
BackgroundSelf-citationUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“In fact, a study detected the presence of gender bias in image search results for a variety of occupations [17].”
BackgroundLearning diverse rankings with multi-armed bandits
Radlinski, F., Kleinberg, R., & Joachims, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390255 · Cited by 503
Self-citationFA*IR: A Fair Top-k Ranking Algorithm
Zehlike, M., Bonchi, F., Castillo, C., Hajian, S., Megahed, M., & Baeza-Yates, R. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132938 · Cited by 496
Building Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · cited 2×
“aint in the form fT Pg = h which one can plug it into the linear program from Section §3.3. We call this a Demographic Parity Constraint similar to an analogous constraint in fair supervised learning [6, 38]. Similar to that setting, in our case also, such a constraint may lead to a big loss in utility in cases when t…”
BackgroundInfluentialRanking with Fairness Constraints
Celis, L. E., Straszak, D., & Vishnoi, N. K. (2017) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.icalp.2018.28 · Cited by 402
Measuring Fairness in Ranked Outputs
Yang, K., & Stoyanovich, J. (2017) · Proceedings of the 29th International Conference on Scientific and Statistical Database Management · doi:10.1145/3085504.3085526 · Cited by 378
Avoiding Discrimination through Causal Reasoning
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358
Equity of Attention: Amortizing Individual Fairness in Rankings
Biega, A. J., Gummadi, K. P., & Weikum, G. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210063 · Cited by 337
The Politics of Search: A Decade Retrospective
Granka, L. A. (2010) · The Information Society · doi:10.1080/01972243.2010.511560 · Cited by 215
Designing Fair Ranking Schemes
Asudeh, A., Jagadish, H. V., Stoyanovich, J., & Das, G. (2019) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3299869.3300079 · Cited by 209
Learning Non-Discriminatory Predictors
Woodworth, B., Gunasekar, S., Ohannessian, M. I., & Srebro, N. (2017) · arXiv · doi:10.48550/arxiv.1702.06081 · Cited by 182
On the relation between accuracy and fairness in binary classificationPreprint
Žliobaitė, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05723 · Cited by 146
Redundancy, diversity and interdependent document relevance
Radlinski, F., Bennett, P. N., Carterette, B., & Joachims, T. (2009) · ACM SIGIR Forum · doi:10.1145/1670564.1670572 · Cited by 112
Self-citationUnbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/738 · Cited by 74
Self-citationSome Skepticism About Search Neutrality
Grimmelmann, J. (2010) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 55
Nabi, R., & Shpitser, I. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11553 · Cited by 20 · cited 1×
“Several recent works have focused on learning algorithms compatible with these definitions of fair classification [29, 31, 33], including causal approaches to fairness [18, 19, 21].”
Background
Cited by
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang, J., Jin, D., He, X., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3568022 · Cited by 1,082
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
Fairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E. H., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330745 · Cited by 362
A Survey on the Fairness of Recommender Systems
Wang, Y., Ma, W., Zhang, M., Liu, Y., & Ma, S. (2023) · ACM Transactions on Information Systems · doi:10.1145/3547333 · Cited by 356
Metadata
- Authors
- Singh, A., & Joachims, T.
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 2219-2228
- OpenAlex
- W2787991113
- Cited by
- 591
- References
- 31
- FWCI
- 98.67
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.