Auditing Race and Gender Discrimination in Online Housing Markets
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
While researchers have developed rigorous practices for offline housing audits to enforce the US Fair Housing Act, the online world lacks similar practices. In this work we lay out principles for developing and performing online fairness audits. We demonstrate a controlled sock-puppet audit technique for building online profiles associated with a specific demographic profile or intersection of profiles, and describe the requirements to train and verify profiles of other demographics. We also present two audits using these sock-puppet profiles. The first audit explores the number and content of housing-related ads served to a user. The second compares the ordering of personalized recommendations on major housing and real-estate sites. We examine whether the results of each of these audits exhibit indirect discrimination: whether there is correlation between the content served and users' protected features, even if the system does not know or use these features explicitly. Our results show differential treatment in the number and type of housing ads served based on the user's race, as well as bias in property recommendations based on the user's gender. We believe this framework provides a compelling foundation for further exploration of housing fairness online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“Previous auditing studies have particularly looked at two categorizations of fairness: direct vs. indirect and individual vs. group fairness (Datta, Tschantz, and Datta 2014; Chen et al. 2018; Hardt, Price, and Srebro 2016; Zemel et al. 2013).”
BackgroundRacial Discrimination in the Sharing Economy: Evidence from a Field Experiment
Edelman, B. G., Luca, M., & Svirsky, D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2701902 · Cited by 999
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 2×
“For example, Google search results for stereotypically African American names returned ads containing the word “arrest” significantly more than for white names (Sweeney 2013).”
BackgroundDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · cited 1×
“Inthe direct/indirect categorization of fairness, the focus is on the method of discrimination (Pedreschi, Ruggieri, and Turini 2008).”
MethodSweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Digital Discrimination: The Case of Airbnb.com
Edelman, B. G., & Luca, M. (2014) · doi:10.2139/ssrn.2377353 · Cited by 471 · cited 4×
“Recent studies of the online rental site Airbnb (Edelman and Luca 2014; Edelman, Luca, and Svirsky 2017) show significant racial bias against those with African American names, while (Edelman, Luca, and Svirsky 2017) demonstrated that African American users were turned down for rentals 58 percent of the time, as oppose…”
BackgroundInfluentialMeasuring price discrimination and steering on e-commerce web sites
Hannak, A., Soeller, G., Lazer, D., Mislove, A., & Wilson, C. (2014) · Proceedings of the 2014 Conference on Internet Measurement Conference · doi:10.1145/2663716.2663744 · Cited by 405 · cited 1×
“use this audit strategy for detecting differential pricing online (Hannk et al. 2014) using a combination of voluntarily submitted browsing profiles and algorithmically generated browsing behavior.”
MethodRental Discrimination and Ethnicity in Names1
Carpusor, A. G., & Loges, W. E. (2006) · Journal of Applied Social Psychology · doi:10.1111/j.0021-9029.2006.00050.x · Cited by 221 · cited 8×
“These were chosen as they are the four largest racial and ethnic groups in the United States.”
BackgroundInfluentialInvestigating the Impact of Gender on Rank in Resume Search Engines
Chen, L., Ma, R., Hannák, A., & Wilson, C. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174225 · Cited by 155 · cited 2×
“A 2018 study analyzed bias in resume search results (Chen et al. 2018).”
Background
Cited by
A Survey on ChatGPT: AI–Generated Contents, Challenges, and Solutions
Wang, Y., Pan, Y., Yan, M., Su, Z., & Luan, T. H. (2023) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2023.3300321 · Cited by 368
Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193
Shen, H., DeVos, A., Eslami, M., & Holstein, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479577 · Cited by 171
“Past research in this domain has uncovered harmful biases across a wide range of algorithmic systems such as search engines [57, 67, 74], housing websites [5], hiring systems [21], employment online services [15] and ecommerce platforms [47] .”
BackgroundSelf-citationBuilding and Auditing Fair Algorithms: A Case Study in Candidate Screening
Wilson, C., Ghosh, A., Jiang, S., Mislove, A., Baker, L., Szary, J., Trindel, K., & Polli, F. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445928 · Cited by 133
“ing [2, 15, 39], housing [3], and child protective services [11].”
BackgroundDeng, W. H., Guo, B., Devrio, A., Shen, H., Eslami, M., & Holstein, K. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581026 · Cited by 103
Self-citation
Metadata
- Authors
- Asplund, J., Eslami, M., Sundaram, H., Sandvig, C., & Karahalios, K.
- Venue
- Proceedings of the International AAAI Conference on Web and Social Media
- Published
- 2020-05-26
- Volume
- 14
- Pages
- 24-35
- ISSN
- 2162-3449 (ISSN-L)
- 2334-0770
- OpenAlex
- W2969884768
- Cited by
- 65
- References
- 14
- FWCI
- 10.77
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Auditing Race and Gender Discrimination in Online Housing Markets. https://app.cochat.ai/papers/10.1609/icwsm.v14i1.7276
- The paper
- Asplund, J., Eslami, M., Sundaram, H., Sandvig, C., & Karahalios, K. (2020). Auditing Race and Gender Discrimination in Online Housing Markets. Proceedings of the International AAAI Conference on Web and Social Media. https://doi.org/10.1609/icwsm.v14i1.7276
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.