Themis: automatically testing software for discrimination
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Bias in decisions made by modern software is becoming a common and serious problem. We present Themis, an automated test suite generator to measure two types of discrimination, including causal relationships between sensitive inputs and program behavior. We explain how Themis can measure discrimination and aid its debugging, describe a set of optimizations Themis uses to reduce test suite size, and demonstrate Themis' effectiveness on open-source software. Themis is open-source and all our evaluation data are available at http://fairness.cs.umass.edu/. See a video of Themis in action: https://youtu.be/brB8wkaUesY
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Causal inference in statistics: An overview
Pearl, J. (2009) · Statistics Surveys · doi:10.1214/09-ss057 · Cited by 2,336 · cited 2×
“Software testing enables a unique opportunity to conduct hypothesis testing to determine statistical causation [26] between inputs and outputs.”
BackgroundA Reductions Approach to Fair Classification
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02453 · Cited by 1,288
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 2×
“…discrimination have emerged, including the correlation or mutual information between inputs and outputs [31], discrepancies in the fractions of inputs that produce a given output [4, 12, 35, 36] (known as group discrimination [10]), or discrepancies in output probability distributions [15].”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“, advertisements [29], hotel bookings [17], and image search [13].”
BackgroundFairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694
Fraser, G., & Arcuri, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.14 · Cited by 586 · cited 2×
“Unlike other automated test generation tools, e.g., Randoop [25] and EvoSuite [8], Themis processes test results to compute discrimination scores.”
MethodFace Recognition Performance: Role of Demographic Information
Klare, B. F., Burge, M. J., Klontz, J. C., Vorder Bruegge, R. W., & Jain, A. K. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2214212 · Cited by 516 · cited 1×
“Facial recognition systems often underperform on female and black faces [14].”
BackgroundUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“Discrimination shows up in many software applications, e.g., advertisements [29], hotel bookings [17], and image search [13].”
BackgroundPreventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 1×
“Reducing discrimination in machine learning classifiers [2, 17, 26, 30, 31, 51, 56, 58] and selection algorithms [47] is important work that is complementary to ours.”
Background
Cited by
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
“As the field matures, integrated toolkits are being developed, with the aim of making these methods more widely accessible and usable [7, 28, 35, 42, 70].”
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
“…in technical review articles (Berk et al. 2017, Chouldechova & Roth 2018, Friedler et al. 2018, Verma & Rubin 2018) and an in-progress textbook (Barocas et al. 2018), and has inspired a number of software packages (Galhotra et al. 2017, Zehlike et al. 2017, Angell et al. 2018, Wexler 2018).”
BackgroundBias in machine learning software: why? how? what to do?
Chakraborty, J., Majumder, S., & Menzies, T. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3468537 · Cited by 224
Balayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
“[3,7,56,186] propose test-suites to evaluate the fairness of software that relies on machine learning models, focusing on individual unfairness and developing methodologies for auto-generation of test inputs.”
Background
Metadata
- Authors
- Angell, R., Johnson, B., Brun, Y., & Meliou, A.
- Venue
- Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2018-10-26
- Pages
- 871-875
- OpenAlex
- W2898851569
- Cited by
- 94
- References
- 24
- FWCI
- 7.81
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Themis: automatically testing software for discrimination. https://app.cochat.ai/papers/10.1145/3236024.3264590
- The paper
- Angell, R., Johnson, B., Brun, Y., & Meliou, A. (2018). Themis: automatically testing software for discrimination. Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. https://doi.org/10.1145/3236024.3264590
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.