The DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index; none of the 45 sources its claims cite is checked for support yet.
Fairness testing: testing software for discrimination
Open accessAbstract
This paper defines software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates, focusing on causality in discriminatory behavior. Evidence of software discrimination has been found in modern software systems that recommend criminal sentences, grant access to financial products, and determine who is allowed to participate in promotions. Our approach, Themis, generates efficient test suites to measure discrimination. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not require an oracle. We evaluate Themis on 20 software systems, 12 of which come from prior work with explicit focus on avoiding discrimination. We find that (1) Themis is effective at discovering software discrimination, (2) state-of-the-art techniques for removing discrimination from algorithms fail in many situations, at times discriminating against as much as 98% of an input subdomain, (3) Themis optimizations are effective at producing efficient test suites for measuring discrimination, and (4) Themis is more efficient on systems that exhibit more discrimination. We thus demonstrate that fairness testing is a critical aspect of the software development cycle in domains with possible discrimination and provide initial tools for measuring software discrimination.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper defines causal software discrimination and presents Themis, a testing method that automatically generates test suites to measure group and causal discrimination without an oracle. Evaluating 20 instances of 8 financial-decision systems, it finds discrimination even in systems designed to avoid it and shows sound pruning reduces test suites by two-to-three orders of magnitude.
Key findings
- Themis detected causal discrimination missed by group discrimination: for `B census gender`, the causal score was 11.27% versus a group score of 0.52%, more than 21× higher.
- Discrimination was present even in systems designed to avoid it: `B census gender` had a causal discrimination score over 11% for gender, and discriminated against gender, marital status, and race with a causal score of 77.2%; `F census` discriminated against country and race with a causal score of 98.1%.
- There was no clear evidence that discrimination-aware methods outperformed discrimination-unaware ones; the discrimination-unaware approaches typically discriminated less than their discrimination-aware counterparts, with the exception of logistic regression.
- Pruning reduced the number of test cases by, on average, a factor of 2,849 for group and 148 for causal discrimination; Themis was more efficient when systems discriminated more, e.g., `B credit gender` needed 3,413 tests at threshold θ = 0.7 but only 10 tests at θ = 0.6.
Methods
- The paper formally defines group, causal, and apparent discrimination, then introduces Themis, which uses an input schema, requires no oracle, and employs test caching, adaptive confidence-driven sampling, and sound pruning based on monotonicity properties.
- Evaluation used 20 instances of 8 software systems making financial decisions: 12 instances from prior discrimination-aware work (4 focused on race, 8 on gender) and 8 standard machine-learning instances that made no attempt to avoid discrimination.
- The datasets were the Adult/Census Income dataset (45K individuals, 14 attributes) and the Statlog German Credit dataset (1,000 individuals, 20 pieces of data); systems were trained on balanced samples of 15.5K census individuals and 600 credit individuals, and Themis measured group and causal discrimination with respect to race and gender, including discrimination-checking experiments with thresholds and pruning comparisons at 99% confidence and 0.05 error bound.
Limitations
- The paper states simplifying assumptions: software is modeled as a black box mapping categorical input characteristics to a categorical output characteristic, and non-categorical inputs/outputs are handled via binning; extending the measures directly to broader data types is left to future work.
- The paper states Themis requires an input schema, and its apparent-discrimination method does not compute confidence because it is only as strong as the developer’s confidence that the test suite or operational profile is representative; apparent discrimination was not directly evaluated.
- Evident from the text: the evaluation is limited to 20 instances of 8 financial-decision systems on two datasets, so the findings may not generalize to other software domains; the paper also leaves integration with combinatorial testing and other automated test-generation improvements to future work.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Own interpretation · 23 Not checked yet · 21 Own result · 5 No basis given · 2 Sources not found · 1 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“(3) Themis optimizations are effective at producing efficient test suites for measuring discrimination”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that (1) Themis is effective at discovering software discrimination”
- Also stated in: § ACMReference format: · p. 2 · plain
“Themis is an effective fairness testing tool.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § ACMReference format: · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“(2) state-of-the-art techniques for removing discrimination from algorithms fail in many situations, at times discriminating against as much as 98% of an input subdomain”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, a discrimination-aware decision tree approach trained not to discriminate against gender, B census gender , had a causal discrimination score over 11%: more than 11% of the individuals had the output flipped just by altering the individual's gender.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“demonstrating that (i) even when fairness is a design goal, developers can easily introduce discrimination in software”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As Section 5.4 will evaluate empirically, pruning leads to, on average, a two-to-three order of magnitude reduction in test suite size.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Discrimination is present even in instances designed to avoid discrimination.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, for B census gender , the causal score was more than 21 × higher than the group score (11.27% vs. 0.52%).”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Reducing discrimination (CV score) in classifiers [18, 40, 88, 91], as our evaluation has shown, often fails to remove causal discrimination and discrimination against certain groups.”
What the sources say
- Not checked yetcites [18] [18] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. In Proceedings of the 2009 IEEE International Conference on Data Mining (ICDM) Workshops , pages 13-18, Miami, FL, USA, December 2009. - source
- Not checked yetcites [40] [40] Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. Discrimination aware decision tree learning. In International Conference on Data Mining (ICDM) , pages 869-874, Sydney, Australia, December 2010. - source
- Not checked yetcites [88] source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Themis was able to discover significant discrimination.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 30
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, the discrimination-unaware approaches typically discriminated less than their discrimination-aware counterparts, with the exception of logistic regression.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pruning reduces the number of test cases by, on average, a factor of 2 , 849 for group and 148 for causal discrimination.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, B census gender discriminates against gender, marital status, and race with a causal score of 77.2%. That means for 77.2% of the individuals, changing only the gender, marital status, or race causes the output of the algorithm to flip.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is the case with B census gender , for example, as its causal discrimination score against gender is 11.2%, but against gender, marital status, and race is 77.2%.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“show that Themis effectively measures discriminations and produces efficient test suites to do so.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Themis is effective at finding both group and causal discrimination.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, Themis needed 3,413 tests to find sets of characteristics that B credit gender discriminated with a score of more than θ = 0 . 7, but only 10 tests when we reduced θ to 0.6.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: Decreasing the discrimination threshold θ effectively simulates increasing system discrimination.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not require an oracle.”
- Also stated in: § ACMReference format: · p. 2 · plain
“Themis requires a schema for generating inputs, but does not require an oracle.”
- Also stated in: § ACMReference format: · p. 2 · plain
“Unlike prior work that requires manually written tests [79], Themis automatically generates test suites that measure discrimination.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § ACMReference format: · p. 2 · plain
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Amazon.com, Inc. software that determined where to offer free same-day delivery did not explicitly consider race but made racecorrelated decisions because of correlations between race and other input characteristics [36, 52].”
What the sources say
- Not matchedcites [36] [36] David Ingold and Spencer Soper. Amazon doesn't consider the race of its customers. Should it? Bloomberg , April 21, 2016. http://www.bloomberg.com/ graphics/2016-amazon-same-day. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Test caching has low storage overhead and offers significant runtime gains.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The output domain distance function generalizes our work beyond binary outputs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These approaches do not capture causality and can miss discrimination that our causal approach detects, e.g., when the software discriminates negatively with respect to a group in one settings, but positively in another.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Themis is effective. Themis is able to (1) verify that many of the software instances do not discriminate against race and gender, and (2) identify the software that does.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In forecasting who would reoffend, the software is 'particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants; white defendants were mislabeled as low risk more often than black defendants' [5].”
What the sources say
- Not checked yetcites [5] [5] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. ProPublica , May 23, 2016. https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing. -
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, for example, while the group and apparent discrimination scores penalize software that gives loans to different fractions of individuals of different races, the causal discrimination score penalizes software that gives loans to individuals of one race but not to otherwise identical individuals of another race.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Our causal fairness definition is designed specifically to be testable, unlike definitions that require probabilistic estimation or knowledge of the future [38].”
What the sources say
- Not checked yetcites [38] [38] Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. CoRR , abs/1207.0016, 2012. https: //arxiv.org/abs/1605.07139. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The causal discrimination score identifies changing which characteristics directly affects the output.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our definition extends to the more broad categorical input types, reflecting the relative complexity of arbitrary decision software.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our definition captures causal relationships between inputs and outputs”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Themis could have analyzed the software and detected the discrimination prior to deployment.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Operational profiles [58] describe the input distributions likely to be observed in the field.”
What the sources say
- Not checked yetcites [58] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Our group discrimination score generalizes the CV score to the software domain with more complex inputs.”
What the sources say
- Not checked yetcites [19] [19] Toon Calders and Sicco Verwer. Three naive Bayes approaches for discriminationfree classification. Data Mining and Knowledge Discovery , 21(2):277-292, 2010. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“operational profiles can significantly improve the effectiveness of testing by more accurately representing real-world system use [7, 50]”
What the sources say
- Not checked yetcites [7] [7] Andrea Arcuri and Lionel Briand. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In ACM/IEEE International Conference on Software Engineering (ICSE) , pages 1-10, Honolulu, HI, USA, 2011. -
- Not checked yetcites [50] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“the use of operational profiles has been shown to more accurately measure system properties, such as reliability [35].”
What the sources say
- Not matchedcites [35] [35] Herman Hartmann. A statistical analysis of operational profile driven testing. In International Conference on Software Quality, Reliability, and Security Companion (QRS-C) , pages 109-116, Viena, Austria, August 2016. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Themis measures apparent discrimination by either executing a provided test suite, or by generating a test suite following a provided operational profile.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Themis can also measure apparent discrimination (Definition 3.8) via a developer-provided test suite or operational profile.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our work does not attempt to remove discrimination but offers developers a tool to identify and measure discrimination”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“input characteristics may correlate, e.g., education correlates with age, so perturbing some characteristics without perturbing others may create inputs not representative of the real world.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have further described Themis, an approach and its open-source implementation for measuring discrimination in software and for generating efficient test suites to perform these measurements.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, focusing on avoiding discriminating against one characteristic may create discrimination against another, e.g., B census gender limits discrimination against gender but discriminates against race with a causal score of 38.40%.”
hedgedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“FairTest, an implementation of the unwarranted associations framework [79] uses manually written tests to measure four kinds of discrimination scores: the CV score and a related ratio, mutual information, Pearson correlation, and a regression between the output and sensitive inputs.”
What the sources say
- Not checked yetcites [79]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The B approach was designed to avoid a variant of group discrimination (as are other discrimination-aware approaches), but this design is, at least in some conditions, insufficient to prevent causal discrimination.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“FairML [1] uses orthogonal projection to co-perturb characteristics, which can mask some discrimination, but find discrimination that is more likely to be observed in real-world scenarios”
What the sources say
- Not checked yetcites [1] [1] Julius Adebayo and Lalana Kagal. Iterative orthogonal feature projection for diagnosing bias in black-box models. CoRR , abs/1611.04967, 2016. -
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Counterfactual fairness is related to causal fairness but can miss some instances of discrimination, e.g., if loan shows preferential treatment for some purple inputs, but at the same time against some other similar purple inputs.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This experiment demonstrates that Themis effectively discovers discrimination and can test software for unexpected software discrimination effects across a wide variety of input partitions.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The work on operational profiles is complementary to ours: Themis uses operational profiles and work on more efficient test generation from operational profiles can directly benefit discrimination testing.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have formally defined software fairness testing and introduced a causality-based measure of discrimination.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The number of tests needed to evaluate every possible value pair can be significantly smaller than the exhaustive testing alternative since each test can simultaneously contribute to multiple value pairs [22, 44, 80].”
What the sources say
- Not checked yetcites [22] [22] Marcelo d'Amorim, Steven Lauterburg, and Darko Marinov. Delta execution for efficient state-space exploration of object-oriented programs. In International Symposium on Software Testing and Analysis (ISSTA) , pages 50-60, London, UK, 2007. - source
- Not checked yetcites [44] [44] Christian Kästner, Alexander von Rhein, Sebastian Erdweg, Jonas Pusch, Sven Apel, Tillmann Rendel, and Klaus Ostermann. Toward variability-aware testing. In International Workshop on Feature-Oriented Software Development (FOSD) , pages 1-8, Dresden, Germany, 2012. - source
- Not checked yetcites [80] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Counterfactual fairness requires output probability distributions to match for input populations that differ only in the label value of a sensitive input characteristic [51].”
What the sources say
- Not checked yetcites [51] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The goal of the generated tests is typically finding bugs [29] or generating specifications [21].”
What the sources say
- Not checked yetcites [29] [29] Gordon Fraser and Andrea Arcuri. Whole test suite generation. IEEE Transactions on Software Engineering (TSE) , 39(2):276-291, February 2013. - source
- Not checked yetcites [21] [21] Valentin Dallmeier, Nikolai Knopp, Christoph Mallon, Gordon Fraser, Sebastian Hack, and Andreas Zeller. Automatically generating test cases for specification mining. IEEE Transactions on Software Engineering (TSE) , 38(2):243-257, March 2012. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our approach generates tests automatically and measures causal discrimination.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“AdFisher computes a variant of group discrimination, but it could be integrated with Themis and its algorithms to measure causal discrimination.”
What the sources say
- Not checked yetcites [23] [23] Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies (PoPETs) , 1:92-112, 2015. - source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our main goal is to develop a method to process test executions to measure software discrimination, whereas that is not a goal of combinatorial testing.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Causal relationships in data management systems [54, 55] can help explain query results [57] and debug errors [82-84] by tracking and using data provenance [56].”
What the sources say
- Not checked yetcites [54]
- Not checked yetcites [55]
- Not checked yetcites [57]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Our prior work on testing software that relies on data management systems has focused on data errors [59, 60], whereas this work focuses on testing fairness.”
What the sources say
- Not checked yetcites [59]
- Not checked yetcites [60] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These tools deal with more complex input spaces than Themis, but none of them focus on testing fairness and they require oracles whereas Themis does not need oracles as it measures discrimination by comparing tests' outputs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: None of the automated test generation tools focus on testing fairness.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our oracle is more precise and easier to compute, but is only applicable to fairness testing.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The authors state this as an assumption.
Software can be modeled as a black box that maps input characteristics to an output characteristic.
“First, we define software as a black box that maps input characteristics to an output characteristic.”
The authors state this as an assumption.
Input characteristics and the output characteristic are categorical variables.
“Second, we assume that the input characteristics and the output characteristic are categorical variables, each having a set of possible values (e.g., race, gender, eye color, age ranges, income ranges).”
The authors state this as an assumption.
For the remainder of the paper, software outputs binary decisions.
“For simplicity of exposition, for the remainder of this paper, we assume software outputs binary decisions - a natural domain for fairness testing.”
The paper relies on this without support.
No prior work automatically generates test suites to measure causal discrimination in software.
“To the best of our knowledge, this work is the first to automatically generate test suites to measure causal discrimination in software.”
“We are aware of no prior testing technique, neither automated nor manual, that produces a measure of a software system's causal discrimination.”
The authors state this as an assumption.
Decreasing the discrimination threshold θ effectively simulates increasing system discrimination.
“decreasing θ effectively simulates increasing system discrimination.”
The paper relies on this without support.
No prior work on operational profile testing has measured software discrimination.
“Meanwhile no prior work on operational profile testing has measured software discrimination.”
The paper relies on this without support.
None of the automated test generation tools focus on testing fairness.
“none of them focus on testing fairness”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Thus, causal methods are speciically well suited to identifying 1) proxies of sensitive variables as discussed in subsection 2.1; 2) which subgroups are most (un)fairly treated and diferentiate the types of bias exhibited [118]; and 3) provide transparency with respect to how (classiication) decisions were made [143].”
From Fairness in machine learning: A survey (2024) · cites this paper as [113]
In the citing paper
“As a result, a classifier trained without regard to the protected attribute not only fails to remove discrimination, but it can complicate the detection and mitigation of discrimination downstream via in-processing or post-processing techniques [9,10,18,24,25,34,43,51], such as those we describe next.”
From Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5.2 Race and Gender Discrimination · p. 9
“| A census race | 2.10% | 1.13% | 8.90% | 7.20% |”
§ 4.2 Sound Pruning · p. 7
“Theorem 4.3. Let X be an input type and let S be a decision software with input type X . Then for all sets of characteristics X ′ ⊆ X , ˜ d X ′ ( S ) ≤ ⃗ d X ′ ( S ) .”
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“nition of discrimination is that a “better” input is never deprived of the “better” output [24].”
BackgroundInternational Conference on Machine Learning (ICML)-2005
津田, 宏. (2006) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 2,558
Causal inference in statistics: An overview
Pearl, J. (2009) · Statistics Surveys · doi:10.1214/09-ss057 · Cited by 2,336 · cited 4×
“Our causal discrimination score is a stronger measure of discrimination, as it seeks out causality in software, measuring the fraction of inputs for which changing specific input characteristics causes the output to change [67].”
BackgroundInfluential(2013) · Cited by 1,126
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104 · cited 1×
“While prior work has tackled the oracle problem [10, 26, 30] typically using inferred pre- and post-conditions or documentation, our oracle is more precise and easier to compute, but is only applicable to fairness testing.”
BackgroundKusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995
Arcuri, A., & Briand, L. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985795 · Cited by 989 · cited 4×
“Apparent discrimination captures this notion, applying group or causal discrimination score measurement to a subset of the input domain, which can be described by an operational profile [7, 58].”
BackgroundMethodInfluentialFeedback-Directed Random Test Generation
Pacheco, C., Lahiri, S. K., Ernst, M. D., & Ball, T. (2007) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2007.37 · Cited by 879 · cited 1×
“Automated testing research has produced tools to generate tests, including random testing, such as Randoop [63, 64], NightHawk [4], JCrasher [20], CarFast [65], and T3 [69]; search-based testing, such as EvoSuite [29], TestFul [9], and eToc [78]; dynamic symbolic execution tools, such as DSC [37], Symbolic PathFinder…”
Method
Research works printed but not linked: 8
- Agitar Technologies . AgitarOne. http://www.agitar.com/solutions/products/ automated_junit_generation.html , 2016 . Agitar Technologies. AgitarOne. http://www.agitar.com/solutions/products/ automated_junit_generation.html, 2016.
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . Machine bias . ProPublica , May 23, 2016 . Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. ProPublica, May 23, 2016.
- 10.5555/1792786.1792789 · DOI
- Katelyn Ferral . Wisconsin supreme court allows state to continue using computer program to assist in sentencing . The Capital Times , July 13, 2016 . http://host.madison.com/ct/news/local/govt-and-politics/ wisconsin-supreme-court-allows-state-to-continue-using-computer-program/ article_7eb67874-bf40-59e3-b62a-923d1626fa0f.html. Katelyn Ferral. Wisconsin supreme court allows state to continue using computer program to assist in sentencing. The Capital Times, July 13, 2016. http://host.madison.com/ct/news/local/govt-and-politics/ wisconsin-supreme-court-allows-state-to-continue-using-computer-program/ article_7eb67874-bf40-59e3-b62a-923d1626fa0f.html.
- Rafi Letzter . Amazon just showed us that ‘unbiased’ algorithms can be inadvertently racist . TECH Insider , April 21, 2016 . http://www.techinsider.io/ how-algorithms-can-be-racist- 2016 - 2014 . Rafi Letzter. Amazon just showed us that ‘unbiased’ algorithms can be inadvertently racist. TECH Insider, April 21, 2016. http://www.techinsider.io/ how-algorithms-can-be-racist-2016-4.
- Satya Nadella . The partnership of the future . Slate , June 28, 2016 . http: //www.slate.com/articles/technology/future_tense/2016/06/microsoft_ceo_ satya_nadella_humans_and_a_i_can_work_together_to_solve_society.html. Satya Nadella. The partnership of the future. Slate, June 28, 2016. http: //www.slate.com/articles/technology/future_tense/2016/06/microsoft_ceo_ satya_nadella_humans_and_a_i_can_work_together_to_solve_society.html.
- Aarti Shahani . Now algorithms are deciding whom to hire, based on voice . NPR All Things Considered , March 2015 . Aarti Shahani. Now algorithms are deciding whom to hire, based on voice. NPR All Things Considered, March 2015.
- 10.5555/1792786.1792798 · DOI
Cited by
Sorted by citations
Items 201–300 of 399
Gunasekara, S., & Saarela, M. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107933 · Cited by 13
Promoting Fairness in Learned Models by Learning to Active Learn under Parity Constraints
Sharaf, A., Daumé, H., & Ni, R. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534632 · Cited by 11
“Finally, there is the broader question of: how does one know what is the right intervention to mitigate disparities? Should we constrain our classifier? Should we collect more data? More features? Change the architecture? These are all important questions that are only beginning to be explored [9, 22].”
Fairness-Aware Training of Decision Trees by Abstract Interpretation
Ranzato, F., Urban, C., & Zanella, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482342 · Cited by 11
Stoyanovich, J., Howe, B., Jagadish, H. V., & Miklau, G. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3229863.3240494 · Cited by 9
“This portion of the panel will be informed by recent work on algorithmic and data transparency [3, 8, 9, 15, 18, 26].”
BackgroundA Methodology for Non-Functional Property Evaluation of Machine Learning Models
Anisetti, M., Ardagna, C. A., Damiani, E., & Panero, P. G. (2020) · International Conference on Management of Emergent Digital EcoSystems (MEDES) · doi:10.1145/3415958.3433101 · Cited by 10
“Many concrete projects have been developed trying to implement evaluation of machine learning models fairness [2, 3, 9, 11, 20].”
BackgroundAlamin, M. A. A., & Uddin, G. (2021) · IEEE International Conference on Autonomous Systems (ICAS) · doi:10.1109/icas49788.2021.9551151 · Cited by 12
On Adaptive Fairness in Software Systems
Farahani, A., Pasquale, L., Bennaceur, A., Welsh, T., & Nuseibeh, B. (2021) · International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) · doi:10.1109/seams51251.2021.00022 · Cited by 10
Removing biased data to improve fairness and accuracyPreprint
Verma, S., Ernst, M. D., & Just, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03054 · Cited by 27
“Our experiments take inspiration from Themis [39], which measures individual discrimination by generating similar individuals, who only differ in a sensitive attribute, sampling individuals uniformly at random from the feature space, which is captured from training data.”
The Impact of Considering Human Values during Requirements Engineering ActivitiesPreprint
Perera, H., Hoda, R., Shams, R. A., Nurwidyantoro, A., Shahin, M., Hussain, W., & Whittle, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.15293 · Cited by 10
“Unsurprisingly, values violations through software applications often end up creating undesired consequences such as financial losses [2], reputation damages [3], and even loss of lives [4].”
Unmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness
Kheya, T. A., Bouadjenek, M. R., & Aryal, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714528 · Cited by 10
“Even if a model is trained only on user-item-rating interactions with no explicit mention of sensitive attributes, the model can still infer this private information due to the correlation between their behavior and their sensitive attributes [6, 9, 22].”
Fair and square? Evaluating fairness of LLM-generated synthetic datasets
Voria, G., Scala, B., Todisco, L., Venditto, C., Giordano, G., Catolino, G., & Palomba, F. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107980 · Cited by 10
Are there gender differences when interacting with social goal models?
Gralha, C., Goulão, M., & Araújo, J. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09883-y · Cited by 9
“Awareness of these gender biases within software systems has increased in recent years (Galhotra et al. 2017; Showkat and Grimm 2018), and analysing gender differences with software is important.”
BackgroundChakraborty, J., Majumder, S., & Tu, H. (2022) · doi:10.1145/3524491.3527305 · Cited by 9
“Most of the prior works [34, 37, 38, 42, 49, 68] used one or two datasets whereas we used ten of them.”
MethodPRoofster: Automated Formal Verification
Agrawal, A., First, E., Kaufman, Z., Reichel, T., Zhang, S., Zhou, T., Sanchez-Stern, A., Ringer, T., & Brun, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion58688.2023.00018 · Cited by 9
“For software properties such as fairness [3], [7], [8], [14], [22] and safety [52], complementary approaches provide high-confidence, probabilistic guarantees based on statistical tests and confidence bounds [2], [16], [21], [31], [52].”
BackgroundSelf-citationMetamorphic Testing and Debugging of Tax Preparation Software
Tizpaz-Niari, S., Monjezi, V., Wagner, M., Darian, S., Reed, K., & Trivedi, A. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis58686.2023.00019 · Cited by 9
“Algorithmic fairness is an important area to promote inclusion, diversity, and accessibility of software solutions [38], [39], [40].”
Search-Based Fairness Testing: An Overview
Mamman, H., Basri, S. B., Balogun, A. O., Imam, A. A., Kumar, G., & Capretz, L. F. (2023) · IEEE International Conference on Computing (ICOCO) · doi:10.1109/icoco59262.2023.10397906 · Cited by 10
“The possibility of using test cases designed for one attribute to assess another emphasises the need for efficient strategies like test case recycling, including sound pruning [2], to optimise resource utilisation.”
BackgroundMethodInfluentialA Large-Scale Empirical Study on Improving the Fairness of Image Classification Models
Yang, J., Jiang, J., Sun, Z., & Chen, J. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652122 · Cited by 8
“Many testing [2, 19, 26, 31, 59, 73, 75, 78, 79, 87] and repair studies [56, 71, 77] aim to automatically identify and fix such issues.”
BackgroundFairquant: Certifying and Quantifying Fairness of Deep Neural Networks
Kim, B. H., Wang, J., & Wang, C. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00016 · Cited by 10
“Testing techniques can quickly detect fairness violations in machine learning models including neural networks [37], [6], [13], [38], [14], [39], but it does not provide formal guarantee that is important for certain applications.”
A Catalog of Fairness-Aware Practices in Machine Learning Engineering
Voria, G., Sellitto, G., Ferrara, C., Abate, F., Lucia, A. D., Ferrucci, F., Catolino, G., & Palomba, F. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3727967.3756824 · Cited by 8
Fairness, explainability, privacy, and robustness for trustworthy algorithmic decision-making
Majumdar, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-85713-0.00017-7 · Cited by 10
DeepGemini: Verifying Dependency Fairness for Deep Neural Network
Xie, X., Zhang, F.-Y., Hu, X., & Ma, L. (2023) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26779 · Cited by 9
“In this work, we take a special focus on the important dependency fairness (Urban et al. 2020; Galhotra, Brun, and Meliou 2017) of DNNs, which is considered as a stronger notion of group fairness because it imposes fairness on every individual sample.”
BackgroundSeldonian Toolkit: Building Software with Safe and Fair Machine Learning
Hoag, A., Kostas, J., da Silva, B. C., Thomas, P. S., & Brun, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion58688.2023.00035 · Cited by 8
“An important complement to building systems that enforce fairness is testing systems for discrimination [4], [17], [46].”
BackgroundSelf-citationFairness Testing of Machine Translation Systems
Sun, Z., Chen, Z., Zhang, J. M., & Hao, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664608 · Cited by 11
“This field has garnered increasing interest within the SE community, particularly following the groundbreaking work by Galhotra et al. [21] at ESEC/FSE 2017, which introduces the first fairness testing approach for ML software in SE and received the Distinguished Paper Award.”
Group Fairness via Group Consensus
Chan, E., Liu, Z., Qiu, R., Zhang, Y., Maciejewski, R., & Tong, H. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659006 · Cited by 9
“Being a broad concept, fairness has been defined in a wide variety of ways, the most popular views being (1) individual fairness [5, 14, 16, 29, 32, 36], in which similar individuals receive similar outcomes, and (2) group fairness, which aims to achieve some statistical parity among groups [11].”
BackgroundContrastive Counterfactual Fairness in Algorithmic Decision-Making
Mutlu, E. Ç., Yousefi, N., & Garibay, Ö. Ö. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534143 · Cited by 11
“Furthermore, similarity-based measures such as causal discrimination [12], fairness through unawareness [24] and fairness through awareness [9] are proposed since the aforementioned statistical methods largely ignore the relationship between other attributes and sensitive attributes by…”
BackgroundApplying Combinatorial Testing to Verification-Based Fairness Testing
Kitamura, T., Zhao, Z., & Toda, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21251-2_7 · Cited by 8
Model-based discrimination analysis
Ramadan, Q., Ahmadian, A. S., Strüber, D., Jürjens, J., & Staab, S. (2018) · doi:10.1145/3194770.3194775 · Cited by 8
“Existing works either focus on the machine learning algorithm [4, 14, 19] or on testing the overall system [9].”
BackgroundStatic Analysis of Data Science Software
Urban, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32304-2_2 · Cited by 8
Addressing multiple metrics of group fairness in data-driven decision makingPreprint
Miron, M., Tolan, S., Gómez, E., & Castillo, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04794 · Cited by 13
“Specifically, we propose a framework to analyze for fairness, similarly to other frameworks which test software for discrimination, such as Themis [16], Aequitas [30], and BlackBoxAuditing [1].”
MethodBiasTrap: Runtime Detection of Biased Prediction in Machine Learning Systems
Mamman, H., Basri, S. B., Balogun, A. O., Imam, A. A., Kumar, G., & Capretz, L. F. (2024) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.40.2.127139 · Cited by 9
Chen, Z., Li, X., Zhang, J. M., Sarro, F., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00122 · Cited by 7
“This operation is grounded in the widely adopted causal fairness principle [14], which directly modifies protected attributes and assesses the impact of this modification on the prediction [32].”
Johnson, B., Bartola, J., Angell, R., Witty, S., Giguere, S., & Brun, Y. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100031 · Cited by 18
“…algorithms ( Galhotra et al., 2017 ); using fairness-aware algorithms to reduce one kind of bias can significantly increase other biases ( Galhotra et al., 2017 ); and learning algorithms that do provide guarantees about their models’ fairness can, under some conditions, break those…”
BackgroundInfluentialSelf-citationCertifying Fairness of Probabilistic Circuits
Selvam, N. R., den Broeck, G. V., & Choi, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i10.26447 · Cited by 6
Diversity-aware fairness testing of machine learning classifiers through hashing-based sampling
Zhao, Z., Toda, T., & Kitamura, T. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107390 · Cited by 7
Human Values Violations in Stack Overflow: An Exploratory Study
Krishtul, S., Shahin, M., Obie, H. O., Khalajzadeh, H., Gai, F., Nasab, A. R., & Grundy, J. (2022) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3530019.3530027 · Cited by 8
“Over the last few years, the media has reported many examples of software-intensive systems that intentionally or unintentionally ignored or violated ethical and human values [1, 17].”
BackgroundAn efficient discrimination discovery method for fairness testing
Sano, S., Kitamura, T., & Takada, S. (2022) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2022-064 · Cited by 7
OBSan: An Out-Of-Bound Sanitizer to Harden DNN Executables
Chen, Y., Yuan, Y., & Wang, S. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.24103 · Cited by 7
An epistemic approach to the formal specification of statistical machine learning
Kawamoto, Y. (2020) · Software & Systems Modeling · doi:10.1007/s10270-020-00825-2 · Cited by 7
“In a recently few years, there have been studies on the testing methods for fairness of machine learning [1,18,42].”
BackgroundSOCRATES: Towards a Unified Platform for Neural Network Verification.Preprint
Pham, L. H., Li, J., & Sun, J. (2020) · arXiv (Cornell University) · Cited by 7
ShapeFlow: Dynamic Shape Interpreter for TensorFlowPreprint
Verma, S., & Su, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13452 · Cited by 14
“Galhotra et al. proposed Themis [23], which uses a random strategy for generating test cases for a biased learned model based on the definition of individual fairness [21].”
BackgroundJohnson, B., Bartola, J., Angell, R., Keith, K. A., Witty, S., Giguere, S., & Brun, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09951 · Cited by 7
“In other words, if two individuals differ only in protected attributes, and are otherwise identical, this definition requires classifiers to predict the same outcome for both individuals [21, 36].”
BackgroundInfluentialSelf-citationFairness Testing of Deep Image Classification with Adequacy MetricsPreprint
Zhang, P., Wang, J., Sun, J., & Wang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08856 · Cited by 12
“Multiple recent works [1, 12, 44, 55] have investigated the fairness 1 testing problem of machine learning models.”
InfluentialQuan, L., Li, T., Xie, X., Chen, Z., Chen, S., Jiang, L., & Li, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00235 · Cited by 6
“In the black-box setting, [20] define fairness and discrimination and develop THEMIS, a tool to generate efficient test suites for measuring discrimination.”
PCIV method for Indirect Bias Quantification in AI and ML Models
Garg, A., & Dr. Rajesh, S. L. (2021) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit217251 · Cited by 6
Causal Inference in Data Analysis with Applications to Fairness and Explanations
Roy, S., & Salimi, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-31414-8_3 · Cited by 7
From Literature to Practice: Exploring Fairness Testing Tools for the Software Industry Adoption
Nguyen, T., Baldassarre, M. T., de Lima, L. F. F. P., & Santos, R. D. S. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3695404 · Cited by 6
“Software fairness is the ethical practice of ensuring that software systems, algorithms, and their outcomes are just, equitable, and unbiased across different groups of people, regardless of characteristics such as race, gender, ethnicity, or socioeconomic status [14].”
Efficient Fairness Testing Through Hash-Based Sampling
Zhao, Z., Toda, T., & Kitamura, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21251-2_3 · Cited by 6
Does the GDPR Help or Hinder Fair Algorithmic Decision-Making?Preprint
Henderson, T. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3140887 · Cited by 7
Approximation-guided Fairness Testing through Discriminatory Space Analysis
Zhao, Z., Toda, T., & Kitamura, T. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695481 · Cited by 8
Endlose Theorie: Zu ihrer Relevanz im Kontext von Big Data
Fuhrmann, J. T. (2023) · Berliner Journal für Soziologie · doi:10.1007/s11609-023-00502-3 · Cited by 5
Isolation-Based Debugging for Neural Networks
Chen, J., Wang, J., Sun, Y., Cheng, P., & Chen, J. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652132 · Cited by 7
From Expectation to Habit: Why do Software Practitioners Adopt Fairness Toolkits?
Voria, G., Lambiase, S., Schiavone, M. C., Catolino, G., & Palomba, F. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00015 · Cited by 5
“Post-processing techniques refine model outputs after training, with Galhotra et al. [35] introducing T HEMIS and Udeshi et al. [36] developing A EQUITAS .”
Fairness Testing Through Extreme Value Theory
Monjezi, V., Trivedi, A., Крейнович, В., & Tizpaz-Niari, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00070 · Cited by 5
“From a practical standpoint, it is crucial to ensure fairness on average outcomes (e.g., T HEMIS [14]) as well as in the tail.”
InfluentialBias-aware guidelines and fairness-preserving Taxonomy in software engineering education
Spoletini, P., & Parizi, R. M. (2018) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2018.8659178 · Cited by 5
“During the Spring Semester 2018, we briefly introduced fairness testing [9] to the graduate students in the Software”
BackgroundMethodInfluentialPrivileged and unprivileged groups
Hort, M., & Sarro, F. (2022) · doi:10.1145/3524491.3527308 · Cited by 8
“To date, the software engineering community has tackled fairness at different stages of the software project lifecycle such as requirements analysis [23], design [16], and testing [1, 2, 25, 54, 55].”
MethodSpecifying and Testing k-Safety Properties for Machine-Learning Models
Christakis, M., Enişer, H. F., Hoffmann, J., Singla, A. K., & Wüstholz, V. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/528 · Cited by 8
Artificial Intelligence Usage Opportunities in Smart City Data Management
Hudasi, L. F., & Ady, L. (2020) · Interdisciplinary Description of Complex Systems · doi:10.7906/indecs.18.3.8 · Cited by 5
“The discrimination factor is too high, and there are already case studies for detecting and removing discriminative part from the software [7].”
BackgroundFairkit-learn: A Fairness Evaluation and Comparison Toolkit
Johnson, B., & Brun, Y. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion55297.2022.9793775 · Cited by 5
Self-citationImproving Patch Quality by Enhancing Key Components of Automatic Program Repair
Soto, M. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00147 · Cited by 5
Bias Mitigation Post-processing for Individual and Group FairnessPreprint
Lohia, P., Ramamurthy, K. N., Bhide, M. A., Saha, D., Varshney, K. R., & Puri, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06135 · Cited by 5
“Checking for group fairness is a fairly straightforward computation of statistical metrics [8], but checking for individual fairness is more computationally involved when there are many protected attributes with many values and scoring samples using a model is expensive [9, 10].”
BackgroundTechnical Challenges for Training Fair Neural NetworksPreprint
Cherepanova, V., Nanda, V., Goldblum, M., Dickerson, J. P., & Goldstein, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06764 · Cited by 25
“In such situations, differences in model behaviors between different social groups may result in disparities in how these groups are treated (Barocas & Selbst, 2016; Galhotra et al., 2017).”
BackgroundThe problem of fairness in tools for algorithmic fairness
Sun, X.-Y., Ye, B., & Xia, B.-H. (2024) · AI and Ethics · doi:10.1007/s43681-024-00533-3 · Cited by 4
“Unlike manually created testing tools, Themis-ML can automatically generate testing suites to assess discrimination [21].”
Optimizing Fairness in Machine Learning: A Hyperparameter Tuning Approach
Nadeem, M. A. (2025) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins65080.2025.11125753 · Cited by 4
“For instance, Galhotra et al. introduced Fairness Testing to identify discrimination in decision-making software [6], while Udeshi et al. developed Directed Fairness Testing to systematically uncover fairness violations [7].”
Sarkar, P., & Liem, C. C. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637360 · Cited by 6
Detecting Bias in LLMs' Natural Language Inference Using Metamorphic Testing
Li, Z., Chen, J., Chen, H., Xu, L., & Guo, W. (2024) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c63300.2024.00015 · Cited by 5
“The significance to detect biases within AI systems has been underscored by many researchers in the field [16][17][18].”
Fairness Testing for Recommender Systems
Guo, H. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3605235 · Cited by 4
“For instance, Galhotra et al. proposed Themis [1, 5], a method that measures group discrimination in software by grouping users based on various sensitive attribute values.”
BackgroundMethodPreliminary Insights on Industry Practices for Addressing Fairness Debt
Santos, R. D. S., de Lima, L. F. F. P., Baldassarre, M. T., & Spinola, R. O. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3695406 · Cited by 4
“Software fairness involves the ethical principle of ensuring that software systems, algorithms, and their outcomes are equitable and unbiased across diverse groups of people, regardless of characteristics such as race, gender, ethnicity, or socioeconomic status [11].”
BackgroundFairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models
Xiao, Y., Liu, A., Liang, S., Liu, X., & Tao, D. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728881 · Cited by 10
“[22] first defined software fairness and discrimination, proposing a random-based fairness testing method.”
InfluentialHunt, L., & Ferrario, M. A. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis55304.2022.9793939 · Cited by 4
FlipTest: Fairness Auditing via Optimal Transport.Preprint
Black, E., Yeom, S., & Fredrikson, M. (2019) · arXiv (Cornell University) · Cited by 4
Testing Monotonicity of Machine Learning ModelsPreprint
Sharma, A., & Wehrheim, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12278 · Cited by 4
“[14] they aim at a more systematic generation of test inputs.”
Can We Achieve Fairness Using Semi-Supervised Learning?Preprint
Chakraborty, J., Tu, H., Majumder, S., & Menzies, T. (2021) · arXiv (Cornell University) · Cited by 4
Beyond the Seeds: Fairness Testing via Counterfactual Analysis of Non-Seed Instances
Mamman, H., Basri, S. B., Balogun, A. O., Gilal, A. R., Imam, A. A., Kumar, G., & Capretz, L. F. (2024) · IEEE Access · doi:10.1109/access.2024.3502164 · Cited by 3
“Fairness testing is a specialized type of software testing that is concerned with examining a machine learning soft-ware’s behavior in terms of its bias towards certain protected attributes, such as race, gender, or age [7], [16], [17].”
InfluentialCounterfactual Sentence Generation with Plug-and-Play Perturbation
Madaan, N., Saha, D., & Bedathur, S. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00028 · Cited by 9
“The notion of counterfactuals [6], [45] and their use in model testing for has also been applied towards testing in models that consume structured inputs [46]–[48]”
MethodBlack-Box Fairness Testing with Shadow Models
Jiang, W., Shen, C. Q., Lin, C., Wang, J., Sun, J., & Gao, X. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7356-9_28 · Cited by 3
Towards Fair Federated Learning via Unbiased Feature Aggregation
He, Z., Wang, Z., Dong, X., Sun, P., Ren, J., & Ren, K. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3541306 · Cited by 3
“Various metrics (e.g., [40], [41], [42], [43]) have been proposed to measure model fairness with different philosophies of how to define discrimination.”
Lourenço, R., Freire, J., Simon, E., Weber, G., & Shasha, D. E. (2022) · The VLDB Journal · doi:10.1007/s00778-022-00733-5 · Cited by 3
“Software testing State-of-the-art techniques for software testing [25,34], statistical debugging [40,60], and bug localization [3,4,29] are often application-specific and/or require a user-defined test suite.”
MethodFair decision making via automated repair of decision trees
Zhang, J., Beschastnikh, I., Mechtaev, S., & Roychoudhury, A. (2022) · doi:10.1145/3524491.3527306 · Cited by 3
“We focus on group fairness, modified from [5] and [9].”
BackgroundBias in human data: A feedback from social sciences
Takan, S., Ergün, D., Yaman, S. G., & Kılınççeker, O. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1498 · Cited by 3
“They also provide an experimental evaluation of eight benchmarks concerning age, gender, and race attributes compared to the Themis tool (Galhotra et al., 2017).”
BackgroundMethodExplainable Global Fairness Verification of Tree-Based Classifiers
Calzavara, S., Cazzaro, L., Lucchese, C., & Marcuzzi, F. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00011 · Cited by 3
“A very popular approach to assess the fairness guarantees of ML models is based on testing [14], [15], [16], [17].”
BackgroundInfluentialReflexive Practices in Software Engineering
Boyd, A. E. (2023) · doi:10.1109/fairware59297.2023.00006 · Cited by 4
“Within the software engineering community, we have recently seen several relevant contributions [14]–[20].”
BackgroundFETA: Fairness Enforced Verifying, Training, and Predicting Algorithms for Neural Networks
Mohammadi, K., Sivaraman, A., & Farnadi, G. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623243 · Cited by 3
“In this paper, we evaluate FETA using the casual discrimination notion of individual fairness as defined in [19] (see Definition 2.1).”
BackgroundMethodInfluentialGeneralizing Group Fairness in Machine Learning via Utilities
Blandin, J., & Kash, I. A. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14238 · Cited by 6
“…that are tightly coupled with a particular problem domain. b Many group fairness works focus only on classification settings [4, 9, 10, 13, 20, 21, 28]. l s r d In this work, we focus on group fairness definitions, where an algorithm is considered fair if its results are independent of…”
BackgroundTeaching Software Ethics to Future Software Engineers
Pant, A., Spiegler, S. V., Hoda, R., Yoon, J., Yusuf, N., Er, T., & Hu, S. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) · doi:10.1145/3639474.3640082 · Cited by 3
“The increasing prevalence of Digital Addiction, as discussed by Ali et al. [4], and the existence of discriminatory software, as outlined by Galhotra et al. [17], clearly illustrate the detrimental impacts of the lack of ethical considerations in software on modern society.”
MethodKeeper: Automated Testing and Fixing of Machine Learning Software
Wan, C., Liu, S., Xie, S., Liu, Y., Hoffmann, H., Maire, M., & Lu, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672451 · Cited by 7
“ML models, typically in the format of neural networks, are widely studied on testing [1, 4, 5, 10, 13, 24, 25, 28, 30, 31, 31, 32, 39, 43, 56, 63, 64, 66, 67, 74, 77, 78, 86, 91, 100, 110, 112, 114, 118, 119, 122], verifying [26, 27, 27, 111], and fixing [46, 58, 94, 97, 98, 115], to improve their…”
Predicting Fairness of ML Software Configurations
Robles, S., Monjezi, V., Крейнович, В., Trivedi, A., & Tizpaz-Niari, S. (2024) · doi:10.1145/3663533.3664040 · Cited by 5
“In summary, our goal is to study the feasibility of training regression models to predict the fairness of ML hyperparameters in two levels: i) fixed dataset over a protected attribute and ii) temporal distribution shifts between training and test data.”
BackgroundQEDCartographer: Automating Formal Verification Using Reward-Free Reinforcement Learning
Sanchez-Stern, A., Varghese, A., Kaufman, Z., Zhang, D., Ringer, T., & Brun, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00033 · Cited by 4
“…specifying properties, automatically generating formal specifications from natural language [16], [24], [51], [93], and extending the types of properties formal languages can capture, including privacy properties [82], data-based properties [54], [55], fairness properties [9], [21], among others.”
Self-citationFundamental Approaches to Software Engineering
Johnsen, E. B., & Wimmer, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99429-7 · Cited by 3
Unbiasing on the Fly: Explanation-Guided Human Oversight of Machine Learning Decisions
Mamman, H., Basri, S. B., Balogun, A. O., Imam, A. A., Kumar, G., & Capretz, L. F. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-70300-3_20 · Cited by 5
Toward Individual Fairness Testing with Data Validity
Kitamura, T., Amasaki, S., Inoue, J., Isobe, Y., & Toda, T. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695300 · Cited by 2
“We review the concept of IDIs by Galhotra et al. [9], the standard definition underlying current individual fairness testing research.”
InfluentialCounterfactual Situation Testing: From Single to Multidimensional Discrimination
Álvarez, J. M., & Ruggieri, S. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17935 · Cited by 4
“These frameworks (Aggarwal et al., 2018; Galhotra et al., 2017; Qureshi et al., 2020) keep the focus on comparing multiple control-test instances for making individual claims, providing evidence similar to that produced by the quantitative tools used in court.”
Less Is More: Feature Engineering for Fairness and Performance of Machine Learning Software
Meng, L., Li, Y., Chen, L., Ma, M., Zhou, Y., & Xu, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3730577 · Cited by 2
“Galhotra et al. [32] proposed the concept of fairness for the first time in the field of SE, and they proposed a method called Themis to effectively measure the bias and generate effective test suites.”
FairSense: Long-Term Fairness Analysis of ML-Enabled Systems
She, Y., Biswas, S., Kästner, C., & Kang, E. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00159 · Cited by 2
“For example, researchers have developed various methods for fairness measures and identification [5–7], verification and testing [5, 8–11], and bias mitigation [12–14].”
Towards Systematic Specification and Verification of Fairness Requirements: A Position Paper
Ramadan, Q., Ruohonen, J., Tiwari, A., Alami, A., & Boukhers, Z. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00059 · Cited by 2
“Third, approaches for discrimination detection focus on testing and verifying whether a system is fair with respect to a specific fairness notion [9], [15], [26].”
InfluentialKaufman, Z., Endres, M., Xiong, C., & Brun, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3634245 · Cited by 3
“Biased model behavior, which is unfortunately common in ML [29, 88, 92] adds an extra layer of complexity; bias can be particularly difficult to formalize and control during training because of a possible trade-off between fairness and accuracy [56], and different notions of bias can be…”
Self-citationAutomatically Engineering Trusted Software: A Research Roadmap
Brun, Y., Chakraborty, S., Goues, C. L., Păsăreanu, C. S., & Singla, A. K. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3779132 · Cited by 2
“Still, other types of desirable properties, such as fairness [28, 65], which is particularly relevant to systems that use machine-learning-based components, can be verified probabilistically instead [3, 69, 81, 129, 170, 177].”
Self-citationPepe, F., Nardone, V., Mastropaolo, A., Canfora, G., Bavota, G., & Penta, M. D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10843-1 · Cited by 2
Exposing hidden bias: A study of fairness debt in gray literature
Sotolani, R. S., Freire, S., Fronchetti, F., Santos, R. D. S., & Spinola, R. O. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112866 · Cited by 2
CILIATE: Towards Fairer Class-Based Incremental Learning by Dataset and Training Refinement
Gao, X., Zhai, J., Ma, S., Shen, C. Q., Chen, Y., & Wang, S. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598071 · Cited by 4
“In the following incremental steps, the same process will repeat (on different datasets and labels) as illustrated in Figure 1.”
Method
Metadata
- Authors
- Galhotra, S., Brun, Y., & Meliou, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2017-08-21
- Pages
- 498-510
- OpenAlex
- W2730550703
- Cited by
- 405
- References
- 91
- Citations by year
- 2026: 26 · 2025: 55 · 2024: 46 · 2023: 56 · 2022: 50 · 2021: 53 · 2020: 57 · 2019: 37 · 2018: 22 · 2017: 3
- FWCI
- 23.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (CCF-1453474, IIS-1453543, CNS-1744471)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic discrimination, fairness testing, automated test generation, THEMIS, algorithmic fairness, automated decision systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
