The DOI matches its Crossref record; 33 of the 37 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
Open accessAbstract
Prediction systems are successfully deployed in applications ranging from disease diagnosis, to predicting credit worthiness, to image recognition. Even when the overall accuracy is high, these systems may exhibit systematic biases that harm specific subpopulations; such biases may arise inadvertently due to underrepresentation in the data used to train a machine-learning model, or as the result of intentional malicious discrimination. We develop a rigorous framework of *multiaccuracy* auditing and post-processing to ensure accurate predictions across *identifiable subgroups*. Our algorithm, MULTIACCURACY-BOOST, works in any setting where we have black-box access to a predictor and a relatively small set of labeled data for auditing; importantly, this black-box framework allows for improved fairness and accountability of predictions, even when the predictor is minimally transparent. We prove that MULTIACCURACY-BOOST converges efficiently and show that if the initial model is accurate on an identifiable subgroup, then the post-processed model will be also. We experimentally demonstrate the effectiveness of the approach to improve the accuracy among minority subgroups in diverse applications (image classification, finance, population health). Interestingly, MULTIACCURACY-BOOST can improve subpopulation accuracy (e.g. for "black women") even when the sensitive features (e.g. "race", "gender") are not given to the algorithm explicitly.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces multiaccuracy, a black-box post-processing framework that audits a classifier for systematic bias across identifiable subpopulations and iteratively updates it with Multiaccuracy Boost. It proves convergence and do-no-harm guarantees and shows improved subgroup accuracy in gender detection, adult income, and semi-synthetic disease prediction without requiring sensitive features.
Key findings
- On PPB gender detection, the initial model had 9.9% overall error but 39.8% error on dark-skinned females and 0.0% on light-skinned males; Multiaccuracy Boost reduced overall error to 3.9% and dark-skinned female error to 12.5%, while light-skinned male error changed from 0.0% to 0.8%.
- On Adult Income, Multiaccuracy Boost improved overall error from 19.3% to 14.7% and improved each subgroup, outperforming subgroup-specific models (SS overall 19.7%); for example, Black female error went from 4.8% to 4.5% (SS 5.5%).
- On semi-synthetic UK Biobank disease prediction, Multiaccuracy Boost lowered overall error from 18.9% to 16.0% and old female error from 36.8% to 26.5%, better than subgroup-specific models (SS old female 37.6%).
- Theoretically, the do-no-harm guarantee bounds post-processed error on an audited subpopulation as er_S(f;y) ≤ 3·er_S(f0;y) + 4β, and Algorithm 1 converges to a (C, α)-multiaccurate hypothesis in T = O(ℓ_D(f0;y)/α²) iterations from m = Õ(T·d(C)+log(1/δ)/α²) samples.
Methods
- Setting: black-box access to f0 and a small labeled validation sample from D; an auditor A learns a class C to find functions h correlating with the residual f−y; Multiaccuracy Boost partitions X by f0 into X0 and X1, audits X, X0, and X1, selects the largest residual, and applies multiplicative weights updates until (C, α)-multiaccuracy is reached.
- Experiments: Gender detection used an inception-resnet-v1 trained on CelebA (>200,000 images; 98.4% test accuracy), audited on PPB (415 test, 855 audit) with ridge regression on a VAE/Facenet representation, compared to white-box retraining of the last two layers. Adult Income used 45,222 samples and 14 attributes, with 27,145 training, 15,060 test, and 3,017 audit samples, decision tree regression (max depth 5), and sensitive features removed. Semi-synthetic disease prediction used 40,000 UK Biobank individuals with 60 phenotype features, four gender-age groups with polynomial label functions of orders 1, 4, 2, and 6, 32,000 training and 4,000 audit samples, and decision tree regression.
- Analysis: theoretical proofs of Proposition 1, Theorem 2, and Theorem 3 are given in the appendix; empirical evaluation reports classification error on overall and subgroup populations in Tables 1–3.
Limitations
- Stated: Multiaccuracy works only to the extent that the auditor can identify subgroups where f0 tends to make mistakes; if the mistakes appear overly complicated to the bounded auditor, the auditor will not identify them. The paper also notes that one must balance the choice of C and the number of iterations to avoid overfitting, and that if the auditor learns an expressive enough class, the algorithm will start to overfit at some point.
- Stated: The theory assumes fresh samples per iteration, and in practice with few samples one may need noise-addition techniques to reduce overfitting. The data distribution D should reflect the true population or be aspirational, and important subpopulations must be sufficiently represented; the classification error guarantee improves as γ, the density of the protected subpopulation, grows. In the gender detection case study, CelebA lacks race labels, so subpopulation accuracy could not be evaluated on that set, and PPB is described as very small and a stress test.
- Evident: Empirical validation is limited to binary classification tasks and specific datasets; effectiveness depends on audit features being correlated with unobserved sensitive categories, as the paper states. Formal guarantees also depend on A agnostically learning the class C.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 19 Own result · 16 No basis given · 14 Not checked yet
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Importantly, if the initial model is accurate on an identifiable subgroup, then the post-processed model will be also.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We prove convergence guarantees for Multiaccuracy Boost and show that post-processing for multiaccuracy may actually improve the overall classification accuracy.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Multiaccuracy Boost can improve subpopulation accuracy (e.g. for”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We prove guarantees on the convergence rate of the algorithm and show that it improves overall accuracy at each step.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As suggested by the theory, enforcing multiaccuracy actually improves the overall accuracy, by identifying subpopulations where the initial models systematically erred”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In a nutshell, the property guarantees that on any subpopulation S ⊆ X that A audits, the classification error cannot increase significantly from f 0 to the post-processed classifier.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Do no harm.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The auditing algorithm A can identify a very rich class of overlapping sets.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“further, post-processing does not significantly affect performance on groups where accuracy was already high.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The resulting test accuracy on CelebA for binary gender classification is 98.4%.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4 CASE STUDY: GENDER DETECTION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After post-processing, the accuracy improves across these minority groups, even though minority-status is not explicitly given to the post-processing algorithm as a feature.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Then the classification error of f on the subset S is bounded as er S (f ; y) ≤ 3 • er S (f 0 ; y) + 4β .”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Do no harm.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 12
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show that Multiaccuracy Boost, which only accesses f 0 as a black-box, performs comparably and sometimes even better than very strong white-box alternatives which has full access to f 0 .”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though the overall accuracy of this f 0 is high, the error rate is much worse for females compared to males and worse for blacks compared to non-blacks”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4 CASE STUDY: GENDER DETECTION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Concretely, in our experiments, we audit with ridge regression and decision tree regression; both auditors are effective at identifying subpopulations on which the model is underperforming.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 2.2 Auditing for multiaccuracy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“indeed, the CelebA images are not labeled with race information, and as such, we were unable to evaluate the subpopulation classification accuracy on this set.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4.0.1 Multiaccuracy auditing as diagnostic.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“For instance, the initial classification error on black females is significant, whereas on white males, it is near 0%.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4.0.1 Multiaccuracy auditing as diagnostic.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Multiaccuracy Boost converged in 5 iterations and substantially reduced error across subpopulations.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4 CASE STUDY: GENDER DETECTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the classification error improves significantly on DF but does not hurt the accuracy on LM significantly.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 4 CASE STUDY: GENDER DETECTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that this subpopulation had the least representation in D 0 .”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Semi-Synthetic Disease Prediction.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The f 0 we discover performs moderately well on every subpopulation, with the exception of old females (OF) where the classification error is significantly higher.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Semi-Synthetic Disease Prediction.
plaincorrelationaln = 32,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 2, multiaccuracy post-processing achieves better accuracy both in aggregate and for each of the subgroups.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Adult Income Prediction.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“While all systems achieved roughly 90% accuracy at gender detection on a popular benchmark”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
What the sources say
- Not checked yetcites [2]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Our notion of multiaccuracy differs from parity-based notions of fairness [7,11,16], and is reasonable in settings such as gender detection where we would like to boost the classifier's accuracy across subgroups.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
What the sources say
- Not checked yetcites [7] source
- Not checked yetcites [11]
- Not checked yetcites [16]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Multiaccuracy auditing makes no assumptions about the original classifier; in particular, it can handle inadvertent and malicious forms of discrimination.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 1 INTRODUCTION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, the proof of Theorem 2 reveals that this worst-case bound is very pessimistic and can be improved with stronger assumptions.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Do no harm.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the classification error guarantee of Proposition 1 improves as γ , the density of the protected subpopulation S, grows.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 2.3 Classification accuracy from multiaccuracy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, as described, the algorithm evaluates statistics like E x ∼D [h(x)•(f (x)-y(x))], which we can estimate accurately and efficiently from a small sample.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Given:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Our post-processing algorithm, Multiaccuracy Boost, builds on the algorithm given in [13], providing the additional”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
What the sources say
- Not checked yetcites [13]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even if this information is available, a priori, it may not always be obvious which subpopulations require special attention.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The multiaccuracy framework can be applied very broadly; importantly, we can post-process any initial model f 0 given only blackbox access to f 0 and a small set of labeled validation data.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.2 Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Our work directly extends the approach of [13], adapting their work to the binary classification setting.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
What the sources say
- Not checked yetcites [13]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“In particular, multiaccuracy post-processing may be applicable in domain adaptation settings, particularly under label distribution shift as studied recently in [22], but when the learner gets a small number of labeled samples from the new distribution.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
What the sources say
- Not checked yetcites [22]
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our setting, we do not restrict ourselves to an explicitly-defined hypothesis class, so we can augment the current model using the auditor; these augmentations improve the accuracy of the model.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“At a technical level, our post-processing algorithm is similar to gradient boosting [10,24].”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
What the sources say
- Not checked yetcites [10]
- Not checked yetcites [24]
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Rather than using an expressive class of predictors as the base classifiers to learn the function directly, we focus on the regime where data is limited and we must restrict our attention to simple classes.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“First, decoupling the classification problem requires that we have the attributes of interest in the data set and that the groups we wish to protect are partitioned by these attributes.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.1 Related works
What the sources say
- Not checked yetcites [8]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate the effectiveness of Multiaccuracy Boost in two other settings: the adult income prediction task and a semisynthetic disease prediction task.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > A ADDITIONAL EXPERIMENTS
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Multiaccuracy provides a framework for fairness in classification by improving the accuracy in identifiable subgroups, in a way that suffers no tradeoff between accuracy and utility.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.2 Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate -both theoretically and empirically -that post-processing serves as an effective tool for improving the accuracy across important subpopulations, and does not harm the populations that are already classified well.”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > 5.2 Conclusion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
The important subpopulations are sufficiently represented in the validation data distribution D.
“we assume that the important subpopulations are sufficiently represented on D”
The authors state this as an assumption.
The auditing algorithm A uses a fresh sample D_t ∼ D^m per iteration.
“we assume that A uses a fresh sample D t ∼ D m per iteration.”
The authors state this as an assumption.
The auditing algorithm A can identify a very rich class of overlapping sets.
“As we assume A can identify a very rich class of overlapping sets”
The paper relies on this without support.
Data from certain minority populations is less available than from the majority population.
“often for historical and social reasons, data from certain minority populations is less available than from the majority population.”
The paper relies on this without support.
In the common setting where the learned model is provided as a service, there may not be sufficient incentive for the service provider to retrain the model.
“there may not be sufficient incentive (financial, social, etc.) for the service provider to retrain the model.”
The paper relies on this without support.
In many applications, race or more granular categories are not given.
“This is not feasible in the many applications where, say, race or more granular categories are not given.”
The paper relies on this without support.
Even when subgroups are identified, there are often not enough samples to train accurate classifiers on each subgroup separately.
“Even when the subgroups are identified, we often do not have enough samples to train accurate classifiers on each subgroup separately.”
The paper relies on this without support.
The auditor is bounded.
“if the mistakes appear overly-complicated to the bounded auditor”
The authors state this as an assumption.
A agnostic learns a class C ⊆ [-1, 1]^X of dimension d(C).
“suppose A agnostic learns a class C ⊆ [-1, 1] X of dimension d(C).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“[170,175] undertake a similar approach under diferent settings.”
From Fairness in machine learning: A survey (2024) · cites this paper as [175]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Adult Income Prediction.
“SS 19.7 9.5 24.6 10.5 19.9 5.5 25.3”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Adult Income Prediction.
“All F M B W BF WM D 100 32.3 67.7 9.7 90.3 4.8 62.9”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > □
“≤ Pr x ∼S [x ∈ S 0 ] • 3τ 0 + 2β Pr x ∼S [x ∈ S 0 ] + Pr x ∼S [x ∈ S 1 ] • 3τ 1 + 2β Pr x ∼S [x ∈ S 1 ](16)”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > □
“= 3 • Pr x ∼S [x ∈ S 0 ] • τ 0 + Pr x ∼S [x ∈ S 1 ] • τ 1 + 4β(17)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Adult Income Prediction.
“MA 14.7 7.2 18.3 9.4 15.0 4.5 18.3”
§ Multiaccuracy: Black-Box Post-Processing for Fairness in Classification > Adult Income Prediction.
“All F M B W BF WM D 100 32.3 67.7 9.7 90.3 4.8 62.9”
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
(2018) · Cited by 3,510
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Research works printed but not linked: 3
- Angwin , J. , Larson , J. , Mattu , S. , and Kirchner , L . Machine bias: There's software used across the country to predict future criminals. and it's biased against blacks. ProPublica ( 2016 ). Angwin, J., Larson, J., Mattu, S., and Kirchner, L. Machine bias: There's software used across the country to predict future criminals. and it's biased against blacks. ProPublica (2016).
- Chouldechova , A. , and G'Sell , M. Fairer and more accurate, but for whom? FATML ( 2016 ). Chouldechova, A., and G'Sell, M. Fairer and more accurate, but for whom? FATML (2016).
- Lipton , Z. C. , Wang , Y. , and Smola , A. J . Detecting and correcting for label shift with black box predictors. ICML ( 2018 ). Lipton, Z. C., Wang, Y., and Smola, A. J. Detecting and correcting for label shift with black box predictors. ICML (2018).
Cited by
Sorted by citations
Items 101–190 of 190
Reducing bias in healthcare artificial intelligence: A white paper
Sun, C., & Harris, S. L. (2024) · Health Informatics Journal · doi:10.1177/14604582241291410 · Cited by 6
When Small Decisions Have Big Impact: Fairness Implications of Algorithmic Profiling Schemes
Kern, C. A., Bach, R. L., Mautner, H., & Kreuter, F. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3689485 · Cited by 10
Who's Responsible? Jointly Quantifying the Contribution of the Learning Algorithm and Data
Yona, G., Ghorbani, A., & Zou, J. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462574 · Cited by 6
Self-citationReduced-Bias Co-trained Ensembles for Weakly Supervised Cyberbullying Detection
Raisi, E., & Huang, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34980-6_32 · Cited by 6
Individually Fair Gradient BoostingPreprint
Vargo, A., Zhang, F., Yurochkin, M., & Sun, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16785 · Cited by 6
VLSlice: Interactive Vision-and-Language Slice Discovery
Slyman, E., Kahng, M., & Lee, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01403 · Cited by 5
Islam, M. A., Nag, A., Roy, N., Dey, A. R., Fahim, S. F. A., & Ghosh, A. (2023) · International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) · doi:10.1109/icccis60361.2023.10425394 · Cited by 5
Leveraging Simulation Data to Understand Bias in Predictive Models of Infectious Disease Spread
Züfle, A. E., Salim, F. D., Anderson, T., Scotch, M., Xiong, L., Sokol, K., Xue, H., Kong, R., Heslop, D. J., Paik, H.-Y., et al. (2024) · ACM Transactions on Spatial Algorithms and Systems · doi:10.1145/3660631 · Cited by 5
Sukumar, A., Desai, A., Singhal, P., Gokhale, S., Jain, D. K., Walambe, R. A., & Kotecha, K. V. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10582007 · Cited by 5
mcboost: Multi-Calibration Boosting for R
Pfisterer, F., Kern, C. A., Dandl, S., Sun, M. Z., Kim, M., & Bischl, B. (2021) · The Journal of Open Source Software · doi:10.21105/joss.03453 · Cited by 5
Self-citationFairness-Aware Vision Transformer via Debiased Self-Attention
Qiang, Y., Li, C., Khanduri, P., & Zhu, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_20 · Cited by 8
Weakly Supervised Machine Learning for Cyberbullying Detection
Raisi, E. (2019) · VTechWorks (Virginia Tech) · Cited by 5
Fairness Meets Cross-Domain Learning: A Benchmark of Models and Metrics
Iurada, L., Bucci, S., Hospedales, T. M., & Tommasi, T. (2024) · IEEE Access · doi:10.1109/access.2024.3383841 · Cited by 5
Wang, H., Chen, W., Luo, X., Xing, Z., Liu, L., Qin, J., Wu, S., & Zhu, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02234 · Cited by 4
On the Richness of Calibration
Höltgen, B. T. A., & Williamson, R. C. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594068 · Cited by 5
Complexity-Theoretic Implications of Multicalibration
Casacuberta, S., Dwork, C., & Vadhan, S. (2024) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3618260.3649748 · Cited by 11
Schenk, P. O., & Kern, C. A. (2024) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-024-00344-2 · Cited by 11
FairPlay: A Collaborative Approach to Mitigate Bias in Datasets for Improved AI Fairness
Behzad, T., Singh, M. K., Ripa, A. J., & Mueller, K. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710982 · Cited by 4
Building Nondiscriminatory Algorithms in Selected Data
Arnold, D. S., Dobbie, W., & Hull, P. (2025) · American Economic Review Insights · doi:10.1257/aeri.20240249 · Cited by 4
Kobayashi, K., & Nakao, Y. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-87687-6_5 · Cited by 4
Multicalibrated Regression for Downstream Fairness
Globus-Harris, I., Gupta, V., Jung, C., Kearns, M. J., Morgenstern, J., & Roth, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604683 · Cited by 4
Barbierato, E., Pozzi, A., & Tessera, D. (2023) · IEEE Access · doi:10.1109/access.2023.3325235 · Cited by 5
Deep learning for biomedical videos: perspective and recommendations
Ouyang, D., Wu, Z., He, B. D., & Zou, J. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-821259-2.00003-x · Cited by 4
Self-citationAn Empirical Study on Group Fairness Metrics of Judicial Data
Li, Y., Huang, H., Guo, X., & Yuan, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3122443 · Cited by 3
Efficient Bias Mitigation Without Privileged Information
Zarlenga, M. E., Sankaranarayanan, S., Andrews, J. T. A., Shams, Z., Jamnik, M., & Xiang, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73220-1_9 · Cited by 5
Guidelines for designing visualization tools for group fairness analysis in binary classification
Cruz, A., Salazar, T., Carvalho, M., Maç�ãs, C., Machado, P., & Abreu, P. H. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11179-w · Cited by 3
Pérez, Á. P., Fernández, M., Al-Madfai, H., Burel, G., & Alani, H. (2023) · Web Science Conference · doi:10.1145/3578503.3583605 · Cited by 1
Fairmod: making predictions fair in multiple protected attributes
Liu, J., Li, J., Liu, L., Le, T. D., Ye, F., & Li, G. (2023) · Knowledge and Information Systems · doi:10.1007/s10115-023-02003-4 · Cited by 3
Poe, R. L., & Mestari, S. Z. E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659015 · Cited by 9
Augmented Fairness: An Interpretable Model Augmenting Decision-Makers' FairnessPreprint
Wang, T., & Saar‐Tsechansky, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08398 · Cited by 3
Dzakpasu, D. Q., Liu, J., Li, J., & Liu, L. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679802 · Cited by 2
Toward substantive intersectional algorithmic fairness: desiderata for a feminist approach
Mirsch, M., Wegner, L., Strube, J., & Leicht-Scholten, C. (2026) · AI and Ethics · doi:10.1007/s43681-025-00926-y · Cited by 3
van Stijn, J., Neerincx, M. A., ten Teije, A., Vethman, S., Martin, A., Hinkelmann, K., Fill, H.-G., Gerber, A., Lenat, D. B., Stolle, R., et al. (2021) · Research Repository (Delft University of Technology) · Cited by 2
Delving into Identify-Emphasize Paradigm for Combating Unknown Bias
Zhao, B., Chen, C., Wang, Q.-W., He, A., & Xia, S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01969-6 · Cited by 2
Model ChangeLists: Characterizing Updates to ML Models
Eyuboglu, S., Goel, K., Desai, A., Chen, L., Monfort, M., Ré, C., & Zou, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659047 · Cited by 6
Self-citationReview on fairness in image recognition
Wang, M., Deng, W., & Su, S. (2024) · Journal of Image and Graphics · doi:10.11834/jig.230226 · Cited by 2
Unsupervised Search for Ethnic Minorities’ Medical Segmentation Training Set
Chen, Y., Yao, Y., Yang, R., Hossain, M. Z., Gupta, A., & Gedeon, T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888593 · Cited by 2
Fairness in Machine Learning for National Statistical Organizations
Schenk, P. O., Kern, C. A., & Kreuter, F. (2025) · Society, environment and statistics · doi:10.1007/978-3-032-10004-7_6 · Cited by 2
A survey on optimization and machine learning-based fair decision making in healthcare
Chen, Z., & Marrero, W. J. (2026) · Health Care Management Science · doi:10.1007/s10729-025-09749-z · Cited by 2
Fairness-Aware Active Learning for Decoupled Model
Cao, Y., & Lan, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892608 · Cited by 2
Tutorial on Fair and Private Deep Learning
Padala, M., Damle, S., & Gujar, S. (2024) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3632410.3633294 · Cited by 2
Fairness Constraints in Semi-supervised LearningPreprint
Zhang, T., Zhu, T., Han, M., Li, J., Zhou, W., & Yu, P. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06190 · Cited by 7
Kobayashi, K., & Nakao, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13494 · Cited by 7
Fair for All: Best-effort Fairness Guarantees for ClassificationPreprint
Krishnaswamy, A. K., Jiang, Z., Wang, K., Cheng, Y., & Munagala, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10216 · Cited by 3
Online Multivalid Learning: Means, Moments, and Prediction IntervalsPreprint
Gupta, V., Jung, C., Noarov, G., Pai, M. M., & Roth, A. (2021) · RePEc: Research Papers in Economics · doi:10.4230/lipics.itcs.2022.82 · Cited by 19
Multi-group Agnostic PAC LearnabilityPreprint
Rothblum, G. N., & Yona, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09989 · Cited by 2
Calibrated Variant Effect Prediction at the Residue Level Using Conditional Score DistributionsPreprint
Passi, G., Amittai, S., & Schneidman‐Duhovny, D. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.24.690189 · Cited by 2
Discover, Explain, Improve: An Automatic Slice Detection Benchmark for Natural Language Processing
Hua, W., Jin, L., Song, L., Mi, H., Zhang, Y., & Yu, D. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00617 · Cited by 4
Reducing Bias in Face Recognition
Gong, S., Liu, X., & Jain, A. K. (2023) · Springer eBooks · doi:10.1007/978-3-031-43567-6_12 · Cited by 1
Suk, Y., & Han, K. T. (2024) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986241269820 · Cited by 3
Bridging Data and AIOps for Future AI Advancements with Human-in-the-Loop. The AI-DAPT Concept
Koussouris, S., Dalamagas, T. M., Figueiras, P. A., Pallis, G., Bountouni, N., Gkolemis, V., Perakis, Κ., Bibikas, D., & Agostinho, C. (2024) · IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) · doi:10.1109/ice/itmc61926.2024.10794334 · Cited by 1
How robust is your fair model? Exploring the robustness of prominent fairness strategies
Small, E. A., Shao, W., Zhang, Z., Liu, P., Chan, J., Sokol, K., & Salim, F. D. (2025) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-025-01112-8 · Cited by 1
Correa, G. L., Campello, B. S. C., & Duarte, L. T. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111552 · Cited by 1
Inference-Time Rule Eraser: Fair Recognition via Distilling and Removing Biased Rules
Zhang, Y., Lu, D., & Sang, J. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3654438 · Cited by 3
Nearest-Neighbor Density Estimation for Dependency Suppression
Anderson, K., & Martinetz, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31663-9_44 · Cited by 1
Personalized Regularization Learning for Fairer Matrix Factorization
Yao, S., & Huang, B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75765-6_48 · Cited by 1
Learning Flexible and Fair Data Representations
Usama, M., & Chang, D. E. (2022) · IEEE Access · doi:10.1109/access.2022.3207747 · Cited by 1
Dzakpasu, D. Q., Liu, J., Li, J., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0348-0_11 · Cited by 1
Predict to Minimize Swap Regret for All Payoff-Bounded Tasks
Hu, L., & Wu, Y. (2024) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs61266.2024.00024 · Cited by 1
LLM Enhanced Machine Learning Estimators for Classification
Wu, Y., Wang, Y., Wang, C., & Zheng, Z. (2024) · Winter Simulation Conference · doi:10.1109/wsc63780.2024.10838779 · Cited by 1
Impact on bias mitigation algorithms to variations in inferred sensitive attribute uncertainty
Wang, Y., & Singh, L. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1520330 · Cited by 5
Omnipredicting Single-Index Models with Multi-index Models
Hu, L., Tian, K., & Yang, C. (2025) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3717823.3718223 · Cited by 1
Fairness-aware Outlier EnsemblePreprint
Liu, H., Ma, F., He, S., Chen, J., & Gao, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.09419 · Cited by 6
Multicalibrated Partitions for Importance WeightsPreprint
Gopalan, P., Reingold, O., Sharan, V., & Wieder, U. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05853 · Cited by 3
Consider the Alternatives: Navigating Fairness-Accuracy Tradeoffs via DisqualificationPreprint
Rothblum, G. N., & Yona, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00813
Meaningfully Explaining Model Mistakes Using Conceptual Counterfactuals.Preprint
Abid, A., Yüksekgönül, M., & Zou, J. (2021) · arXiv (Cornell University) · Cited by 1
Self-citationEmpirical analysis of fairness-aware data segmentation
Okura, S., & Mohri, T. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00029
Indistinguishable Predictions and Multi-group Fair Learning
Rothblum, G. N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30545-0_1
FairFix: Enhancing Fairness of Pre-Trained Deep Neural Networks with Scarce Data Resources
Li, Z., Zhu, R., Wang, Z., Li, J., Liu, K., Qin, Y., Fan, Y., Gu, M., Lu, Z., Wu, J., et al. (2024) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/ids62739.2024.00010
Mutual Information-Based Counterfactuals for Fair Graph Neural Networks
Chen, L. F., Chen, W., Li, J., & Yu, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650076
Intersectional consequences for marginal fairness in prediction models for emergency admissionsPreprint
Lett, E., Shahbandegan, S., Barak‐Corren, Y., Fine, A., & Cava, W. L. (2024) · medRxiv · doi:10.1101/2024.11.05.24316769
He, F., & Tao, D. (2025) · Machine learning · doi:10.1007/978-981-16-8233-9_12
Liu, Y., Yao, R., Liu, Y., Zong, R., Li, Z., Zhang, Y., & Wang, D. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732066
Towards a fairer vision: addressing class imbalance in image gender recognition
Barahona, D., Quijano-Sánchez, L., & Liberatore, F. (2025) · Machine Vision and Applications · doi:10.1007/s00138-025-01732-6 · Cited by 1
Multi-Group Proportional Representation for Text-to-Image Models
Jung, S., Oesterling, A. X., Verdun, C. M., Vithana, S., Moon, T., & Calmon, F. (2025) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.02211 · Cited by 6
Multicalibration in Fair Link Prediction
Pal, M., Sikdar, S., & Ganguly, N. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3600263
Value of diversity characteristics in predictive modeling: ACS screening as a case study
Bunney, G., Miller, K., Ryu, K., Kabeer, R., Graber‐Naidich, A., Pasao, M. A., Rizvi, M., & Yiadom, M. Y. A. B. (2025) · npj Cardiovascular Health · doi:10.1038/s44325-025-00088-0
Generalized and Unified Equivalences Between Hardness and Pseudoentropy
Hu, L., & Vadhan, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-12290-2_9
A Credit Score in Pay Later Service by Using Machine Learning Optimization: A Bibliometric Analysis
Arisandy, Y., Dasri, Y., & Salahudin, S. N. (2026) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-032-01309-5_3
Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
Du, X., Ramamoorthy, S., Duivesteijn, W., Tian, J. W., & Pechenizkiy, M. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00126
Fair Learning : an optimal transport based approach
Gordaliza, P. (2020) · theses.fr (ABES) · doi:10.70675/9b44aae6z5b37z47d7zae40ze61595fa014d
Semi-supervised learning in insurance : fairness and active learning
Hu, F. (2022) · theses.fr (ABES) · doi:10.70675/f9cec72az9beaz4c9bz9436z152694ee9162 · Cited by 1
Toward Robust and Fair Partial Label Federated Learning Service
Fu, S., Bao, W., Yang, Z., Sheng, X., Guo, Y., Liu, Y., & Zhou, B. B. (2026) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2026.3686458
Fairness-Aware Machine Learning Framework for Bias Mitigation in Large-Scale Data Analytics
Salotagi, S., Sardar, T. H., Pal, D., Madhumala, R. B., Hole, S. R., & Sridhar, S. K. (2026) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei68323.2026.11542187
Bothmann, L., Boustani, P. A., Álvarez, J. M., Casalicchio, G., Bischl, B., & Dandl, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812340 · Cited by 2
Multicalibration for LLM-based Code Generation
Campos, V., Kuschnereit, R., & Ulges, A. (2026) · doi:10.1145/3786175.3788347
Lee, Y.-C., Li, J., Zhang, R., & Liu, N. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009625180.002
Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts
Wilkins‐Reeves, S., Darmon, A. N. M., & Sinha, D. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818094
Wang, T., Vivas-Valencia, C., Bauer, C., Gong, Y., Choo, K.-K. R., & Guo, Y. (2026) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-026-00251-x
Dual-Calibrated Venn Predictors for Detecting Localized Bias
Ståhl, N., Löfström, T., García‐Galindo, A., & Johansson, U. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37976-4_20
Metadata
- Authors
- Kim, M. P., Ghorbani, A., & Zou, J.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2019-01-27
- Pages
- 247-254
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W2962925443
- Cited by
- 193
- References
- 37
- Citations by year
- 2026: 13 · 2025: 24 · 2024: 34 · 2023: 21 · 2022: 32 · 2021: 35 · 2020: 25 · 2019: 8
- FWCI
- 25.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, other-oa)
- Funders
- NSF (CCF-1763299)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, algorithmic auditing, post-processing, black-box access, algorithmic bias, fairness and accountability, image classification, credit scoring, population health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
