CoChat
Sign Up
Cited by 85Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Machine Learning and Data Classification

Model Multiplicity: Opportunities, Concerns, and Solutions

Open access
Black, Emily, Raghavan, Manish, Barocas, Solon·2022·2022 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3531146.3533149
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent scholarship has brought attention to the fact that there often exist multiple models for a given prediction task with equal accuracy that differ in their individual-level predictions or aggregate properties. This phenomenon---which we call model multiplicity---can introduce a good deal of flexibility into the model selection process, creating a range of exciting opportunities. By demonstrating that there are many different ways of making equally accurate predictions, multiplicity gives model developers the freedom to prioritize other values in their model selection process without having to abandon their commitment to maximizing accuracy. However, multiplicity also brings to light a concerning truth: model selection on the basis of accuracy alone---the default procedure in many deployment scenarios---fails to consider what might be meaningful differences between equally accurate models with respect to other criteria such as fairness, robustness, and interpretability. Unless these criteria are taken into account explicitly, developers might end up making unnecessary trade-offs or could even mask intentional discrimination. Furthermore, the prospect that there might exist another model of equal accuracy that flips a prediction for a particular individual may lead to a crisis in justifiability: why should an individual be subject to an adverse model outcome if there exists an equally accurate model that treats them more favorably? In this work, we investigate how to take advantage of the flexibility afforded by model multiplicity while addressing the concerns with justifiability that it might raise.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3531146.3533149

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • If some Rashomon models predict positive and others negative for an individual, this provides a meaningful basis for contesting the decision.

    “If some predictions are positive while others are negative, this provides a meaningful basis for contesting the decision, as argued by Black, Raghavan, and Barocas (2022) and formalized in the following theorem.”

    From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Black, Raghavan, and Barocas 2022

  • Non-epistemic values, such as fairness or interpretability concerns, can guide model selection in such situations.

    “Indeed, non-epistemic values(Longino 2025), such as fairness or interpretability concerns, can guide model selection in such situations, an idea now widely discussed in the model multiplicity literature(Black, Raghavan, and Barocas 2022;Xin et al. 2022;Rudin et al. 2024;Meyer et al. 2025;Höltgen and Oliver 2025).”

    From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Black, Raghavan, and Barocas 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 50

  1. Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. 2018. A Reductions Approach to Fair Classification. In International Conference on Machine Learning. 60-69.
  2. Christopher Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Müller, and Pan Kessel. 2020. Fairwashing explanations with off-manifold detergent. In International Conference on Machine Learning. PMLR, 314-323.
  3. Solon Barocas and Andrew D Selbst. 2016. Big data's disparate impact. California Law Review 104 (2016), 671-732.
  4. Solon Barocas, Andrew D Selbst, and Manish Raghavan. 2020. The hidden assump- tions behind counterfactual explanations and principal reasons. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 80-89.
  5. Dimitris Bertsimas, Arthur Delarue, Patrick Jaillet, and Sebastien Martin. 2019. The price of interpretability. arXiv preprint arXiv:1907.03419 (2019). · arXiv
  6. Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2021. The values encoded in machine learning research. arXiv preprint arXiv:2106.15590 (2021). · arXiv
  7. Emily Black and Matt Fredrikson. 2021. Leave-one-out Unfairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 285-295.
  8. Emily Black, Klas Leino, and Matt Fredrikson. 2022. Selective Ensembles for Consistent Predictions. In International Conference on Learning Representations.
  9. Emily Black, Zifan Wang, Matt Fredrikson, and Anupam Datta. 2022. Consistent Counterfactuals for Deep Models. In International Conference on Learning Representations.
  10. Leo Breiman. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 (2001), 199-231.
  11. Lisa Schultz Bressman. 2003. Beyond accountability: Arbitrariness and legitimacy in the administrative state. NYUL Rev. 78 (2003), 461.
  12. Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang. 2018. An interpretable model with globally consistent explanations for credit risk. arXiv preprint arXiv:1811.12615 (2018). · arXiv
  13. Danielle Keats Citron. 2007. Technological due process. Wash. L Rev. 85 (2007), 1249.
  14. Danielle Keats Citron and Frank Pasquale. 2014. The scored society: Due process for automated predictions. Wash. L. Rev. 89 (2014), 1.
  15. A Feder Cooper and Ellen Abrams. 2021. Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 46-54.
  16. Amanda Coston, Ashesh Rambachan, and Alexandra Chouldechova. 2021. Characterizing Fairness Over the Set of Good Models Under Selective Labels. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139). PMLR, 2144-2155.
  17. Kate Crawford and Jason Schultz. 2014. Big data and due process: Toward a framework to redress predictive privacy harms. BCL Rev. 55 (2014), 93.
  18. Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. 2020. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 (2020). · arXiv
  19. Anupam Datta, Matt Fredrikson, Gihyuk Ko, Piotr Mardziel, and Shayak Sen. 2017. Proxy Discrimination in Data-Driven Systems. arXiv preprint arXiv:1707.08120 (2017). · arXiv
  20. Pedro Domingos. 2000. A unified bias-variance decomposition. In Proceedings of 17th International Conference on Machine Learning. 231-238.
  21. Jiayun Dong and Cynthia Rudin. 2019. Variable importance clouds: A way to explore variable importance for the set of good models. arXiv preprint arXiv:1901.03209 (2019). · arXiv
  22. David Donoho. 2017. 50 years of data science. Journal of Computational and Graphical Statistics 26, 4 (2017), 745-766.
  23. Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, and Kush Varshney. 2020. Is there a trade-off between fairness and accuracy? a perspective using mismatched hypothesis testing. In International Conference on Machine Learning. PMLR, 2803-2813.
  24. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference. 214-226.
  25. Aaron Fisher, Cynthia Rudin, and Francesca Dominici. 2019. All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously. J. Mach. Learn. Res. 20, 177 (2019), 1-81.
  26. Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2020. A survey of algorithmic recourse: definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04050 (2020). · arXiv
  27. Pauline T. Kim. 2022. Race-aware algorithms: Fairness, nondiscrimination and affirmative action. California Law Review 110 (2022).
  28. Jon Kleinberg and Manish Raghavan. 2021. Algorithmic monoculture and social welfare. Proceedings of the National Academy of Sciences 118, 22 (2021).
  29. Ron Kohavi, David H Wolpert, et al. 1996. Bias plus variance decomposition for zero-one loss functions. In ICML, Vol. 96. 275-83.
  30. David Lehr and Paul Ohm. 2017. Playing with the data: what legal scholars should learn about machine learning. UCDL Rev. 51 (2017), 653.
  31. Charles T. Marx, Flávio P. Calmon, and Berk Ustun. 2020. Predictive Multiplicity in Classification. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). PMLR, 6765-6774.
  32. Johannes Mehrer, Courtney J Spoerer, Nikolaus Kriegeskorte, and Tim C Kietzmann. 2020. Individual differences among deep neural network models. Nature communications 11, 1 (2020), 1-12.
  33. Aditya Krishna Menon and Robert C Williamson. 2018. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency. PMLR, 107-118.
  34. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447-453.
  35. Martin Pawelczyk, Klaus Broelemann, and Gjergji. Kasneci. 2020. On Counter- factual Explanations under Predictive Multiplicity. In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) (Proceedings of Machine Learning Research).
  36. Ronen Perry and Tal Z Zarsky. 2014. May the Odds Be Ever in Your Favor: Lotteries in Law. Ala. L. Rev. 66 (2014), 1035.
  37. Manish Raghavan and Solon Barocas. 2019. Challenges for mitigating bias in algorithmic hiring. Brookings. Retrieved February 25 (2019), 2020.
  38. Xavier Renard, Thibault Laugel, and Marcin Detyniecki. 2021. Understanding Prediction Discrepancies in Machine Learning Classifiers. arXiv preprint arXiv:2104.05467 (2021). · arXiv
  39. Michael L Rich. 2016. Machine learning, automated suspicion algorithms, and the fourth amendment. University of Pennsylvania Law Review (2016), 871-929.
  40. Kit T Rodolfa, Erika Salomon, Lauren Haynes, Iván Higuera Mendieta, Jamie Larson, and Rayid Ghani. 2020. Case study: predictive fairness to reduce misdemeanor recidivism through social service interventions. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 142-153.
  41. Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (2019), 206-215.
  42. Andrew D Selbst and Solon Barocas. 2018. The intuitive appeal of explainable machines. Fordham L. Rev. 87 (2018), 1085.
  43. Lesia Semenova, Cynthia Rudin, and Ronald Parr. 2019. A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:1908.01755 (2019). · arXiv
  44. Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding machine learning: From theory to algorithms. Cambridge university press.
  45. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. 2019. Robustness may be at odds with accuracy. In International Conference on Learning Representations.
  46. Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable Recourse in Linear Classification. In Conference on Fairness, Accountability, and Transparency. 10-19.
  47. Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2018. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology 31, 2 (2018), 841-887.
  48. Tong Wang. 2019. Gaining free or low-cost interpretability with interpretable par- tial substitute. In International Conference on Machine Learning. PMLR, 6505-6514.
  49. Michael Wick, swetasudha panda, and Jean-Baptiste Tristan. 2019. Unlocking Fairness: a Trade-off Revisited. In Advances in Neural Information Processing Systems, Vol. 32.
  50. Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. 2019. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning. PMLR, 7472-7482.

Cited by

Sorted by citations

Authors
Black, E., Raghavan, M., & Barocas, S.
Affiliations
  • Carnegie Mellon University (US)
  • Harvard University (US)
  • Microsoft (United States) (US)
ORCID
Venue
2022 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2022-06-20
Pages
850-863
DOI
10.1145/3531146.3533149
OpenAlex
W4283169532
Cited by
85
References
5
Citations by year
2026: 23 · 2025: 24 · 2024: 26 · 2023: 10 · 2022: 1 · 2021: 1
FWCI
13.94
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-sa)
  • SSRN Electronic Journal (acceptedVersion)
Funders
  • Harvard School of Engineering and Applied Sciences
  • Amazon Graduate Research Fellowship
  • Microsoft Research
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
model multiplicity, model selection, fairness, robustness, interpretability, justifiability, personalized prediction, algorithmic discrimination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review