CoChat
Sign Up
PreprintCited by 2Cryptography and Data SecurityEthics and Social Impacts of AIPrivacy-Preserving Technologies in Data

FairProof : Confidential and Certifiable Fairness for Neural Networks

Open access
Yadav, Chhavi, Chowdhury, Amrita Roy, Boneh, Dan, Chaudhuri, Kamalika·2024·arXiv
DOI 10.48550/arxiv.2402.12572arXiv 2402.12572
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning models are increasingly used in societal applications, yet legal and privacy concerns demand that they very often be kept confidential. Consequently, there is a growing distrust about the fairness properties of these models in the minds of consumers, who are often at the receiving end of model predictions. To this end, we propose \name -- a system that uses Zero-Knowledge Proofs (a cryptographic primitive) to publicly verify the fairness of a model, while maintaining confidentiality. We also propose a fairness certification algorithm for fully-connected neural networks which is befitting to ZKPs and is used in this system. We implement \name in Gnark and demonstrate empirically that our system is practically feasible. Code is available at https://github.com/infinite-pursuits/FairProof.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.12572

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 50

  1. Aws Albarghouthi, Loris D'Antoni, Samuel Drews, and Aditya V. Nori. Fairsquare: Probabilistic verification of program fairness. Proc. ACM Program. Lang., 1(OOPSLA), oct 2017. doi: 10.1145/3133904. URL https://doi.org/10.1145/3133904. · DOI · Paper page
  2. Osbert Bastani, Xin Zhang, and Armando Solar-Lezama. Probabilistic verification of fairness properties via concentration. Proc. ACM Program. Lang., 3(OOPSLA), oct 2019. doi: 10.1145/ 3360544. URL https://doi.org/10.1145/3360544. · DOI · Paper page
  3. Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5XW20. · DOI
  4. Elias Benussi, Andrea Patané, Matthew Wicker, Luca Laurenti, Marta Kwiatkowska University of Oxford, and Tu Delft. Individual fairness guarantees for neural networks. In International Joint Conference on Artificial Intelligence, 2022. URL https://api.semanticscholar.org/ CorpusID:248722046.
  5. Marianne Bertrand and Sendhil Mullainathan. Are emily and greg more employable than lakisha and jamal? a field experiment on labor market discrimination. American Economic Review, 94(4): 991-1013, September 2004. doi: 10.1257/0002828042002561. URL https://www.aeaweb. org/articles?id=10.1257/0002828042002561. · DOI · Paper page
  6. Sumon Biswas and Hridesh Rajan. Fairify: Fairness verification of neural networks. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pp. 1546-1558, 2023. doi: 10.1109/ICSE48619.2023.00134. · DOI
  7. Fabian Boemer, Rosario Cammarota, Daniel Demmler, Thomas Schneider, and Hossein Yalame. Mp2ml: A mixed-protocol machine learning framework for private inference. In Proceedings of the 15th international conference on availability, reliability and security, pp. 1-10, 2020.
  8. Tim Brennan, William Dieterich, and Beate Ehret. Evaluating the predictive validity of the compas risk and needs assessment system. Criminal Justice and Behavior, 36(1):21-40, 2009. doi: 10.1177/0093854808326545. URL https://doi.org/10.1177/0093854808326545. · DOI
  9. Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, et al. Black-box access is insufficient for rigorous ai audits. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pp. 2254-2272, 2024.
  10. Francesco Croce, Maksym Andriushchenko, and Matthias Hein. Provable robustness of relu networks via maximization of linear regions. In the 22nd International Conference on Artificial Intelligence and Statistics, pp. 2057-2066. PMLR, 2019.
  11. Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination. ArXiv, abs/1408.6491, 2014. URL https://api.semanticscholar.org/CorpusID:6817607. · arXiv
  12. Cynthia Dwork and Martha Minow. Distrust of artificial intelligence: Sources & responses from computer science & law. Daedalus, 151(2):309-321, 2022.
  13. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS '12, pp. 214-226, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450311151. doi: 10.1145/2090236.2090255. URL https://doi.org/10.1145/ 2090236.2090255. · DOI · Paper page
  14. Boyuan Feng, Lianke Qin, Zhenfei Zhang, Yufei Ding, and Shumo Chu. Zen: Efficient zero- knowledge proofs for neural networks. IACR Cryptol. ePrint Arch., 2021:87, 2021. URL https: //api.semanticscholar.org/CorpusID:231731893.
  15. Bishwamittra Ghosh, D. Basu, and Kuldeep S. Meel. Justicia: A stochastic sat approach to formally verify fairness. In AAAI Conference on Artificial Intelligence, 2020. URL https://api. semanticscholar.org/CorpusID:221655566.
  16. Oded Goldreich, Silvio Micali, and Avi Wigderson. Proofs that yield nothing but their validity or all languages in np have zero-knowledge proof systems. J. ACM, 38(3):690-728, jul 1991. ISSN 0004-5411. doi: 10.1145/116825.116852. URL https://doi.org/10.1145/116825. 116852. · DOI
  17. S Goldwasser, S Micali, and C Rackoff. The knowledge complexity of interactive proof-systems. In Proceedings of the Seventeenth Annual ACM Symposium on Theory of Computing, STOC '85, pp. 291-304, New York, NY, USA, 1985. Association for Computing Machinery. ISBN 0897911512. doi: 10.1145/22145.22178. URL https://doi.org/10.1145/22145.22178. · DOI
  18. Faisal Hamman, Jiahao Chen, and Sanghamitra Dutta. Can querying for bias leak protected attributes? achieving privacy with smooth sensitivity. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 1358-1368, 2023.
  19. Boris Hanin and David Rolnick. Deep relu networks have surprisingly few activation patterns. Advances in neural information processing systems, 32, 2019.
  20. Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994. DOI: https://doi.org/10.24432/C5NC77. · DOI · Paper page
  21. Rashidul Islam, Shimei Pan, and James R Foulds. Can we obtain fairness for free? In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 586-596, 2021.
  22. Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. {GAZELLE}: A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1651-1669, 2018.
  23. Mintong Kang, Linyi Li, Maurice Weber, Yang Liu, Ce Zhang, and Bo Li. Certifying some distributional fairness with subpopulation decomposition, 2022b. Amir E. Khandani, Adlar J. Kim, and Andrew W. Lo. Consumer credit-risk models via machine- learning algorithms. Journal of Banking & Finance, 34(11):2767-2787, 2010. ISSN 0378-4266. doi: https://doi.org/10.1016/j.jbankfin.2010.06.001. URL https://www.sciencedirect. com/science/article/pii/S0378426610002372. · DOI · Paper page
  24. Niki Kilbertus, Adria Gascon, Matt Kusner, Michael Veale, Krishna Gummadi, and Adrian Weller. Blind justice: Fairness with encrypted sensitive attributes. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2630-2639. PMLR, 10-15 Jul 2018. URL https://proceedings.mlr.press/v80/kilbertus18a.html.
  25. Seunghwan Lee, Hankyung Ko, Jihye Kim, and Hyunok Oh. vcnn: Verifiable convolutional neural net- work. IACR Cryptol. ePrint Arch., 2020:584, 2020. URL https://api.semanticscholar. org/CorpusID:218895602.
  26. Jian Liu, Mika Juuti, Yao Lu, and Nadarajah Asokan. Oblivious neural network predictions via minionn transformations. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp. 619-631, 2017.
  27. Tianyi Liu, Xiang Xie, and Yupeng Zhang. zkcnn: Zero knowledge proofs for convolutional neural network predictions and accuracy. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021. URL https://api.semanticscholar. org/CorpusID:235349006.
  28. Pranav Maneriker, Codi Burley, and Srinivasan Parthasarathy. Online fairness auditing through iterative refinement. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining, KDD '23, pp. 1665-1676, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701030. doi: 10.1145/3580305.3599454. URL https://doi.org/10.1145/3580305.3599454. · DOI
  29. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Comput. Surv., 54(6), jul 2021. ISSN 0360-0300. doi: 10.1145/3457607. URL https://doi.org/10.1145/3457607. · DOI · Paper page
  30. Payman Mohassel and Peter Rindal. Aby3: A mixed protocol framework for machine learning. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp. 35-52, 2018.
  31. Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy-preserving machine learning. In 2017 IEEE symposium on security and privacy (SP), pp. 19-38. IEEE, 2017.
  32. Saerom Park, Seongmin Kim, and Yeon-sup Lim. Fairness audit of machine learning models with con- fidential computing. In Proceedings of the ACM Web Conference 2022, WWW '22, pp. 3488-3499, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450390965. doi: 10.1145/3485447.3512244. URL https://doi.org/10.1145/3485447.3512244. · DOI
  33. Haakon Robinson, Adil Rasheed, and Omer San. Dissecting deep neural networks. arXiv preprint arXiv:1910.03879, 2019. · arXiv
  34. Anian Ruoss, Mislav Balunovic, Marc Fischer, and Martin Vechev. Learning certified individually fair representations. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 7584-7596.
  35. Curran Asso- ciates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/55d491cf951b1b920900684d71419282-Paper.pdf.
  36. Shahar Segal, Yossi Adi, Benny Pinkas, Carsten Baum, Chaya Ganesh, and Joseph Keshet. Fairness in the eyes of the data: Certifying machine-learning models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES '21, pp. 926-935, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384735. doi: 10.1145/3461702.3462554. URL https://doi.org/10.1145/3461702.3462554. · DOI
  37. Thiago Serra, Christian Tjandraatmadja, and Srikumar Ramalingam. Bounding and counting linear regions of deep neural networks. In International Conference on Machine Learning, pp. 4558-4566.
  38. PMLR, 2018.
  39. Ali Shahin Shamsabadi, Sierra Calanda Wyllie, Nicholas Franzese, Natalie Dullerud, Sébastien Gambs, Nicolas Papernot, Xiao Wang, and Adrian Weller. Confidential proof of fair training of trees. ICLR, 2023.
  40. Ioana Baldini Soares, Chhavi Yadav, Payel Das, and Kush Varshney. Keeping up with the language models: Robustness-bias interplay in nli data and models. In Annual Meeting of the Association for Computational Linguistics, 2023.
  41. Wenting Zheng Srinivasan, PMRL Akshayaram, and Popa Raluca Ada. Delphi: A cryptographic inference service for neural networks. In Proc. 29th USENIX Secur. Symp, pp. 2505-2522, 2019.
  42. Ehsan Toreini, Maryam Mehrnezhad, and Aad van Moorsel. Verifiable fairness: Privacy- preserving computation of fairness for machine learning systems. 2023. URL https: //api.semanticscholar.org/CorpusID:261696588.
  43. Caterina Urban, Maria Christakis, Valentin Wüstholz, and Fuyuan Zhang. Perfectly parallel fairness certification of neural networks. Proc. ACM Program. Lang., 4(OOPSLA), nov 2020. doi: 10.1145/3428253. URL https://doi.org/10.1145/3428253. · DOI
  44. Jiasi Weng, Jian Weng, Gui Tang, Anjia Yang, Ming Li, and Jia-Nan Liu. Pvcnn: Privacy-preserving and verifiable convolutional neural network testing. Trans. Info. For. Sec., 18:2218-2233, mar 2023. ISSN 1556-6013. doi: 10.1109/TIFS.2023.3262932. URL https://doi.org/10. 1109/TIFS.2023.3262932. · DOI
  45. Shaojie Xu, Joel Vaughan, Jie Chen, Aijun Zhang, and Agus Sudjianto. Traversing the local polytopes of relu neural networks: A unified approach for network verification. arXiv preprint arXiv:2111.08922, 2021. · arXiv
  46. I-Cheng Yeh. default of credit card clients. UCI Machine Learning Repository, 2016. DOI: https://doi.org/10.24432/C55S3H. · DOI
  47. Samuel Yeom and Matt Fredrikson. Individual fairness revisited: Transferring techniques from adversarial robustness. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI'20, 2021. ISBN 9780999241165.
  48. Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=B1gdkxHFDH.
  49. Jiaheng Zhang, Zhiyong Fang, Yupeng Zhang, and Dawn Song. Zero knowledge proofs for decision tree predictions and accuracy. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, CCS '20, pp. 2039-2053, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450370899. doi: 10.1145/3372297.3417278. URL https://doi.org/10.1145/3372297.3417278. · DOI
  50. Lingchen Zhao, Qian Wang, Cong Wang, Qi Li, Chao Shen, and Bo Feng. Veriml: Enabling integrity assurances and fair payments for machine learning as a service. IEEE Transactions on Parallel and Distributed Systems, 32(10):2524-2540, 2021. doi: 10.1109/TPDS.2021.3068195. · DOI

Cited by

Sorted by citations

Authors
Yadav, C., Chowdhury, A. R., Boneh, D., & Chaudhuri, K.
Affiliations
  • University of California San Diego (US)
  • Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.12572
arXiv
2402.12572
OpenAlex
W4392019798
Cited by
2
References
0
Citations by year
2026: 1 · 2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (1804829, 2241100, 2217058)
  • Defense Advanced Research Projects Agency
  • Multidisciplinary University Research Initiative (W911NF-21-1-0317)
  • Office of Naval Research (N00014)
  • Army Research Office
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
algorithmic fairness, zero-knowledge proofs, model confidentiality, fully connected neural network, privacy-preserving verification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review