Software fairness
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
A goal of software engineering research is advancing software quality and the success of the software engineering process. However, while recent studies have demonstrated a new kind of defect in software related to its ability to operate in fair and unbiased manner, software engineering has not yet wholeheartedly tackled these new kinds of defects, thus leaving software vulnerable. This paper outlines a vision for how software engineering research can help reduce fairness defects and represents a call to action by the software engineering research community to reify that vision. Modern software is riddled with examples of biased behavior, from automated translation injecting gender stereotypes, to vision systems failing to see faces of certain races, to the US criminal justice sytem relying on biased computational assessments of crime recidivism. While systems may learn bias from biased data, bias can also emerge from ambiguous or incomplete requirement specification, poor design, implementation bugs, and unintended component interactions. We argue that software fairness is analogous to software quality, and that numerous software engineering challenges in the areas of requirements, specification, design, testing, and verification need to be tackled to solve this problem.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Causal inference in statistics: An overview
Pearl, J. (2009) · Statistics Surveys · doi:10.1214/09-ss057 · Cited by 2,336 · cited 1×
“legal nonimmigrants, than for natural-born citizens [59].”
A Reductions Approach to Fair Classification
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02453 · Cited by 1,288
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 1×
“Fairness in machine learning research is also concerned with causal measures of discrimination, e.g., coun-terfactual fairness [50].”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Want to upload a video to YouTube and have closed captions generated automatically? If your video features a man’s voice, the closed captions are likely to be more accurate than one with a woman’s voice [65].”
BackgroundFairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694 · cited 2×
“every pair of input characteristics, which can be particularly helpful when testing software product lines [5, 35, 37, 38].”
BackgroundFraser, G., & Arcuri, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.14 · Cited by 586 · cited 1×
“Causal relationships in data management systems [26, 46, 47] can help explain query results [51] and debug errors [70–72] by tracking and using data provenance [49].”
BackgroundFace Recognition Performance: Role of Demographic Information
Klare, B. F., Burge, M. J., Klontz, J. C., Vorder Bruegge, R. W., & Jain, A. K. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2214212 · Cited by 516 · cited 1×
“, as a result of implementation bugs, as an unintended property of self-organizing systems [9, 11, 13, 14], as an emergent property of component interaction [10, 12, 15, 41], or as an automatically learned property from biased data [16, 17, 31– 34, 73–75].”
BackgroundUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“every pair of input characteristics, which can be particularly helpful when testing software product lines [5, 35, 37, 38].”
BackgroundPreventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 3×
“Reducing discrimination in machine learning classifiers [2, 22, 38, 43, 44, 76, 76, 82, 84] and selection algorithms [72] is important work that is complementary to the effort the software engineering community needs to put in to create tools and other support for developing fair software.”
BackgroundMethodBuilding Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · cited 1×
“Group discrimination is a generalization of the Calders-Verwer (CV) score [17], used frequently in prior work on algorithmic fairness, particularly in the context of fair machine learning [16, 32, 73, 75].”
MethodClassifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · cited 6×
“The group fairness definition [16, 22, 32, 73, 75] states that the same fraction of the applicants of each race should get loans.”
BackgroundMethodInfluentialFairness testing: testing software for discrimination
Galhotra, S., Brun, Y., & Meliou, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106277 · Cited by 405 · cited 1×
“, using static or dynamic analyses for vacuity testing [6, 29] or to identify configuration op-”
MethodSelf-citationGender and Dialect Bias in YouTube’s Automatic Captions
Tatman, R. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1606 · Cited by 338
Discrimination Aware Decision Tree Learning
Kamiran, F., Calders, T., & Pechenizkiy, M. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.50 · Cited by 303
Decision Theory for Discrimination-Aware Classification
Kamiran, F., Karim, A., & Zhang, X. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.45 · Cited by 300
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · cited 2×
“But while each of these definitions is appropriate in a given context, many are impossible to satisfy simultaneously [33].”
BackgroundFairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2015) · arXiv · doi:10.48550/arxiv.1510.02377 · Cited by 171
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2017) · 2017 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2017.29 · Cited by 171 · cited 1×
“tests needed to evaluate every possible value pair can be significantly smaller than the exhaustive testing alternative since each test can simultaneously contribute to multiple value pairs [20, 35, 69].”
BackgroundBlodgett, S. L., & O'Connor, B. (2017) · arXiv · doi:10.48550/arxiv.1707.00061 · Cited by 153
Handling Conditional Discrimination
liobaite, I., Kamiran, F., & Calders, T. (2011) · 2011 IEEE 11th International Conference on Data Mining · doi:10.1109/icdm.2011.72 · Cited by 125
FairSquare: probabilistic verification of program fairness
Albarghouthi, A., D'Antoni, L., Drews, S., & Nori, A. V. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133904 · Cited by 94 · cited 4×
“Recent work proposing verification of fairness properties highlights relevant challenges [3].”
BackgroundInfluentialEarly Detection of Collaboration Conflicts and Risks
Brun, Y., Holmes, R., Ernst, M. D., & Notkin, D. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.28 · Cited by 82 · cited 1×
“…unintended property of self-organizing systems [11, 16, 18, 19], as an emergent property of component interaction [12, 17, 21, 49], as conflicting logic from multiple developers’ changes [13–15, Existing work on test generation focuses on two specific measures of discrimination, group and causal.”
BackgroundSelf-citationCausality and explanations in databases
Meliou, A., Roy, S., & Suciu, D. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2733004.2733070 · Cited by 73
Self-citationToward variability-aware testing
Kästner, C., von Rhein, A., Erdweg, S., Pusch, J., Apel, S., Rendel, T., & Ostermann, K. (2012) · Proceedings of the 4th International Workshop on Feature-Oriented Software Development - FOSD '12 · doi:10.1145/2377816.2377817 · Cited by 63
Synthesizing partial component-level behavior models from system specifications
Krka, I., Brun, Y., Edwards, G., & Medvidovic, N. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595756 · Cited by 40 · cited 1×
“, as a result of implementation bugs, as an unintended property of self-organizing systems [11, 16, 18, 19], as an emergent property of component interaction [12, 17, 21, 49], as conflicting logic from multiple developers’ changes [13–15, 20] or as an automatically learned property from biased data [22, 23, 37– 40, 82…”
BackgroundSelf-citationDelta execution for efficient state-space exploration of object-oriented programs
d'Amorim, M., Lauterburg, S., & Marinov, D. (2007) · Proceedings of the 2007 international symposium on Software testing and analysis · doi:10.1145/1273463.1273472 · Cited by 26 · cited 3×
“For example, AdFisher [27] collects information on how changes in Google ad settings and prior visited webpages affect the ads Google serves.”
InfluentialSelf-Adapting Reliability in Distributed Software Systems
Brun, Y., Bang, J. Y., Edwards, G., & Medvidovic, N. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2412134 · Cited by 16 · cited 5×
“The group fairness definition [16, 22, 32, 73, 75] states that the same fraction of the applicants of each race should get loans.”
BackgroundMethodInfluentialSelf-citation
Cited by
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“Development Functional Testing [164, 235, 381] Performance Benchmarking [40, 56, 88, 93, 123, 127, 153, 238, 248, 307, 327]”
BackgroundBalayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
“Roadmaps to develop ethical software are proposed [13,28], where the needs formethods to build ethical software, to evaluate the compatibility of the softwarewith human values, and to help stakeholders formulate their values are highlighted.”
BackgroundGeorgieva, I., Lazo, C., Timan, T., & van Veenstra, A. F. (2022) · AI and Ethics · doi:10.1007/s43681-021-00127-3 · Cited by 89
Fairness-Aware Machine Learning Engineering: How Far Are We?
Ferrara, C., Sellitto, G., Ferrucci, F., Palomba, F., & De Lucia, A. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10402-y · Cited by 60
“Confirming the findings of previous work (Brun and Meliou 2018; Verma and Rubin 2018), our empirical investigation highlighted that the ethical aspects of a machine learning solution are difficult to generalize and treat in a standardized way.”
BackgroundInfluential
Metadata
- Authors
- Brun, Y., & Meliou, A.
- Venue
- Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2018-10-26
- Pages
- 754-759
- OpenAlex
- W2913771223
- Cited by
- 138
- References
- 29
- FWCI
- 9.05
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.