Measurement and Fairness
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; 7 of the 70 sources its claims cite were checked for support.
Abstract
We propose measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems. Computational systems often involve unobservable theoretical constructs, such as socioeconomic status, teacher effectiveness, and risk of recidivism. Such constructs cannot be measured directly and must instead be inferred from measurements of observable properties (and other unobservable theoretical constructs) thought to be related to them---i.e., operationalized via a measurement model. This process, which necessarily involves making assumptions, introduces the potential for mismatches between the theoretical understanding of the construct purported to be measured and its operationalization. We argue that many of the harms discussed in the literature on fairness in computational systems are direct results of such mismatches. We show how some of these harms could have been anticipated and, in some cases, mitigated if viewed through the lens of measurement modeling. To do this, we contribute fairness-oriented conceptualizations of construct reliability and construct validity that unite traditions from political science, education, and psychology and provide a set of tools for making explicit and testing assumptions about constructs and their operationalizations. We then turn to fairness itself, an essentially contested construct that has different theoretical understandings in different contexts. We argue that this contestedness underlies recent debates about fairness definitions: although these debates appear to be about different operationalizations, they are, in fact, debates about different theoretical understandings of fairness. We show how measurement modeling can provide a framework for getting to the core of these debates.
Overview
CoChat Summary
TL;DR
The paper proposes measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems, arguing that many fairness harms stem from mismatches between unobservable theoretical constructs and their operationalizations. It contributes fairness-oriented construct reliability and construct validity tools and argues that debates over fairness definitions are really debates about different theoretical understandings of fairness—i.e., values.
Key findings
- Many fairness-related harms arise because computational systems operationalize unobservable constructs—such as socioeconomic status, teacher effectiveness, risk of recidivism, and patient benefit—via measurement models whose assumptions are left implicit or untested.
- The authors contribute fairness-oriented conceptualizations of construct reliability and construct validity, including seven aspects of construct validity: face, content, convergent, discriminant, predictive, hypothesis, and consequential validity.
- Recent fairness-definition debates—such as individual vs. group fairness and COMPAS’s error-rate balance vs. predictive parity—are framed as operationalization disputes but are actually debates about different theoretical understandings of fairness, and therefore about values.
- Examples include Obermeyer et al.’s finding that only 18% of patients identified for high-risk care management enrollment were Black; EVAAS teacher-effectiveness scores varying from 6/100 to 96/100 for one teacher; and the Universal Credit system treating £1000 paychecks on March 30 and April 29 as a £2000 monthly wage.
Methods
- Conceptual/position paper; no empirical data collection or new dataset.
- Uses five illustrative examples: measuring height, socioeconomic status, teacher effectiveness via EVAAS MRM, risk of recidivism via COMPAS, and patient benefit via high-risk care management models.
- Analysis is conceptual synthesis, uniting traditions from political science, education, and psychology to apply construct reliability and construct validity to fairness in computational systems.
Limitations
- Stated: the authors acknowledge that assessing construct reliability and construct validity can be time-consuming, and they omit inter-rater reliability and inter-item reliability from their conceptualization.
- Stated: they emphasize that values play a role in the measurement modeling process, not that measurement modeling is value-free.
- Evident: as a conceptual paper, it offers no new empirical validation or dataset; its examples and numbers rely on prior literature, documentation, and reported cases.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
5 Valid · 13 Not valid · 48 Not checked · 116 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In contrast to computer scientists, who focus primarily on out-ofsample prediction, quantitative social scientists typically test their assumptions by assessing construct reliability and construct validity.”
§ Measurement and Fairness > 3 TESTING ASSUMPTIONS
What the sources say
- Partly supportedcites [49] matched to “How to analyze political attention with minimal assumptions and costs” (2010) sourceExperimental — accuracy not yet measured
The 'Reliability, Validity, Interpretation, and Application' section supports the substance of the claim: 'The evaluation of any measurement is generally based on its reliability (can it be repeated?) and validity (is it right?),' and the paper then assesses convergent and discriminant construct validity (with Weber 1990 cited) plus semantic, predictive and hypothesis validity. The companion 'Categorizing Texts: Methods, Assumptions, and Costs' section does describe supervised learning as evaluated by 'summaries of out-of-sample predictive accuracy,' which underwrites the contrasting half of the citing sentence. What our copy does not state is the disciplinary generalization itself: it never says that quantitative social scientists 'typically' test assumptions this way, nor that computer scientists focus primarily on out-of-sample prediction — the reliability/validity framing is presented as a general principle of measurement evaluation in content analysis, not as a characterization of a research community. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Prompted by this, many academic papers began to discuss the properties and shortcomings of different definitions of group fairness, with some papers demonstrating that it is impossible for a system like COMPAS to possess both error-rate balance and predictive parity when the probability of recidivism for Black defendants is not the same as the probability of recidivism for white defendants [9,11,1”
§ Measurement and Fairness > 4.4 Group Fairness
What the sources say
- Partly supportedcites [9] matched to “Fairness in Criminal Justice Risk Assessments: The State of the Art” (2021) sourceExperimental — accuracy not yet measured
Section 6.1 (p. 18) states that equal error rates and conditional use accuracy cannot both be achieved when groups have different base rates and classification is not perfect. The claim omits the paper’s qualification: with perfect separation, both can be achieved despite different base rates. (LLM, conf=0.88)
- Supportedcites [11] matched to “Fair prediction with disparate impact: a study of bias in recidivism prediction instruments” (2017) source
Section 2.1 (p. 2) says test fairness implies equal positive predictive value across groups, and that when recidivism prevalence differs, a test-fair score cannot have equal false-positive and false-negative rates across those groups. It applies this result to Black and White defendants assessed with COMPAS. (LLM, conf=0.93)
- Not checked yetcites [13]
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“by using arrests as a proxy for crimes committed, COMPAS can only exacerbate racist policing practices, rather than transcending them [7,13,23,37,39].”
§ Measurement and Fairness > 3.2.7 Consequential validity.
What the sources say
- Not checked yetcites [7]
- Not checked yetcites [13]
- Partly supportedcites [23] matched to “The false promise of risk assessments: epistemic reform and the limits of fairness” (2020) sourceExperimental — accuracy not yet measured
Section 5 says risk assessments use arrest data that typically provide a racially biased measure of crime, supporting the claim's concern about using arrests as a proxy. Sections 3 and 3.3 argue that these tools do not necessarily reduce racial discrimination and are poorly suited to structural reform. The paper does not establish the stronger claim that COMPAS can only exacerbate racist policing practices. (LLM, conf=0.84)
- Partly supportedcites [37] matched to “To Predict and Serve?” (2016) sourceExperimental — accuracy not yet measured
The ‘Bias in police-recorded data’ and ‘Discussion’ sections say that police records reflect unequal policing and that predictive policing can reproduce and magnify those biases. But the paper’s case study examines PredPol, not COMPAS; it does not establish that COMPAS uses arrests as a proxy for crimes committed or that it can only exacerbate racist policing. (LLM, conf=0.87)
- Not checked yetcites [39]
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, because most computational systems are developed by computer scientists, the practice of collapsing these distinctions is widespread [47].”
§ Measurement and Fairness > 5 DISCUSSION
What the sources say
- Partly supportedcites [47] matched to “Problem Formulation and Fairness” (2019) sourceExperimental — accuracy not yet measured
Our copy of Passi & Barocas (2019) does support the descriptive half of the claim: the Abstract states that 'the specification and operationalization of the problem are always negotiated and elastic, and rarely worked out with explicit normative considerations in mind,' and normative assessments 'often take them for granted'; the case-study section (4 CASE-STUDY: SPECIAL FINANCING) shows practitioners equating lead quality with financeability, financeability with dealer decisions, and dealer decisions with credit score ranges, i.e. collapsing a construct into its proxy/measure. What the paper nowhere says is the causal premise the citing sentence attaches to it — that this practice is widespread 'because most computational systems are developed by computer scientists'; no passage makes any claim about who develops most computational systems, and the BACKGROUND section instead stresses that applied data science is 'inherently collaborative,' with 'project managers, product designers, and business analysts ... as much a part of ... corporate data science as are data scientists' [36:16]. The paper's own explanation for such collapses is practical and organizational (business requirements, data availability, analytic cost), not the disciplinary composition of developers, so the attributing mechanism and the 'widespread' generalization beyond one ethnographed team are unsupported. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We emphasize that even with careful consideration of different theoretical understandings and careful assessment of both construct reliability and construct validity, this approach is fraught and likely to cause a variety of fairness-related and other harms [e.g., 8, 22, 33, 35, 44], despite good intentions.”
§ Measurement and Fairness > 4.5 Demographic Factors
What the sources say
- Not checked yetcites [8] matched to “What is the point of fairness?” (2020) source
- Partly supportedcites [22] matched to “The Profiling Potential of Computer Vision and the Challenge of Computational Empiricism” (2019) sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Not checked yetcites [33] matched to “The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition” (2018) source
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
77 more claims
Create a free account to see all 182 claims and to verify the 63 references still open.
- It may state two separate claims as one. 42
- Probably not a substantial claim. 82
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When measurement bias is associated with a sensitive characteristic, in this case gender, the model is likely to replicate the pattern which can result in an unfair allocation of jobs [59].”
From Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree (2023) · cites this paper as [59]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Essentially, equal calibration requires that the meaning of predicted scores is equal across groups [59]: receiving a score of 0.7 corresponds to a probability of 0.7, irrespective of sensitive group membership.”
From Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree (2023) · cites this paper as [59]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These evaluations should also demonstrate that the outcome of interest can be measured with reasonable accuracy and validity, as low-quality data and bad proxy variables present a significant limit on an algorithm's reliability ( Jacobs & Wallach, 2021 ).”
From The flaws of policies requiring human oversight of government algorithms (2022) · cites this paper as Jacobs & Wallach, 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Framing bias as something to be measured objectively ignores the assumptions made in the operationalization of the measurement tool (Jacobs and Wallach 2021).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Jacobs and Wallach 2021
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“5Error-rate balance is also known as equalized odds [28].”
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“r operationalization, i.e., calibration. The fairness in machine learning community responded, largely re ecting the incompatibilities and shortcomings of dierent fairness operationalizations (e.g., [8, 11, 15, 13, 39, 54]). Meanwhile, Chouldechova [11] and Kleinberg et al. [39] proved that the two prominent and desir…”
BackgroundInfluentialMachine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989 · cited 1×
“Despite the name, predictive validity is not the same as out-of-sample prediction [29, 47].”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 2×
“modeling brings the biggest value to fairness. In practice, evaluation of computational systems can fail to characterize potential harms which are otherwise captured by failures of construct validity [27, 53, 59]. The language of measurement, with the tools of construct validity and reliability, provide a concrete fra…”
BackgroundObjective Tests as Instruments of Psychological Theory
Loevinger, J. (1957) · Psychological Reports · doi:10.2466/pr0.1957.3.3.635 · Cited by 1,473
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · cited 3×
“explicitly consider how to minimize or compensate for historical injustices [11]. This nontrivial measurement task is deeply complex and requires signicant debate about justice, policy, and fairness [8, 13, 15, 37]. 4.2 Value-Added Modeling in Education We now look to the well-known example of value-added modeling for…”
BackgroundInfluentialInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 2×
“, [27, 41, 50].”
Background
Research works printed but not linked: 17
- Audrey Amrein-Beardsley and Tray Geiger. 2020. Methodological Concerns About the Education Value-Added Assessment System (EVAAS): Validity, Relia- bility, and Bias. SAGE Open 10, 2 (2020), 2158244020922224.
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (May 2016).
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: from allocative to representational harms in machine learning. Special Interest Group for Computing. Information and Society (SIGCIS) (2017).
- Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting things out: Classification and its consequences. MIT Press. · doi:10.5555/380681
- Sam Corbett-Davies, Emma Pierson, Avi Feller, and Sharad Goel. 2016. A com- puter program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Washington Post (October 2016).
- Kate Crawford. 2017. The Trouble with Bias. NIPS Keynote, https://www. youtube.com/watch?v=fMym_BKWQzk, accessed 08-22-2018.
- Walter Bryce Gallie. 1955. Essentially contested concepts. In Proceedings of the Aristotelian Society, Vol. 56. JSTOR, 167-198.
- Justin Grimmer. 2012. Comment: Evaluating model performance in fictitious prediction problems. Technical Report.
- Simon Jackman. 2008. Measurement. In The Oxford Handbook of Political Method- ology. Oxford Univ. Press, Chapter 9.
- Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. 2016. Fairness in learning: Classic and contextual bandits. NIPS (2016).
- Sandra G Mayson. 2018. Bias in, bias out. Yale Law Journal 128 (2018), 2218.
- Samuel Messick. 1987. Validity. In ETS Research Report Series.
- Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447-453.
- Cathy O'Neil. 2016. Weapons of Math Destruction. Broadway Books.
- Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. NIPS (2017).
- Anna Roberts. 2018. Arrests as guilt. Ala. L. Rev. 70 (2018), 987.
- Human Rights Watch. 2020. UK: Automated Benefits System Failing People in Need. https://www.hrw.org/news/2020/09/29/uk-automated-benefits-system- failing-people-need
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“…al., 2021b), there is a lack of standardization of fairness benchmarks, an understanding of what harms different bias measures in NLP relate to (Blodgett et al., 2020, 2021; Jacobs & Wallach, 2021), and coverage of identities in fluid, comprehensive ways (Cao & Daumé III, 2020; Dev et al., 2021a).”
BackgroundAnil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
“The data that is collected can differ significantly from what occurs in the real world [77, 78, 119].”
BackgroundMadaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436
“However, AI ethics involves both sociocultural and technical factors, and concepts like fairness are often essentially contested [54].”
BackgroundSelf-citationTowards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295
“hich stem from bureaucratic processes. We know from thehistoriesoutlinedabovethatthesecategoriesareunstable,contingent, and rooted in racial inequality. Followingrecentworkaroundmeasurementandfairness[5,57], we suggest taking seriously the problems of measurement modeling when considering racial variables. Race, like…”
BackgroundAuditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“One bottleneck to developing effective auditing procedures is the difficulty of operationalising normative concepts like robustness and truthfulness [250].”
BackgroundCan HR adapt to the paradoxes of artificial intelligence?
Charlwood, A., & Guenole, N. (2022) · Human Resource Management Journal · doi:10.1111/1748-8583.12433 · Cited by 271
“…outlined what we believe is a strong case for a positive future for AI in HR. Domain knowledge and expertise is essential for the development of AI tools that work as intended, are fair and which do not reproduce existing organisational biases (van den Broek et al., 2021; Jacobs & Wallach, 2021).”
BackgroundThe flaws of policies requiring human oversight of government algorithms
Green, B. (2022) · Computer Law & Security Review · doi:10.1016/j.clsr.2022.105681 · Cited by 263
The Flaws of Policies Requiring Human Oversight of Government Algorithms
Green, B. (2021) · doi:10.2139/ssrn.3921216 · Cited by 259
The Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251
“As a construct validity issue, the constructs of the built model could not possibly meaningfully represent those relevant to the task at hand [98, 99].”
BackgroundSafe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
Schramowski, P., Brack, M., Deiseroth, B., & Kersting, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02157 · Cited by 219
The Ethics of Emotion in Artificial Intelligence Systems
Stark, L., & Hoey, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445939 · Cited by 179
“However, it is also important to note that such manipulations do not imply that these companies can build an accurate model of a person based on their affective responses, nor that they necessarily understand the causal relationships between the responder and the modified elements of their interface [56].”
BackgroundAlderman, J. E., Palmer, J., Laws, E., McCradden, M. D., Ordish, J., Ghassemi, M., Pfohl, S. R., Rostamzadeh, N., Cole-Lewis, H., Glocker, B., et al. (2025) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00224-3 · Cited by 170
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
“As a result, algorithmic fairness criteria may be misleading in several situations, including when the observed outcome is a biased surrogate for the construct of interest [3, 49] and when predictive models are not appropriately contextualized in terms of the heterogeneous and dynamic impact of the complex interventio…”
Background(2019) · doi:10.1145/3351095.3372862 · Cited by 167
Gender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 165
“For a broader discussion on measurement and bias we refer the reader also to (Jacobs, 2021; Jacobs et al., 2020).”
BackgroundAssessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156
Self-citationDExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.522 · Cited by 148
“However, the mismatch between the construct of toxicity and its operationalization through an automatic classi-fier can cause biased or unintended model behavior (Jacobs and Wallach, 2021).”
BackgroundDrukker, K., Chen, W., Gichoya, J. W., Gruszauskas, N. P., Kalpathy-Cramer, J., Koyejo, S., Myers, K. J., Sá, R. C., Sahiner, B., Whitney, H. M., et al. (2023) · Journal of Medical Imaging · doi:10.1117/1.jmi.10.6.061104 · Cited by 149
“46 When training bias occurs, an AI/ML model may skew its output based on the prevalence of disease(s) in the training data, potentially over or underrepresenting the prevalence in the target population on which the AI/ML will be deployed.”
BackgroundDisability, fairness, and algorithmic bias in AI recruitment
Tilmes, N. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09633-2 · Cited by 257
“While anti-classification, calibration, and classification parity all aim to operationalize distinct notions of fairness, it is mathematically impossible to satisfy each of these definitions simultaneously in the real world (Jacobs & Wallach, 2021).”
BackgroundREFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 129
“This is especially important because mismatches between variables and the constructs they are purported to represent can create fairness issues ( 120 ).”
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.81 · Cited by 147
“Measurement Modeling We apply a measurement modeling lens by viewing each benchmark as a measurement model (MM) (e.g., Quinn et al., 2010; Jacobs and Wallach, 2021).”
BackgroundMethodSelf-citationMeasuring algorithmically infused societies
Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021) · Nature · doi:10.1038/s41586-021-03666-1 · Cited by 144
“Yet they often come as ‘black boxes’ that do not make their assumptions explicit and thereby introduce or exacerbate potential mismatches between the theoretical construct of interest and its operationalization 63,67 .”
BackgroundMethodA survey on bias in visual datasets
Fabbrizzi, S., Papadopoulos, S., Ntoutsi, E., & Kompatsiaris, I. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103552 · Cited by 137
Harms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“In particular, many have identified the impact of computational modeling and development in social change [Abebe et al. 2020; Jacobs and Wallach 2021; Selbst et al. 2019] and scoped numerous taxonomies of risks, harms, and failures of algorithmic systems [Raji et al. 2022; Shelby et al. 2022;…”
BackgroundSeeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579621 · Cited by 135
“10Note that Fairlearn [T5] has—since we conducted the data analysis for this paper—published resources in its user guide for understanding social science concepts such as construct validity for concepts such as fairness [34] and explanations of sociotechnical abstraction traps [72].”
BackgroundPhysiognomic Artificial Intelligence
Stark, L., & Hutson, J. (2021) · doi:10.2139/ssrn.3927300 · Cited by 139
AI and the Everything in the Whole Wide World Benchmark
Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021) · arXiv · doi:10.48550/arxiv.2111.15366 · Cited by 127
“…been acknowledged broadly by the machine learning community as a particularly difficult task in benchmark design and development [Mitchell, 2021, Jacobs and Wallach, 2021, Malik, 2020] — however as Bowman and Dahl note, it remains a generally neglected issue as “[t]his criterion is difficult to…”
BackgroundBuilding Human Values into Recommender Systems: An Interdisciplinary Synthesis
Stray, J., Halevy, A., Assar, P., Hadfield-Menell, D., Boutilier, C., Ashar, A., Bakalar, C., Beattie, L., Ekstrand, M., Leibowicz, C., et al. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3632297 · Cited by 122
“Jacobs and Wallach (2021) propose several other types of validity in the context of measurements within computational systems, including face validity (is the metric plausible?)”
BackgroundThe life cycle of large language models in education: A framework for understanding sources of bias
Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13505 · Cited by 119
“Overall, this inherent ambiguity in quantifying abstract constructs can introduce measurement bias when operation-alizing these constructs during data pre-processing, and ultimately lead to harm (Jacobs & Wallach, 2021).”
BackgroundAlgorithms and Decision-Making in the Public Sector
Levy, K., Chasalow, K. E., & Riley, S. (2021) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-041221-023808 · Cited by 108
“Measurability and construct validity are common concerns across empirical social science, where the variables we can measure or access are proxies for true outcomes of interest (Jacobs & Wallach 2021, Singleton & Straits 2005).”
BackgroundFairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 160
“Furthermore, what makes an algorithm biased and its outcomes unfair is the subject of a contested debate (Rovatsos et al., 2019; Barocas & Selbst, 2016; Jacobs, 2021; Wachter et al., 2021a).”
Geiger, R. S., Cope, D., Ip, J., Lotosh, M., Shah, A., Weng, J., & Tang, R. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00144 · Cited by 142
“Another key issue in data labeling are issues of construct validity and operationalization ( Jacobs and Wallach, 2019 ): is the labeling process actually capturing the theoretical construct that the authors are claiming to capture?”
BackgroundWang, A., Ramaswamy, V. V., & Russakovsky, O. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533101 · Cited by 87
“8 The foundation of this problem is that categorizing people into discrete, socially constructed groups, while tenuous, is often necessary for machine learning systems to make sense of socially relevant distinctions [35, 56, 67].”
BackgroundEscaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness
Green, B. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00584-6 · Cited by 90
“As with fairness more generally [13, 47, 53, 85], attempting to reduce substantive equality to mathematical definitions is likely to narrow and distort the concept.”
InfluentialMethodology and evaluation in sports analytics: challenges, approaches, and lessons learned
Davis, J. J., Bransen, L., Devos, L., Jaspers, A., Meert, W., Robberechts, P., Haaren, J. V., & Roy, M. V. (2024) · Machine Learning · doi:10.1007/s10994-024-06585-0 · Cited by 154
“This is also called face validity (Jacobs & Wallach, 2021).”
Reliance on metrics is a fundamental challenge for AI
Thomas, R. L., & Uminsky, D. (2020) · Patterns · doi:10.1016/j.patter.2022.100476 · Cited by 123
Understanding and Evaluating Racial Biases in Image Captioning
Zhao, D., Wang, A., & Russakovsky, O. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01456 · Cited by 83
“This is consistent with the argument of Jacobs and Wallach [37]: gender is operationalized differently in caption-derived versus human-collected annotations.”
ResultCollective Constitutional AI: Aligning a Language Model with Public Input
Huang, S., Siddarth, D., Lovitt, L., Liao, T. I., Durmus, E., Tamkin, A., & Ganguli, D. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658979 · Cited by 76
“When adjudicating some of the trade-offs in the process we ran, one principle that guided our decision-making was aiming to not bias the resulting constitution (e.g. minimizing editorialization of the principles) to maintain construct validity [32].”
BackgroundMethodInfluentialAligning AI Optimization to Community Well-Being
Stray, J. (2020) · International Journal of Community Well-Being · doi:10.1007/s42413-020-00086-3 · Cited by 115
“The particular well-being aspect of interest must first be “ operationalized ” and tested for reliability and validity (Jacobs and Wallach 2019).”
BackgroundREVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 77
“…dynamic and non-discrete concepts such as gender and using skin tone as a proxy for race, we reify certain conceptions of these concepts (Hanna et al., 2020; Jacobs and Wallach, 2021) that harm certain groups, e.g., non-binary individuals (Hamidi et al., 2018; Scheuerman et al., 2020).”
BackgroundDesigning Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., Wadsworth, W. D., & Wallach, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462610 · Cited by 72
“However, in many scenarios, the labels may be inaccurate, either due to measurement issues (Jacobs and Wallach 2021) or due to discrimina-”
BackgroundSelf-citationA Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability
Sloane, M., Moss, E., & Chowdhury, R. (2022) · Patterns · doi:10.1016/j.patter.2021.100425 · Cited by 65
“Validity generally refers to the extent to which a statistical tool measures what it is supposed to measure.(5) Of particular relevance here is construct validity, which is ‘‘the extent to which the measure ‘behaves’ in a way consistent with theoretical hypotheses and represents how well scores on the instrument are i…”
BackgroundEvaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533233 · Cited by 62
Li, Y., Wang, H., & Luo, Y. (2022) · Circulation: Heart Failure · doi:10.1161/circheartfailure.122.009473 · Cited by 64
“46%) Electrocardiography QRS Duration (10-300), ms 114.”
A scoping review and evidence gap analysis of clinical AI fairness
Liu, M., Ning, Y., Teixayavong, S., Liu, X., Mertens, M., Shang, Y., Li, X., Miao, D., Liao, J., Xu, J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01667-2 · Cited by 113
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 59
“Moreover, bias measures may cause further epistemic violence onto the marginalized by creating a veneer of fairness, in spite of ongoing marginalization (Gonen and Goldberg, 2019; Talat et al., 2021; Jacobs and Wallach, 2021).”
BackgroundKawakami, A., Chowdhary, S., Iqbal, S. T., Liao, Q. V., Olteanu, A., Suh, J., & Saha, K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610207 · Cited by 103
“Research has revealed other limitations of these sensing-based data-driven wellbeing measurements, including how poor proxies could lead to fairness and bias issues [79, 123, 125].”
BackgroundPicking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?
Bommasani, R., Creel, K. A., Kumar, A., Jurafsky, D., & Liang, P. (2022) · arXiv · doi:10.48550/arxiv.2211.13972 · Cited by 60
“This speaks to the convergent and divergent validity of our metrics [Campbell and Fiske, 1959, Messick, 1987, Jacobs and Wallach, 2021], i.e. whether they are adequately correlated with metrics of similar constructs and adequately uncorrelated with metrics of dissimilar constructs.”
BackgroundHartmann, D., Wang, S. M., Pohlmann, L., & Berendt, B. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00381-z · Cited by 59
“Like Jacobs and Wallach [112] in the case of fairness, we propose to use measurement modeling from the quantitative social sciences [111] to discuss the echo chamber construct and to compare conceptualizations and operationalizations of echo chambers.”
MethodWang, A., Kapoor, S., Barocas, S., & Narayanan, A. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3636509 · Cited by 57
“without Machine Learning Predictive systems can only meet their goals, such as minimizing crime or hiring good employees, to the extent that their predictions are accurate.”
BackgroundTheory In, Theory Out: The Uses of Social Theory in Machine Learning for Social Science
Radford, J., & Joseph, K. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00018 · Cited by 58
“ues fall into four broad categories. First, Hanna Wallach and her colleagues have argued that many machine learning papers focus too heavily on prediction relative to explanation [115] or measurement [56]. The prioritization of prediction leads to models that perform well for unknown reasons, leading to ad hoc justica…”
BackgroundInfluentialPower and Public Participation in AI
Corbett, E., Denton, R., & Erete, S. (2023) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3617694.3623228 · Cited by 56
“…what data to include (or exclude), what categories should structure the data, and how key variables should be operationalized also inject normative political beliefs into the process [27, 36] underscoring the importance of dataset development as an important entry point for public participation.”
BackgroundInsolvent: How to Reorient Computing for Just Sustainability
Becker, C. (2023) · doi:10.7551/mitpress/14668.001.0001 · Cited by 56
Interpretable Data-Based Explanations for Fairness Debugging
Pradhan, R., Zhu, J., Glavic, B., & Salimi, B. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517886 · Cited by 82
“Furthermore, sources of bias and discrimination in training data are typically not randomly distributed across different sub-populations; rather they manifest systematic errors in data collection, selection, feature engineering, and curation [29, 35, 42, 62, 63, 70, 94].”
Can Workers Meaningfully Consent to Workplace Wellbeing Technologies?
Chowdhary, S., Kawakami, A., Gray, M. L., Suh, J., Olteanu, A., & Saha, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594023 · Cited by 86
“Prior work has noted that sensing-driven measurements of wellbeing constructs suffer limitations such as data and measurement biases, poor proxies, unintended consequences [46, 79, 85] and mismatches in self-perceived versus passively sensed wellbeing [25, 50, 91].”
BackgroundGender stereotype reinforcement: Measuring the gender bias conveyed by ranking algorithms
Fabris, A., Purpura, A., Silvello, G., & Susto, G. A. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102377 · Cited by 52
“the phenomenon we want to study, from its subsequent operationalization, which details how the phenomenon can be measured from observed data [42, 52].”
BackgroundMethodInfluentialSocial impacts of algorithmic decision-making: A research agenda for the social sciences
Gerdon, F., Bach, R. L., Kern, C. A., & Kreuter, F. (2022) · Big Data & Society · doi:10.1177/20539517221089305 · Cited by 88
“Such measurement bias can be directly connected to social science work on measurement errors and thus represents one example of how social science already contributes to researching social impacts of ADM (Boeschoten et al., 2020; Jacobs and Wallach, 2021).”
BackgroundThey shall be fair, transparent, and robust: auditing learning analytics systems
Simbeck, K. (2023) · AI and Ethics · doi:10.1007/s43681-023-00292-7 · Cited by 83
“The fairness of LA systems’ outcome will be negatively affected if the constructs used do not meet the requirements of validity and reliability [50].”
Sociotechnical Safety Evaluation of Generative AI Systems
Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., Bergman, S., Kay, J., Griffin, C., Bariach, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.11986 · Cited by 49
Ehsan, U., Singh, R., Metcalf, J., & Riedl, M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533186 · Cited by 40
“Similarly, Jacobs & Wallach [50] argue that many contestations about algorithmic fairness are actually conflicts over the validity of inferring unobservable theoretical constructs (e.g., worker effectiveness) from observable features that are readily available in datasets.”
BackgroundKawakami, A., Sivaraman, V., Stapleton, L., Cheng, H.-F., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533556 · Cited by 42
“By contrast, the ground truth labels in many social decision-making tasks—for example, whether an observed behavior is considered socially “harm-ful”—may be socially contested [19, 34] or represent an imperfect proxy for what decision-makers actually predict [18, 28].”
BackgroundEnvisioning Equitable Speech Technologies for Black Older Adults
Brewer, R. N., Harrington, C., & Heldreth, C. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594005 · Cited by 42
“We take a qualitative approach to data collection and analysis as recent arguments made in the FAccT community discuss how race and fairness are“contested” constructs that may not be quantifiable [21, 27, 50].”
MethodRobust Optimization for Fairness with Noisy Protected Groups
Wang, S., Guo, W., Narasimhan, H., Cotter, A., Gupta, M. R., & Jordan, M. I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09343 · Cited by 37
“Jacobs and Wallach [36] provide a general framework for understanding measurement issues for these sensitive attributes in the machine-learning setting, building on foundational work from the social sciences [5].”
BackgroundEvaluating the impact of human explanation strategies on human-AI visual decision-making
Morrison, K., Shin, D., Holstein, K., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579481 · Cited by 37
“1Throughout this paper, we distinguish between the terms reliance and trust: reliance is an observable human behavior that can be measured, whereas human trust is a latent factor which cannot directly be measured from observable human behaviors [14, 25, 36, 61]”
BackgroundBerman, G., Goyal, N., & Madaio, M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642398 · Cited by 40
“…changes in system development practices or outcomes (e.g., reduction in data misuse); although, as we discuss in Section 5.2, such measures may not currently exist for many responsible AI goals, or may be difcult to develop due to contestation in operationalizing constructs such as fairness [107].”
BackgroundInfluentialAdler, D. A., Stamatis, C. A., Meyerhoff, J., Mohr, D. C., Wang, F., Aranovich, G. J., Sen, S., & Choudhury, T. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00057-y · Cited by 52
“Given these de fi nitions, researchers in AI fairness have argued that aspects of psychometric reliability are important in an AI context: similar inputs (e.g. sensed-behaviors) to an AI model should yield similar outputs (e.g. estimated depression risk) 16 .”
Result(2021) · doi:10.1145/3442188.3445933 · Cited by 44
“status” that includes income, but does not account for wealth, accurately captures the content of the construct [9].”
BackgroundSurvey on fairness notions and related tensions
Alves, G. T., Bernier, F., Couceiro, M., Makhlouf, K., Palamidessi, C., & Zhioua, S. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100033 · Cited by 59
Sloane, M., & Zakrzewski, J. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533156 · Cited by 59
“[53, 76], deploying structural causal models [55] or introducing traceability [65] to define, assess and improve models of accountability, nutritional labels [49, 120], algorithmic impact assessment [56, 74, 97, 107], or algorithmic auditing [5, 18, 59, 63, 93, 98, 117].”
MethodInequality and inequity in network-based ranking and recommendation algorithms
Espín-Noboa, L., Wagner, C., Strohmaier, M., & Karimi, F. (2022) · Scientific Reports · doi:10.1038/s41598-022-05434-1 · Cited by 62
A Systematic Review of Ethical Concerns with Voice Assistants
Seymour, W., Zhan, X., Coté, M., & Such, J. M. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604679 · Cited by 68
“blages of data processing [50, 64], researchers should be cautious when attempting to quantify the lived experiences that relate to the concerns described in the review.”
BackgroundDesigning recommender systems to depolarize
Stray, J. (2022) · First Monday · doi:10.5210/fm.v27i5.12604 · Cited by 75
Hullman, J. R., Kapoor, S., Nanayakkara, P., Gelman, A., & Narayanan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534196 · Cited by 59
“, across social groups) [41, 162, 197, 223] which is not modeled [125, 133] Label errors [38, 158] and disagreement [57, 97] Data transformations decided contingent on performance com-”
BackgroundWatkins, E. A., McKenna, M., & Chen, J. (2022) · arXiv · doi:10.48550/arxiv.2202.09519 · Cited by 35
A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms
Coston, A., Kawakami, A., Zhu, H., Holstein, K., & Heidari, H. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00050 · Cited by 79
“Our work seeks to center validity considerations, a crucial criterion for the justified use of algorithmic tools in real-world decision-making [26, 27, 28].”
BackgroundInfluentialThe Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool
Gerchick, M., Jegede, T., Shah, T., Gutierrez, A., Beiers, S., Shemtov, N., Xu, K., Samant, A., & Horowitz, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594081 · Cited by 34
“surements used to assess ML systems [44, 45, 65], and shaping how system outputs are communicated and explained [12, 14, 58].”
BackgroundIt's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness BenchmarksPreprint
Bao, M., Zhou, A., Zottola, S. A., Brubach, B., Desmarais, S. L., Horowitz, A., Lum, K., & Venkatasubramanian, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05498 · Cited by 137
“Issues of measurement relate broadly to both measurement theory in the social sciences [24] and measurement error in econometrics.”
BackgroundHigh Tech, High Risk: Tech Ethics Lessons for the COVID-19 Pandemic Response
Moss, E., & Metcalf, J. (2020) · Patterns · doi:10.1016/j.patter.2020.100102 · Cited by 37
Evaluating Proposed Fairness Models for Face Recognition Algorithms
Howard, J. J., Laird, E., Rubin, R. E., Sirotin, Y. B., Tipton, J. L., & Vemury, A. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37660-3_31 · Cited by 55
“Additionally, as observed by individuals, fairness is not primarily a mathematical construct but a social and perceptual one [75], [76].”
BackgroundWeerts, H., Xenidis, R., Tarissan, F., Olsen, H. P., & Pechenizkiy, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594044 · Cited by 34
“pattern which can result in an unfair allocation of jobs [59].”
BackgroundGhasemaghaei, M., & Kordzadeh, N. (2024) · Information Systems Journal · doi:10.1111/isj.12572 · Cited by 46
“The seminal work by Jacobs and Wallach (2021) underscores the importance of measuring these dimensions to ensure that algorithms perform ethically across different contexts.”
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging
Katzman, J., Wang, A., Scheuerman, M., Blodgett, S. L., Laird, K., Wallach, H., & Barocas, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26670 · Cited by 32
Self-citationPfohl, S., Xu, Y., Foryciarz, A., Ignatiadis, N., Genkins, J., & Shah, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533166 · Cited by 30
“For our analysis, we assume that the observed outcomes are not subject to unobserved differential measurement error across patient subgroups [45, 64], that the choice of decision threshold used to allocate a clinical intervention on the basis of the output of a predictive model is not constrained by resource or operat…”
BackgroundAI and discriminative decisions in recruitment: Challenging the core assumptions
Seppälä, P., & Małecka, M. (2024) · Big Data & Society · doi:10.1177/20539517241235872 · Cited by 95
“This kind of argument has also been advanced within computer science by Jacobs and Wallach (2021) who have stressed the importance of making explicit the political and philosophical assumptions that underlie different mathematical operationalizations of fairness.”
Adapting Ethical Sensitivity as a Construct to Study Technology Design Teams
Boyd, K., & Shilton, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3463929 · Cited by 41
Reimagining the machine learning life cycle to improve educational outcomes of students
Liu, L. T., Wang, S., Britton, T., & Abebe, R. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2204781120 · Cited by 70
Pursuing Equity With Artificial Intelligence in Health Care
Johnson, K. B., Horn, I. B., & Horvitz, E. (2025) · JAMA Health Forum · doi:10.1001/jamahealthforum.2024.5031 · Cited by 31
A Method to Analyze Multiple Social Identities in Twitter Bios
Pathak, A., Madani, N., & Joseph, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479502 · Cited by 48
“hypothesis validity, are aimed specifically at utility [34].”
BackgroundGround(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
Guerdan, L., Coston, A., Wu, Z. S., & Holstein, K. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594036 · Cited by 28
“” These latent constructs are not directly observable in the world, but can be operationalized via a measurement model [42, 49].”
BackgroundMethodInfluentialEquity and Access in Algorithms, Mechanisms, and Optimization
(2021) · doi:10.1145/3465416 · Cited by 29
Differential Fairness: An Intersectional Framework for Fair AI
Islam, R., Keya, K. N., Pan, S., Sarwate, A. D., & Foulds, J. R. (2023) · Entropy · doi:10.3390/e25040660 · Cited by 45
“This does not account for additional harms due to its interaction with a larger sociotechnical system [22] or harms that are not captured by the fairness metric itself [23].”
BackgroundInfluentialKlumbytė, G., Draude, C., & Taylor, A. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533207 · Cited by 46
“The FAccT community in particular has showcased research that brings together approaches, concepts and methods from social sciences and humanities together with ML research and design, or that explicitly starts from humanities and social sciences perspectives when analyzing implications of ML systems and proposing new…”
BackgroundAugmented Datasheets for Speech Datasets and Ethical Decision-Making
Papakyriakopoulos, O., Choi, A. S. G., Thong, W. E., Zhao, D., Andrews, J. T. A., Bourke, R., Xiang, A., & Koenecke, A. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594049 · Cited by 59
“In this way, we bring awareness to the lack of content validity [108] in datasets that can result in representational harm [24].”
Background
Metadata
- Authors
- Jacobs, A. Z., & Wallach, H.
- Affiliations
- University of Michigan (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 375-385
- OpenAlex
- W2995006168
- Semantic Scholar
- 681f4fbce872f138cbac9cdd92e8f6ed89ba6f8d
- Cited by
- 362
- References
- 24
- Citations by year
- 2026: 65 · 2025: 55 · 2024: 54 · 2023: 82 · 2022: 47 · 2021: 41
- FWCI
- 50.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- measurement modeling, fairness, construct reliability, construct validity, operationalization, fairness definitions, algorithmic fairness, socioeconomic status, teacher effectiveness, recidivism risk, essentially contested concepts, measurement assumptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.