ANOTHER LOOK AT “CULTURAL FAIRNESS”1
Abstract
Four definitions of “cultural fairness” are examined and found to be not only mutually contradictory (for reasons which are explained), but all based on the false view that optimum treatment of cultural factors in test construction or test selection can be reduced to completely mechanical procedures. If a conflict arises between the two goals of maximizing a test's validity and minimizing the test's discrimination against certain cultural groups, then a subjective, policy‐level decision must be made concerning the relative importance of the two goals. The terms in which this judgment should be made are described, and methods are described for entering the result of this judgment into mechanical procedures for constructing a “culturally optimum” test. Such a test will not necessarily fit any of the four definitions of “cultural fairness.”
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Much of the early literature into fairness considered regression and correlation-based scenarios, compared to the recent emphasis on classification.
“It is noteworthy that much of the early literature into fairness considered regression and correlation-based scenarios (standardized test scores, see e.g., [81]), compared to the recent emphasis on classiication (binary decisions).”
From Fairness in machine learning: A survey (2024) · cites this paper as [81]
The general notions from early fairness literature transfer to (binary) classification settings and define essential concepts such as protected/demographic variables, notions of group vs individual fairness, impossibility conditions between fairness conditions, and metric-based fairness quantification.
“However, the general notions transfer to (binary) classiication settings and thus deine essential concepts such as protected/demographic variables (e.g.[72,73]), notions of group vs individual fairness (e.g.[257,278]), impossibility conditions between fairness conditions ( [81]), and metric-based fairness quantiication (e.g., true positive rates, [74]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [81]
Cites
TEST BIAS: PREDICTION OF GRADES OF NEGRO AND WHITE STUDENTS IN INTEGRATED COLLEGES
Cleary, T. A. (1968) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1968.tb00613.x · Cited by 658 · see citing works
THORNDIKE, R. L. (1971) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1971.tb00907.x · Cited by 265 · see citing works
CONSIDERATIONS FOR STUDIES OF TEST BIAS
LINN, R. L., & WERTS, C. E. (1971) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1971.tb00898.x · Cited by 113 · see citing works
Methodological considerations relevant to discrimination in employment testing.
Einhorn, H. J., & Bass, A. R. (1971) · Psychological Bulletin · doi:10.1037/h0030871 · Cited by 75 · see citing works
An Introduction to Psychological Statistics.
Maxwell, A. E., & DuBois, P. H. (1965) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343482 · Cited by 65 · see citing works
An Introduction to Psychological Statistics.
Brewer, J. K., & DuBois, P. H. (1966) · Journal of the American Statistical Association · doi:10.2307/2283073 · Cited by 45 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993 · see citing works
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
UTILITIES AND THE ISSUE OF FAIRNESS IN A DECISION THEORETIC MODEL FOR SELECTION
SAWYER, R. L., COLE, N. S., & COLE, J. W. L. (1976) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1976.tb00182.x · Cited by 47 · see citing works
Reeb, M. (1976) · Journal of Applied Psychology · doi:10.1037//0021-9010.61.3.253 · Cited by 10 · see citing works
Non-Discriminatory Personalized Pricing
Strack, P., & Yang, K. H. (2025) · arXiv · doi:10.48550/arxiv.2506.20925
- Authors
- DARLINGTON, R. B.
- Affiliations
- Cornell University (US)
- Corresponding author
- DARLINGTON, RICHARD B.
- Venue
- Journal of Educational Measurement
- Publisher
- Wiley
- Published
- 1971-06
- Volume
- 8
- Issue
- 2
- Pages
- 71-82
- ISSN
- 0022-0655 (ISSN-L)
- 1745-3984
- OpenAlex
- W2055851514
- Cited by
- 199
- References
- 6
- Citations by year
- 2026: 8 · 2025: 2 · 2024: 4 · 2023: 6 · 2022: 4 · 2021: 7 · 2020: 8 · 2019: 8 · 2018: 2 · 2015: 2 · 2014: 7 · 2013: 2 · 2012: 2
- FWCI
- 13.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Measurement (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Educational Assessment and Pedagogy
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Keywords
- cultural fairness, test development, test selection, test validity, cultural bias in testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review