The many dimensions of algorithmic fairness in educational applications
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The issues of algorithmic fairness and bias have recently featured prominently in many publications highlighting the fact that training the algorithms for maximum performance may often result in predictions that are biased against various groups. Educational applications based on NLP and speech processing technologies often combine multiple complex machine learning algorithms and are thus vulnerable to the same sources of bias as other machine learning systems. Yet such systems can have high impact on people’s lives especially when deployed as part of high-stakes tests. In this paper we discuss different definitions of fairness and possible ways to apply them to educational applications. We then use simulated and real data to consider how test-takers’ native language backgrounds can affect their automated scores on an English language proficiency assessment. We illustrate that total fairness may not be achievable and that different definitions of fairness may require different solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · cited 6×
“In their review of algorithmic bias in the context of criminal justice risk assessment, Berk et al. (2018) offer five definitions of fairness: (1) Overall accuracy equality: overall classification accuracy is the same for all groups; (2) Statistical parity: the marginal distribution of predicted…”
BackgroundResultInfluentialFairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694 · cited 1×
“…publications highlighting the fact that training the algorithms for maximum performance may often result in predictions that are biased against various groups (Kamiran and Calders, 2009; Kamishima et al., 2012; Luong et al., 2011; Zemel et al., 2013; Feldman et al., 2015; Friedler et al., 2016).”
BackgroundA framework for evaluation and use of automated scoring
Williamson, D. M., Xi, X., & Breyer, F. J. (2012) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2011.00223.x · Cited by 430 · cited 4×
“Absolute values above 0.1 are considered an issue of concern and require further evaluation (Williamson et al., 2012).”
BackgroundMethodInfluentialClassifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · cited 1×
“…publications highlighting the fact that training the algorithms for maximum performance may often result in predictions that are biased against various groups (Kamiran and Calders, 2009; Kamishima et al., 2012; Luong et al., 2011; Zemel et al., 2013; Feldman et al., 2015; Friedler et al., 2016).”
BackgroundThe social impact of natural language processing
Hovy, D., & Spruit, S. L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2096 · Cited by 329 · cited 1×
“Like any algorithm, NLP systems are not immune to such bias (Hovy and Spruit, 2016; Caliskan et al., 2017).”
k-NN as an implementation of situation testing for discrimination discovery and prevention
Luong, B. T., Ruggieri, S., & Turini, F. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020488 · Cited by 263 · cited 1×
“…publications highlighting the fact that training the algorithms for maximum performance may often result in predictions that are biased against various groups (Kamiran and Calders, 2009; Kamishima et al., 2012; Luong et al., 2011; Zemel et al., 2013; Feldman et al., 2015; Friedler et al., 2016).”
BackgroundOn the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247
Automated essay scoring: Psychometric guidelines and practices
Ramineni, C., & Williamson, D. M. (2013) · Assessing Writing · doi:10.1016/j.asw.2012.10.004 · Cited by 127 · cited 3×
“The two most common analyses are standardized mean score differences and overall model performance for different groups with human scores (predictive ability) (Ramineni and Williamson, 2013; Williamson et al., 2012).”
Background
Cited by
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
“…(2019); Bhargava and Forsyth (2019); Cao and Daumé (2019); Font and Costa-jussà (2019); Garg et al. (2019); Garimella et al. (2019); Liu et al. (2019); Loukina et al. (2019); Mehrabi et al. (2019); Nozza et al. (2019); Sap et al. (2019); Sheng et al. (2019); Stanovsky et al. (2019); Vaidya et al.…”
BackgroundInfluentialBaker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
An automated essay scoring systems: a systematic literature review
Ramesh, D., & Sanampudi, S. K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10068-2 · Cited by 550
Assessing Algorithmic Fairness in Automatic Classifiers of Educational Forum Posts
Sha, L., Rakovic, M., Whitelock-Wainwright, A., Carroll, D., Yew, V. M., Gasevic, D., & Chen, G. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-78292-4_31 · Cited by 34
Metadata
- Authors
- Loukina, A., Madnani, N., & Zechner, K.
- Venue
- Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications
- Published
- 2019
- Pages
- 1-10
- OpenAlex
- W2970052781
- Cited by
- 62
- References
- 9
- FWCI
- 6.55
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The many dimensions of algorithmic fairness in educational applications. https://app.cochat.ai/papers/10.18653/v1/w19-4401
- The paper
- Loukina, A., Madnani, N., & Zechner, K. (2019). The many dimensions of algorithmic fairness in educational applications. Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications. https://doi.org/10.18653/v1/w19-4401
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.