The politics and reciprocal (re) configuration of accountability and fairness in data-driven education
Open accessAbstract
As awareness of bias in educational machine learning applications increases, accountability for technologies and their impact on educational equality is becoming an increasingly important constituent of ethical conduct and accountability in education. This article critically examines the relationship between so-called algorithmic fairness and algorithmic accountability in education. I argue that operative political meanings of accountability and fairness are constructed, operationalized, and reciprocally configured in the performance of algorithmic accountability in education. Tools for measuring forms of unwanted bias in machine learning systems, and technical fixes for mitigating them, are value-laden and may conceal the politics behind quantifying educational inequality. Crucially, some approaches may also disincentivize systemic reforms for substantive equality in education in the name of accountability.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The risks with AI systems mainly come from data accountability and algorithm accountability.
“The risks with AI systems mainly come from data accountability [24,59] and algorithm accountability [60][61][62].”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [24]
Processes for data accuracy and accountability require resource allocation for auditing.
“Requires resource allocation for auditing”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [24]
Processes for data accuracy and accountability ensure reliability of data.
“Ensures reliability of data”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [24]
Cites
null (2016) · Philosophy Study · doi:10.17265/2159-5313/2016.09.003 · Cited by 28,317 · see citing works
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Analysing and Assessing Accountability: A Conceptual Framework <sup>1</sup>
Bovens, M. (2007) · European Law Journal · doi:10.1111/j.1468-0386.2007.00378.x · Cited by 2,667 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
Lascoumes, P., & Galès, P. L. (2007) · Governance · doi:10.1111/j.1468-0491.2007.00342.x · Cited by 1,490 · see citing works
(2016) · Open Research Online (The Open University) · Cited by 1,410 · see citing works
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149 · see citing works
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 2× · see citing works
“It cannot usually be satis fi ed simultaneously with other formal equality metrics, such as Equalized Odds , however (Kleinberg, Mullainathan, and Raghavan 2017).”
BackgroundThe Oxford Handbook of Ethics of AI
(2020) · doi:10.1093/oxfordhb/9780190067397.001.0001 · Cited by 459 · see citing works
Algorithmic Accountability and Public Reason
Binns, R. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0263-5 · Cited by 457 · cited 2× · see citing works
“What is called algorithmic accountability (Binns 2018; Reisman et al. 2018) thereby emerges as a nascent constituent of educational institutions ’ ethical conduct and accountability.”
BackgroundBoaler, J., Wiliam, D., & Brown, M. (2000) · British Educational Research Journal · doi:10.1080/713651583 · Cited by 428 · see citing works
High-Stakes Accountability in Urban Elementary Schools: Challenging or Reproducing Inequality? 1
Diamond, J. B., & Spillane, J. P. (2004) · Teachers College Record The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2004.00375.x · Cited by 363 · see citing works
Decision Theory for Discrimination-Aware Classification
Kamiran, F., Karim, A., & Zhang, X. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.45 · Cited by 300 · see citing works
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · see citing works
Towards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295 · see citing works
Editorial: the datafication of education
Jarke, J., & Breiter, A. (2019) · Learning Media and Technology · doi:10.1080/17439884.2019.1573833 · Cited by 286 · see citing works
Ability Grouping as a Self-Fulfilling Prophecy: A Micro-Analysis of Teacher-Student Interaction
Eder, D. J. (1981) · Sociology of Education · doi:10.2307/2112327 · Cited by 253 · see citing works
Toward multi-label sentiment analysis: a transfer learning based approach
Tao, J., & Fang, X. (2020) · Journal Of Big Data · doi:10.1186/s40537-019-0278-0 · Cited by 251 · see citing works
Making Sense of Test-Based Accountability in Education
Hamilton, L. S., Stecher, B. M., & Klein, S. P. (2002) · Defense Technical Information Center (DTIC) · Cited by 233 · see citing works
Verger, A., Fontdevila, C., & Parcerisa, L. (2019) · Discourse Studies in the Cultural Politics of Education · doi:10.1080/01596306.2019.1569882 · Cited by 187 · see citing works
How policy instruments are chosen: patterns of decision makers’ choices
Capano, G., & Lippi, A. A. (2016) · Policy Sciences · doi:10.1007/s11077-016-9267-8 · Cited by 164 · see citing works
Objectivity as standardization in data-scientific education policy, technology and governance
Williamson, B., & Piattoeva, N. (2018) · Learning Media and Technology · doi:10.1080/17439884.2018.1556215 · Cited by 161 · see citing works
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · see citing works
Opening or closing doors for students? Equity and data use in schools
Datnow, A., & Park, V. (2018) · Journal of Educational Change · doi:10.1007/s10833-018-9323-6 · Cited by 139 · see citing works
Algorithmic Fairness from a Non-ideal Perspective
Fazelpour, S., & Lipton, Z. C. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375828 · Cited by 119 · see citing works
50 Years of Test (Un)fairness: Lessons for Machine LearningPreprint
Hutchinson, B., & Mitchell, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10104 · Cited by 118 · see citing works
Algorithmic Bias in EducationPreprint
Baker, R. S., & Hawn, A. (2021) · doi:10.35542/osf.io/pbmvz · Cited by 117 · see citing works
The Politics of Accountability
Lewis, W. D., & Young, T. V. (2013) · Educational Policy · doi:10.1177/0895904812472725 · Cited by 100 · see citing works
A Conceptual Framework for Accountability in Education
Levin, H. M. (1974) · The School Review · doi:10.1086/443136 · Cited by 85 · see citing works
Sellar, S. (2015) · Discourse: Studies in the Cultural Politics of Education · doi:10.1080/01596306.2014.931117 · Cited by 84 · see citing works
Elastic numbers: national examinations data as a technology of government
Piattoeva, N. (2015) · Governing by Numbers · doi:10.1080/02680939.2014.937830 · Cited by 82 · see citing works
The false promise of risk assessments: epistemic reform and the limits of fairness
Green, B. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372869 · Cited by 80 · see citing works
Equal Educational Opportunity: A Definition
Coleman, J. S. (1975) · Oxford Review of Education · doi:10.1080/0305498750010104 · Cited by 71 · see citing works
The Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning
Green, B., & Hu, L. J. (2018) · International Conference on Machine Learning · Cited by 66 · see citing works
The politics of accountability
Ozga, J. (2019) · Journal of Educational Change · doi:10.1007/s10833-019-09354-2 · Cited by 64 · see citing works
School tracking and educational inequality: a comparison of 12 education systems in Switzerland
Felouzis, G., & Charmillot, S. (2012) · Comparative Education · doi:10.1080/03050068.2012.706032 · Cited by 57 · see citing works
Ongoing issues in test fairness
Camilli, G. (2013) · Educational Research and Evaluation · doi:10.1080/13803611.2013.767602 · Cited by 55 · see citing works
Accountability as a policy technology: accounting for education performance in Europe
Ozga, J. (2013) · International Review of Administrative Sciences · doi:10.1177/0020852313477763 · Cited by 54 · see citing works
The COVID-19 exams fiasco across the UK: four nations and two windows of opportunity
Kippin, S., & Cairney, P. (2021) · British Politics · doi:10.1057/s41293-021-00162-y · Cited by 51 · see citing works
The value of sharing: Branding and behaviour in a life and health insurance company
Jeanningros, H., & McFall, L. (2020) · Big Data & Society · doi:10.1177/2053951720950350 · Cited by 48 · see citing works
Ethics of Artificial Intelligence in Education
Kurni, M., Mohammed, M. S., & Srinivasa, K. G. (2023) · Springer eBooks · doi:10.1007/978-3-031-32653-0_12 · Cited by 48 · see citing works
“We called that a behavior”: The making of institutional data
Whitman, M. (2020) · Big Data & Society · doi:10.1177/2053951720932200 · Cited by 44 · see citing works
The Contestation of Tech Ethics: A Sociotechnical Approach to Ethics and Technology in ActionPreprint
Green, B. (2021) · Social Science Research Network · doi:10.2139/ssrn.3859358 · Cited by 20 · see citing works
Hartong, S., & Breiter, A. (2020) · Routledge eBooks · doi:10.4324/9781003014164-6 · Cited by 11 · see citing works
Reshaping Ability Grouping Through Big DataPreprint
Carmel, Y. H., & Ben-Shahar, T. H. (2017) · SSRN Electronic Journal · Cited by 4 · see citing works
The Myth of Accountability: How Data (Mis)Use Is Reinforcing the Problems of Public EducationPreprint
Fontaine, C. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3126460 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
- Authors
- Sahlgren, O.
- Affiliations
- Tampere University (FI)
- Corresponding author
- Sahlgren, Otto
- ORCID
- Sahlgren, Otto
- Venue
- Learning, Media and Technology
- Publisher
- Informa UK Limited
- Published
- 2021-10-01
- Volume
- 48
- Issue
- 1
- Pages
- 95-108
- ISSN
- 1743-9884 (ISSN-L)
- 1743-9892
- OpenAlex
- W3204629139
- Semantic Scholar
- 58999945dcdce99900012f46a894d0db30c28217
- Cited by
- 30
- References
- 50
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 6 · 2023: 6 · 2022: 6
- FWCI
- 5.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Learning Media and Technology (publishedVersion, cc-by)
- Tampere University Institutional Repository (Tampere University) (submittedVersion, cc-by)
- Funders
- Academy of Finland (326584)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Information Systems › Digital Education and Society
- Social Sciences › Political Science and International Relations › Global Educational Policies and Reforms
- Keywords
- algorithmic accountability, algorithmic fairness, machine learning bias, educational inequality, substantive equality, bias mitigation, systemic reform
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review