The DOI matches its Crossref record; 50 of the 53 references it cites were found in a registry or an index.
The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates
Open accessAbstract
The transition of Artificial Intelligence (AI) from a lab-based science to live human contexts brings into sharp focus many historic, socio-cultural biases, inequalities, and moral dilemmas. Many questions that have been raised regarding the broader ethics of AI are also relevant for AI in Education (AIED). AIED raises further specific challenges related to the impact of its technologies on users, how such technologies might be used to reinforce or alter the way that we learn and teach, and what we, as a society and individuals, value as outcomes of education. This chapter discusses key ethical dimensions of AI and contextualises them within AIED design and engineering practices to draw connections between the AIED systems we build, the questions about human learning and development we ask, the ethics of the pedagogies we use, and the considerations of values that we promote in and through AIED within a wider socio-technical system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Crucial AI ethics considerations are equally underexamined in AIEd [60, 103].”
From AI Sensing and Intervention in Higher Education: Student Perceptions of Learning Impacts, Affective Responses, and Ethical Priorities (2026) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2018) · Cited by 3,510
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2018) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,320
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 1×
“This approach is deeply ingrained in traditional educational practices focused on school-based subject-domain teaching, which was further boosted by Bloom’s influential “2-sigma effect” studies (Bloom, 1984).”
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631 · cited 2×
“…framework, data, models, and education constitute the foundational level; while at the overlaps is a second level: the ethics of data in AI (e.g., Floridi and Cowls, 2019), the ethics of data in education (e.g., Ferguson et al. 2016), and the ethics of models and algorithms applied in…”
BackgroundEthics of AI in Education: Towards a Community-Wide Framework
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., et al. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00239-1 · Cited by 1,437
Self-citationLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 1×
“Bias in AI has long been the subject of research, revealing overlapping definitions and different points at which it may enter the AI development and deployment pipeline (Crawford, 2017a; Blodgett, Barocas, Daumé & Wallach 2020; Baker, Ocumpaugh & Andres, 2021).”
Background(2019) · Open Research Online (The Open University) · Cited by 1,004
Self-citationBaker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
AI and education: guidance for policy-makers
UNESCO (2021) · doi:10.54675/pcsp7350 · Cited by 929
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783
(2019) · Cited by 771
Algorithmic Bias in Autonomous Systems
Danks, D., & London, A. J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/654 · Cited by 472 · cited 1×
“Danks & London (2017) distinguish between: (a) moral bias , i.e., a deviation from certain moral principles related to equity, autonomy and human rights; (b) legal bias , i.e., undue prejudice, such as judgments based on preconceived notions of particular groups, which violates written legal norms…”
BackgroundGuha, M. (2000) · Reference Reviews · doi:10.1108/rr.2000.14.4.8.171 · Cited by 302
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247
Susan Bull, A., & Kay, J. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0090-8 · Cited by 180
Research Evidence on the Use of Learning Analytics: Implications for Education Policy
Ferguson, R., Andrew Brasher, A., Clow, D., Adam, C., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T., & Riina, V. (2016) · Research Explorer (The University of Manchester) · doi:10.2791/955210 · Cited by 179
Carter, L. D., Liu, D., & Cantrell, C. (2020) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00138 · Cited by 172
AI in Education needs interpretable machine learning: Lessons from Open Learner ModellingPreprint
Conati, C., Porayska‐Pomsta, K., & Mavrikis, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00154 · Cited by 163
Self-citationResponses to Critiques on Machine Learning of Criminality Perceptions (Addendum of arXiv:1611.04135)Preprint
Wu, X., & Zhang, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04135 · Cited by 140
Opportunity for all? Technology and learning in lower-income families
Rideout, V. J., & Katz, V. S. (2016) · Cited by 138
Challenges for the Future of Educational Data Mining: The Baker Learning Analytics Prizes
Baker, R. S. (2019) · Journal of Educational Data Mining · doi:10.5281/zenodo.3554745 · Cited by 135
The Wisdom of Nature: An Evolutionary Heuristic for Human Enhancement
Bostrom, N., & Sandberg, A. (2009) · doi:10.1093/oso/9780199299720.003.0019 · Cited by 135
Digital Technology and the Futures of Education: Towards ‘Non-Stupid’ Optimism
Facer, K., & Selwyn, N. (2021) · Bristol Research (University of Bristol) · Cited by 129
Chinese Number Words, Culture, and Mathematics Learning
Ng, S. S. N., & Rao, N. (2010) · Review of Educational Research · doi:10.3102/0034654310364764 · Cited by 128
Pedagogical agents for learning by teaching: Teachable agents
Blair, K. P. (2007) · Cited by 91
Kulynych, B., Overdorf, R., Troncoso, C., & Gürses, S. F. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372853 · Cited by 87
The UnLocke Team, Wilkinson, H. R., Smid, C. R., Morris, S., Farran, E. K., Dumontheil, I., Mayer, S., Tolmie, A. K., Bell, D., Porayska‐Pomsta, K., et al. (2019) · Journal of Cognitive Enhancement · doi:10.1007/s41465-019-00161-4 · Cited by 76
Self-citationYu, R., Li, Q., Fischer, C., Doroudi, S., & Xu, D. (2020) · Educational Data Mining · Cited by 71
The Effects of Culturally Congruent Educational Technologies on Student Achievement
Finkelstein, S., Yarzebinski, E., Vaughn, C., Ogan, A., & Cassell, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39112-5_50 · Cited by 68
Williamson, B., Eynon, R., Knox, J., & Davies, H. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781800375413.00037 · Cited by 57
Porayska‐Pomsta, K., & Rajendran, G. (2019) · Perspectives on rethinking and reforming education · doi:10.1007/978-981-13-8161-4_3 · Cited by 56
Self-citationTeaching Machines: The History of Personalized Learning, 2021.
Tursunkulova, I. (2022) · Canadian Journal of Learning and Technology · doi:10.21432/cjlt28248 · Cited by 52
Evaluation of Fairness Trade-offs in Predicting Student SuccessPreprint
Lee, H., & Kizilcec, R. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00088 · Cited by 49
Designing for “challenge” in a large‐scale adaptive literacy game for primary school children
Benton, L., Mavrikis, M., Vasalou, A., Joye, N., Sumner, E., Herbert, E., Révész, A., Symvonis, A., & Raftopoulou, C. N. (2021) · British Journal of Educational Technology · doi:10.1111/bjet.13146 · Cited by 45
Beyond "Fairness:" Structural (In)justice Lenses on AI for EducationPreprint
Madaio, M., Blodgett, S. L., Mayfield, E., & Dixon‐Román, E. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08847 · Cited by 39
Discrimination in the Age of AlgorithmsPreprint
Kleinberg, J. M., Ludwig, J., Mullainathan, S., & Sunstein, C. R. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3329669 · Cited by 38
Guldberg, K., Parsons, S., Porayska‐Pomsta, K., & Keay‐Bright, W. (2017) · British Educational Research Journal · doi:10.1002/berj.3275 · Cited by 36
Self-citationUnderstanding Potential Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2021) · doi:10.21428/2c646de5.c16a07bb · Cited by 34
Designing Appropriate Learning Technologies for School vs Home Settings in Tanzanian Rural Villages
Uchidiuno, J., Yarzebinski, E., Madaio, M., Maheshwari, N., Koedinger, K., & Ogan, A. (2018) · Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies · doi:10.1145/3209811.3209881 · Cited by 34
Building an Intelligent, Authorable Serious Game for Autistic Children and Their Carers
Porayska‐Pomsta, K., Anderson, K., Bernardini, S., Guldberg, K., Smith, T. J., Kossivaki, L., Hodgins, S., & Lowe, I. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-03161-3_34 · Cited by 30
Self-citationThe Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates
Rose, J. (2024) · The Serials Librarian · doi:10.1080/0361526x.2024.2427948 · Cited by 23
Gauthier, A., Porayska‐Pomsta, K., Dumontheil, I., Mayer, S., & Mareschal, D. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3485168 · Cited by 22
Self-citationBuilding Autonomous Social Partners for Autistic Children
Bernardini, S., Porayska‐Pomsta, K., Smith, T. J., & Avramides, K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33197-8_5 · Cited by 14
Self-citationTreviranus, J. (2022) · Routledge eBooks · doi:10.4324/9780429329067-3 · Cited by 14
Tang, Y., & Liu, Y. (2009) · Progress in brain research · doi:10.1016/s0079-6123(09)17810-x · Cited by 12
Rashedi, R. N., Bonnet, K. R., Schulte, R. J., Schlundt, D., Swanson, A. R., Kinsman, A., Bardett, N., Warren, Z. E., Juárez, P., Biswas, G., et al. (2020) · Proceedings. · Cited by 2
The UnLocke Project Team, Gauthier, A., Porayska‐Pomsta, K., & Mareschal, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-52240-7_19 · Cited by 1
Self-citationAn implementation of a self-improving teaching program
O’Shea, T. (1979) · Birkhäuser Basel eBooks · doi:10.1007/978-3-0348-6553-1_4 · Cited by 1
Flagged first, then by citations
3 of the 53 references couldn’t be matched to a known paper.
Research works printed but not linked: 24
- Barocas, S., Crawford, K., Shapiro, A. and Wallach, H. (2017). The Problem With Bias: Allocative Versus Representational Harms in Machine Learning. In SIGCIS Conference. http://meetings.sigcis.org/uploads/6/3/6/8/6368912/program.pdf (accessed 1 September 2021).
- Blikstein, P. (2018). Time to Make Hard Choices for AI in Education. Keynote at the 19 th International Conference on Artificial Intelligence in Education, London 2018.
- Bogdan, K., Overdorf, R., Troncoso, C., and Gürses, S. (2018). POTs: Protective Optimization Technologies. arXiv.org. https://arxiv.org/abs/1806.02711. · arXiv
- Bolukbasi, T, Chang, K.-W., Zou, J.Y., Saligrama, V. and Kalai, A. T. (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pages 4349-4357, Curran Associates, Inc.
- Cramer, H., Vaughan, W. J., & Holstein, K. (2019). Challenges of incorporating algorithmic 'fairness' into practice. Tutorial at the ACM Conference on Fairness, Accountability, and Transparency (FAT* 2019). (accessed 1 September 2021).
- du Boulay, B. (2022). Artificial Intelligence in Education: A brief discussion about ethics. In W. Holmes and K. Porayska-Pomsta (Eds.) The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates. Routledge.
- Floridi, L. (2013). The Ethics of Information, Oxford, Oxford University Press.
- Grawemayer, B., Mavrikis, M., Holmes, W., Gutierrez-Santos, S., Wiedmann, M., Rummel, N. (2017). Affective Learning: Improving engagement and enhancing learning with affect- aware feedback, User modelling and User-Adapted Interaction, 27(1), 119-158.
- Heaven, W D. (2020). Predictive Policing Algorithms are Racist. They need to be dismantled. MIT Technology Review, July 17, 2020 (accessed 1 September 2021).
- Holstein, K. and Doroudi, S. (2022). In W. Holmes and K. Porayska-Pomsta (Eds.) The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates. Routledge.
- Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudik, M., & Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1- 16) (accessed 1 September 2021).
- IEEE General Principles (2017) https://standards.ieee.org/content/dam/ieee- standards/standards/web/documents/other/ead1e_general_principles.pdf
- Jobin, A., Ienca, M., & Vayena, E. (2019). Artificial Intelligence: The global landscape of ethics guidelines. Nature Machine Intelligence, 1(9), 389-399. https://doi.org/10.1038/s42256-019-0088-2Kant, 1785. · DOI
- Kelly, K. (2010). What Technology Wants, Penguin Books.
- Kurzweil, R. (2014). The singularity is near. In Ethics and emerging technologies (pp. 393-406). Palgrave Macmillan, London.
- Madaio, M., Yarzebinski, E., Zinszer, B., Kamath, V., Akpe, H., Seri, A.B., Tanoh, F., Hannon-Cropp, J., Cassell, J., Jasinska, K., Ogam, A. (2020). Motivations and Barriers for Family Engagement with Home Literacy Technology in Rural Communities. CHI Conference on Human Factors in Computing Systems, 2020, ACM.
- Narayanan, A. (2018). 21 Fairness Definitions and their Politics. Tutorial at the FACCT Conference, 2018. Video available at https://www.youtube.com/watch?v=jIXIuYdnyyk (accessed 1 September 2021).
- Noble, S. (2018). Algorithms of Oppression. New York, NY: NYU Press.
- Porayska-Pomsta, K., Alcorn, A M., Avramides, K., Beale, S., Bernardini, S., Foster, M-E., Frauenberger, C., Good, J., Guldberg, K., Keay-Bright, W., Kossyvaki, L., Lemon, O., Mademtzi, M., Menzies, R., Pain, H., Rajendran, G., Waller, A., Sam Wass, Smith, T.J. (2018). Blending human and artificial intelligence to support autistic children of social communication skills, TOCHI, 25(6), 1-35, ACM
- Rawls, J. (1958). Justice as Fairness, The Philosophical Review, 67(2).
- Ross, W. D. (1930). The Right and the Good. Clarendon Press. The Royal Society.
- Saxena, M., Pillai, R. K., Mostow, J. (2018). Relating Children's Automatically Detected Facial Expressions to Their Behaviour in RoboTutor. In 32 nd AAAI Conference on Artificial Intelligence.
- Sweeney, Latanya. May 1, 2013. Discrimination in Online Ad Delivery. ACM 56(5), 44-54.
- The Royal Society Workshop Report (2019): Digital technology and human transformation, November 2019 DES6259, https://royalsociety.org/- /media/policy/Publications/2020/human-transformation-summary.pdf
Cited by
Sorted by citations
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 1,435
“There are ongoing debates regarding AI ethics in education with a mixture of focuses on algorithmic and human ethics among educational data mining and AI in education communities (Holmes and Porayska-Pomsta, 2022).”
BackgroundExplainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603197 · Cited by 350
Bulathwela, S., Pérez-Ortiz, M., Holloway, C., Cukurova, M., & Shawe-Taylor, J. (2024) · Sustainability · doi:10.3390/su16020781 · Cited by 269
Zhang, J., & Zhang, Z. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12988 · Cited by 263
Metadata
- Authors
- Holmes, W., & Porayska-Pomsta, K.
- Affiliations
- University College London (GB)
- ORCID
- Holmes, Wayne
- Venue
- Routledge eBooks
- Publisher
- Routledge
- Published
- 2022-08-11
- OpenAlex
- W4301626130
- Cited by
- 258
- References
- 53
- Citations by year
- 2026: 66 · 2025: 83 · 2024: 45 · 2023: 59 · 2022: 4
- FWCI
- 67.15
- Type
- book
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Routledge eBooks (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- artificial intelligence in education, AI ethics, educational technology design, educational inequality, pedagogy, educational values, socio-technical systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
