CoChat
Sign Up
Cited by 142Ethics and Social Impacts of AIArtificial Intelligence in EducationDigital Education and Society

The cyclical ethical effects of using artificial intelligence in education

Open access
Dieterle, Edward, Dede, Chris, Walker, Michael·2024·AI & SOCIETY
DOI 10.1007/s00146-022-01497-wPMC PMC9513289
Open this paper in CoChat — free accountOpen PDF

Abstract

Our synthetic review of the relevant and related literatures on the ethics and effects of using AI in education reveals five qualitatively distinct and interrelated divides associated with access, representation, algorithms, interpretations, and citizenship. We open our analysis by probing the ethical effects of algorithms and how teams of humans can plan for and mitigate bias when using AI tools and techniques to model and inform instructional decisions and predict learning outcomes. We then analyze the upstream divides that feed into and fuel the algorithmic divide, first investigating access (who does and does not have access to the hardware, software, and connectivity necessary to engage with AI-enhanced digital learning tools and platforms) and then representation (the factors making data either representative of the total population or over-representative of a subpopulation's preferences, thereby preventing objectivity and biasing understandings and outcomes). After that, we analyze the divides that are downstream of the algorithmic divide associated with interpretation (how learners, educators, and others understand the outputs of algorithms and use them to make decisions) and citizenship (how the other divides accumulate to impact interpretations of data by learners, educators, and others, in turn influencing behaviors and, over time, skills, culture, economic, health, and civic outcomes). At present, lacking ongoing reflection and action by learners, educators, educational leaders, designers, scholars, and policymakers, the five divides collectively create a vicious cycle and perpetuate structural biases in teaching and learning. However, increasing human responsibility and control over these divides can create a virtuous cycle that improves diversity, equity, and inclusion in education. We conclude the article by looking forward and discussing ways to increase educational opportunity and effectiveness for all by mitigating bias through a cycle of progressive improvement.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s00146-022-01497-w

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 55 · Cited by 142

Research works printed but not linked: 59

  1. Adjei S, Ostrow K, Erickson, E, Heffernan N (2017) Clustering students in ASSISTments: exploring system and school-level traits to advance personalization. In: Proceedings of the 10th international conference on Educational Data Mining, Hubei Province, China
  2. Bakhshi H, Downing J, Osborne M, Schneider P (2017). The future of skills: employment in 2030. Pearson and Nesta. https://futureskills.pearson.com/research/assets/pdfs/technical-report.pdf
  3. Buolamwini J, Gebru T (2018) Gender shades: Intersectional accuracy disparities in commercial gender classification. In: Proceedings of the 1st conference on fairness, accountability and transparency, New York, NY. http://proceedings.mlr.press
  4. Caliskan A Bryson JJ Narayanan A Semantics derived automatically from language corpora contain human-like biases Science 2017 356 6334 183 186 10.1126/science.aal4230 28408601 · DOI · Paper page
  5. Chardin M Novak K Equity by design: Delivering on the power and promise of UDL 2020 California Corwin Press
  6. Culp KM Honey M Mandinach E A retrospective on twenty years of education technology policy J Educ Comput Res 2005 32 3 279 307 10.2190/7w71-qvt2-pap2-udx7 · DOI
  7. D’Angelo C, Rutstein D, Harris C, Bernard R, Borokhovski E, Haertel G (2014) Simulations for STEM learning: systematic review and meta-analysis. SRI International. https://www.sri.com/wp-content/uploads/pdf/simulations-for-stem-learning-full-report.pdf
  8. Daniel BK Big data and data science: a critical review of issues for educational research Br J Edu Technol 2019 50 1 101 113 10.1111/bjet.12595 · DOI · Paper page
  9. Decuypere M Grimaldi E Landri P Introduction: critical studies of digital education platforms Crit Stud Educ 2021 62 1 1 16 10.1080/17508487.2020.1866050 · DOI
  10. Dede C Dede C Richards J Saxberg B Improving efficiency and effectiveness through learning engineering Learning engineering for online education: theoretical contexts and design-based examples 2019 UK Routledge 1 14
  11. Dede C Dede C Richards J Introduction: Reconceptualizing higher education and lifelong learning in the era of the synergistic digital economy The 60-year curriculum: new models for lifelong learning in the digital economy 2020 UK Routledge 1 24
  12. Dede C Richards J Saxberg B Learning engineering for online education: theoretical contexts and design-based examples 2019 UK Routledge
  13. Díaz P Ioannou A Bhagat KK Spector JM Learning in a digital world: perspective on interactive technologies for formal and informal education 2019 Berlin Springer
  14. D'Mello S Dieterle E Duckworth A Advanced, analytic, automated (AAA) measurement of engagement during learning Educ Psychol 2017 52 2 104 123 10.1080/00461520.2017.1281747 29038607 PMC5640167 · DOI
  15. Ercikan K Pellegrino JW Validation of score meaning for the next generation of assessments: the use of response processes 2017 UK Routledge
  16. Eubanks V (2017) Automating in equality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press
  17. European Economic and Social Committee The ethics of big data: balancing economic benefits and ethical questions of big data in the EU policy context European Union 2017 10.2864/18937 · DOI
  18. Ferrero F, Barujel AG (2019) Algorithmic driven decision-making systems in education: analyzing bias from the sociocultural perspective. In: 2019 XIV Latin American Conference on Learning Technologies (LACLO), 10.1109/LACLO49268.2019.00038 · DOI
  19. Fischer C Pardos ZA Baker RS Williams JJ Smyth P Yu R Slater S Baker R Warschauer M Mining big data in education: affordances and challenges Rev Res Educ 2020 44 1 130 160 10.3102/0091732x20903304 · DOI
  20. Giannakos MN Sharma K Pappas IO Kostakos V Velloso E Multimodal data as a means to understand the learning experience Int J Inf Manage 2019 48 108 119 10.1016/j.ijinfomgt.2019.02.003 · DOI
  21. Gilman M (2020) Poverty lawgorithms: a poverty lawyer’s guide to fighting automated decision-making harms on low-income communities. https://ssrn.com/abstract=3699650
  22. Guo H, Rios JA, Haberman S, Liu OL, Wang J, Paek I (2016) A new procedure for detection of students’ rapid guessing responses using response time. Appl Meas Educ 29(3):173–183. 10.1080/08957347.2016.1171766 · DOI
  23. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019) Ethically aligned design: a vision for prioritizing human well-being with autonomous and intelligent systems, 1st edn. IEEE. https://standards.ieee.org/content/dam/ieee-standards/standards/web/documents/other/ead1e.pdf
  24. Jiang W, Pardos ZA, Wei Q (2019) Goal-based course recommendation. In: Proceedings of the 9th international conference on learning analytics & knowledge, Tempe, AZ. 10.1145/3303772.3303814 · DOI
  25. Kirkpatrick K Battling algorithmic bias Commun ACM 2016 59 10 16 17 10.1145/2983270 · DOI
  26. Kostyuk V, Almeda MV, Baker RS (2018) Correlating affect and behavior in reasoning mind with state test achievement. In: Proceedings of the 8th international conference on learning analytics and knowledge, Sydney, Australia
  27. LearnPlatform (2021) Edtech engagement & digital learning equity gaps. Retrieved July 14, 2021. https://learnplatform.com/insights/infographic/2020-digital-equity-in-review
  28. Lee Y-H Haberman SJ Investigating test-taking behaviors using timing and process data Int J Test 2016 16 3 240 267 10.1080/15305058.2015.1085385 · DOI
  29. Levy R Dynamic Bayesian network modeling of game-based diagnostic assessments Multivar Behav Res 2019 54 6 771 794 10.1080/00273171.2019.1590794 30942094 · DOI
  30. Luckin R, Holmes W, Griffiths M, Forcier LB (2016) Intelligence unleashed: an argument for AI in education. Pearson. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/about-pearson/innovation/open-ideas/Intelligence-Unleashed-v15-Web.pdf
  31. MacLean CL Dror IE Robertson CT Kesselheim AS A primer on the psychology of cognitive bias Blinding as a solution to bias: strengthening biomedical science, forensic science, and law 2016 Cambridge Academic Press 13 24
  32. Mandinach EB Miskell RC Blended learning and data use in three technology-infused charter schools Learn Landsc 2018 11 1 183 198 10.36510/learnland.v11i1.931 · DOI
  33. Marco CA Larkin GL Research ethics: ethical issues of data reporting and the quest for authenticity Acad Emerg Med 2000 7 6 691 694 10.1111/j.1553-2712.2000.tb02049.x 10905651 · DOI
  34. Nance JP (2019) Implicit racial bias and students' fourth amendment rights. University of Florida Levin College of Law Research Paper No. 19-26. https://ssrn.com/abstract=3140638
  35. National Academies of Sciences, Engineering, and Medicine (2000). How people learn: brain, mind, experience, and school (expanded ed.). National Academy Press. 10.17226/9853 · DOI · Paper page
  36. National Academies of Sciences, Engineering, and Medicine How people learn II: learners, contexts, and cultures 2018 Washington The National Academy Press
  37. National Academies of Sciences, Engineering, and Medicine Climate change: evidence and causes Update 2020 2020 Washington The National Academies Press
  38. Nielsen MW Bloch CW Schiebinger L Making gender diversity work for scientific discovery and innovation Nat Hum Behav 2018 2 10 726 734 10.1038/s41562-018-0433-1 31406295 · DOI
  39. Osei Bonsu K Song J Turbulence on the global economy influenced by artificial intelligence and foreign policy inefficiencies J Liberty Int Affairs 2020 6 2 113 122 10.47305/JLIA2020113ob · DOI
  40. Pardos ZA Baker RS San Pedro M Gowda SM Gowda SM Affective states and state tests: Investigating how affect and engagement during the school year predict end-of-year learning outcomes J Learn Anal 2014 1 1 107 128 10.18608/jla.2014.11.6 · DOI
  41. Perkins D Smart schools: better thinking and learning for every child 1992 New York Free Press
  42. Ramsetty A Adams C Impact of the digital divide in the age of COVID-19 J Am Med Inform Assoc 2020 27 7 1147 1148 10.1093/jamia/ocaa078 32343813 PMC7197532 · DOI · Paper page
  43. Ritzhaupt AD Cheng L Luo W Hohlfeld TN Bishop MJ Boling E Elen J Svihla V The digital divide in formal educational settings: The past, present, and future relevance Handbook of research in educational communications and technology 2020 5 New York Springer 483 504
  44. Sao Pedro MA Baker RS Gobert JD Montalvo O Nakama A Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill User Model User-Adap Inter 2013 23 1 1 39 10.1007/s11257-011-9101-0 · DOI
  45. Schiff D Out of the laboratory and into the classroom: the future of artificial intelligence in education AI Soc 2021 36 1 331 348 10.1007/s00146-020-01033-8 32836908 PMC7415331 · DOI · Paper page
  46. Schildkamp K Lai MK Earl LM Data-based decision making in education: challenges and opportunities 2013 New York Springer
  47. Schildkamp K Poortman CL Ebbeler J Pieters JM How school leaders can build effective data teams: five building blocks for a new wave of data-informed decision making J Educ Change 2019 20 3 283 325 10.1007/s10833-019-09345-3 · DOI
  48. Shah DS, Schwartz HA, Hovy D (2020) Predictive biases in natural language processing models: a conceptual framework and overview. In: Proceedings of the 58th annual meeting of the association for computational linguistics
  49. Shao E Guo S Pardos ZA Degree planning with PLAN-BERT: multi-semester recommendation using future courses of interest Proc AAAI Conf Artif Intell 2021 35 17 14920 14929
  50. Shepard LA Classroom assessment to support teaching and learning Ann Am Acad Pol Soc Sci 2019 683 1 183 200 10.1177/0002716219843818 · DOI
  51. Shohamy E Democratic assessment as an alternative Lang Test 2016 18 4 373 391 10.1177/026553220101800404 · DOI
  52. Shute V Wang L Rupp AA Leighton JP Assessing and supporting hard-to-measure constructs in video games The handbook of cognition and assessment: frameworks, methodologies, and applications 2017 New Jersey John Wiley & Sons 535 562
  53. Shute V Rahimi S Smith G Ke F Almond R Dai CP Kuba R Liu Z Yang X Sun C Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games J Comput Assist Learn 2020 10.1111/jcal.12473 · DOI
  54. Stadler M Fischer F Greiff S Taking a closer look: an exploratory analysis of successful and unsuccessful strategy use in complex problems Front Psychol 2019 10.3389/fpsyg.2019.00777 PMC6514184 31133910 · DOI
  55. Valadez JR Duran R Redefining the digital divide: beyond access to computers and the internet High School J 2007 90 3 31 44 10.1353/hsj.2007.0013 · DOI
  56. van Rijn PW Ali US A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing Br J Math Stat Psychol 2017 70 2 317 345 10.1111/bmsp.12101 28474769 · DOI
  57. Wang Y Kai S Baker RS Bittencourt II Cukurova M Muldner K Luckin R Millán E Early detection of wheel-spinning in ASSISTments Artificial intelligence in education. AIED 2020. Lecture notes in computer science 2020 New York Springer International Publishing 574 585
  58. Yapo A, Weiss J (2018) Ethical implications of bias in machine learning. In: Hawaii international conference on system sciences 2018, Waikoloa Village, HI
  59. Završnik A (2019) Algorithmic justice: Algorithms and big data in criminal justice settings. Eur J Criminol. 10.1177/1477370819876762 · DOI

Authors
Dieterle, E., Dede, C., & Walker, M.
Affiliations
  • Educational Testing Service (US)
  • Harvard University (US)
Corresponding author
Dieterle, Edward
ORCID
Venue
AI & SOCIETY
Publisher
Springer Science and Business Media LLC
Published
2022-09-27
Volume
39
Issue
2
Pages
633-643
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
DOI
10.1007/s00146-022-01497-w
PMC
PMC9513289
OpenAlex
W4297371803
Cited by
142
References
55
Citations by year
2026: 41 · 2025: 61 · 2024: 34 · 2023: 5 · 2021: 1
FWCI
29.00
Type
journal article
Language
en
Open access
Licence
Locations
  • AI & Society (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Information Systems › Digital Education and Society
Keywords
artificial intelligence in education, algorithmic bias, algorithmic fairness, access to AI, algorithmic interpretation, digital citizenship, diversity, equity, and inclusion, learning outcome prediction, human oversight of AI, educational equity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review