Potential Risks of Artificial Intelligence Integration into School Education: A Systematic Review
The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Currently, artificial intelligence (AI) is being rapidly incorporated into K-12 education because of its increasing social and pedagogical importance. The integration of AI in K-12 education is likely to have a profound influence on the lives and learning styles of learners, teaching approaches of teachers, and the whole mechanism of school management systems. As AI technologies are new to K-12 school curriculum, the research on AI for K-12 classrooms is under explored. In this study, the current state of AI integration in school education and risks associated with it were explored. The study using a systematic review method attempted to explore the findings, observations, and results of recent research regarding the possible risk factors of AI integration in K-12 education. At initial search using predefined key terms, 390 articles were recorded. Using inclusion and exclusion criteria, 71 articles covering 34 journals and other publications were selected for final analysis. Selected 71 articles reported that AI innovation incorporation into K-12 education is associated with certain risks and challenges. Through a systematic review technique, we categorized them into major 6 risk areas namely Privacy and Autonomy Risks, AI Biases, Accuracy and Functional Risks, Deepfakes and FATE Risks, Social Knowledge and Skill Building Risks, and Risk in Shifting Teacher's Role. The study also explored these risk areas to provide an overview of how these are connected with teaching and learning process of K-12 education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“Various social media (e.g., Instagram) has been overwhelmed with procre-ative AI system such as deepfakes and creative fi lters (Ali et al., 2021; Kaya, 2019), which can mislead young learners through sharing fake information faster than valid information (Vosoughi et al., 2018).”
BackgroundZawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1×
“Therefore, the privacy issue must be taken care before applying AI-based personalized teaching and learning technologies (Zawacki-Richter et al., 2019).”
BackgroundArtificial Intelligence in Education: A Review
Chen, L., Chen, P., & Lin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2988510 · Cited by 4,133
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 2×
“Although, advances in AI automation increases productivity in all sectors, it reduces the need of human workers and heighten the concern of replacing human workforce (Long & Magerko, 2020).”
BackgroundVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Exploring the impact of artificial intelligence on teaching and learning in higher education
Popenici, S. A. D., & Kerr, S. (2017) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-017-0062-8 · Cited by 2,019 · cited 1×
“In this vein, teaching job in education necessitates a substantial change of teaches ’ roles and pedagogies due to the rapid rate of AI integration and its in fl uence in the fi eld (Popenici & Kerr, 2017).”
BackgroundArtificial intelligence in education: Addressing ethical challenges in K-12 settings
Akgun, S., & Greenhow, C. (2022) · AI and Ethics · doi:10.1007/s43681-021-00096-7 · Cited by 1,473 · cited 8×
“…2022), Assessment and Grading (Feathers, 2019; Lin et al., 2021), Natural Language Processing (Wang & Cheng, 2021; Zanetti et al., 2019), AI surveillance (Akgun & Greenhow, 2022; Seo et al., 2021) are associated with several risks for both teachers and students in particular and school in general.”
BackgroundInfluentialEthics of AI in Education: Towards a Community-Wide Framework
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., et al. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00239-1 · Cited by 1,437
Vision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435 · cited 11×
“ITS and adaptive learning are signi fi cantly used to observe students ’ learning process, analyse their learning performances, and provide instant support, hints and guidance according to their needs.”
BackgroundInfluentialA Review of Artificial Intelligence (AI) in Education from 2010 to 2020
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., & Li, Y. (2021) · Complexity · doi:10.1155/2021/8812542 · Cited by 1,364 · cited 2×
“…rst century ’ s digital skills (Dermeval et al., 2018), improve learning performances (Zhang et al., 2022), provide smart learning environment (Zhai et al., 2021), solve science and mathematical problems (Yannier et al., 2020), get customized learning assistant (Xu & Ouyang, 2022), and help…”
BackgroundDeepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Application and theory gaps during the rise of artificial intelligence in education
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100002 · Cited by 1,003
AI and education: guidance for policy-makers
UNESCO (2021) · doi:10.54675/pcsp7350 · Cited by 929
Envisioning AI for K-12: What Should Every Child Know about AI?
Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019795 · Cited by 918
The impact of artificial intelligence on learner–instructor interaction in online learning
Seo, K., Tang, J., Roll, I., Fels, S., & Yoon, D. (2021) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-021-00292-9 · Cited by 788 · cited 4×
“…2022), Assessment and Grading (Feathers, 2019; Lin et al., 2021), Natural Language Processing (Wang & Cheng, 2021; Zanetti et al., 2019), AI surveillance (Akgun & Greenhow, 2022; Seo et al., 2021) are associated with several risks for both teachers and students in particular and school in general.”
BackgroundInfluentialHwang, G.-J., & Chien, S.-Y. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100082 · Cited by 759 · cited 1×
“Hwang and Chien (2022) also observed potential ethical issues of using AI in teaching and learning, such as privacy offending, bullying, cheating, and educational inequality.”
BackgroundKim, J., Lee, H., & Cho, Y. H. (2022) · Education and Information Technologies · doi:10.1007/s10639-021-10831-6 · Cited by 669 · cited 2×
“AI system needs high volumes of data to run and support AIEd, eventually there may have high privacy risk if data goes wrong, and can threaten children ’ s rights to privacy, safety, and security (Holmes et al., 2022; Kim et al., 2022; Su & Zhong, 2022; Yau et al., 2022).”
BackgroundHeffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Viewpoint: when will AI exceed human performance? evidence from AI experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11222 · Cited by 579
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare
Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0288-5 · Cited by 553
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Chiu, T. K. F., & Chai, C.-S. (2020) · Sustainability · doi:10.3390/su12145568 · Cited by 448
Empowering Educators to Be AI-Ready
Luckin, R., Cukurova, M., Kent, C., & du Boulay, B. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100076 · Cited by 422 · cited 2×
“An AI system turn bias in any form because it is trained inadequately with poor data or due to biased data feed in its system (Luckin et al., 2022).”
BackgroundZhang, H., Lee, I., Ali, S., DiPaola, D., Cheng, Y., & Breazeal, C. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00293-3 · Cited by 396
Ethics and Privacy in AI and Big Data: Implementing Responsible Research and Innovation
Stahl, B. C., & Wright, D. (2018) · IEEE Security & Privacy · doi:10.1109/msp.2018.2701164 · Cited by 374 · cited 1×
“These biases embedded into algorithmic model with no explicit intention appears in the form of racial discrimination (Andrejevic & Selwyn, 2020; Stahl & Wright, 2018).”
BackgroundKim, N. J., & Kim, M. K. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.755914 · Cited by 298 · cited 1×
“The over emphasis on AI system in teaching tasks will replace human teachers, decrease the role of teachers, and create bias in all forms (Kim & Kim, 2022).”
BackgroundRegan, P. M., & Jesse, J. (2019) · Ethics and Information Technology · doi:10.1007/s10676-018-9492-2 · Cited by 286 · cited 1×
“…tracks students and teachers ’ activities but also reveals students ’ future preferences, strengths, weaknesses, and learning patterns and actions (Regan & Jesse, 2019), which threatens students ’ rights of autonomy, and kept them under fear of speech, and holds them back from active…”
BackgroundWhen Will AI Exceed Human Performance? Evidence from AI Experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017) · arXiv · doi:10.48550/arxiv.1705.08807 · Cited by 285 · cited 1×
“Though everyone is not in position to agree, the rapid progress in AI and robotics completely take over human jobs in near future (Grace et al., 2018).”
BackgroundFacial recognition technology in schools: critical questions and concerns
Andrejevic, M., & Selwyn, N. (2020) · Learning, Media and Technology · doi:10.1080/17439884.2020.1686014 · Cited by 284 · cited 3×
“…performances, strengths, and weaknesses of learners, is sometimes intrinsically oppressive because it is most likely to apply with the intention of controlling students and teachers ’ natural expression (Andrejevic & Selwyn, 2020; Dewan et al., 2019; Montgomery & Marais, 2014; Valera et al., 2015).”
BackgroundAI in education: learner choice and fundamental rights
Berendt, B., Littlejohn, A., & Blakemore, M. (2020) · Learning, Media and Technology · doi:10.1080/17439884.2020.1786399 · Cited by 281 · cited 1×
“Berendt et al. (2020), in this context, identi fi ed that the risks of AI in school are not always apparent due to its unseen consequences.”
BackgroundUsing digital story writing as a pedagogy to develop AI literacy among primary students
Ng, D. T. K., Luo, W., Chan, H. M. Y., & Chu, S. K. W. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100054 · Cited by 266 · cited 1×
“Due to its fool proof biasness, Ng et al. (2022) revealed that students are reluctant to share their personal information in AI system.”
BackgroundA systematic review of AI role in the educational system based on a proposed conceptual framework
Xu, W., & Ouyang, F. (2022) · Education and Information Technologies · doi:10.1007/s10639-021-10774-y · Cited by 230 · cited 2×
“…smart learning environment (Zhai et al., 2021), solve science and mathematical problems (Yannier et al., 2020), get customized learning assistant (Xu & Ouyang, 2022), and help teachers develop digital pedagogy skills (De la Higuera, 2019; Williams et al., 2022), provide smart teaching contents…”
BackgroundThe use of AI in education: Practicalities and ethical considerations
Reiss, M. J. (2021) · London Review of Education · doi:10.14324/lre.19.1.05 · Cited by 211 · cited 2×
“According to Reiss (2021), AI monitoring and tracking system in school premises will cause a lot of inconvenience to the work and life of teachers and students.”
BackgroundAims for cultivating students’ key competencies based on artificial intelligence education in China
Huang, X. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10530-2 · Cited by 187 · cited 1×
“The integration of AI in education (AIEd) becomes an important subject of growing attention (Bogina et al., 2022; Huang, 2021).”
BackgroundEdwards, A., Edwards, C., Spence, P. R., Harris, C., & Gambino, A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.06.005 · Cited by 174
Lin, P.-Y., Chai, C.-S., Jong, M. S.-Y., Dai, Y., Guo, Y., & Qin, J. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100006 · Cited by 171
A Conceptual Framework for Human–AI Hybrid Adaptivity in Education
Holstein, K., Aleven, V., & Rummel, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-52237-7_20 · Cited by 168 · cited 1×
“Holstein et al. (2020) emphasis on incorporating hybrid human-AI approaches in education, including co-teaching scenarios and integration of learning technologies in class teaching.”
BackgroundMarda, V. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0087 · Cited by 161
A differentiated discussion about AI education K-12
Steinbauer, G., Kandlhofer, M., Chklovski, T., Heintz, F., & Koenig, S. (2021) · KI - Künstliche Intelligenz · doi:10.1007/s13218-021-00724-8 · Cited by 160 · cited 2×
“Studies (e.g., Manyika et al., 2017; Powell, 2014; Steinbauer et al., 2021), in this context, reported that AI will not only serve as a classroom assistant or student ’ s peer, but also will be capable of carrying affective relationships with learners, and handle classroom management as expected…”
BackgroundYau, K. W., CHAI, C. S., Chiu, T. K. F., Meng, H., King, I., & Yam, Y. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11161-x · Cited by 160 · cited 1×
“AI system needs high volumes of data to run and support AIEd, eventually there may have high privacy risk if data goes wrong, and can threaten children ’ s rights to privacy, safety, and security (Holmes et al., 2022; Kim et al., 2022; Su & Zhong, 2022; Yau et al., 2022).”
BackgroundCorrection to: Excavating AI: the politics of images in machine learning training sets
Crawford, K., & Paglen, T. (2021) · AI & SOCIETY · doi:10.1007/s00146-021-01301-1 · Cited by 153 · cited 1×
“For instance, students of different color in K-12 schools are subjected to the discrimination at entry for incorrect identi fi cation by AI (Crawford & Paglen, 2021).”
BackgroundBogina, V., Hartman, A., Kuflik, T., & Shulner-Tal, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00248-0 · Cited by 142 · cited 3×
“According to Bogina et al. (2022), AI ethics require more responsibility and accountability from teachers, students, and organizations where it is supposed to apply.”
BackgroundWang, T., & Cheng, E. C. K. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100031 · Cited by 141 · cited 3×
“Studies reported that the incorporation of AI system into K-12 education in forms of Face Recognition (Borenstein & Howard, 2021; Noble, 2018), Students Classi fi cation (Bogina et al., 2022; Brandom, 2018), Intelligent Tutoring System (ITS) (Hwang et al., 2020; Westenberger et al., 2022), Student Tracking (Akgun & Gre…”
BackgroundVisual tools for teaching machine learning in K-12: A ten-year systematic mapping
Gresse von Wangenheim, C., Hauck, J. C. R., Pacheco, F. S., & Bertonceli Bueno, M. F. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10570-8 · Cited by 130
Preparing students for future learning with Teachable Agents
Chin, D. B., Dohmen, I. M., Cheng, B. H., Oppezzo, M. A., Chase, C. C., & Schwartz, D. L. (2010) · Educational Technology Research and Development · doi:10.1007/s11423-010-9154-5 · Cited by 119 · cited 1×
“Henceforth, some instructional designers and stake-holders believed that robots in class are neither about to or ever going to be capable of taking over the role of human teachers (Chin et al., 2010, 2013; Lee et al., 2008; You et al., 2006).”
BackgroundDeep learning for deepfakes creation and detection: a survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4030341 · Cited by 112
Exploring Generative Models with Middle School Students
Ali, S., DiPaola, D., Lee, I., Hong, J., & Breazeal, C. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445226 · Cited by 82
Young Children Can Learn Scientific Reasoning with Teachable Agents
Chin, D. B., Dohmen, I. M., & Schwartz, D. L. (2013) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2013.24 · Cited by 54
Cited by
Preparing future educators for AI-enhanced classrooms: Insights into AI literacy and integration
Kohnke, L., Zou, D., Ou, A. W., & Gu, M. M. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100398 · Cited by 86
What’s the Chatter About AI and School Leaders?
Richardson, J. W., Vedder, B. C., Roberts, A. B., & McLeod, S. (2025) · Leadership and Policy in Schools · doi:10.1080/15700763.2024.2409307 · Cited by 24
Nguyen (Ruby), N. N., & Barbieri, W. (2025) · Education Sciences · doi:10.3390/educsci15060642 · Cited by 15
Metadata
- Authors
- Karan, B., & Angadi, G. R.
- Venue
- Bulletin of Science, Technology & Society
- Publisher
- SAGE Publications
- Published
- 2023-12
- Volume
- 43
- Issue
- 3-4
- Pages
- 67-85
- ISSN
- 0270-4676 (ISSN-L)
- 1552-4183
- OpenAlex
- W4390543494
- Cited by
- 87
- References
- 51
- FWCI
- 4.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.