The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
The integration of artificial intelligence (AI) in educational measurement has transformed assessment methods, allowing for automated scoring, swift content analysis, and personalized feedback through machine learning and natural language processing. These advancements provide valuable insights into student performance while also enhancing the overall assessment experience. However, the implementation of AI in education also raises significant ethical concerns regarding validity, reliability, transparency, fairness, and equity. Issues such as algorithmic bias and the opacity of AI decision-making processes risk perpetuating inequalities and affecting assessment outcomes. In response, various stakeholders, including educators, policymakers, and testing organizations, have developed guidelines to ensure the ethical use of AI in education. The National Council of Measurement in Education’s Special Interest Group on AI in Measurement and Education (AIME) is dedicated to establishing ethical standards and advancing research in this area. In this paper, a diverse group of AIME members examines the ethical implications of AI-powered tools in educational measurement, explores significant challenges such as automation bias and environmental impact, and proposes solutions to ensure AI’s responsible and effective use in education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 1×
“Some data augmentation methods, such as oversampling (Chawla et al., 2002), can be applied to re-balance the small subgroups class in the training data.”
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“LLMs may also generate items containing unfamiliar or inappropriate terms for underrepresented groups or groups using low-resource languages, raising fairness concerns (Bender et al., 2021).”
(2023) · doi:10.1126/science · Cited by 6,063
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 1×
“Automation bias denotes the phenomenon where decisions are influenced by an overreliance or excessive dependence on AI-based systems, even when these systems may be flawed or incorrect (Bond et al., 2019; Lyell & Coiera, 2017; Parasuraman & Riley, 1997).”
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 1×
“They can then undergo a subsequent fine-tuning phase to tune the parameters of the LLMs or receive task-relevant prompts without tuning the parameters to perform specific downstream tasks (Liu et al., 2023).”
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 1×
“They can then undergo a subsequent fine-tuning phase to tune the parameters of the LLMs or receive task-relevant prompts without tuning the parameters to perform specific downstream tasks (Liu et al., 2023).”
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 3,832
Developing the theory of formative assessment
Black, P., & Wiliam, D. (2009) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-008-9068-5 · Cited by 3,301 · cited 1×
“Indeed, a central premise of formative assessment is clarifying and making the criteria for success for learners explicit (Black & Wiliam, 2009).”
Rethinking models of feedback for learning: the challenge of design
Boud, D., & Molloy, E. (2013) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2012.691462 · Cited by 1,636 · cited 1×
“Its purpose is to inform individuals about their current ability levels and provide them with guidelines to enhance their knowledge for improved future learning outcomes (Boud & Molloy, 2013).”
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245 · cited 3×
“To address this ethical issue, several techniques can be implemented during multiple stages of training LLMs, including pre-processing, in-training, and post-processing (Gallegos et al., 2024).”
Ethical principles for artificial intelligence in education
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.-P. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11316-w · Cited by 1,228 · cited 3×
“Inclusiveness stands out as one of the fundamental ethical principles in AI, encompassing the assurance of non-discrimination and the promotion of unbiased algorithms (Nguyen et al., 2023).”
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863 · cited 1×
“The lack of transparency can lead to mistrust and skepticism, particularly among disadvantaged subgroups who may feel treated unfairly by an opaque system (Khosravi et al., 2022).”
(2016) · doi:10.3389/fpsyg · Cited by 801
Energy and Policy Considerations for Modern Deep Learning Research
Strubell, E., Ganesh, A., & McCallum, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7123 · Cited by 774 · cited 1×
“This accuracy comes at a significant increase in computational requirements, which, in turn, necessitates similar increases in carbon emissions (Strubell et al., 2020).”
An automated essay scoring systems: a systematic literature review
Ramesh, D., & Sanampudi, S. K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10068-2 · Cited by 550
APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS
WEISS, D. J., & KINGSBURY, G. G. (1984) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1984.tb01040.x · Cited by 501
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · cited 2×
“However, this adoption of automation introduces its own possible errors (Lyell & Coiera, 2017; Jones-Jang & Park, 2023).”
A framework for evaluation and use of automated scoring
Williamson, D. M., Xi, X., & Breyer, F. J. (2012) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2011.00223.x · Cited by 430 · cited 3×
“This level of differences in human and AI scores is typically considered large enough to be concerned, and further investigation is warranted (as per Williamson et al., 2012).”
Quantifying the Carbon Emissions of Machine Learning
Lacoste, A., Luccioni, A., Schmidt, V., & Dandres, T. (2019) · arXiv · doi:10.48550/arxiv.1910.09700 · Cited by 408 · cited 1×
“This focus on the emissions from AI has prompted Lacoste et al. (2019) to develop a tool quantifying emissions by cloud computing provider, hardware, and region.”
Feedback loops and the longer-term: towards feedback spirals
Carless, D. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1531108 · Cited by 328 · cited 1×
“However, to foster SRL, feedback must be contextualized to the students’ circumstances, thereby promoting introspection on their task approach (Carless, 2019).”
Chen, C.-F. E., & Cheng, W.-Y. E. C. (2008) · Language Learning & Technology · doi:10.64152/10125/44145 · Cited by 304
Wilson, J., & Czik, A. (2016) · Computers & Education · doi:10.1016/j.compedu.2016.05.004 · Cited by 297
A Systematic Review on AI-based Proctoring Systems: Past, Present and Future
Nigam, A., Pasricha, R., Singh, T., & Churi, P. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10597-x · Cited by 267 · cited 5×
“First, facial recognition and AI audio detection tools are not exempt from producing false positives and can, therefore, mistakenly assign such flags for academic dishonesty (Nigam et al., 2021; Slusky, 2020).”
InfluentialAutomated Writing Assessment in the Classroom
Warschauer, M., & Grimes, D. (2008) · Pedagogies: An International Journal · doi:10.1080/15544800701771580 · Cited by 243
Bilen, E., & Matros, A. (2021) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2020.12.004 · Cited by 237
Automation Bias and Assistive AI: Risk of Harm From AI-Driven Clinical Decision Support
Khera, R., Simon, M. A., & Ross, J. S. (2023) · JAMA · doi:10.1001/jama.2023.22557 · Cited by 222 · cited 1×
“One prominent issue about automation bias is blindly accepting AI-based tools’ outcomes without critical examination (e.g., Khera et al., 2023; Kupfer et al., 2023).”
Good proctor or “big brother”? Ethics of online exam supervision technologies
Coghlan, S., Miller, T., & Paterson, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00476-1 · Cited by 219 · cited 2×
“Since AI proctoring typically involves some form of analyzing biometric data and often even video feed from the test-taker environment, it can also be perceived as an invasion of privacy (Coghlan et al., 2021), which could sometimes lead to legal action (Dyer, 2024).”
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12249-8 · Cited by 217
The use of AI in education: Practicalities and ethical considerations
Reiss, M. J. (2021) · London Review of Education · doi:10.14324/lre.19.1.05 · Cited by 211
Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E., Liu, K., Chen, L., Tran, S., Cheng, N., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2123433119 · Cited by 187
ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Review
Khowaja, S. A., Khuwaja, P., Dev, K., Wang, W., & Nkenyereye, L. (2023) · doi:10.36227/techrxiv.22619932.v2 · Cited by 183
Jones-Jang, S. M., & Park, Y. J. (2022) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmac029 · Cited by 178
Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy
Wilson, J., & Roscoe, R. D. (2020) · Journal of Educational Computing Research · doi:10.1177/0735633119830764 · Cited by 176
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
Fleckenstein, J., Liebenow, L. W., & Meyer, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1162454 · Cited by 145
Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. D. L., Hanna, E. B., Bressand, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.04088 · Cited by 143
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137 · cited 1×
“In fact, not even experts and LLM developers can reliably interpret how a specific output was reached (Bowman, 2023).”
Trustworthy artificial intelligence (AI) in education: Promises and challenges
(2020) · OECD Education Working Papers · doi:10.1787/a6c90fa9-en · Cited by 135
End-to-end generation of multiple-choice questions using text-to-text transfer transformer models
Rodriguez-Torrealba, R., Garcia-Lopez, E., & Garcia-Cabot, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118258 · Cited by 118
Machine learning based approach to exam cheating detection
Kamalov, F., Sulieman, H., & Santandreu Calonge, D. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254340 · Cited by 102 · cited 1×
“Kamalov et al. (2021) presented an algorithm using sequences of grades from students’ continuous assessment results to identify cheating on the final exam.”
Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI
Hao, J., von Davier, A. A., Yaneva, V., Lottridge, S., von Davier, M., & Harris, D. J. (2024) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12602 · Cited by 93 · cited 1×
“As previously stated, using a human-in-the-loop framework is crucial for multiple stages of assessment design, such as item quality review and sensitivity review (Hao et al., 2024).”
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82 · cited 1×
“The literature on bias in machine learning (Mehrabi et al., 2021; Suresh & Guttag, 2021a, 2021b) focuses on seven sources of bias that are particularly relevant in the AI scoring context: historical bias, representation bias, measurement bias, aggregation bias, learning/algorithmic bias, evaluation…”
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82
Check the box! How to deal with automation bias in AI-based personnel selection
Kupfer, C., Prassl, R., Fleiß, J., Malin, C., Thalmann, S., & Kubicek, B. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1118723 · Cited by 59 · cited 1×
“One prominent issue about automation bias is blindly accepting AI-based tools’ outcomes without critical examination (e.g., Khera et al., 2023; Kupfer et al., 2023).”
Racial, skin tone, and sex disparities in automated proctoring software
Yoder-Himes, D. R., Asif, A., Kinney, K., Brandt, T. J., Cecil, R. E., Himes, P. R., Cashon, C., Hopp, R. M. P., & Ross, E. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.881449 · Cited by 49 · cited 1×
“For instance, in a study on automated proctoring software, Yoder-Himes et al. (2022) found evidence of race, skin tone, and gender bias in the facial detection algorithm, which was significantly more likely to flag women with darker skin tones for review than men or women with lighter skin tones.”
Cited by
Opportunities and challenges of using generative AI to personalize educational assessment
Arslan, B., Lehman, B., Tenison, C., Sparks, J. R., López, A. A., Gu, L., & Zapata-Rivera, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1460651 · Cited by 52
Integrating Generative AI in Hackathons: Opportunities, Challenges, and Educational Implications
Sajja, R., Ramirez, C. E., Li, Z., Demiray, B. Z., Sermet, Y., & Demir, I. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8120188 · Cited by 28
Shomotova, A., ElSayary, A., & Husain, S. (2026) · Education and Information Technologies · doi:10.1007/s10639-025-13832-x · Cited by 19
“Lastly, Ethics emphasizes the responsible and ethical use of AI technologies to address issues like data privacy, algorithmic bias, and the broader societal impact of AI, ensuring that students understand the ethical considerations that come with AI adoption (Bulut & Beiting-Parrish, 2024).”
InfluentialAI in assessment analysis and improvement
Trajkovski, G., & Hayes, H. (2025) · Digital Education and Learning · doi:10.1007/978-3-031-88252-4_4 · Cited by 4
Metadata
- Authors
- Bulut, O., & Beiting-Parrish, M.
- Venue
- Chinese/English Journal of Educational Measurement and Evaluation
- Published
- 2024-12-11
- Volume
- 5
- Issue
- 3
- ISSN
- 2837-0899 (ISSN-L)
- OpenAlex
- W4405276448
- Cited by
- 126
- References
- 48
- FWCI
- 48.22
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.