Exploring the potential of artificial intelligence tools in educational measurement and assessment

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

AI-powered educational assessment tools provide benefits including improving the accuracy and efficiency of assessments, generating personalized feedback for students, and enabling teachers to adapt their teaching strategies to meet individual student needs.

“AI-powered educational assessment tools provide numerous benefits, including improving the accuracy and efficiency of assessments, generating personalized feedback for students, and enabling teachers to adapt their teaching strategies to meet the unique needs of each student.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The development of AI-powered tools and applications has revolutionized the field of education by providing educators with valuable insights into student performance, learning outcomes, and instructional effectiveness.

“Therefore, the development of AI-powered tools and applications has revolutionized the field of education by providing educators with valuable insights into student performance, learning outcomes, and instructional effectiveness.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Using AI in educational assessment has benefits and limitations.

“In conclusion, using AI in educational assessment has benefits and limitations.”

May combine two claims.

AI-powered grading tools can analyze students' essays and provide feedback on grammar, structure, and content, reducing teachers' time grading assignments.

“For example, AI-powered grading tools can analyze students essays and provide feedback on grammar, structure, and content, reducing teachers' time grading assignments (Huang et al., 2023b).”

• cites Huang et al., 2023bNot checked yet

  Not checked in the free run.

May combine two claims.

AI-powered tools can help identify students at risk of falling behind or who would benefit from additional assistance or remediation.

“AI-powered tools can also help identify students at risk of falling behind or benefit from additional assistance or remediation (Delgado et al., 2020).”

• cites Delgado et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The use of AI in teaching and learning has limitations.

“Nevertheless, the use of AI in teaching and learning has its limitations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

User privacy is a significant concern regarding AI use.

“Moreover, the concern regarding user privacy is another significant matter (Elliott & Soifer, 2022;Hu & Min, 2023;Saura et al., 2022).”

• cites Elliott & Soifer, 2022Not checked yet

  Not checked in the free run.

• cites Hu & Min, 2023Not checked yet

  Not checked in the free run.

• cites Saura et al., 2022Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

AI-powered tools and applications are now being used in many industries, including education, to enhance the quality of services provided to students and teachers.

“AI-powered tools and applications are now being used in many industries (Suh & Ahn, 2022), including education, to enhance the quality of services provided to students and teachers.”

• cites Suh & Ahn, 2022Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

AI can help create and curate learning materials tailored to individual student's needs and learning styles.

“AI can help create and curate learning materials tailored to individual student's needs and learning styles.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can generate test items that align with specific learning objectives and outcomes.

“LLMs can also generate test items that align with specific learning objectives and outcomes.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

A test blueprint is essential to ensure that a test accurately measures the intended learning outcomes.

“A test blueprint is essential to ensure that a test accurately measures the intended learning outcomes.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Using LLMs in test administration can improve the testing experience's quality and increase test results' accuracy and reliability.

“Using LLMs in test administration can improve the testing experience's quality and increase test results' accuracy and reliability.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can generate distractors or incorrect options for multiple-choice questions, ensuring they are plausible but incorrect.

“LLMs can also generate distractors or incorrect options for multiple-choice questions, ensuring they are plausible but incorrect.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can assist with translating test items into different languages, ensuring that the translated items accurately convey the intended meaning of the original items.

“Furthermore, they can assist with translating test items into different languages, ensuring that the translated items accurately convey the intended meaning of the original items.”

May combine two claims.

AI can identify students' learning needs by analyzing data from multiple sources such as assessments, homework, and quizzes and provide targeted feedback.

“AI can identify students' learning needs by analyzing data from multiple sources such as assessments, homework, and quizzes and provide targeted feedback.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One of the key features of LLMs is their ability to understand and conversationally generate text.

“One of the key features of LLMs is their ability to understand and conversationally generate text (Kasneci et al., 2023).”

• cites Kasneci et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can help ensure that test instructions are unambiguous.

“Regarding testing instructions, LLMs can help ensure that the instructions are unambiguous.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can assist in developing test blueprints by analyzing the content areas and the levels of instructional objectives in the cognitive domain.

“They can assist in developing test blueprints by analyzing the content areas and the levels of instructional objectives in the cognitive domain.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can analyze course content and other relevant sources to generate a large pool of potential test items.

“These models can analyze course content and other relevant sources to generate a large pool of potential test items.”

Classified as probably not a substantial claim.

There are currently no methods available to distinguish between content generated by AI tools and content written by humans.

“This is because there are currently no methods available to distinguish between content generated by such a tool and those written by humans (Tovani-Palone, 2023).”

• cites Tovani-Palone, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

LLMs can be extremely useful in the preparation of test instructions.

“LLMs can be extremely useful in the preparation of test instructions.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can be useful in developing essay test items.

“Moreover, LLMs can also be useful in developing essay test items.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can analyze the coherence and consistency of test items to ensure that they assess the intended construct or skill fairly and validly.

“LLMs can analyze the coherence and consistency of test items to ensure that they assess the intended construct or skill fairly and validly.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

LLMs are part of the family of generative models, meaning they can generate new text based on patterns and structures learned from training data.

“LLMs are part of the family of generative models (Ingraham et al., 2019), which means they can generate new text based on the patterns and structures learned from the data used to train them.”

• cites Ingraham et al., 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

AI can help transcribe and translate lectures and other educational materials, making them accessible to a wider range of students.

“AI can help transcribe and translate lectures and other educational materials, making them accessible to a wider range of students.”

Probably not a substantial claim: may not pass S3 (load-bearing).

LLMs can be useful in the test specification of purpose by analyzing the course content and identifying key topics or concepts that need to be assessed.

“Since purpose of a test is built from the course content or subject (Joshua, 2012), LLMs can be useful in the test specification of purpose by analyzing the course content and identifying key topics or concepts that need to be assessed.”

• cites Joshua, 2012Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can help ensure the instructions are culturally appropriate and sensitive to different audiences.

“Moreover, LLMs can help ensure the instructions are culturally appropriate and sensitive to different audiences.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

AI-powered NLP tools can help students learn languages and improve their writing skills by providing grammar, spelling, and punctuation feedback.

“AI-powered NLP tools can help students learn languages and improve their writing skills by providing grammar, spelling, and punctuation feedback.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can be used to support remote proctoring for electronically taken tests.

“LLMs can be used to support remote proctoring for electronically taken tests.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

LLMs can provide a more engaging and interactive testing experience.

“Additionally, LLMs can provide a more engaging and interactive testing experience.”

Probably not a substantial claim: may not pass S4 (based).

By analyzing relevant texts, LLMs can assist in identifying the most relevant and appropriate items from the item pool based on the test blueprint and objectives.

“By analyzing relevant texts, LLMs can assist in identifying the most relevant and appropriate items from the item pool based on the test blueprint and objectives.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can assist in creating relevant, clear, specific, and unambiguous test items.

“LLMs can assist in creating relevant, clear, specific, and unambiguous test items.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can assist in developing objective test items such as multiple-choice, true/false, or matching questions.

“LLMs can also assist in developing objective test items such as multiple-choice, true/false, or matching questions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some main purposes of testing revolve around evaluating teachers' effectiveness and students' motivation, judging students' learning proficiency, their acquisition of essential skills and knowledge, diagnosing students' learning difficulties, ranking students' learning achievement, and measuring their growth over time.

“To Joshua (2012), some of the main purposes of testing revolve around evaluating teachers' effectiveness and students' motivation, judging students' learning proficiency, their acquisition of essential skills and knowledge, diagnosing students' learning difficulties, ranking students' learning achievement, and measuring their growth over time.”

• cites Joshua, 2012Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

One of the most significant benefits of LLMs in test administration is their ability to provide immediate feedback to test-takers.

“One of the most significant benefits of LLMs in test administration is their ability to provide immediate feedback to test-takers.”

May combine two claims.

In an experiment, questions generated by AI were satisfactorily clear, acceptable and favorable to students, and relevant to the subject matter.

“In an experiment, a study found that questions generated by AI were satisfactorily clear, acceptable and favorable to students, and relevant to the subject matter (Nasution, 2023).”

• cites Nasution, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

LLMs can help with the localization of test instructions.

“LLMs can also help with the localization of test instructions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Concerns have been raised regarding plagiarism detection in content produced by ChatGPT and the challenge of distinguishing between factual and fictional text generated.

“Furthermore, concerns have been raised regarding plagiarism detection in content produced by ChatGPT, as well as the challenge of distinguishing between factual and fictional text generated (Chatterjee & Dethlefs, 2023;King & ChatGPT, 2023).”

• cites Chatterjee & Dethlefs, 2023Not checked yet

  Not checked in the free run.

• cites King & ChatGPT, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

LLMs are designed to process and interpret vast quantities of text data.

“By leveraging NLP principles and machine learning principles, LLMs are designed to process and interpret vast quantities of text data (Dergaa et al., 2023).”

• cites Dergaa et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

AI-powered virtual assistants can help students with administrative tasks such as scheduling, reminders, and task management.

“AI-powered virtual assistants can help students with administrative tasks such as scheduling, reminders, and task management.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can monitor test-taking behavior during the test administration to detect unusual patterns that might suggest cheating or other misconduct.

“LLMs can monitor test-taking behavior during the test administration to detect unusual patterns that might suggest cheating or other misconduct.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can help to ensure that test items are fair and unbiased.

“Additionally, LLMs can help to ensure that test items are fair and unbiased, which is essential in ensuring that the test results accurately reflect students' knowledge and skills.”

May combine two claims.

AI can help automate the grading process, saving time for teachers and providing students with immediate feedback on their assignments.

“AI can help automate the grading process, saving time for teachers and providing students with immediate feedback on their assignments.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

AI-powered tutoring systems can provide personalized support and feedback to students.

“tutoring systems can provide personalized support and feedback to students.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can help ensure the security of tests by providing features such as password protection, encryption, and monitoring tools to prevent cheating and unauthorized access to test content.

“Furthermore, LLMs can help ensure the security of tests by providing features such as password protection, encryption, and monitoring tools to prevent cheating and unauthorized access to test content.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs can play a significant role in various stages of the educational assessment process, such as test planning, item generation, preparation of test instruction, item assembly/selection, test administration, test scoring, test analysis, interpretation, appraisal, reporting, and follow-up.

“LLMs can play a significant role in various stages of educational assessment process, such as test planning, item generation, preparation of test instruction, item assembly/selection, test administration, test scoring, test analysis, interpretation, appraisal, reporting, and followup.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs can provide more accurate and consistent scoring of tests than human graders, because they are not subject to biases or errors arising from fatigue, distraction, or personal preferences.

“LLMs can provide more accurate and consistent scoring of tests than human graders, as they are not subject to biases or errors arising from fatigue, distraction, or personal preferences.”

May combine two claims.

AI tools can customize learning experiences, boost teacher productivity, and heighten student engagement.

“Tools and systems based on AI have the potential to customize learning experiences, boost the productivity of teachers, and heighten student engagement (Mena-Guacas et al., 2023).”

• cites Mena-Guacas et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs can be very useful in the test assembly process.

“LLMs can be very useful in the test assembly process.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

LLMs can generate human-like text and are designed to understand and generate text in a contextually relevant and coherent way.

“LLMs can generate human-like text and are designed to understand and generate text in a contextually relevant and coherent way.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can assist in delivering tests to many test-takers simultaneously, which can be particularly useful in high-stakes testing situations.

“LLMs can also assist in delivering tests to many test-takers simultaneously, which can be particularly useful in high-stakes testing situations such as college entrance exams or professional certifications.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

LLMs can be useful in various ways during test administration to ensure that all students have a fair chance to manifest the desired behavior being measured and to minimize cheating.

“LLMs can be useful in various ways during test administration to ensure that all students have a fair chance to manifest the desired behavior being measured and to minimize cheating.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can help to ensure that test items are valid, reliable, and relevant to the educational issues being addressed.

“Furthermore, LLMs can help to ensure that test items are valid, reliable, and relevant to the educational issues being addressed.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs can aid in developing a test blueprint by electronically analyzing relevant texts to identify key concepts, skills, and knowledge areas that need to be assessed and help determine appropriate weightage or distribution of items.

“aid in developing a test blueprint by electronically analyzing relevant texts to identify the key concepts, skills, and knowledge areas that need to be assessed and help determine the appropriate weightage or distribution of these items in the test.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can be used to create adaptive tests that adjust the difficulty level of questions based on students' responses.

“Moreover, LLMs can be used to create adaptive tests that adjust the difficulty level of questions based on students' responses.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs can be programmed to recognize and account for common mistakes or misconceptions made by students, which can help identify areas where further teaching or support may be needed.

“They can also be programmed to recognize and account for common mistakes or misconceptions made by students, which can help to identify areas, where further teaching or support may be needed.”

May combine two claims.

Developers of AI have limited knowledge about learning sciences and insufficient pedagogical understanding for the successful integration of AI into teaching.

“According to Luckin and Cukurova (2019), developers of AI have limited knowledge about learning sciences and insufficient pedagogical understanding for the successful integration of AI into teaching.”

• cites Luckin and Cukurova (2019)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

AI algorithms may not consider non-cognitive factors that can influence academic performance.

“AI algorithms can be biased and may not consider non-cognitive factors that can influence academic performance.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Integrating AI-powered assessment tools with existing systems can be a challenge.

“Integrating AI-powered assessment tools with existing systems can be a challenge.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It is essential to take teachers' opinions, past experiences, and expectations into account to ensure the successful integration of AI in schools.

“therefore, it is essential to take their opinions, past experiences, and expectations into account to ensure the successful integration of AI in schools (Holmes et al., 2019).”

• cites Holmes et al., 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs can provide personalized feedback and recommendations based on individual test results, which can help guide further learning and development.

“LLMs can provide personalized feedback and recommendations based on individual test results, which can help to guide further learning and development.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can use student responses to specific items to highlight items that may be too difficult, too easy, or poorly worded.

“LLMs can use student responses to specific items to highlight areas, where the item may be too difficult, too easy, or poorly worded.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs have the potential to be valuable tools in test appraisals.

“LLMs have the potential to be valuable tools in test appraisals.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

AI-based assessment can identify areas of weakness and strength, allowing educators to tailor their teaching methods to individual needs.

“It can also identify areas of weakness and strength, allowing educators to tailor their teaching methods to individual needs.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

AI tools, due to their inanimate nature, may not be well-suited to measure and grade affective attributes and psychomotor skills.

“Furthermore, AI-tools, due to their inanimate nature, may not be well-suited to measure and grade affective attributes (such as students' feelings, attitude, interest and anxiety) and psychomotor skills (such as driving, footballing, cooking, weaving, laboratory work, and dancing, among others).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can generate comprehensive test reports summarizing results, including descriptive statistics, performance profiles, and graphical representations.

“By analyzing the assessment data, LLMs can generate comprehensive reports summarizing the test results, including descriptive statistics, performance profiles, and graphical representations.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Some students may be uncomfortable with being assessed by machines, which could impact their motivation and engagement.

“Additionally, some students may be uncomfortable with being assessed by machines, which could impact their motivation and engagement.”

Probably not a substantial claim: may not pass S2 (checkable).

In educational assessment, teachers still have a critical role in ensuring that AI is appropriately used for measurement and evaluation purposes.

“Therefore, in educational assessment, teachers still have a critical role in ensuring that AI is appropriately used for measurement and evaluation purposes.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Focus on standardization in AI-powered assessment tools may not account for the diversity of students' learning experiences and backgrounds, leading to a one-size-fits-all approach to education.

“But focus on standardization may not account for the diversity of students' learning experiences and backgrounds, leading to a onesize-fits-all approach to education.”

Probably not a substantial claim: may not pass S2 (checkable).

The role of teachers in AI-based assessments is still critical and cannot be undermined.

“However, the role of teachers in AI-based assessments is still critical and cannot be undermined.”

• cites Dillenbourg (2016)Not checked yet

  Not checked in the free run.

• cites Hrastinski et al. (2019)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

LLMs trained on a large corpus of text can be fine-tuned to provide scores for open-ended questions or essays based on criteria such as content, language use, and organization.

“Training LLMs on a large corpus of text can be fine-tuned to provide scores for open-ended questions or essays based on various criteria, such as content, language use, and organization.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can provide a more detailed and nuanced analysis of test results than traditional methods.

“Apart from their ability to automate the interpretation of a large volume of test results in record time, LLMs can provide a more detailed and nuanced analysis of test results than traditional methods.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs can aid in interpreting assessment results by providing insights into the meaning and implications of the data.

“LLMs can aid in interpreting assessment results by providing insights into the meaning and implications of the data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Teachers are highly important in AI-based teaching.

“Teachers are highly important in AI-based teaching (Seufert et al., 2020);”

• cites Seufert et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The use of AI in educational assessment can lead to a lack of human interaction in the assessment process, which can be detrimental to students' learning experiences.

“The use of AI in educational assessment can lead to a lack of human interaction in the assessment process, which can be detrimental to students' learning experiences.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The use of AI in educational assessment raises ethical concerns such as issues around data privacy and ownership, as well as concerns around using technology to replace human educators.

“The use of AI in educational assessment raises ethical concerns, such as issues around data privacy and ownership, as well as concerns around using technology to replace human educators.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can identify potential sources of confusion or misunderstanding in test questions, such as multiple word or phrase meanings.

“LLMs can also identify potential sources of confusion or misunderstanding in test questions, such as multiple word or phrase meanings.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can be employed to analyze assessment data, including item statistics, item difficulty, discrimination indices, and other performance metrics.

“LLMs can be employed to analyze assessment data, including item statistics, item difficulty, discrimination indices, and other performance metrics.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Without adequate training, educators may be unable to use AI-powered assessment tools effectively, leading to inaccurate assessments.

“Without adequate training, educators may be unable to use the tools effectively, leading to inaccurate assessments.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can be programmed to recognize patterns or trends in scores that may not be immediately apparent to human graders or analysts.

“They can be programmed to recognize patterns or trends in the scores that may not be immediately apparent to human graders or analysts.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs can provide more detailed feedback and analysis of test scores than traditional scoring methods.

“LLMs can provide more detailed feedback and analysis of test scores than traditional scoring methods.”