CoChat
Sign Up
Cited by 52Educational Technology and AssessmentArtificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive Learning

Automatic item generation: foundations and machine learning-based approaches for assessments

Open access
Circi, Ruhan, Hicks, Juanita, Sikali, Emmanuel·2023·Frontiers in Education
DOI 10.3389/feduc.2023.858273
Open this paper in CoChat — free accountOpen PDF

Abstract

This mini review summarizes the current state of knowledge about automatic item generation in the context of educational assessment and discusses key points in the item generation pipeline. Assessment is critical in all learning systems and digitalized assessments have shown significant growth over the last decade. This leads to an urgent need to generate more items in a fast and efficient manner. Continuous improvements in computational power and advancements in methodological approaches, specifically in the field of natural language processing, provide new opportunities as well as new challenges in automatic generation of items for educational assessment. This mini review asserts the need for more work across a wide variety of areas for the scaled implementation of AIG.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/feduc.2023.858273

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Using AI and AI-driven applications can decrease the challenge of traditional methods in item construction and item stability.

    “Using AI and AI-driven applications, tools, and techniques can decrease the challenge of traditional methods presented as item construction and item stability [23].”

    From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [23]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 15

  1. Baghaee, T. (2017). Automatic neural question generation using community-based question answering systems. Unpublished master's thesis. University of Lethbridge.
  2. Bo, E., He, W., Javurel, A., Miller, S., Scheuring, M. S., and Simpson, M. A. (2020). Items and item models: AIG traditional and modern. In P. Kaliski (Chair), Challenges with automatic item generation implementation: Research, strategies, and lessons learned [virtual symposium]. Annual meeting of the National Council on measurement in education.
  3. Chan, Y.-H., and Fan, Y.-C. (2019). A recurrent BERT-based model for question generation. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering. Association for Computational Linguistics.
  4. Harrison, B., Purdy, C., and Riedl, M. (2021). Toward automated story generation with markov chain Monte Carlo methods and deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 191-197.
  5. Kaliski, P., Clauser, J., and Burke, M. (2020). Exploring the utility of semantic similarity indices for automated item generation. In P. Kaliski (Chair), Challenges with automatic item generation implementation: Research, strategies, and lessons learned [virtual symposium]. Annual meeting of the National Council on measurement in education.
  6. Kumar, V., Muneeswaran, S., Ramakrisnan, G., and Li, Y.-G.. (2019). ParaQG: a system for generating questions and answers from paragraphs. arXiv [Epub ahead of preprint]: 1-6.
  7. Pan, L., Lei, W., Chua, T. S., and Kan, M. Y. (2019). Recent advances in neural question generation. arXiv [Epub ahead of preprint].
  8. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv [Epub ahead of preprint].
  9. Sun, X., Liu, J., Lyu, Y., He, W., Ma, Y., and Wang, S. (2018). Answer-focused and position-aware neural question generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3930-3939.
  10. von Davier, M. (2019). Training optimus prime, M.D.: generating medical certification items by fine tuning OpenAI's gpt2 transformer model. arXiv [Epub ahead of preprint], pp. 1-19.
  11. Wang, Z., Lan, A. S., Nie, W., Waters, A. E., Grimaldi, P. J., and Baraniuk, R. G. (2018). QG net: a data driven question generation model for educational content. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale, pp. 1-10.
  12. Wang, H., Zhang, X., Ma, S., Sun, X., Wang, H., and Wang, M. (2018). A neural question answering model based on semi-structured tables. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 1941-1951.
  13. Zhao, Y., Ni, X., Ding, Y., and Ke, Q. (2018). Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3901-3910.
  14. Zhou, Q., and Huang, D. (2019). Towards generating math word problems from equations and topics. In Proceedings of the 12th international conference on natural language generation, pp. 494-503.
  15. Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., and Zhou, M. (2017). Neural question generation from text: a preliminary study. arXiv [Epub ahead of preprint].

Cited by

Sorted by citations

Related

Authors
Circi, R., Hicks, J., & Sikali, E.
Affiliations
American Institutes for Research (US)
Corresponding author
Circi, Ruhan
ORCID
Venue
Frontiers in Education
Publisher
Frontiers Media SA
Published
2023-05-10
Volume
8
ISSN
2504-284X (ISSN-L)
DOI
10.3389/feduc.2023.858273
OpenAlex
W4381573643
Cited by
52
References
34
Citations by year
2026: 19 · 2025: 23 · 2024: 9 · 2023: 1
FWCI
12.21
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Education (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
automatic item generation, educational assessment, machine learning, natural language processing, test item generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review