The DOI matches its Crossref record; 100 of the 106 references it cites were found in a registry or an index.
Automatic Multiple Choice Question Generation From Text: A Survey
Abstract
Automatic multiple choice question (MCQ) generation from a text is a popular research area. MCQs are widely accepted for large-scale assessment in various domains and applications. However, manual generation of MCQs is expensive and time-consuming. Therefore, researchers have been attracted toward automatic MCQ generation since the late 90's. Since then, many systems have been developed for MCQ generation. We perform a systematic review of those systems. This paper presents our findings on the review. We outline a generic workflow for an automatic MCQ generation system. The workflow consists of six phases. For each of these phases, we find and discuss the list of techniques adopted in the literature. We also study the evaluation techniques for assessing the quality of the system generated MCQs. Finally, we identify the areas where the current research focus should be directed toward enriching the literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, while existing systems rely on textual context to generate reading-comprehension-like questions, [4, 30] we focus on generating a question well-aligned with the provided LO, allowing for easier large-scale evaluation.”
From A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education (2024) · cites this paper as [4]
Citations
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“Kumar, Banchs, and D’Haro [46] selected the distractors using semantic similarity computed through the word2vec tool [110].”
MethodAn introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · cited 1×
“Aldabe and Maritxalar [27] used Latent Semantic Analysis [108] to generate distractors.”
MethodIntroduction to WordNet: An On-line Lexical Database*
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197 · cited 1×
“WordNet: WordNet [105] is a lexical database that groups the words into sets of synonyms (called synsets), and records the relations among these synonym sets or their members.”
BackgroundJurafsky, D. S., & Martin, J. (2000) · Cited by 4,132
Automatic Labeling of Semantic Roles
Gildea, D., & Jurafsky, D. (2002) · Computational Linguistics · doi:10.1162/089120102760275983 · Cited by 1,626
Information filtering and information retrieval: two sides of the same coin?
Belkin, N. J., & Croft, W. B. (1992) · Communications of the ACM · doi:10.1145/138859.138861 · Cited by 1,308
Antoniou, G., & van Harmelen, F. (2004) · Handbook on Ontologies · doi:10.1007/978-3-540-24750-0_4 · Cited by 1,017
Interpreting TF-IDF term weights as making relevance decisions
Wu, H. C., Luk, R. W. P., Wong, K. F., & Kwok, K. L. (2008) · ACM Transactions on Information Systems · doi:10.1145/1361684.1361686 · Cited by 821 · cited 1×
“Statistical Analysis: Several systems used certain word level analysis including word frequency, n-gram frequency, Term frequency * inverse document frequency (Tf*Idf) [85], and co-occurrence statistics in various modules ([1], [6], [14], [16], [47], [49], [69], [76], [77], [86], [87], [88]).”
MethodStanford’s Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task
Lee, H., Peirsman, Y., Chang, A. L. S., Chambers, N., Surdeanu, M., & Jurafsky, D. (2011) · Lirias · Cited by 504
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366
Research works printed but not linked: 11
- hoshino, 2010, Predicting the difficulty of multiple-choice close questions for computer-adaptive testing, Proc CICLing, 279
- correia, 2010, Automatic generation of cloze question distractors, Proc Interspeech Satellite Workshop 2nd Lang Stud Acquisition Learn Edu Technol, 1
- stancheva, 2017, A model for generation of test questions, Comptes Rendus de L'Academie Bulgare des Sciences, 70, 619
- bhale, 0, Automatic question generation using Wikipedia, Imperial Journal of Interdisciplinary Research (IJIR), 2, 943
- agarwal, 2012, Cloze and open cloze question generation systems and their evaluation guidelines, MS Thesis IIIT Hyderabad Hyderabad INDIA
- pino, 2008, A selection strategy to improve cloze question quality, Proc Workshop Intell Tutoring Syst Ill-Defined Domains 9th Int Conf Intell Tutoring Syst, 22
- kojiri, 2010, Statistical learning-based approach for automatic generation system of multiple-choice cloze questions, Proc Workshop Proc 18th Int Conf Comput Edu
- sung, 2007, The design of automatic quiz generation for ubiquitous English e-learning system, Proc Conf Technol Enhanced Learn, 161
- nielsen, 2008, A taxonomy of questions for question generation, Workshop on the Question Generation Shared Task and Evaluation Challenge, 15
- goto, 2009, An automatic generation of multiple-choice cloze questions based on statistical learning, Proc 17th Int Conf Comput Edu, 415
- lin, 2007, An automatic multiple-choice question generation scheme for English adjective understanding, Proc Workshop Model Manage Gener Problems/Questions eLearn 15th Int Conf Comput Edu, 137
Cited by
A Systematic Review of Automatic Question Generation for Educational Purposes
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00186-y · Cited by 514
Diwan, C., Srinivasa, S., Suri, G., Agarwal, S., & Ram, P. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100110 · Cited by 176
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., et al. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636256 · Cited by 132
“Note that some of the LOs extracted from the courses were not well-defined (e.g., no action verb) and, hence, it was not possible to categorize them in terms of Bloom’s taxonomy.”
BackgroundMCQGen: A Large Language Model-Driven MCQ Generator for Personalized Learning
Hang, C. N., Tan, C. W., & Yu, P.-D. (2024) · IEEE Access · doi:10.1109/access.2024.3420709 · Cited by 150
“Research on automatic MCQ generation began over two decades ago, and since then, a significant amount of effort has been dedicated to its development [5].”
BackgroundAutomatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123
A Review on Question Generation from Natural Language Text
Zhang, R., Guo, J., Chen, L., Fan, Y., & Cheng, X. (2021) · ACM Transactions on Information Systems · doi:10.1145/3468889 · Cited by 153
“For instance, Kurdi et al. [111] reviewed the automatic question generation for educational purposes, while Ch and Saha [24] performed a systematic review of automatic multiple choice question generation.”
BackgroundAutomatic Distractor Generation for Multiple Choice Questions in Standard Tests
Qiu, Z., Wu, X., & Fan, W. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.189 · Cited by 53
Multiple-Choice Question Generation Using Large Language Models: Methodology and Educator Insights
Biancini, G., Ferrato, A., & Limongelli, C. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3665233 · Cited by 71
“We conduct an experiment with 21 domain experts, including high school and university professors, utilizing several established metrics to assess the quality of generated MCQs from [5].”
InfluentialQuiz-Style Question Generation for News Stories
Lelkes, A. D., Tran, V. Q., & Yu, C. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449892 · Cited by 66
“For a comprehensive review of the field, we refer the reader to [23] as well as earlier surveys such as [7] and [38].”
A Survey on Recent Approaches to Question Difficulty Estimation from Text
Benedetto, L., Cremonesi, P., Caines, A., Buttery, P. J., Cappelli, A., Giussani, A., & Turrin, R. (2022) · ACM Computing Surveys · doi:10.1145/3556538 · Cited by 49
“Two existing surveys focused on question generation have referred to some works that addressed the task of QDET [15, 57].”
BackgroundA review of automatic item generation techniques leveraging large language models
Tan, B., Armoush, N., Mazzullo, E., Bulut, O., & Gierl, M. J. (2025) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.1602294 · Cited by 50
Exam paper generation based on performance prediction of student group
Wu, Z., He, T., Mao, C., & Huang, C. (2020) · Information Sciences · doi:10.1016/j.ins.2020.04.043 · Cited by 45
Kumar, A. P., Nayak, A., K, M. S., Chaitanya, & Ghosh, K. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00333-6 · Cited by 41
“Hence the unstructured data can be suitably used to generate Cloze questions using ML. Based on the review by Ch and Saha (2018), ML generates reasonably good Cloze questions but fails to generate grammatically correct Wh-type questions compared to the ontology technique.”
Generation of Multiple-Choice Questions From Textbook Contents of School-Level Subjects
Rao, D., & Saha, S. K. (2022) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3224232 · Cited by 38
“The evaluation metrics are adopted from the literature, primarily from Gronlund [14], Mitkov and Ha [3], Le and Pinkwart [15], and CH and Saha [11].”
BackgroundMethodSelf-citationLearning to Reuse Distractors to Support Multiple-Choice Question Generation in Education
Bitew, S. K., Hadifar, A., Sterckx, L., Deleu, J., Develder, C., & Demeester, T. (2022) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3226523 · Cited by 35
“MCQs are categorized as objective types of questions because they primarily deal with the facts or knowledge embedded in a text rather than subjective opinions [7].”
BackgroundCo-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension
Zhou, X., Luo, S., & Wu, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6522 · Cited by 34
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11222-2 · Cited by 34
“Generating distractors for multiword answer key is more complex than the unigram key [12].”
BackgroundInfluentialA novel approach to generate distractors for Multiple Choice Questions
Kumar, A. P., Nayak, A., Manjula Shenoy, K., Goyal, S., & Chaitanya (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120022 · Cited by 43
Automatic item generation for educational assessments: a systematic literature review
Song, Y., Du, J., & Zheng, Q. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2482588 · Cited by 31
Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs
Wang, X., Fan, S., Houghton, J., & Wang, L. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.22 · Cited by 40
Maurya, K. K., & Desarkar, M. S. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411997 · Cited by 30
“In the classical convention, MCQ consists of a triplet: (1) question, (2) correct answer, and (3) distractors or the incorrect answers to confuse examinees [2].”
BackgroundLu, O. H. T., Huang, A. Y. Q., Tsai, D. C. L., & Yang, S. J. H. (2021) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 30
Empowering Scientific Workflows with Federated Agents
Kamatar, A., Pauloski, J. G., Babuji, Y., Chard, R., Sakarvadia, M., Babnigg, D., Foster, I., & Chard, K. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00114 · Cited by 28
“Completion time remains constant for Academy and Ray up to 3328 agents while the performance of Dask degrades starting at 104 actors.”
Bachiri, Y.-A., & Mouncif, H. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i05.36589 · Cited by 31
English grammar multiple-choice question generation using Text-to-Text Transfer Transformer
Chomphooyod, P., Suchato, A., Tuaycharoen, N., & Punyabukkana, P. (2023) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2023.100158 · Cited by 36
Diverse Distractor Generation for Constructing High-Quality Multiple Choice Questions
Xie, J., Peng, N., Cai, Y., Wang, T., & Huang, Q. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138706 · Cited by 27
Automated Test Creation Using Large Language Models: A Practical Application
Hadzhikoleva, S., Rachovski, T., Ivanov, I., Hadzhikolev, E. N., & Dimitrov, G. P. (2024) · Applied Sciences · doi:10.3390/app14199125 · Cited by 31
“Older applications rely mainly on basic statistical and syntactic information, while more recent ones increasingly use semantic information [55].”
Automatic distractor generation in multiple-choice questions: a systematic literature review
Awalurahman, H. W., & Budi, I. D. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2441 · Cited by 24
Leveraging Large Language Models to Generate Course‐Specific Semantically Annotated Learning Objects
Lohr, D., Berges, M., Chugh, A., Kohlhase, M., & Müller, D. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13101 · Cited by 25
“…the literature for measuring the quality of questions [23], we reviewed metric lists from literature reviews on automatic question generation [24, 23] to identify relevant criteria that determine the quality of the questions we generate, namely: • correct technical language, • appropriateness…”
Awalurahman, H. W., Aji, R. F., & Budi, I. D. (2025) · IEEE Access · doi:10.1109/access.2025.3590423 · Cited by 22
“Madri and Meruva [8] provided similar insight to Ch and Saha [7] but covered newer methods such as machine learning and deep learning.”
InfluentialDevelopment of system for generating questions, answers, distractors using transformers
Barlybayev, A., & Matkarimov, B. T. (2024) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v14i2.pp1851-1863 · Cited by 17
“Crafting and evaluating such questions can be a tedious and drawn out of activity, eating up a lot of time [4].”
BackgroundVideoDL: Video-Based Digital Learning Framework Using AI Question Generation and Answer Assessment
Forkan, A. R. M., Kang, Y., Jayaraman, P. P., Du, H., Thomson, S., Kollias, E., & Wieland, N. (2023) · International Journal of Advanced Corporate Learning (iJAC) · doi:10.3991/ijac.v16i1.35207 · Cited by 16
A comprehensive review on MCQ generation from text
Madri, V. R., & Meruva, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14768-5 · Cited by 15
Hai-yang, Y., Wang, E., Lang, Q., & Wang, J. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3364155 · Cited by 15
“Many advanced educational information retrieval and question-and-answer models provide technical support for intelligent learning assistants in entrepreneurship education [20], [21], [22].”
BackgroundAutomated multiple-choice question generation in Spanish using neural language models
de-Fitero-Dominguez, D., García‐Cabot, A., & García‐López, E. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10076-7 · Cited by 15
“Traditionally, ontology and similarity measures have been employed for this task [6, 23, 28], often using lexical knowledge bases like WordNet [40].”
Ten Natural Language Processing Tasks with Generative Artificial Intelligence
Golec, J., & Hachaj, T. (2025) · Applied Sciences · doi:10.3390/app15169057 · Cited by 19
Towards development of a system for automatic assessment of the quality of a question paper
Saha, S. K. (2021) · Smart Learning Environments · doi:10.1186/s40561-021-00148-9 · Cited by 15
Self-citationEvaluating the performance of multilingual models in answer extraction and question generation
Moreno-Cediel, A., del-Hoyo-Gabaldon, J.-A., García‐López, E., García‐Cabot, A., & de-Fitero-Dominguez, D. (2024) · Scientific Reports · doi:10.1038/s41598-024-66472-5 · Cited by 17
Clark, A. K., Hirt, A., Whitcomb, D., Thompson, W. J., Wine, M., & Karvonen, M. (2025) · Education Sciences · doi:10.3390/educsci15020233 · Cited by 14
Automated Generation and Evaluation of MultipleChoice Quizzes using Langchain and Gemini LLM
Pawar, P., Dube, R., Joshi, A., Gulhane, Z., & Patil, R. (2024) · International Conference on Electrical Electronics and Computing Technologies (ICEECT) · doi:10.1109/iceect61758.2024.10739326 · Cited by 13
TreeQuestion: Assessing Conceptual Learning Outcomes with LLM-Generated Multiple-Choice Questions
Cheng, Z., Xu, J., & Jin, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686970 · Cited by 13
“However, existing automatic question generation techniques are good at creating factual questions [9, 14, 35], while not being able to generate questions that target educational goals lying in the higher levels of Bloom’s Taxonomy [5, 8].”
Applicable strategy to choose and deploy a MOOC platform with multilingual AQG feature
Bachiri, Y.-A., & Mouncif, H. (2020) · International Arab Conference on Information Technology (ACIT) · doi:10.1109/acit50332.2020.9300051 · Cited by 13
“Our AQG system follows the generic workflow as described by D. R. Ch and S. K. Saha [8].”
MethodBachiri, Y.-A., & Mouncif, H. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-031-06458-6_11 · Cited by 13
Yao, Z., Parashar, A., Zhou, H., Jang, W. S., Ouyang, F., Yang, Z., & Yu, H. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.538 · Cited by 27
Unified Question Generation with Continual Lifelong Learning
Yuan, W., Yin, H., He, T., Chen, T., Wang, Q., & Cui, L. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511930 · Cited by 15
“Such inflexibility also exists in models built for other QG formats like multi-choice [9].”
BackgroundLarrañaga, M., Aldabe, I., Arruarte, A., Elorriaga, J. A., & Maritxalar, M. (2022) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3171589 · Cited by 13
Towards the Automatic Generation of Pedagogical Conversational Agents from Lecture Slides
Wölfel, M. (2021) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-82565-2_18 · Cited by 14
Leveraging Semantic Facets for Automatic Assessment of Short Free Text Answers
Qiao, C., & Hu, X. (2022) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3199469 · Cited by 10
“Besides, these technologies have also been heavily applied in closely related tasks such as automated test question generation and classification [34], [35].”
MethodExtractive Summarization and Multiple Choice Question Generation using XLNet
Pochiraju, D., Chakilam, A., Betham, P., Chimulla, P., & Rao, S. G. (2023) · International Conference on Intelligent Computing and Control Systems (ICICCS) · doi:10.1109/iciccs56967.2023.10142220 · Cited by 10
Song, T., Tian, Q., Xiao, Y., & Liu, S. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0730-0_28 · Cited by 15
Automatic Multiple Choice Question Generation: A Systematic Literature Review
Henry, M. M., Heryanto, N. A., Isnan, M., Nugroho, K. S., & Pardamean, B. (2024) · International Conference on Information Technology and Digital Applications (ICITDA) · doi:10.1109/icitda64560.2024.10810102 · Cited by 10
Cognitive Evaluation of Examinees by Dynamic Question Set Generation based on Bloom’s Taxonomy
Dutta, A., Chatterjee, P., Dey, N., Pablo Moreno-ger, A., & Sen, S. (2023) · IETE Journal of Research · doi:10.1080/03772063.2023.2175060 · Cited by 9
“A systematic review of the automatic generation of the MCQ question-based assessment system is given in [37].”
Pirozelli, P., José, M. M., Silveira, I. C., Cação, F. N., Peres, S. M., Brandão, A. A. F., Costa, A. H. R., & Cozman, F. G. (2024) · Data Intelligence · doi:10.1162/dint_a_00245 · Cited by 15
“0, taking inspiration from the long-standing tradition in AI of automatically generating alternatives for MCQA [40, 41].”
Comparative analysis of NLP-driven MCQ generators from text sources
Azzi, A., Erdős, F., Németh, R., Varadarajan, V., & Afrifa, S. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100440 · Cited by 9
Factual Question Generation for the Portuguese Language
Leite, B., Cardoso, H. L., Reis, L. P., & Soares, C. (2020) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista49547.2020.9194631 · Cited by 9
“In education, factual questions such as those starting with Who , When , Where , Why or What , are useful to recall specific information and to assess student’s knowledge given the information embedded in text [4].”
BackgroundChapman, K. E., Davidson, M. E., Azuka, N., & Liberatore, M. W. (2023) · Computer Applications in Engineering Education · doi:10.1002/cae.22614 · Cited by 9
A Lecture Centric Automated Distractor Generation for Post-Graduate Software Engineering Courses
Chughtai, R., Azam, F., Anwar, M. W., Butt, W. H., & Farooq, M. U. (2022) · International Conference on Frontiers of Information Technology (FIT) · doi:10.1109/fit57066.2022.00028 · Cited by 11
Virtual Dialogue Assistant for Remote Exams
Matveev, A., Makhnytkina, O. V., Matveev, Y. N., Svischev, A., Korobova, P., Rybin, A., & Akulov, A. (2021) · Mathematics · doi:10.3390/math9182229 · Cited by 8
“At least in the past decade, there were multiple attempts to approach the closed question generation problem, resulting in a commonly acknowledged solution [32].”
BackgroundAutomatic Item Generation with Reading Passages: A Systematic Literature Review
Setiawan, H. C., Hidayah, I., & Kusumawardani, S. S. (2022) · doi:10.1109/icet56879.2022.9990738 · Cited by 8
“And then AIG, especially in Multiple Choice Questions (MCQ), gets special attention at least in the last 20 years [10].”
MethodClosing the Gap: Automated Distractor Generation in Japanese Language Testing
Andersson, T., & Picazo‐Sanchez, P. (2023) · Education Sciences · doi:10.3390/educsci13121203 · Cited by 7
AQG: Arabic Question Generator
Bousmaha, K. Z., Chergui, N. H., Mbarek, M. S. A., & Belguith, L. H. (2020) · Revue d intelligence artificielle · doi:10.18280/ria.340606 · Cited by 7
“According to recent research [5, 6], the majority of QG systems focus on generating questions for the purpose of assessment.”
BackgroundGenerating Answer Candidates for Quizzes and Answer-Aware Question Generators
Vachev, K., Hardalov, M., Karadzhov, G., Georgiev, G., Releva AI, S., Koychev, I. K., & Nakov, P. (2021) · Student Research Workshop .../Proceedings of the Student Research Workshop ... · doi:10.26615/issn.2603-2821.2021_028 · Cited by 8
Pattern-Based Syntactic Simplification of Compound and Complex Sentences
Kumar, A. P., Nayak, A., Manjula Shenoy, K., Manoj, R. J., & Priyadarshi, A. (2022) · IEEE Access · doi:10.1109/access.2022.3174846 · Cited by 7
“…the review of this manuscript and approving it for publication was Sergio Consoli . summarization [1], [8] [9] which benefits applications like question answering [10], [11], opinion mining, etc. Due to this reason, the last decade has seen many research efforts targeted in this direction.”
BackgroundA Systematic Review and Bibliometric Analysis of Automated Multiple-Choice Question Generation
Mitroulias, D., & Sioutas, S. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10010035 · Cited by 6
Virtual Assistant for Enhancing English Speaking Skills
Desai, A., Gandhi, Y., Gaglani, J., & Mulla, N. (2021) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca51532.2021.9544877 · Cited by 6
Extractive Text and Video Summarization using TF-IDF Algorithm
Gothankar, A., Gupta, L., Bisht, N., Nehe, S., & Bansode, P. M. (2022) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2022.40775 · Cited by 6
NLP-Based Management of Large Multiple-Choice Test Item Repositories
Albano, V., Firmani, D., Laura, L., Mathew, J. G., Paoletti, A. L., & Torrente, I. (2023) · Journal of Learning Analytics · doi:10.18608/jla.2023.7897 · Cited by 6
Toward Personalised Learning Experiences: Beyond Prompt Engineering
Kruis, J., Pera, M. S., ten Napel, Z., Landoni, M., Murgia, E., Huibers, T. W. C., & Feskens, R. C. W. (2024) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3628516.3659367 · Cited by 6
Automatic Question Generator Using Natural Language Processing
Puneeth Thotad, S. K. (2022) · Journal of Pharmaceutical Negative Results · doi:10.47750/pnr.2022.13.s10.330 · Cited by 6
Can deep learning solve a preschool image understanding problem?
Das, B., Sekh, A. A., Majumder, M., & Phadikar, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06080-w · Cited by 6
“Question generation (QG) has been studied for many years in natural language processing (NLP) [3, 6–8].”
BackgroundUniPoll: A Unified Social Media Poll Generation Framework via Multiobjective Optimization
Li, Y., Xiang, R., Song, Y., & Li, J. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3512868 · Cited by 6
“As for methodology, many early MCQG systems employ heuristic or rule-based generation [28].”
An AI-based Solution for Enhancing Delivery of Digital Learning for Future TeachersPreprint
Kang, Y., Forkan, A. R. M., Jayaraman, P. P., Wieland, N., Kollias, E., Du, H., Thomson, S., & Li, Y.-F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01229 · Cited by 6
“Abbreviations Abbreviation Meaning AI Artificial Intelligence API Application Programming Interface AQG Automatic Question Generation AWS Amazon Web Services BERT Bidirectional Encoder Representations from Transformers BLQ Boolean Question C4 Colossal Clean Crawled Corpus DL Deep Learning DNN Deep Neural Network E-Lea…”
MethodAutomated Question Paper Generation using Natural Language Processing
Narayana, B. J. L., Harsha, V. N. S. S., Prudhvi, K., Vardhan, B. G. H., & Sravika, A. (2024) · International Conference on Computational Intelligence for Green and Sustainable Technologies (ICCIGST) · doi:10.1109/iccigst60741.2024.10717510 · Cited by 5
“D. R. CH et al. [4] explores into the realm of automated Multiple Choice Question (MCQ) generation, emphasizing its significance in large-scale assessments and addressing the challenges posed by the costly and time-consuming manual generation process.”
BackgroundAutomatic Question Generation from Youtube Lectures using Deep Learning
Jasuja, H., Negi, U., Vibhav, & Kaur, G. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10726107 · Cited by 5
Secure Automatic Question Paper with Reconfigurable Constraints
Ragasudha, I., & Saravanan, D. (2021) · International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) · doi:10.1109/wispnet51692.2021.9419410 · Cited by 5
Söbke, H. (2022) · Education Sciences · doi:10.3390/educsci12050297 · Cited by 5
“Although MCQs asking for numerical answers are also to be regarded as MCQs of high quality, focusing solely on MCQs with numerical answers underrepresent specific domains as numerical answers that are more common in specific domains, such as mathematics or physics [68].”
BackgroundKang, Y., Forkan, A. R. M., Banerjee, A., Jayaraman, P. P., McCosker, A., Kim, S.-S., Wieland, N., & Kollias, L. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3620274 · Cited by 6
“With the rise of large pretrained language models (PLMs) such as BERT [23], T5 [24], GPT-2, GPT-3 [25], and Chat-GPT [26], question generation capabilities have reached new levels of contextual richness and flexibility.”
Automated Question Generator using NLP
Chakankar, T., Shinkar, T., Waghdhare, S., Waichal, S., & Phadtare, M. M. M. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.49390 · Cited by 4
Popescu, D. A., Bold, N., & Stefanidakis, M. (2025) · Applied Sciences · doi:10.3390/app15074039 · Cited by 4
Deep Learning and Linguistic Feature Based Automatic Multiple Choice Question Generation from Text
Agarwal, R., Negi, V., Kalra, A., & Mittal, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-94876-4_18 · Cited by 4
Generating Adequate Distractors for Multiple-Choice Questions
Zhang, C., Sun, Y., Chen, H., & Wang, J. (2020) · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management · doi:10.5220/0010148303100315 · Cited by 4
Evolving a Programming CS2 Course: A Decade-Long Experience Report
Giacaman, N., Roop, P. S., & Terragni, V. (2023) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3545945.3569831 · Cited by 4
“Given the large number of students, MCQs have the benefits of quick evaluation, reliable scoring, affordability, andmarking automation [7].”
BackgroundA Dataset for Learning University STEM Courses at Scale and Generating Questions at a Human Level
Drori, I., Zhang, S., Chin, Z., Shuttleworth, R., Lu, A., Chen, L., Birbo, B., He, M., Lantigua, P., Tran, S., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.27091 · Cited by 4
“Before then (Ch and Saha 2018) and (Papasalouros and Chatzigiannakou 2018) did a related review of research done up until 2018 and before them (Alsubait, Parsia, and Sattler 2016) and (Rakangor and Ghodasara 2015) reviewed the literature up until 2014.”
BackgroundAI-Powered Resume Based QA Tailoring for Success in Interviews
Varalakshmi, P., & Bugatha, N. M. K. (2024) · IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS) · doi:10.1109/incos59338.2024.10527664 · Cited by 4
LLM-Based Quiz Generation for Assessments in Learning Management System
Shirkar, S., Langhi, D., & Agrawal, S. (2025) · International Conference on Cognitive Robotics and Intelligent Systems (ICC-ROBINS) · doi:10.1109/icc-robins64345.2025.11086273 · Cited by 3
“Rao et al. [2] executed an extensive evaluation of automatic MCQ generation approaches with special emphasis on their applications in standardized assessments.”
LLM-Based Automatic Generation of Multiple-Choice Questions With Meaningful Distractors
Chico, V. J. S., Regino, A. G., Bonacin, R., & dos Reis, J. C. (2025) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2025.12675 · Cited by 3
“Multiple-choice questions (MCQs) are a type of assessment in which a person is asked to identify the correct answer from a list of possible options [CH and Saha 2020].”
Rockembach, G. R., & Thom, L. H. (2026) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2026.6093 · Cited by 3
“Questomatic”: Automated Question Formulation System
Jadhav, P., Deore, S., Aute, S., Gaikwad, R., Barphe, S. S., & Meshram, E. (2024) · International Conference on Sustainable Expert Systems (ICSES) · doi:10.1109/icses63445.2024.10763124 · Cited by 3
Lin, R., Xu, D., Wang, H., Chen, Z., Wang, Y., & Liu, S. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831835 · Cited by 3
“Multi-choice testing (MCT) is widely recognized as one of the most enduring and successful forms of educational technology [1]–[3].”
BackgroundAutomatic Question Paper Generation using ML: A Review
Dhomse, E. A. K. B. (2021) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.17762/turcomat.v12i1s.1763 · Cited by 3
“⚫ Dhawaleswar Rao Ch & Sujan Kumar Saha [6] programmed different optional address (MCQ) era to their”
BackgroundInfluentialSyntagmatic Distractor Generation for Vocabulary Multiple Choice Questions
Yusuf, H. F., Hidayah, I., & Adji, T. B. (2023) · doi:10.1109/icet59790.2023.10453362 · Cited by 3
“This research will be focused on standardized English tests as the most common type for second language learning [12].”
BackgroundMethodMultiple Choice Question Generation Based on the Improved TextRank
Wei, L., Hao, G., Wang, X., Meng, S., Yang, X., & Zhu, Y. Q. (2024) · International Conference on Internet Technology and Educational Informatization (ITEI) · doi:10.4108/eai.24-11-2023.2343612 · Cited by 3
Automatic Question Generation from Video
Priya, T. J., Priya, K. P. S., Jenneyl, L. R., & Uma, K. V. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-3089-8_35 · Cited by 3
Snegaa, A., Fatima, N. S., Karthiga, I., Amsavalli, S., Nazreen, A., & Hassan, J. J. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-6883-0_42 · Cited by 2
Anwar, M. R., Khalid, S., Alshahrani, S., Bilal, H. S. M., & Aldawsari, M. F. (2026) · IEEE Access · doi:10.1109/access.2026.3652006 · Cited by 2
“However, the manual creation of high-quality MCQs is a significant challenge for test creators, as it is laborious, time-consuming and requires strong domain and pedagogical expertise to ensure accuracy, relevance, and balanced difficulty while avoiding ambiguity and bias [1], [49], [69].”
InfluentialAI Based Monitoring System for Social Engineering
Yapa, K., Udara, I., Wijayawardane, U. P. B., Kularatne, K. N. P., Navaratne, N. M. P. P., & Dharmaphriya, W. G. V. U. (2021) · International Conference on Advancements in Computing (ICAC) · doi:10.1109/icac54203.2021.9671218 · Cited by 2
Experiences in the use of a unique open-book exam paper for every student in a cohort
Ward, A. (2022) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet56107.2022.10031955 · Cited by 2
“A substantial body of literature exists on the use of automated marking in the Computer Science arena [2], image processing [3] and mathematics [4] and in the use of MultipleChoice Questions (MCQs) [5].”
BackgroundQ-GENius: A GPT Based Modified MCQ Generator for Identifying Learner Deficiency
Prakash, V., Agrawal, K., & Das, S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-36336-8_98 · Cited by 6
Question Generation System Using NLP
Devi, M. D. R. N. (2023) · International Journal of Computational Sciences and Engineering · doi:10.37622/ijcse/13.1.2023.1-9 · Cited by 2
Multisource Soft Labeling and Hard Negative Sampling for Retrieval Distractor Ranking
Wang, J., Rong, W., Bai, J., Sun, Z., Ouyang, Y., & Xiong, Z. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3325549 · Cited by 2
“However, such strategies are time consuming and require heavy expert efforts [3].”
Background
Metadata
- Authors
- CH, D. R., & Saha, S. K.
- Affiliations
- Birla Institute of Technology, Mesra (IN)
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2020-01-01
- Volume
- 13
- Issue
- 1
- Pages
- 14-25
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W2906432682
- Cited by
- 144
- References
- 106
- Citations by year
- 2026: 15 · 2025: 29 · 2024: 29 · 2023: 25 · 2022: 20 · 2021: 17 · 2020: 8
- FWCI
- 3.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Learning Technologies (publishedVersion)
- Funders
- Science and Engineering Research Board (YSS/2015/001948)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Keywords
- large-scale assessment, automated assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.