A Systematic Review of Automatic Question Generation for Educational Purposes
Open accessAbstract
While exam-style questions are a fundamental educational tool serving a variety of purposes, manual construction of questions is a complex process that requires training, experience, and resources. This, in turn, hinders and slows down the use of educational activities (e.g. providing practice questions) and new advances (e.g. adaptive testing) that require a large pool of questions. To reduce the expenses associated with manual construction of questions and to satisfy the need for a continuous supply of new questions, automatic question generation (AQG) techniques were introduced. This review extends a previous review on AQG literature that has been published up to late 2014. It includes 93 papers that were between 2015 and early 2019 and tackle the automatic generation of questions for educational purposes. The aims of this review are to: provide an overview of the AQG community and its activities, summarise the current trends and advances in AQG, highlight the changes that the area has undergone in the recent years, and suggest areas for improvement and future opportunities for AQG. Similar to what was found previously, there is little focus in the current literature on generating questions of controlled difficulty, enriching question forms and structures, automating template construction, improving presentation, and generating feedback. Our findings also suggest the need to further improve experimental reporting, harmonise evaluation metrics, and investigate other evaluation methods that are more feasible.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior systematic reviews have focused on investigating issues related to one specific application scenario of LLMs-based innovations.
“Prior systematic reviews have focused on investigating these issues related to one specific application scenario of LLMsbased innovations (eg, question generation, essay scoring, chatbots or automated feedback) (Cavalcanti et al., 2021;Kurdi et al., 2020;Ramesh & Sanampudi, 2022;Wollny et al., 2021).”
From Practical and ethical challenges of large language models in education: A systematic scoping review (2024) · cites this paper as Kurdi et al., 2020
The Introduction and Conclusion and Future Work identify this as a systematic review focused on automatic question generation for educational purposes, supporting its use as an example of a review of one application. The cited passages do not establish that the review concerns LLM-based innovations; that characterization is unsupported by this source. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA few specific reviews exist for particular domain areas of question generation.
“There are a few specific reviews, in particular domain areas of question generation. For example, the work done for question generation in the educational domain is discussed in [15].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [15]
Some existing AI tools for item generation are costly, need high technical support, and have good results.
“Some of these are costly, need high technical support, and have good results [24–27].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
(2004) · Cited by 9,718 · see citing works
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · see citing works
(2005) · PubMed · Cited by 7,011 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Taxonomy of Educational Objectives. Handbook I: Cognitive Domain
Bloom, B., & Krathwohl, D. R. (1966) · Cited by 4,624 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · see citing works
Introduction to WordNet: An On-line Lexical Database *
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197 · see citing works
(2005) · Cited by 4,152 · see citing works
International Journal of Scientific and Research Publications
(2022) · International Journal of Scientific and Research Publications · doi:10.29322/ijsrp · Cited by 2,691 · see citing works
Critical appraisal skills programme
Singh, J. (2013) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.107697 · Cited by 2,571 · see citing works
Studies in Health Technology and Informatics
Aphinyanaphongs, Y., & Aliferis, C. (2007) · Cited by 1,247 · see citing works
Evaluating the Quality of Learning: The Solo Taxonomy : Structure of the Observed Learning Outcome
Biggs, J., & Collis, K. F. (1977) · Cited by 1,087 · see citing works
SNOMED-CT: The advanced terminology and coding system for eHealth.
Donnelly, K. (2006) · PubMed · Cited by 541 · see citing works
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366 · see citing works
An Overview of the World of MOOCs
Baturay, M. H. (2015) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2015.01.685 · Cited by 310 · see citing works
Determining Alignment of Expectations and Assessments in Mathematics and Science Education.
Webb, N. L. (1997) · Cited by 281 · see citing works
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Serban, I. V., García-Durán, A., Gulcehre, C., Ahn, S., Chandar, S., Courville, A., & Bengio, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1056 · Cited by 280 · see citing works
Computer-aided generation of multiple-choice tests
Mitkov, R., & Ha, L. A. (2003) · doi:10.3115/1118894.1118897 · Cited by 258 · see citing works
Crowdsourcing Multiple Choice Science Questions
Welbl, J., Liu, N. F., & Gardner, M. (2017) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w17-4413 · Cited by 254 · see citing works
Tarrant, M., Ware, J. S., & Mohammed, A. M. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-40 · Cited by 238 · see citing works
A computer-aided environment for generating multiple-choice test items
MITKOV, R., AN HA, L., & KARAMANIS, N. (2006) · Natural Language Engineering · doi:10.1017/s1351324906004177 · Cited by 229 · see citing works
Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency.
Hingorjo, M. R., & Jaleel, F. (2012) · PubMed · Cited by 191 · see citing works
Automatic factual question generation from text
Smith, N. A., & Heilman, M. (2011) · Cited by 168 · see citing works
Tarrant, M., Knierim, A., Hayes, S. K., et al. (2006) · Nurse Education in Practice · doi:10.1016/j.nepr.2006.07.002 · Cited by 157 · see citing works
Automatic Multiple Choice Question Generation From Text: A Survey
CH, D. R., & Saha, S. K. (2020) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2018.2889100 · Cited by 144 · see citing works
Han, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0183591 · Cited by 131 · see citing works
Rush, B. R., Rankin, D. C., & White, B. J. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0773-3 · Cited by 126 · see citing works
Bias in the reporting of sex and age in biomedical research on mouse models
Flórez-Vargas, O., Brass, A., Karystianis, G., Bramhall, M., Stevens, R., Cruickshank, S., & Nenadic, G. (2016) · eLife · doi:10.7554/elife.13615 · Cited by 124 · see citing works
Aid to the Evaluation of Therapeutic Studies
REISCH, J. S., Tyson, J. E., & Mize, S. G. (1989) · PEDIATRICS · doi:10.1542/peds.84.5.815 · Cited by 123 · see citing works
Gaebel, M., Kupriyanova, V., de Morais, R. D. C. M., & Colucci, E. (2014) · Cited by 114 · see citing works
Generating Instruction Automatically for the Reading Strategy of Self-Questioning
Jack, M., & Wei, C. (2009) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/978-1-60750-028-5-465 · Cited by 101 · see citing works
Linguistic Considerations in Automatic Question Generation
Mazidi, K., & Nielsen, R. D. (2014) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/v1/p14-2053 · Cited by 88 · see citing works
G-Asks: An Intelligent Automatic Question Generation System for Academic Writing Support
Liu, M., Calvo, R. A., & Rus, V. (2012) · Dialogue & Discourse · doi:10.5087/dad.2012.205 · Cited by 77 · see citing works
Ontology-Based Generation of Medical, Multi-term MCQs
Leo, J., Kurdi, G. R., Matentzoglu, N., Parsia, B., Sattler, U., Forge, S., Donato, G., & Dowling, W. (2019) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-018-00172-w · Cited by 73 · see citing works
Self-citationSemantics-based Question Generation and Implementation
Yao, X., Bouma, G., & Zhang, Y. (2012) · Dialogue & Discourse · doi:10.5087/dad.2012.202 · Cited by 72 · see citing works
Wang, Z., Lan, A., Nie, W., Waters, A. E., Grimaldi, P. J., & Baraniuk, R. G. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231654 · Cited by 71 · see citing works
Hansen, J. D., & Lee, D. (1997) · Journal of Education for Business · doi:10.1080/08832329709601623 · Cited by 69 · see citing works
Recognizing Question Entailment for Medical Question Answering.
Abacha, A. B., & Demner‐Fushman, D. (2016) · PubMed Central · Cited by 67 · see citing works
Distractor Generation for Multiple Choice Questions Using Learning to Rank
Liang, C., Yang, X., Dave, N., Wham, D. C., Pursel, B., & Giles, C. L. (2018) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0533 · Cited by 67 · see citing works
Chen, C.-Y., Liou, H., & Chang, J. S. (2006) · doi:10.3115/1225403.1225404 · Cited by 66 · see citing works
Multiple Choice Question Generation Utilizing An Ontology
Stasaski, K., & Hearst, M. A. (2017) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w17-5034 · Cited by 65 · see citing works
Ontology-Based Multiple Choice Question Generation
Alsubait, T., Parsia, B., & Sattler, U. (2015) · Künstliche Intell. · doi:10.1007/s13218-015-0405-9 · Cited by 64 · see citing works
Self-citationAutomatic Question Generation for Educational Applications – The State of Art
Le, N.-T., Kojiri, T., & Niels Pinkwart, C. (2014) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-06569-4_24 · Cited by 59 · see citing works
Automatic generation of multiple choice questions using dependency-based semantic relations
Afzal, N., & Mitkov, R. (2013) · Soft Computing · doi:10.1007/s00500-013-1141-4 · Cited by 58 · see citing works
Knowledge Questions from Knowledge Graphs
Seyler, D., Yahya, M., & Berberich, K. (2017) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3121050.3121073 · Cited by 57 · see citing works
A System for Generating Multiple Choice Questions: With a Novel Approach for Sentence Selection
Majumder, M., & Saha, S. K. (2015) · Workshop on Natural Language Processing Techniques for Educational Applications (NLP-TEA) · doi:10.18653/v1/w15-4410 · Cited by 54 · see citing works
Ware, J. S., & Vik, T. (2008) · Medical Teacher · doi:10.1080/01421590802155597 · Cited by 52 · see citing works
Automatic Chinese Factual Question Generation
Liu, M., Rus, V., & Liu, L. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2565477 · Cited by 52 · see citing works
RevUP: Automatic Gap-Fill Question Generation from Educational Texts
Kumar, G., Banchs, R. E., & D’Haro, L. F. (2015) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w15-0618 · Cited by 51 · see citing works
A Review of Recent Advances in Adaptive Assessment
Vie, J.-J., Popineau, F., Bruillard, É., & Bourda, Y. (2017) · Studies in systems, decision and control · doi:10.1007/978-3-319-52977-6_4 · Cited by 50 · see citing works
Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts.
Araki, J., Rajagopal, D., Sankaranarayanan, S., Holm, S., Yamakawa, Y., & Mitamura, T. (2016) · International Conference on Computational Linguistics · Cited by 49 · see citing works
Automating Reading Comprehension by Generating Question and Answer Pairs
Kumar, V., Boorla, K., Meena, Y. K., Ramakrishnan, G., & Li, Y.-F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93040-4_27 · Cited by 49 · see citing works
Hill, J., & Simha, R. (2016) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w16-0503 · Cited by 47 · see citing works
Generating multiple choice questions from ontologies: Lessons learnt
Alsubait, T., Parsia, B., & Sattler, U. (2014) · Cited by 44 · see citing works
Self-citationAutomatic Chinese Multiple Choice Question Generation Using Mixed Similarity Strategy
Liu, M., Rus, V., & Liu, L. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2679009 · Cited by 44 · see citing works
Open and Distance Education in Australia, Europe and the Americas
Qayyum, A., & Zawacki‐Richter, O. (2018) · Springer briefs in education · doi:10.1007/978-981-13-0298-5 · Cited by 43 · see citing works
Personalized mathematical word problem generation
Polozov, O., O’Rourke, E., Smith, A. M., Zettlemoyer, L., Gulwani, S., & Popović, Z. (2015) · Cited by 42 · see citing works
Distractor Generation for Chinese Fill-in-the-blank Items
Jiang, S., & Lee, J. (2017) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w17-5015 · Cited by 42 · see citing works
A Semantic Role-based Approach to Open-Domain Automatic Question Generation
Flor, M., & Riordan, B. (2018) · Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0530 · Cited by 39 · see citing works
Venugopal, V. E., & Sreenivasa Kumar, P. (2015) · Journal of Web Semantics · doi:10.1016/j.websem.2015.05.005 · Cited by 37 · see citing works
A similarity-based theory of controlling MCQ difficulty
Alsubait, T., Parsia, B., & Sattler, U. (2013) · International Conference on E-Learning and E-Technologies in Education · doi:10.1109/icelete.2013.6644389 · Cited by 37 · see citing works
Self-citationMostow, J., Beck, J. E., Bey, J., Cuneo, A., Sison, J., Tobin, B., & Valeri, J. (2004) · Cited by 36 · see citing works
Infusing NLU into Automatic Question Generation
Mazidi, K., & Tarau, P. (2016) · Proceedings of the 9th International Natural Language Generation conference · doi:10.18653/v1/w16-6609 · Cited by 36 · see citing works
On the Use of Semantic-Based AIG to Automatically Generate Programming Exercises
Zavala, L., & Mendoza, B. (2018) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3159450.3159608 · Cited by 35 · see citing works
A Detailed Account of The First Question Generation Shared Task Evaluation Challenge
Rus, V., Brendan Wyse, A., Piwek, P., Lintean, M. C., Stoyanchev, S., & Moldovan, C. (2012) · Dialogue & Discourse · doi:10.5087/dad.2012.208 · Cited by 33 · see citing works
Automatic Generation of English Vocabulary Tests
Susanti, Y., Iida, R., & Tokunaga, T. (2015) · International Conference on Computer Supported Education · doi:10.5220/0005437200770087 · Cited by 32 · see citing works
Factual open cloze question generation for assessment of learner’s knowledge
Das, B., & Majumder, M. (2017) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-017-0060-3 · Cited by 32 · see citing works
Computational Intelligence Framework for Automatic Quiz Question Generation
Killawala, A., Khokhlov, I., & Reznik, L. (2018) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2018.8491624 · Cited by 32 · see citing works
Automatic fill-the-blank question generator for student self-assessment
Kumar, G., Banchs, R. E., & D’Haro, L. F. (2015) · 2022 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2015.7344291 · Cited by 31 · see citing works
TEDQuiz: Automatic Quiz Generation for TED Talks Video Clips to Assess Listening Comprehension
Huang, Y.-T., Tseng, Y.-M., Sun, Y. S., et al. (2014) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2014.105 · Cited by 30 · see citing works
Using Wikipedia and Conceptual Graph Structures to Generate Questions for Academic Writing Support
Liu, M., Calvo, R. A., Aditomo, A., et al. (2012) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2012.5 · Cited by 30 · see citing works
Automatic generation of short answer questions for reading comprehension assessment
HUANG, Y., & HE, L. (2016) · Natural Language Engineering · doi:10.1017/s1351324915000455 · Cited by 30 · see citing works
Automatic question generation for intelligent tutoring systems
Shah, R., Shah, D., & Kurup, L. (2017) · doi:10.1109/cscita.2017.8066538 · Cited by 30 · see citing works
Implementation and assessment of the automatic question generation module
Bednarik, L., & Kovács, L. (2012) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom.2012.6421938 · Cited by 27 · see citing works
Leveraging Multiple Views of Text for Automatic Question Generation
Mazidi, K., & Nielsen, R. D. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_26 · Cited by 26 · see citing works
Drill and Practice in Learning (and Beyond)
Lim, C. S., Tang, K. N., & Kor, L. K. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_706 · Cited by 26 · see citing works
Ha, L. A., & Yaneva, V. (2018) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0548 · Cited by 25 · see citing works
Automatic Question Generation from Children’s Stories for Companion Chatbot
Lee, C.-H., Chen, T.-Y., Chen, L.-P., Yang, P.-C., & Tsai, R. T. (2018) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri.2018.00078 · Cited by 25 · see citing works
Evaluation of automatically generated English vocabulary questions
Susanti, Y., Tokunaga, T., Nishikawa, H., et al. (2017) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-017-0051-y · Cited by 24 · see citing works
Liang, C., Yang, X., Wham, D., Pursel, B., Passonneaur, R., & Giles, C. L. (2017) · Proceedings of the Knowledge Capture Conference · doi:10.1145/3148011.3154463 · Cited by 23 · cited 1× · see citing works
“Others, including Liang et al. (2017, 2018) and Liu et al. (2018), used ML-models to rank distractors based on a combination of the previous features.”
MethodAutomatic generation of analogy questions for student assessment: an Ontology-based approach
Alsubait, T., Parsia, B., & Sattler, U. (2012) · Research in Learning Technology · doi:10.3402/rlt.v20i0.19198 · Cited by 22 · see citing works
Self-citationAutomated generation of assessment tests from domain ontologies
Venugopal, V. E., & Kumar, P. S. (2017) · Semantic Web · doi:10.3233/sw-170252 · Cited by 20 · see citing works
Question Generation for Language Learning: From ensuring texts are read to supporting learning
Chinkina, M., & Meurers, D. (2017) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w17-5038 · Cited by 20 · see citing works
Controlling item difficulty for automatic vocabulary question generation
Susanti, Y., Tokunaga, T., Nishikawa, H., et al. (2017) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-017-0065-5 · Cited by 19 · see citing works
Generating story problems via controlled parameters in a web-based intelligent tutoring system
Khodeir, N., Elazhary, H., & Wanas, N. (2018) · International Journal of Information and Learning Technology · doi:10.1108/ijilt-09-2017-0085 · Cited by 19 · see citing works
A hybrid approach for automatic generation of named entity distractors for multiple choice questions
Patra, R., & Saha, S. K. (2018) · Education and Information Technologies · doi:10.1007/s10639-018-9814-3 · Cited by 19 · see citing works
Learning to Automatically Generate Fill-In-The-Blank Quizzes
Marrese-Taylor, E., Nakajima, A., Matsuo, Y., et al. (2018) · doi:10.18653/v1/w18-3722 · Cited by 19 · see citing works
Generating Multiple Choice Questions From Ontologies: How Far Can We Go?
Alsubait, T., Parsia, B., & Sattler, U. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-17966-7_7 · Cited by 18 · see citing works
Self-citationHow do machine-generated questions compare to human-generated questions?
Zhang, L., & VanLehn, K. A. (2016) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-016-0031-7 · Cited by 18 · see citing works
Mostow, J., Huang, Y.-T., Jang, H., et al. (2016) · Natural Language Engineering · doi:10.1017/s1351324916000024 · Cited by 18 · see citing works
Mining Ontologies for Analogy Questions: A Similarity-based Approach.
Alsubait, T., Parsia, B., & Sattler, U. (2012) · Research Explorer (The University of Manchester) · Cited by 18 · see citing works
Self-citationItem Difficulty Analysis of English Vocabulary Questions
Susanti, Y., Nishikawa, H., Tokunaga, T., & Obari, H. (2016) · International Conference on Computer Supported Education · doi:10.5220/0005775502670274 · Cited by 18 · see citing works
Automatic Question Generation: From NLU to NLG
Mazidi, K., & Tarau, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-39583-8_3 · Cited by 18 · see citing works
Stochastic Tree-Based Generation of Program-Tracing Practice Questions
Thomas, A., Stopera, T., Frank-Bolton, P., & Simha, R. (2019) · Technical Symposium on Computer Science Education · doi:10.1145/3287324.3287492 · Cited by 18 · see citing works
Automatic generation of multiple choice questions for e-assessment
Santhanavijayan, A., Balasundaram, S. R., Narayanan, S. H., Kumar, S. V., & Prasad, V. V. (2017) · International Journal of Signal and Imaging Systems Engineering · doi:10.1504/ijsise.2017.084571 · Cited by 18 · see citing works
Content-Dependent Question Generation for History Learning in Semantic Open Learning Space
Jouault, C., & Seta, K. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_37 · Cited by 17 · see citing works
Difficulty Controllable Question Generation for Reading Comprehension.Preprint
Gao, Y., Wang, J., Bing, L., et al. (2018) · arXiv (Cornell University) · Cited by 17 · see citing works
Automatic Generation of Multiple Choice Questions from Slide Content using Linked Data
Faizan, A., & Lohmann, S. (2018) · International Conference on Web Intelligence, Mining and Semantics · doi:10.1145/3227609.3227656 · Cited by 17 · see citing works
Content-Dependent Question Generation Using LOD for History Learning in Open Learning Space
Jouault, C., Seta, K., & Hayashi, Y. (2016) · New Generation Computing · doi:10.1007/s00354-016-0404-x · Cited by 16 · see citing works
Can Automated Questions Scaffold Children’s Reading Comprehension?
Beck, J. E., Mostow, J., & Bey, J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30139-4_45 · Cited by 15 · see citing works
Automatic Generation and Ranking of Questions for Critical Review
Liu, M., Calvo, R. A., & Rus, V. (2014) · Educational Technology & Society · Cited by 15 · see citing works
Automatic multiple-choice question generation from Thai text
Kwankajornkiet, C., Suchato, A., & Punyabukkana, P. (2016) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse.2016.7748891 · Cited by 15 · see citing works
Automatic Generation of English Reference Question by Utilising Nonrestrictive Relative Clause
Satria, A. Y., & Tokunaga, T. (2017) · International Conference on Computer Supported Education · doi:10.5220/0006320203790386 · Cited by 15 · see citing works
Item analysis of published MCQs.
Sarin, Y. K., Khurana, M., Natu, M., Thomas, A. G., & Singh, T. (1998) · PubMed · Cited by 14 · see citing works
User-Defined Difficulty Levels for Automated Question Generation
Singhal, R., Goyal, S., & Henz, M. (2016) · Proceedings - International Conference on Tools with Artificial Intelligence, TAI · doi:10.1109/ictai.2016.0129 · Cited by 13 · see citing works
Difficulty-level modeling of ontology-based factual questions
Venugopal, V. E., & Kumar, P. S. (2020) · Semantic Web · doi:10.3233/sw-200381 · Cited by 13 · see citing works
Next generation of e-assessment: automatic generation of questions
Alsubait, T., Parsia, B., & Sattler, U. (2012) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2012.051580 · Cited by 12 · see citing works
Self-citationDimensionally guided synthesis of mathematical word problems
Wang, K., & Su, Z. (2016) · International Joint Conference on Artificial Intelligence · Cited by 12 · see citing works
Judging the Quality of Automatically Generated Gap-fill Question using Active Learning
Niraula, N. B., & Rus, V. (2015) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w15-0623 · Cited by 11 · see citing works
Question Generation from a Knowledge Base with Web ExplorationPreprint
Song, L., & Zhao, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03807 · Cited by 11 · see citing works
Automatic Generation of Quizzes from DBpedia According to Educational Standards
Rocha, O. R., & Zucker, C. F. (2018) · doi:10.1145/3184558.3191534 · Cited by 11 · see citing works
Semantic Web and Question Generation: An Overview of the State of the Art.
Papasalouros, A., & Chatzigiannakou, M. A. (2018) · International Association for Development of the Information Society · Cited by 11 · see citing works
Basuki, S., & Kusuma, S. F. (2018) · UMM Institutional Repository (University of Maine at Machias) · Cited by 11 · see citing works
Huang, Y.-T., & Mostow, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_16 · Cited by 10 · see citing works
A web-based system for automatic language skill assessment
Fairon, C. (1999) · doi:10.3115/1598834.1598846 · Cited by 10 · see citing works
Semantic Question Generation Using Artificial Immunity
Fattoh, I. E., Aboutabl, A. E., & Haggag, M. H. (2015) · International Journal of Modern Education and Computer Science · doi:10.5815/ijmecs.2015.01.01 · Cited by 10 · see citing works
Improving Reading Comprehension with Automatically Generated Cloze Item Practice
Olney, A. M., Pavlik, P. I., & Maass, J. K. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-61425-0_22 · Cited by 10 · see citing works
Survey on e-Learning Implementation in Eastern-Europe - Spotlight on Romania
Goldbach, I. R., & Hamza-Lup, F. G. (2017) · Cited by 10 · see citing works
Automatically Predicting Quiz Difficulty Level Using Similarity Measures
Lin, C., Liu, D., Pang, W., et al. (2015) · International Conference on Knowledge Capture · doi:10.1145/2815833.2815842 · Cited by 9 · see citing works
Semi-automatic Generation of Multiple-Choice Tests from Mentions of Semantic Relations
Ai, R., Krause, S., Kasper, W., et al. (2015) · doi:10.18653/v1/w15-4405 · Cited by 9 · see citing works
An Experimental Evaluation of Automatically Generated Multiple Choice Questions from Ontologies
Kurdi, G. R., Parsia, B., & Sattler, U. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54627-8_3 · Cited by 9 · see citing works
Self-citationAutomatic generation of multiple choice questions for e-assessment
Kumar, S. V., Prasad, V. V. S. H., Santhanavijayan, A., et al. (2017) · International Journal of Signal and Imaging Systems Engineering · doi:10.1504/ijsise.2017.10005435 · Cited by 9 · see citing works
Generating Quizzes for History Learning Based on Wikipedia Articles
Tamura, Y., Takase, Y., Hayashi, Y., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20609-7_32 · Cited by 8 · see citing works
Aligning automatically generated questions to instructor goals and learner behaviour
Odilinye, L., Popowich, F. P., Zhang, E., Nesbit, J. C., & Winne, P. H. (2015) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icosc.2015.7050809 · Cited by 8 · see citing works
Domain-specific Question Generation from a Knowledge Base.Preprint
Song, L., & Zhao, L. (2016) · arXiv (Cornell University) · Cited by 8 · see citing works
Gupta, M., Gantayat, N., & Sindhgatta, R. (2017) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3051457.3053995 · Cited by 8 · see citing works
Evaluation of Automatically Generated Pronoun Reference Questions
Satria, A. Y., & Tokunaga, T. (2017) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w17-5008 · Cited by 8 · see citing works
Generating Indonesian Question Automatically Based on Bloom’s Taxonomy Using Template Based Method
Kusuma, S. F., & Alhamri, R. Z. (2018) · Kinetik Game Technology Information System Computer Network Computing Electronics and Control · doi:10.22219/kinetik.v3i2.650 · Cited by 8 · see citing works
Distance Education in Australia, Europe and the Americas
Qayyum, A., & Zawacki‐Richter, O. (2018) · Springer briefs in education · doi:10.1007/978-981-13-0298-5_14 · Cited by 7 · see citing works
Automated Question Generation tool for structured data
Shirude, A., Totala, S., Nikhar, S., et al. (2015) · International Conference on Advances in Computing, Communications and Informatics (ICACCI) · doi:10.1109/icacci.2015.7275833 · Cited by 6 · see citing works
Afzal, N., Mitkov, R., & Farzindar, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21043-3_4 · Cited by 6 · see citing works
A Framework for Automated Generation of Questions Based on First-Order Logic
Singhal, R., Henz, M., & Goyal, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_114 · Cited by 6 · see citing works
Automated EA-type question generation from annotated texts
Bednarik, L., & Kovács, L. (2012) · IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI) · doi:10.1109/saci.2012.6250000 · Cited by 6 · see citing works
Modeling of Item-Difficulty for Ontology-based MCQsPreprint
Vinu, E., Alsubait, T., & Kumar, P. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.00869 · Cited by 6 · see citing works
Machine Learning Approach to the Process of Question Generation
Blšták, M., & Rozinajová, V. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-64206-2_12 · Cited by 6 · see citing works
Using Information Extraction to Generate Trigger Questions for Academic Writing Support
Liu, M., & Calvo, R. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30950-2_47 · Cited by 5 · see citing works
A Kanji learning system based on automatic question sentence generation
Zhang, J., & Jun, T. (2015) · International Conference on Asian Language Processing · doi:10.1109/ialp.2015.7451552 · Cited by 5 · see citing works
Generating DFA Construction Problems Automatically
Shenoy, V., Aparanji, U., Sripradha, K., et al. (2016) · International Conference on Learning and Teaching in Computing and Engineering (LaTICE) · doi:10.1109/latice.2016.8 · Cited by 5 · see citing works
Chinkina, M., Ruíz, S., & Meurers, D. (2017) · doi:10.14705/rpnet.2017.eurocall2017.692 · Cited by 5 · see citing works
Complexity-based generation of multi-choice tests in AQG systems
Kovács, L., & Szemán, G. (2013) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom.2013.6719278 · Cited by 4 · see citing works
Jouault, C., Seta, K., & Hayashi, Y. (2017) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2016edp7417 · Cited by 4 · see citing works
Automatic Question Generation From Passages
Mazidi, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-77116-8_49 · Cited by 4 · see citing works
Automatic Question Generation based on Sentence Structure Analysis
Sewunetie, W. T., & Kovács, L. (2023) · International Carpathian Control Conference · doi:10.1109/iccc57093.2023.10178946 · Cited by 4 · see citing works
Automated Generation of Region Based Geometric Questions
Singhal, R., & Henz, M. (2014) · Proceedings - International Conference on Tools with Artificial Intelligence, TAI · doi:10.1109/ictai.2014.129 · Cited by 3 · see citing works
Automated question generator for Tagalog informational texts using case markers
Montenegro, C. S., Engle, V. G., Acuba, M. G. J., et al. (2012) · doi:10.1109/tencon.2012.6412273 · Cited by 3 · see citing works
Lopetegui, M. A., Lara, B. A., Yen, P., et al. (2015) · PubMed · Cited by 3 · see citing works
Automatic Generation of Multiple-Choice Fill-in-the-Blank Question Using Document Embedding
Park, J., Cho, H., & Lee, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93846-2_48 · Cited by 3 · see citing works
A Semantic Graph-Based Japanese Vocabulary Learning Game
Wita, R., Oly, S., Choomok, S., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-96565-9_14 · Cited by 3 · see citing works
Building an Agent for Factual Question Generation Task
Blšták, M., & Rozinajová, V. (2018) · World Symposium on Digital Intelligence for Systems and Machines (DISA) · doi:10.1109/disa.2018.8490637 · Cited by 3 · see citing works
Domain specific automatic Chinese multiple-type question generation
Zhang, T., Liu, Y., & Quan, P. (2018) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621162 · Cited by 3 · see citing works
A Method for Generating History Questions using LOD and Its Evaluation
Jouault, C., Seta, K., & Hayashi, Y. (2015) · 先進的学習科学と工学研究会 · Cited by 2 · see citing works
Algorithms for Automatic Generation of Logical Questions on Mobile Devices
Wang, K., Li, T., Han, J., et al. (2012) · IERI Procedia · doi:10.1016/j.ieri.2012.06.085 · Cited by 2 · see citing works
Quality of LOD Based Semantically Generated Questions
Jouault, C., Seta, K., & Hayashi, Y. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_86 · Cited by 2 · see citing works
Web authoriser tool to build assessments using Wikipedia articles
Adithya, S. S. R., & Singh, P. K. (2017) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon.2017.8227909 · Cited by 2 · see citing works
Automatic Generation of Named Entity Distractors of Multiple Choice Questions Using Web Information
Patra, R., & Saha, S. K. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-981-10-7871-2_49 · Cited by 2 · see citing works
Can LOD Based Question Generation Support Work in a Learning Environment for History Learning
Jouault, C., Seta, K., & Hayashi, Y. (2016) · doi:10.11517/jsaialst.76.0_07 · Cited by 1 · see citing works
Biology Question Generation from a Semantic Network
Zhang, L. (2015) · PhDT · Cited by 1 · see citing works
Generating Multiple Questions From Ontologies: How Far Can We Go?
Alsubait, T., Parsia, B., & Sattler, U. (2014) · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 15
- Afzal, N., & Mitkov, R. (2014). Automatic generation of multiple choice questions using dependency- based semantic relations. Soft Computing, 18(7), 1269-1281. https://doi.org/10.1007/s00500-013- 1141-4. · DOI
- Alsubait, T. (2015). Ontology-based question generation. PhD thesis: University of Manchester.
- Boland, A., Cherry, M. G., Dickson, R. (2013). Doing a systematic review: A student's guide. Sage.
- Chen, C.Y., Liou, H.C., Chang, J.S. (2006). Fast: an automatic generation system for grammar tests. In: the COLING/ACL on interactive presentation sessions, association for computational linguistics, pp. 1-4.
- Faizan, A., Lohmann, S., Modi, V. (2017). Multiple choice question generation for slides. In: Computer Science Conference for University of Bonn Students, pp. 1-6.
- Heilman, M., & Smith, N.A. (2009). Ranking automatically generated questions as a shared task. In: The 2nd Workshop on Question Generation, pp. 30-37.
- Heilman, M., & Smith, N.A. (2010b). Rating computer-generated questions with mechanical turk. In: the NAACL HLT 2010 workshop on creating speech and language data with amazon's mechanical turk, association for computational linguistics, pp. 35-40.
- Kaur, A., & Singh, S. (2017). Automatic question generation system for Punjabi. In: The international conference on recent innovations in science, Agriculture, Engineering and Management.
- Kaur, J., & Bathla, A. K. (2015). A review on automatic question generation system from a given Hindi text. International Journal of Research in Computer Applications and Robotics (IJRCAR), 3(6), 87-92.
- Kurdi, G., Leo, J., Matentzoglu, N., Parsia, B., Forege, S., Donato, G., Dowling, W. (2019). A comparative study of methods for a priori prediction of MCQ difficulty. the Semantic Web journal, In press.
- ACL.
- Singhal, R., Henz, M., Goyal, S. (2015a). A framework for automated generation of questions across formal domains. In: the 17th international conference on artificial intelligence in education, pp. 776- 780.
- Soonklang, T., & Muangon, W. (2017). Automatic question generation system for English exercise for secondary students. In: the 25th international conference on computers in education.
- Vinu, E.V., & Kumar, P.S. (2015a). Improving large-scale assessment tests by ontology based approach. In: the 28th international florida artificial intelligence research society conference, pp. 457- 462.
- Wang, Z., Lan, A.S., Nie, W., Waters, A.E., Grimaldi, P.J., Baraniuk, R.G. (2018). QG-net: a data-driven question generation model for educational content. In: the 5th Annual ACM Conference on Learning at Scale, pp. 15-25.
Cited by
Sorted by citations
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 935 · see citing works
“…reviews have focused on investigating these issues related to one specific application scenario of LLMs-based innovations (e.g., question generation, essay scoring, chatbots, or automated feedback) (Kurdi et al., 2020; Cavalcanti et al., 2021; Wollny et al., 2021; Ramesh and Sanampudi, 2022).”
BackgroundInfluentialLarge language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395 · see citing works
Human-centered Artificial Intelligence in education: Seeing the invisible through the visible
Yang, S. J. H., Ogata, H., Matsui, T., & Chen, N.-S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100008 · Cited by 379 · see citing works
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12249-8 · Cited by 217 · see citing works
Affordances and challenges of artificial intelligence in K-12 education: a systematic review
Crompton, H., Jones, M. V., & Burke, D. (2024) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2121344 · Cited by 213 · see citing works
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
Xiao, C., Xu, S. X., Zhang, K., Wang, Y., & Xia, L. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.52 · Cited by 138 · see citing works
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., et al. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636256 · Cited by 132 · see citing works
Automatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123 · see citing works
Hahn, M. G., Navarro, S. M. B., De La Fuente Valentin, L., & Burgos, D. (2021) · IEEE Access · doi:10.1109/access.2021.3100890 · Cited by 111 · see citing works
“However, automatic feedback has some drawbacks, including the complexity of measuring the feedback quality compared to a manual grader [28].”
BackgroundA bibliometric analysis of artificial intelligence chatbots in educational contexts
Lin, Y., & Yu, Z. (2024) · Interactive Technology and Smart Education · doi:10.1108/itse-12-2022-0165 · Cited by 99 · see citing works
Can large language models write reflectively
Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100140 · Cited by 99 · see citing works
Role of AI in Blended Learning: A Systematic Literature Review
Park, Y., & Doo, M. Y. (2024) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v25i1.7566 · Cited by 81 · see citing works
Coşkun, Ö., Kıyak, Y. S., & Budakoğlu, I. İ. (2025) · Medical Teacher · doi:10.1080/0142159x.2024.2327477 · Cited by 64 · see citing works
Bachiri, Y.-A., Mouncif, H., & Bouikhalene, B. (2023) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v13i8.40853 · Cited by 59 · see citing works
Automatic item generation: foundations and machine learning-based approaches for assessments
Circi, R., Hicks, J., & Sikali, E. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.858273 · Cited by 52 · see citing works
Kumar, A. P., Nayak, A., K, M. S., Chaitanya, & Ghosh, K. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00333-6 · Cited by 41 · see citing works
Hwang, G.-J., Cheng, P.-Y., & Chang, C.-Y. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104266 · Cited by 44 · see citing works
Leaf: Multiple-Choice Question Generation
Vachev, K., Hardalov, M., Karadzhov, G., Georgiev, G., Koychev, I., & Nakov, P. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-99739-7_41 · Cited by 45 · see citing works
AI in medical education: uses of AI in construction type A MCQs
Rezigalla, A. A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05250-3 · Cited by 36 · see citing works
“Mathematical word problem generation from commonsense knowledge graph and equations,”
Liu, T., Fang, Q., Ding, W., Li, H., Wu, Z., & Liu, Z. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.348 · Cited by 29 · see citing works
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation
Yuan, X., Wang, T., Wang, Y.-H., Fine, E., Abdelghani, R., Sauzéon, H., & Oudeyer, P.-Y. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.820 · Cited by 21 · see citing works
Content and Item Response Theory Analysis of ChatGPT-4-Generated Multiple-Choice Items
Young, R., Courtney, E., Kah, A., Wilkerson, M., & Chen, Y.-H. (2025) · Teaching of Psychology · doi:10.1177/00986283241311220 · Cited by 6 · see citing works
- Authors
- Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S.
- Affiliations
- University of Manchester (GB)
- Corresponding author
- Kurdi, Ghader
- Venue
- International Journal of Artificial Intelligence in Education
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 30
- Issue
- 1
- Pages
- 121-204
- ISSN
- 1560-4292 (ISSN-L)
- OpenAlex
- W2989613245
- Cited by
- 514
- References
- 159
- Citations by year
- 2026: 96 · 2025: 111 · 2024: 105 · 2023: 89 · 2022: 53 · 2021: 42 · 2020: 18
- FWCI
- 1.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Artificial Intelligence in Education (publishedVersion, cc-by)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Keywords
- automatic question generation, adaptive testing, question templates, automated feedback, evaluation metrics, practice questions, educational technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
