A computer-aided environment for generating multiple-choice test items
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This paper describes a novel computer-aided procedure for generating multiple-choice test items from electronic documents. In addition to employing various Natural Language Processing techniques, including shallow parsing, automatic term extraction, sentence transformation and computing of semantic distance, the system makes use of language resources such as corpora and ontologies. It identifies important concepts in the text and generates questions about these concepts as well as multiple-choice distractors, offering the user the option to post-edit the test items by means of a user-friendly interface. In assisting test developers to produce items in a fast and expedient manner without compromising quality, the tool saves both time and production costs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy
Jiang, J. J., & Conrath, D. W. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9709008 · Cited by 2,743
Assessing Agreement on Classification Tasks: The Kappa Statistic
Carletta, J. (1996) · arXiv · doi:10.48550/arxiv.cmp-lg/9602004 · Cited by 2,088
Lesk, M. (1986) · Proceedings of the 5th annual international conference on Systems documentation - SIGDOC '86 · doi:10.1145/318723.318728 · Cited by 1,770
Combining Local Context and WordNet Similarity for Word Sense Identification
Leacock, C. (1998) · WordNet · doi:10.7551/mitpress/7287.003.0018 · Cited by 1,662
Technical terminology: some linguistic properties and an algorithm for identification in text
Justeson, J. S., & Katz, S. M. (1995) · Natural Language Engineering · doi:10.1017/s1351324900000048 · Cited by 820
Using Measures of Semantic Relatedness for Word Sense Disambiguation
Patwardhan, S., Banerjee, S., & Pedersen, T. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36456-0_24 · Cited by 494
A non-projective dependency parser
Tapanainen, P., & Järvinen, T. (1997) · Proceedings of the fifth conference on Applied natural language processing - · doi:10.3115/974557.974568 · Cited by 369
Computer-aided generation of multiple-choice tests
Mitkov, R., & Ha, L. A. (2003) · Proceedings of the HLT-NAACL 03 workshop on Building educational applications using natural language processing - · doi:10.3115/1118894.1118897 · Cited by 252
Self-citationAutomatic question generation for vocabulary assessment
Brown, J. C., Frishkoff, G. A., & Eskenazi, M. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220678 · Cited by 247
Using syntactic dependency as local context to resolve word sense ambiguity
Lin, D. (1997) · Proceedings of the 35th annual meeting on Association for Computational Linguistics - · doi:10.3115/976909.979626 · Cited by 242
Cited by
A Systematic Review of Automatic Question Generation for Educational Purposes
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00186-y · Cited by 514
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366
Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017) · Review of Educational Research · doi:10.3102/0034654317726529 · Cited by 219
Metadata
- Authors
- MITKOV, R., AN HA, L., & KARAMANIS, N.
- Affiliations
- University of Wolverhampton (GB)
- Venue
- Natural Language Engineering
- Publisher
- Cambridge University Press (CUP)
- Published
- 2006-05-22
- Volume
- 12
- Issue
- 2
- Pages
- 177-194
- ISSN
- 1351-3249 (ISSN-L)
- 1469-8110
- OpenAlex
- W2069032672
- Cited by
- 229
- References
- 16
- Citations by year
- 2026: 8 · 2025: 5 · 2024: 7 · 2023: 11 · 2022: 14 · 2021: 14 · 2020: 15 · 2019: 8 · 2018: 13 · 2017: 12 · 2016: 14 · 2015: 19 · 2014: 16 · 2013: 14 · 2012: 16
- FWCI
- 6.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Natural Language Engineering (publishedVersion)
- Wolverhampton Intellectual Repository and E-Theses (University of Wolverhampton) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- shallow parsing, term extraction, sentence transformation, semantic distance, ontology (information science), corpora, distractor generation, electronic documents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A computer-aided environment for generating multiple-choice test items. https://app.cochat.ai/papers/10.1017/s1351324906004177
- The paper
- MITKOV, R., AN HA, L., & KARAMANIS, N. (2006). A computer-aided environment for generating multiple-choice test items. Natural Language Engineering. https://doi.org/10.1017/s1351324906004177
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.