Distractor Generation for Multiple Choice Questions Using Learning to Rank
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
We investigate how machine learning models, specifically ranking models, can be used to select useful distractors for multiple choice questions.Our proposed models can learn to select distractors that resemble those in actual exam questions, which is different from most existing unsupervised ontology-based and similarity-based methods.We empirically study feature-based and neural net (NN) based ranking models with experiments on the recently released SciQ dataset and our MCQL dataset.Experimental results show that feature-based ensemble learning methods (random forest and LambdaMART) outperform both the NN-based method and unsupervised baselines.These two datasets can also be used as benchmarks for distractor generation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Liang, C., Yang, X., Wham, D., Pursel, B., Passonneaur, R., & Giles, C. L. (2017) · Proceedings of the Knowledge Capture Conference · doi:10.1145/3148011.3154463 · Cited by 23 · cited 6×
“Prior methods were evaluated on different question sets collected from textbooks (Agarwal and Mannem, 2011), Wikipedia (Liang et al., 2017), ESL corpuses (Sakaguchi et al.”
MethodInfluentialSelf-citation
Cited by
A Systematic Review of Automatic Question Generation for Educational Purposes
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00186-y · Cited by 514
Ferreira‐Mello, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1332 · Cited by 193
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., et al. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636256 · Cited by 132
Automatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123
End-to-end generation of multiple-choice questions using text-to-text transfer transformer models
Rodriguez-Torrealba, R., Garcia-Lopez, E., & Garcia-Cabot, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118258 · Cited by 118
Metadata
- Authors
- Liang, C., Yang, X., Dave, N., Wham, D., Pursel, B., & Giles, C. L.
- Venue
- Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications
- Published
- 2018
- Pages
- 284-290
- OpenAlex
- W2805070499
- Cited by
- 66
- References
- 3
- FWCI
- 2.29
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Distractor Generation for Multiple Choice Questions Using Learning to Rank. https://app.cochat.ai/papers/10.18653/v1/w18-0533
- The paper
- Liang, C., Yang, X., Dave, N., Wham, D., Pursel, B., & Giles, C. L. (2018). Distractor Generation for Multiple Choice Questions Using Learning to Rank. Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications. https://doi.org/10.18653/v1/w18-0533
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.