On the Generation of Medical Question-Answer Pairs
Open accessAbstract
Question answering (QA) has achieved promising progress recently. However, answering a question in real-world scenarios like the medical domain is still challenging, due to the requirement of external knowledge and the insufficient quantity of high-quality training data. In the light of these challenges, we study the task of generating medical QA pairs in this paper. With the insight that each medical question can be considered as a sample from the latent distribution of questions given answers, we propose an automated medical QA pair generation framework, consisting of an unsupervised key phrase detector that explores unstructured material for validity, and a generator that involves a multi-pass decoder to integrate structural knowledge for diversity. A series of experiments have been conducted on a real-world dataset collected from the National Medical Licensing Examination of China. Both automatic evaluation and human annotation demonstrate the effectiveness of the proposed method. Further investigation shows that, by incorporating the generated QA pairs for training, significant improvement in terms of accuracy can be achieved for the examination QA system. 1
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Closed-domain question generation targets a specific domain and depends on domain-specific knowledge limited by an ontology.
“In closed-domain question generation, questions are generated for a specific domain like medicine [5, 6], educational text [7]. Here, the questions usually rely on some domain-specific knowledge restricted by an ontology.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · see citing works
Zhao, T., Zhao, R., & Eskenazi, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1061 · Cited by 736 · see citing works
Learning to Ask: Neural Question Generation for Reading Comprehension
Du, X., Shao, J., & Cardie, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1123 · Cited by 696 · see citing works
Computer Intensive Methods for Testing Hypotheses: An Introduction
Evans, K. J., & Noreen, E. W. (1990) · Journal of the Operational Research Society · doi:10.2307/2582908 · Cited by 666 · see citing works
Liu, C., Lowe, R., Serban, I. V., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08023 · Cited by 603 · see citing works
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366 · see citing works
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Serban, I. V., García-Durán, A., Gulcehre, C., Ahn, S., Chandar, S., Courville, A., & Bengio, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1056 · Cited by 280 · see citing works
Question Generation for Question Answering
Duan, N., Tang, D., Chen, P., & Zhou, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1090 · Cited by 278 · see citing works
Computer-aided generation of multiple-choice tests
Mitkov, R., & Ha, L. A. (2003) · doi:10.3115/1118894.1118897 · Cited by 258 · see citing works
Lagging Inference Networks and Posterior Collapse in Variational AutoencodersPreprint
He, J., Spokoyny, D., Neubig, G., & Berg-Kirkpatrick, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.05534 · Cited by 203 · see citing works
A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU
Chen, B., & Cherry, C. (2014) · Proceedings of the Ninth Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3346 · Cited by 191 · see citing works
emrQA: A Large Corpus for Question Answering on Electronic Medical Records
Pampari, A., Raghavan, P., Liang, J., & Peng, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1258 · Cited by 191 · see citing works
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
Xia, Y., Tian, F., Wu, L., Lin, J., Qin, T., Yu, N., & Liu, T. (2017) · neural information processing systems · Cited by 177 · see citing works
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia
Du, X., & Cardie, C. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1177 · Cited by 168 · see citing works
Zhao, T., Zhao, R., & Eskénazi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10960 · Cited by 164 · see citing works
Machine Comprehension by Text-to-Text Neural Question Generation
Yuan, X., Wang, T., Gülçehre, Ç., Sordoni, A., Bachman, P., Zhang, S., Subramanian, S., & Trischler, A. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2603 · Cited by 158 · see citing works
Leveraging Context Information for Natural Question Generation
Song, L., Wang, Z., Hamza, W., Zhang, Y., & Gildea, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2090 · Cited by 148 · see citing works
Semi-Supervised QA with Generative Domain-Adaptive Nets
Yang, Z., Hu, J., Salakhutdinov, R., & Cohen, W. W. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1096 · Cited by 142 · see citing works
Deep Questions without Deep Understanding
Labutov, I., Basu, S., & Vanderwende, L. (2015) · doi:10.3115/v1/p15-1086 · Cited by 141 · see citing works
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-EncoderPreprint
Gu, X., Cho, K., Ha, J.-W., & Kim, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12352 · Cited by 116 · see citing works
Learning to Ask: Neural Question Generation for Reading ComprehensionPreprint
Du, X., Shao, J., & Cardie, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.00106 · Cited by 110 · see citing works
Computer Intensive Methods for Testing Hypotheses: An Introduction
Evans, K. J. (1990) · Journal of the Operational Research Society · doi:10.1057/jors.1990.171 · Cited by 90 · see citing works
Improving Neural Abstractive Document Summarization with Explicit Information Selection Modeling
Li, W., Xiao, X., Lyu, Y., et al. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1205 · Cited by 66 · see citing works
Building Language Models for Text with Named Entities
Parvez, M. R., Chakraborty, S., Ray, B., & Chang, K.-W. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1221 · Cited by 36 · see citing works
Semi-Supervised QA with Generative Domain-Adaptive NetsPreprint
Yang, Z., Hu, J., Salakhutdinov, R., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.02206 · Cited by 29 · see citing works
Harvesting Paragraph-Level Question-Answer Pairs from WikipediaPreprint
Du, X., & Cardie, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.05942 · Cited by 18 · see citing works
Building Language Models for Text with Named EntitiesPreprint
Parvez, R., Chakraborty, S., Ray, B., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04836 · Cited by 7 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 14
- Ali, H.; Chali, Y.; and Hasan, S. A. 2010. Automation of question generation from sentences. In Proceedings of QG Workshop .
- Cui, Y.; Chen, Z.; Wei, S.; Wang, S.; Liu, T.; and Hu, G. 2017. Attention-over-attention neural networks for reading comprehension. In Proceedings of the ACL .
- Du, X., and Cardie, C. 2018. Harvesting paragraph-level questionanswer pairs from wikipedia. In Proceedings of ACL .
- Forgues, G.; Pineau, J.; Larchevˆ eque, J.-M.; and Tremblay, R. 2014. Bootstrapping dialog systems with word embeddings. In Proceedings of NIPS Workshop .
- Mitkov, R., et al. 2003. Computer-aided generation of multiplechoice tests. In Proceedings of NAACL-HLT Workshop , 17-22.
- Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL .
- Rus, V., and Lintean, M. 2012. A comparison of greedy;optimal assessment of natural language student input using word-to-word similarity metrics. In Proceedings of NAACL-HLT Workshop .
- Serban, I. V.; Sordoni, A.; Lowe, R.; Charlin, L.; Pineau, J.; Courville, A.; and Bengio, Y. 2017. A hierarchicazl latent variable encoder-decoder model for generating dialogues. In Proceedings of AAAI .
- Shen, D.; Wang, G.; Wang, W.; Min, M. R.; Su, Q.; Zhang, Y.; Li, C.; Henao, R.; and Carin, L. 2018. Baseline needs more love: on simple word-embedding-based models;associated pooling mechanisms. In Proceedings of ACL .
- Sohn, K.; Lee, H.; and Yan, X. 2015. Learning structured output representation using deep conditional generative models. In Proceedings of NIPS .
- Subramanian, S.; Wang, T.; Yuan, X.; Zhang, S.; Bengio, Y.; and Trischler, A. 2018. Neural models for key phrase extraction;question generation. In Proceedings of ACL Workshop .
- Walonoski, J.; Kramer, M.; Nichols, J.; Quina, A.; Moesel, C.; Hall, D.; Duffett, C.; Dube, K.; Gallagher, T.; and McLachlan, S. 2017. Synthea: An approach, method, software mechanism for generating synthetic patients records. Journal of the AMIA .
- Xia, Y.; Tian, F.; Wu, L.; Lin, J.; Qin, T.; Yu, N.; and Liu, T.-Y. 2017. Deliberation networks: Sequence generation beyond onepass decoding. In Proceedings of NIPS .
- Yang, Z.; Hu, J.; Salakhutdinov, R.; and Cohen, W. W. 2017. Semisupervised qa with generative domain-adaptive nets. In Proceedings of ACL .
Cited by
Sorted by citations
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
- Authors
- Shen, S., Li, Y., Du, N., Wu, X., Xie, Y., Ge, S., Yang, T., Wang, K., Liang, X., & Fan, W.
- Affiliations
- University of California, Berkeley (US)
- Alibaba Group (China) (CN)
- Tencent (China) (CN)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 05
- Pages
- 8822-8829
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2997894965
- Cited by
- 25
- References
- 36
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 3 · 2023: 5 · 2022: 2 · 2021: 6 · 2020: 2 · 2019: 2
- FWCI
- 1.43
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Keywords
- medical question answering, question-answer pair generation, national medical licensing examination, limited training data, external knowledge integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
