CoChat

Infusing NLU into Automatic Question Generation

Open access
Mazidi, Karen, Tarau, Paul · 2016 · Proceedings of the 9th International Natural Language Generation conference

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index; none of the 10 sources its claims cite is checked for support yet.

Abstract

We present a fresh approach to automatic question generation that significantly increases the percentage of acceptable questions compared to prior state-of-the-art systems.In our evaluation of the top 20 questions, our system generated 71% more acceptable questions by informing the generation process with Natural Language Understanding techniques.The system also introduces our DeconStructure algorithm which creates an intuitive and practical structure for easily accessing sentence functional constituents in NLP applications.

Overview

CoChat Summary

## TL;DR

The paper presents an automatic question generation system that infuses Natural Language Understanding into generation by classifying sentence patterns with DeconStructure before applying templates. In evaluation on the top 20 questions per passage, it raised acceptable questions from 42% to 72% versus Heilman and Smith (2011), a 71% increase.

## Key findings

  • Across 10 open-source textbook passages, the system’s average MTurk rating for the top 20 questions was 3.7, compared with 2.9 for Heilman & Smith (2011), and the difference was statistically significant (p < 0.001).
  • Using an acceptability threshold of > 3.0, 72% of the paper’s questions were acceptable versus 42% for the prior system.
  • The paper reports that this is a 71% increase in acceptable questions among the top-ranked questions.
  • The system generated far fewer questions than Heilman & Smith: the dataset table reports an average of 50 questions per passage for this system versus 406 for H&S, with generation percentages of 60% and 488%, respectively.

## Methods

  • The system first applies the DeconStructure algorithm: it parses each sentence with dependency parsing and semantic role labeling, and also gathers lemmas, POS tags, and named entities.
  • It then divides sentences into independent clauses, labels clause components, and matches the resulting sentence patterns against approximately 60 templates; filtering conditions check features such as verb type and voice.
  • Evaluation used 10 science and humanities passages from OpenStax and Saylor textbooks, with each generated question rated on a 1–5 scale by 4 Amazon Mechanical Turk workers and compared with Heilman & Smith (2011) on the top 20 questions per input file.

## Limitations

  • The paper states that some errors come from idiomatic language, where literal parsing yields awkward questions; it gives “What do few members spend?” as an example.
  • It also says some templates work better for some topics than others, and some generated questions are vague out of context when a sentence is analyzed in isolation.
  • Evident from the text: the evaluation is limited to 10 textbook passages and the top 20 questions from each system, so the reported results reflect a narrow slice of output rather than all generated questions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/w16-6609

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 1 Not valid · 9 Not checked · 30 Own results and interpretations

8 more claims

Create a free account to see all 40 claims and to verify the 10 references still open.

See all claims — free account
  • Probably not a substantial claim. 23
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Citations

Cites

Show all 29 references

Research works printed but not linked: 5

  1. Husam Ali, Yllias Chali, and Sadid A Hasan. 2010. Automation of question generation from sentences. In Proceedings of QG2010: The Third Workshop on Question Generation , pages 58-67.
  2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2012. Abstract meaning representation (amr) 1.0 specification. In Parsing on Freebase from Question-Answer Pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle: ACL , pages 1533-1544.
  3. Kristy Elizabeth Boyer and Paul Piwek. 2010. Proceedings. In Proceedings of QG2010: The Third Workshop on Question Generation. Pittsburgh: questiongeneration. org .
  4. D Gates. 2008. Generating look-back strategy questions from expository texts. In The Workshop on the Question Generation Shared Task and Evaluation Challenge, NSF, Arlington, VA. http://www. cs. memphis. edu/˜ vrus/questiongeneration//1-Gates-QG08. pdf .
  5. Paul Kroeger. 2004. Analyzing syntax: a lexicalfunctional approach . Cambridge University Press.

Cited by

See all 43

Metadata

Authors
Mazidi, K., & Tarau, P.
Affiliations
University of North Texas (US)
Venue
Proceedings of the 9th International Natural Language Generation conference
Published
2016
Pages
51-60
OpenAlex
W2562215231
Cited by
36
References
29
Citations by year
2025: 2 · 2024: 3 · 2023: 6 · 2022: 4 · 2021: 5 · 2020: 2 · 2019: 4 · 2018: 5 · 2017: 4 · 2016: 1
FWCI
2.51
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • International Conference on Natural Language Generation (publishedVersion, cc-by)
  • University of North Texas Digital Library (University of North Texas) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
automatic question generation, natural language understanding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Infusing NLU into Automatic Question Generation. https://app.cochat.ai/papers/10.18653/v1/w16-6609
The paper
Mazidi, K., & Tarau, P. (2016). Infusing NLU into Automatic Question Generation. Proceedings of the 9th International Natural Language Generation conference. https://doi.org/10.18653/v1/w16-6609

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.