Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 63

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.

ASQA: Factoid Questions Meet Long-Form Answers

Stelmakh, Ivan, Luan, Yi, Dhingra, Bhuwan, Chang, Ming-Wei·2022·Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2022.emnlp-main.566

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 The input questions in ASQA are sourced from AMBIGQA. Long-form answers must be sufficient to answer disambiguated questions from AMBIGQA (short answers are marked in blue and green ), and should introduce additional knowledge from Wikipedia (highlighted in red ) to resolve ambiguity and clarify the relationship between different short answers. The DR score we propose combines ROUGE and Disambiguation-accuracy (Disambig-Acc) metrics, overcoming the issues with long-form QA evaluation outlined by Krishna et al. (2021).
Figure 1 The input questions in ASQA are sourced from AMBIGQA. Long-form answers must be sufficient to answer disambiguated questions from AMBIGQA (short answers are marked in blue and green ), and should introduce additional knowledge from Wikipedia (highlighted in red ) to resolve ambiguity and clarify the relationship between different short answers. The DR score we propose combines ROUGE and Disambiguation-accuracy (Disambig-Acc) metrics, overcoming the issues with long-form QA evaluation outlined by Krishna et al. (2021).CC-BY · From the PDF
Figure 2 Schematic representation of the annotation interface.
Figure 2 Schematic representation of the annotation interface.CC-BY · From the PDF
Figure 3 Disambig-F1 of different methods with a varying number of retrieved passages. Marker sizes are proportional to the answer lengths. The T5-O-K score increases with K but there is also an increasing gap between T5-O-K and JPR@K. Passages from the latter are used as input for the former.
Figure 3 Disambig-F1 of different methods with a varying number of retrieved passages. Marker sizes are proportional to the answer lengths. The T5-O-K score increases with K but there is also an increasing gap between T5-O-K and JPR@K. Passages from the latter are used as input for the former.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 27 references

Research works printed but not linked: 16

  1. Abdalghani Abujabal, Rishiraj Saha Roy, Mohamed Yahya, and Gerhard Weikum. 2019. ComQA: A community-sourced dataset for complex factoid ques- tion answering with paraphrase clusters. In Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 307-317, Min- neapolis, Minnesota. Association for Computational Linguistics. · DOI
  2. Hoa Trang Dang. 2005. Overview of duc 2005. In Pro- ceedings of the document understanding conference, volume 2005, pages 1-12.
  3. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-Wei Chang. 2020. REALM: retrieval- augmented language model pre-training. CoRR, abs/2002.08909. · arXiv
  4. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 874-880, Online. Association for Computa- tional Linguistics. · DOI · Paper page
  5. Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
  6. Peter J Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018. Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198. · arXiv
  7. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. · arXiv
  8. Sewon Min, Kenton Lee, Ming-Wei Chang, Kristina Toutanova, and Hannaneh Hajishirzi. 2021. Joint passage ranking for diverse multi-answer retrieval. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 6997-7008, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics. · DOI
  9. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering am- biguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP), pages 5783- 5797, Online. Association for Computational Lin- guistics. · DOI · Paper page
  10. Preksha Nema, Mitesh M. Khapra, Anirban Laha, and Balaraman Ravindran. 2017. Diversity driven atten- tion model for query-based abstractive summariza- tion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 1063-1072, Vancouver, Canada. Association for Computational Linguistics. · DOI · Paper page
  11. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated MA- chine Reading COmprehension dataset. CoRR, abs/1611.09268. · arXiv
  12. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the param- eters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418-5426, Online. Association for Computational Linguistics. · DOI · Paper page
  13. Claude Sammut and Geoffrey I. Webb, editors. 2010. TF-IDF, pages 986-987. Springer US, Boston, MA. · doi:10.1007/978-0-387-30164-8_832
  14. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learn- ing to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008- 3021.
  15. Haitian Sun, William W Cohen, and Ruslan Salakhutdi- nov. 2021. Conditionalqa: A complex reading com- prehension dataset with conditional answers. arXiv preprint arXiv:2110.06884. · arXiv
  16. Ellen M. Voorhees and Dawn M. Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, SIGIR '00, page 200-207, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page

Cited by

See all 63

Metadata

Authors
Stelmakh, I., Luan, Y., Dhingra, B., & Chang, M.-W.
Venue
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Published
2022
Pages
8273-8288
Cited by
63
References
27

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: ASQA: Factoid Questions Meet Long-Form Answers. https://app.cochat.ai/papers/10.18653/v1/2022.emnlp-main.566
The paper
Stelmakh, I., Luan, Y., Dhingra, B., & Chang, M.-W. (2022). ASQA: Factoid Questions Meet Long-Form Answers. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/2022.emnlp-main.566

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.