WikiHow: A Large Scale Text Summarization Dataset
Open accessAbstract
Sequence-to-sequence models have recently gained the state of the art performance in summarization. However, not too many large-scale high-quality datasets are available and almost all the available ones are mainly news articles with specific writing style. Moreover, abstractive human-style systems involving description of the content at a deeper level require data with higher levels of abstraction. In this paper, we present WikiHow, a dataset of more than 230,000 article and summary pairs extracted and constructed from an online knowledge base written by different human authors. The articles span a wide range of topics and therefore represent high diversity styles. We evaluate the performance of the existing methods on WikiHow to present its challenges and set some baselines to further improve it.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
WikiHow article titles number 180,000+.
“We complement the activities from the time use surveys with WikiHow article titles, of which there are 180,000+.”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [74]
Section 3.2 (p. 3) reports 142,783 unique WikiHow articles at the time of crawling, not 180,000-plus distinct articles or titles. Its larger figure of 204,004 comes from treating separate methods within articles as separate dataset entries; the paper also notes that new articles are added regularly. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredMost text compression datasets are abstractive, treating prompt compression as a generative task where original prompts are rephrased into condensed ones.
“most text compression datasets are abstractive (Toutanova et al., 2016; Koupaee and Wang, 2018; Kim et al., 2019), meaning that they treat prompt compression as a generative task where the original prompts are rephrased into condensed ones.”
From LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (2024) · cites this paper as Koupaee and Wang, 2018
The Introduction (p. 1) identifies WikiHow as a summarization dataset, and §4.1 (p. 3) describes its summaries as highly abstractive. That supports WikiHow as an example of an abstractive dataset, but the paper does not establish that most text-compression datasets are abstractive or discuss compressing prompts by rephrasing them. (LLM, conf=0.85)
Experimental — accuracy not yet measuredWikiHow articles include activities that are important for humans where they seek guidance, even if they are not as frequent as the ones included in the time-use surveys.
“WikiHow articles include activities that are important for humans where they seek guidance, even if they are not as frequent as the ones included in the time-use surveys.”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [74]
Section 3.1 (p. 2) describes WikiHow articles as step-by-step guidance for procedural tasks across varied topics, supporting the claim that they cover activities people seek guidance on. The paper does not assess those activities’ importance to humans or compare their frequency with activities in time-use surveys. (LLM, conf=0.84)
Experimental — accuracy not yet measuredWikiHow is a how-to database with a wide following and community as well as expert- and peer-review.
“Since WikiHow is a how-to database with a wide following and community as well as expert- and peer-review, the article texts document objects that are used and acted on in an activity.”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [74]
Section 3.1 (p. 2) describes WikiHow as a knowledge base of how-to articles, and Section 3 (p. 2) says ordinary people write them. The paper does not establish a wide following or say the articles undergo expert or peer review. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 15
- Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , pages 65-72.
- Federico Barrios, Federico L´ opez, Luis Argerich, and Rosa Wachenchauzer. 2016. Variations of the similarity function of textrank for automated summarization. arXiv preprint arXiv:1602.03606 . · arXiv
- Sumit Chopra, Michael Auli, Alexander M Rush, and SEAS Harvard. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In HLT-NAACL , pages 93-98.
- Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016. Learning-based single-document summarization with compression and anaphoricity constraints. arXiv preprint arXiv:1603.08887 . · arXiv
- Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , volume 1, pages 708-719.
- Kai Hong and Ani Nenkova. 2014. Improving the estimation of word importance for news multidocument summarization. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics , pages 712721.
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop , volume 8. Barcelona, Spain.
- Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing .
- Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. AAAI .
- Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge
- Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304 . · arXiv
- Horst Po¨ ttker. 2003. News and its communicative quality: The inverted pyramidwhen and why did it appear? Journalism Studies , 4(4):501-511.
- Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685 . · arXiv
- Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia , 6(12):e26752.
- Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointergenerator networks. ACL .
Cited by
Sorted by citations
GSum: A General Framework for Guided Neural Abstractive Summarization
Dou, Z.-Y., Liu, P., Hayashi, H., Jiang, Z., & Neubig, G. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.384 · Cited by 210 · see citing works
AlignScore: Evaluating factual consistency with a unified alignment function
Zha, Y., Yang, Y., Li, R., & Hu, Z. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.634 · Cited by 81 · see citing works
LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
Pan, Z., Wu, Q., Jiang, H., Xia, M., Luo, X., Zhang, J., Lin, Q., Rühle, V., Yang, Y., Lin, C.-Y., et al. (2024) · Findings of the Association for Computational Linguistics: ACL 2024 · doi:10.18653/v1/2024.findings-acl.57 · Cited by 58 · see citing works
Ladhak, F., Durmus, E., He, H., Cardie, C., & McKeown, K. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.100 · Cited by 46 · see citing works
Task-aware Retrieval with Instructions
Asai, A., Schick, T., Lewis, P., Chen, X., Izacard, G., Riedel, S., Hajishirzi, H., & Yih, W.-T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.225 · Cited by 46 · see citing works
RefSum: Refactoring Neural Summarization
Liu, Y., Dou, Z.-Y., & Liu, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.113 · Cited by 36 · see citing works
Does Vision-and-Language Pretraining Improve Lexical Grounding?
Yun, T., Sun, C., & Pavlick, E. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.370 · Cited by 22 · see citing works
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Sasaki, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.14902 · Cited by 19 · see citing works
“WikiHow The WikiHow dataset (Koupaee and Wang, 2018) is built from the online WikiHow knowledge base and consists of articles with a title, a headline, which is the concatenation of all bold lines of all paragraphs, and text, which is the concatenation of all paragraphs (except the bold lines).”
MethodInfluentialPudasaini, S., Miralles-Pechuán, L., Lillis, D., & Salvador, M. L. (2024) · arXiv · doi:10.48550/arxiv.2407.13105 · Cited by 8 · see citing works
“First is directly taking from other open-source datasets such as the wikiHow text dataset [88].”
BackgroundLi, C., Zhang, R., Wong, J., Gokmen, C., Srivastava, S., Martín-Martín, R., Wang, C., Levine, G., Ai, W., Martinez, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.09227 · Cited by 6 · see citing works
“We source activities from a combination of time-use surveys and WikiHow [74].”
MethodLLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
Pan, Z., Wu, Q., Jiang, H., Xia, M., Luo, X., Zhang, J., Lin, Q., Rühle, V., Yang, Y., Lin, C.-Y., et al. (2024) · arXiv · Cited by 2 · see citing works
“For Q1, most text compression datasets are ab-stractive (Toutanova et al., 2016; Koupaee and Wang, 2018; Kim et al., 2019), meaning that they treat prompt compression as a generative task where the original prompts are rephrased into condensed ones.”
BackgroundFlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents
Xiao, R., Ma, W., Wang, K., Wu, Y., Zhao, J., Wang, H., Huang, F., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.14884 · Cited by 15 · see citing works
- Authors
- Koupaee, M., & Wang, W. Y.
- ORCID
- Wang, William Yang
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1810.09305
- OpenAlex
- W2896739098
- Cited by
- 166
- References
- 0
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 10 · 2023: 49 · 2022: 27 · 2021: 36 · 2020: 24 · 2019: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- text summarization, abstractive summarization, WikiHow, summarization evaluation, sequence-to-sequence models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
