CoChat

A Neural CRF-based Hierarchical Approach for Linear Text Segmentation

Nair, Inderjeet, Garimella, Aparna, Srinivasan, Balaji Vasan, Modani, Natwar, Chhaya, Niyati, Karanam, Srikrishna, Shekhar, Sumit · 2023 · Findings of the Association for Computational Linguistics: EACL 2023
Cited by 7

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

We consider the problem of segmenting unformatted text and transcripts linearly based on their topical structure.While prior approaches explicitly train to predict segment boundaries, we propose to address this task by inferring the hierarchical segmentation structure associated with the input text.For this purpose, we present a data curation strategy to obtain the hierarchical segmentation structure annotations for over 700K Wikipedia articles.We then propose the first supervised approach to generate hierarchical segmentation structures for given text based on a neural conditional random field (CRF) that explicitly models the statistical dependencies between nodes and their constituent children.We introduce a novel data augmentation scheme as part of our model training, which involves sampling a variety of node aggregations, permutations, and removals, all of which help capture fine-grained and coarse topical shifts in the data and improve model performance.Extensive experiments show that our model outperforms or achieves competitive performance when compared to previous state-of-the-art algorithms in the following settings: rich-resource, cross-domain transferability, few-shot supervision, and segmentation when topic label annotations are provided.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.findings-eacl.65

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Transformation of a sequence of linear segmentations to a hierarchical tree.
Figure 1 Transformation of a sequence of linear segmentations to a hierarchical tree.CC-BY · From the PDF
Figure 2 Binarization: Transformation of a tree having nodes containing more than 2 children to a binarized form.
Figure 2 Binarization: Transformation of a tree having nodes containing more than 2 children to a binarized form.CC-BY · From the PDF
Figure 3 Data Augmentation: First, hierarchical structures are randomly sampled from the corpus; then, some nodes are removed from the sampled structures and the outcome is combined
Figure 3 Data Augmentation: First, hierarchical structures are randomly sampled from the corpus; then, some nodes are removed from the sampled structures and the outcome is combinedCC-BY · From the PDF
Figure 4 Pipeline: Our pipeline is divided into two stages. We extract contextualised embeddings for each sentence during the feature extraction stage. These representations are then fed into the structure prediction module, which first infers the binarized tree and label for each node. This inference is used to construct the actual hierarchy.
Figure 4 Pipeline: Our pipeline is divided into two stages. We extract contextualised embeddings for each sentence during the feature extraction stage. These representations are then fed into the structure prediction module, which first infers the binarized tree and label for each node. This inference is used to construct the actual hierarchy.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 7 references

Research works printed but not linked: 35

  1. Pinkesh Badjatiya, Litton J. Kurisinkel, Manish Gupta, and Vasudeva Varma. 2018. Attention-based neural text segmentation. In Advances in Information Re- trieval, pages 180-193, Cham. Springer International Publishing.
  2. Joe Barrow, Rajiv Jain, Vlad Morariu, Varun Manju- natha, Douglas Oard, and Philip Resnik. 2020. A joint model for document segmentation and segment labeling. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 313-322, Online. Association for Computa- tional Linguistics.
  3. Doug Beeferman, Adam Berger, and John Lafferty. 1999. Statistical models for text segmentation. Ma- chine learning, 34(1):177-210.
  4. David M. Blei and Pedro J. Moreno. 2001. Topic seg- mentation with an aspect hidden markov model. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '01, page 343-348, New York, NY, USA. Association for Computing Machinery.
  5. Freddy Y. Y. Choi. 2000. Advances in domain inde- pendent linear text segmentation. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics.
  6. Noam Chomsky. 1959. On certain formal properties of grammars. Information and Control, 2(2):137-167.
  7. Lan Du, Wray Buntine, and Mark Johnson. 2013. Topic segmentation with a structured topic model. In Pro- ceedings of the 2013 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 190-200, Atlanta, Georgia. Association for Compu- tational Linguistics.
  8. Jacob Eisenstein. 2009. Hierarchical text segmentation from multi-scale lexical cohesion. In Proceedings of Human Language Technologies: The 2009 An- nual Conference of the North American Chapter of the Association for Computational Linguistics, pages 353-361, Boulder, Colorado. Association for Com- putational Linguistics.
  9. Jacob Eisenstein and Regina Barzilay. 2008. Bayesian unsupervised topic segmentation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, page 334-343, USA. Association for Computational Linguistics.
  10. Pavlina Fragkou, Vassilios Petridis, and Ath Kehagias. 2004. A dynamic programming algorithm for linear text segmentation. Journal of Intelligent Information Systems, 23(2):179-197.
  11. Goran Glavaš, Federico Nanni, and Simone Paolo Ponzetto. 2016. Unsupervised text segmentation us- ing semantic relatedness graphs. In Proceedings of the Fifth Joint Conference on Lexical and Computa- tional Semantics, pages 125-130, Berlin, Germany. Association for Computational Linguistics.
  12. Barbara J. Grosz and Candace L. Sidner. 1986. At- tention, intentions, and the structure of discourse. Comput. Linguist., 12(3):175-204.
  13. Marti A. Hearst. 1997. Texttiling: Segmenting text into multi-paragraph subtopic passages. Comput. Lin- guist., 23(1):33-64.
  14. John E Hopcroft, Rajeev Motwani, and Jeffrey D Ullman. 2001. Introduction to automata theory, languages, and computation. Acm Sigact News, 32(1):60-65.
  15. Anna Kazantseva and Stan Szpakowicz. 2011. Linear text segmentation using affinity propagation. In Pro- ceedings of the 2011 Conference on Empirical Meth- ods in Natural Language Processing, pages 284-293, Edinburgh, Scotland, UK. Association for Computa- tional Linguistics.
  16. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling se- quence data. In Proceedings of the Eighteenth In- ternational Conference on Machine Learning, ICML '01, page 282-289, San Francisco, CA, USA.
  17. Martin Lange and Hans Leiß. 2009. To cnf or not to cnf? an efficient yet presentable version of the cyk algorithm. Informatica Didactica, 8(2009):1-21.
  18. K. Lari and S.J. Young. 1990. The estimation of stochas- tic context-free grammars using the inside-outside algorithm. Computer Speech Language, 4(1):35-56.
  19. Jing Li, Aixin Sun, and Shafiq Joty. 2018. Segbot: A generic neural text segmentation model with pointer network. In Proceedings of the Twenty-Seventh Inter- national Joint Conference on Artificial Intelligence, IJCAI-18, pages 4166-4172. International Joint Con- ferences on Artificial Intelligence Organization.
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692. · arXiv
  21. Kelvin Lo, Yuan Jin, Weicong Tan, Ming Liu, Lan Du, and Wray Buntine. 2021. Transformer over pre- trained transformer for neural text segmentation with enhanced topic coherence. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2021, pages 3334-3340, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  22. Igor Malioutov and Regina Barzilay. 2006. Minimum cut model for spoken lecture segmentation. In Pro- ceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meet- ing of the Association for Computational Linguis- tics, pages 25-32, Sydney, Australia. Association for Computational Linguistics.
  23. Igor Malioutov, Alex Park, Regina Barzilay, and James Glass. 2007. Making sense of sound: Unsupervised topic segmentation over acoustic input. In Proceed- ings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 504-511, Prague, Czech Republic. Association for Computational Lin- guistics.
  24. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated cor- pus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330.
  25. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia. Association for Computational Linguistics.
  26. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Pro- cessing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
  27. Lev Pevzner and Marti A. Hearst. 2002. A critique and improvement of an evaluation metric for text segmentation. Computational Linguistics, 28(1):19- 36.
  28. Matthew Purver, Konrad P. Körding, Thomas L. Grif- fiths, and Joshua B. Tenenbaum. 2006. Unsupervised topic modelling for multi-party spoken discourse. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meet- ing of the Association for Computational Linguis- tics, pages 17-24, Sydney, Australia. Association for Computational Linguistics.
  29. Itiroo Sakai. 1961. Syntax in universal translation. In Proceedings of the International Conference on Ma- chine Translation and Applied Language Analysis.
  30. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 818-827, Vancouver, Canada. Association for Computational Linguistics.
  31. Masao Utiyama and Hitoshi Isahara. 2001. A statistical model for domain-independent text segmentation. In Proceedings of the 39th Annual Meeting of the Associ- ation for Computational Linguistics, pages 499-506, Toulouse, France. Association for Computational Lin- guistics.
  32. Teun A Van Dijk. 1982. Episodes as units of discourse analysis. Analyzing discourse: Text and talk, pages 177-195.
  33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  34. Naiwen Xue, Fei Xia, Fu-dong Chiou, and Marta Palmer. 2005. The penn chinese treebank: Phrase struc- ture annotation of a large corpus. Nat. Lang. Eng., 11(2):207-238.
  35. Yu Zhang, Houquan Zhou, and Zhenghua Li. 2021. Fast and accurate neural crf constituency parsing. In Pro- ceedings of the Twenty-Ninth International Joint Con- ference on Artificial Intelligence, IJCAI'20.

Cited by

See all 7

Metadata

Authors
Nair, I., Garimella, A., Srinivasan, B. V., Modani, N., Chhaya, N., Karanam, S., & Shekhar, S.
Venue
Findings of the Association for Computational Linguistics: EACL 2023
Published
2023
Pages
883-893
Cited by
7
References
7

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Neural CRF-based Hierarchical Approach for Linear Text Segmentation. https://app.cochat.ai/papers/10.18653/v1/2023.findings-eacl.65
The paper
Nair, I., Garimella, A., Srinivasan, B. V., Modani, N., Chhaya, N., Karanam, S., & Shekhar, S. (2023). A Neural CRF-based Hierarchical Approach for Linear Text Segmentation. Findings of the Association for Computational Linguistics: EACL 2023. https://doi.org/10.18653/v1/2023.findings-eacl.65

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.