A Neural CRF-based Hierarchical Approach for Linear Text Segmentation
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
We consider the problem of segmenting unformatted text and transcripts linearly based on their topical structure.While prior approaches explicitly train to predict segment boundaries, we propose to address this task by inferring the hierarchical segmentation structure associated with the input text.For this purpose, we present a data curation strategy to obtain the hierarchical segmentation structure annotations for over 700K Wikipedia articles.We then propose the first supervised approach to generate hierarchical segmentation structures for given text based on a neural conditional random field (CRF) that explicitly models the statistical dependencies between nodes and their constituent children.We introduce a novel data augmentation scheme as part of our model training, which involves sampling a variety of node aggregations, permutations, and removals, all of which help capture fine-grained and coarse topical shifts in the data and improve model performance.Extensive experiments show that our model outperforms or achieves competitive performance when compared to previous state-of-the-art algorithms in the following settings: rich-resource, cross-domain transferability, few-shot supervision, and segmentation when topic label annotations are provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The one we tackle is that most existing methods retrieve only a few short, contiguous text chunks, which limits their ability to represent and leverage large-scale discourse structure.”
From RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024) · cites this paper as Nair et al., 2023
Experimental — accuracy not yet measuredIn the citing paper
“Although this approach is widely adopted, Nair et al. (2023) highlights a potential shortcoming: contiguous segmentation might not capture the complete semantic depth of the text.”
From RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024) · cites this paper as Nair et al., 2023
In the citing paper
“However, most existing methods retrieve only short contiguous chunks from a retrieval corpus, limiting holistic understanding of the overall document context.”
From RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024) · cites this paper as Nair et al., 2023
In the citing paper
“Despite a diversity in methods, the retrieving components of models predominantly rely on standard approaches, i.e., chunking corpora and encoding with BERT-based retrievers.”
From RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024) · cites this paper as Nair et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · cited 1×
“We believe that using a medical domain specific encoder (Gu et al., 2021) would improve our model’s performance in the Disease domain.”
MethodMiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020) · arXiv · doi:10.48550/arxiv.2002.10957 · Cited by 703 · cited 3×
“The parameters of this model are fine-tuned using self-attention distillation (Wang et al., 2022) for the compression of large language models like RoBERTa-Large (Liu et al.”
MethodText Segmentation as a Supervised Learning Task
Koshorek, O., Cohen, A., Mor, N., Rotman, M., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2075 · Cited by 167 · cited 10×
“As done in (Koshorek et al., 2018), the articles in the HTML form are preprocessed using WikiExtractor 4 to remove (a) non-text elements such as tables and figures, and (b) very short sections and sub-sections spanning fewer than three sentences.”
BackgroundMethodInfluentialSECTOR: A Neural Model for Coherent Topic Segmentation and Classification
Arnold, S., Schneider, R., Cudré-Mauroux, P., Gers, F. A., & Löser, A. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00261 · Cited by 85 · cited 3×
“Finally, we evaluate how our model utilises topic label information for segmentation using WikiSection (Arnold et al., 2019).”
MethodText Segmentation by Cross Segment Attention
Lukasik, M., Dadachev, B., Papineni, K., & Simões, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.380 · Cited by 68
Research works printed but not linked: 35
- Pinkesh Badjatiya, Litton J. Kurisinkel, Manish Gupta, and Vasudeva Varma. 2018. Attention-based neural text segmentation. In Advances in Information Re- trieval, pages 180-193, Cham. Springer International Publishing.
- Joe Barrow, Rajiv Jain, Vlad Morariu, Varun Manju- natha, Douglas Oard, and Philip Resnik. 2020. A joint model for document segmentation and segment labeling. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 313-322, Online. Association for Computa- tional Linguistics.
- Doug Beeferman, Adam Berger, and John Lafferty. 1999. Statistical models for text segmentation. Ma- chine learning, 34(1):177-210.
- David M. Blei and Pedro J. Moreno. 2001. Topic seg- mentation with an aspect hidden markov model. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '01, page 343-348, New York, NY, USA. Association for Computing Machinery.
- Freddy Y. Y. Choi. 2000. Advances in domain inde- pendent linear text segmentation. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics.
- Noam Chomsky. 1959. On certain formal properties of grammars. Information and Control, 2(2):137-167.
- Lan Du, Wray Buntine, and Mark Johnson. 2013. Topic segmentation with a structured topic model. In Pro- ceedings of the 2013 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 190-200, Atlanta, Georgia. Association for Compu- tational Linguistics.
- Jacob Eisenstein. 2009. Hierarchical text segmentation from multi-scale lexical cohesion. In Proceedings of Human Language Technologies: The 2009 An- nual Conference of the North American Chapter of the Association for Computational Linguistics, pages 353-361, Boulder, Colorado. Association for Com- putational Linguistics.
- Jacob Eisenstein and Regina Barzilay. 2008. Bayesian unsupervised topic segmentation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, page 334-343, USA. Association for Computational Linguistics.
- Pavlina Fragkou, Vassilios Petridis, and Ath Kehagias. 2004. A dynamic programming algorithm for linear text segmentation. Journal of Intelligent Information Systems, 23(2):179-197.
- Goran Glavaš, Federico Nanni, and Simone Paolo Ponzetto. 2016. Unsupervised text segmentation us- ing semantic relatedness graphs. In Proceedings of the Fifth Joint Conference on Lexical and Computa- tional Semantics, pages 125-130, Berlin, Germany. Association for Computational Linguistics.
- Barbara J. Grosz and Candace L. Sidner. 1986. At- tention, intentions, and the structure of discourse. Comput. Linguist., 12(3):175-204.
- Marti A. Hearst. 1997. Texttiling: Segmenting text into multi-paragraph subtopic passages. Comput. Lin- guist., 23(1):33-64.
- John E Hopcroft, Rajeev Motwani, and Jeffrey D Ullman. 2001. Introduction to automata theory, languages, and computation. Acm Sigact News, 32(1):60-65.
- Anna Kazantseva and Stan Szpakowicz. 2011. Linear text segmentation using affinity propagation. In Pro- ceedings of the 2011 Conference on Empirical Meth- ods in Natural Language Processing, pages 284-293, Edinburgh, Scotland, UK. Association for Computa- tional Linguistics.
- John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling se- quence data. In Proceedings of the Eighteenth In- ternational Conference on Machine Learning, ICML '01, page 282-289, San Francisco, CA, USA.
- Martin Lange and Hans Leiß. 2009. To cnf or not to cnf? an efficient yet presentable version of the cyk algorithm. Informatica Didactica, 8(2009):1-21.
- K. Lari and S.J. Young. 1990. The estimation of stochas- tic context-free grammars using the inside-outside algorithm. Computer Speech Language, 4(1):35-56.
- Jing Li, Aixin Sun, and Shafiq Joty. 2018. Segbot: A generic neural text segmentation model with pointer network. In Proceedings of the Twenty-Seventh Inter- national Joint Conference on Artificial Intelligence, IJCAI-18, pages 4166-4172. International Joint Con- ferences on Artificial Intelligence Organization.
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692. · arXiv
- Kelvin Lo, Yuan Jin, Weicong Tan, Ming Liu, Lan Du, and Wray Buntine. 2021. Transformer over pre- trained transformer for neural text segmentation with enhanced topic coherence. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2021, pages 3334-3340, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Igor Malioutov and Regina Barzilay. 2006. Minimum cut model for spoken lecture segmentation. In Pro- ceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meet- ing of the Association for Computational Linguis- tics, pages 25-32, Sydney, Australia. Association for Computational Linguistics.
- Igor Malioutov, Alex Park, Regina Barzilay, and James Glass. 2007. Making sense of sound: Unsupervised topic segmentation over acoustic input. In Proceed- ings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 504-511, Prague, Czech Republic. Association for Computational Lin- guistics.
- Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated cor- pus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330.
- Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia. Association for Computational Linguistics.
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Pro- cessing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
- Lev Pevzner and Marti A. Hearst. 2002. A critique and improvement of an evaluation metric for text segmentation. Computational Linguistics, 28(1):19- 36.
- Matthew Purver, Konrad P. Körding, Thomas L. Grif- fiths, and Joshua B. Tenenbaum. 2006. Unsupervised topic modelling for multi-party spoken discourse. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meet- ing of the Association for Computational Linguis- tics, pages 17-24, Sydney, Australia. Association for Computational Linguistics.
- Itiroo Sakai. 1961. Syntax in universal translation. In Proceedings of the International Conference on Ma- chine Translation and Applied Language Analysis.
- Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 818-827, Vancouver, Canada. Association for Computational Linguistics.
- Masao Utiyama and Hitoshi Isahara. 2001. A statistical model for domain-independent text segmentation. In Proceedings of the 39th Annual Meeting of the Associ- ation for Computational Linguistics, pages 499-506, Toulouse, France. Association for Computational Lin- guistics.
- Teun A Van Dijk. 1982. Episodes as units of discourse analysis. Analyzing discourse: Text and talk, pages 177-195.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
- Naiwen Xue, Fei Xia, Fu-dong Chiou, and Marta Palmer. 2005. The penn chinese treebank: Phrase struc- ture annotation of a large corpus. Nat. Lang. Eng., 11(2):207-238.
- Yu Zhang, Houquan Zhou, and Zhenghua Li. 2021. Fast and accurate neural crf constituency parsing. In Pro- ceedings of the Twenty-Ninth International Joint Con- ference on Artificial Intelligence, IJCAI'20.
Cited by
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024) · arXiv · doi:10.48550/arxiv.2401.18059 · Cited by 36
“Although this approach is widely adopted, Nair et al. (2023) highlights a potential shortcoming: contiguous segmentation might not capture the complete semantic depth of the text.”
Metadata
- Authors
- Nair, I., Garimella, A., Srinivasan, B. V., Modani, N., Chhaya, N., Karanam, S., & Shekhar, S.
- Venue
- Findings of the Association for Computational Linguistics: EACL 2023
- Published
- 2023
- Pages
- 883-893
- Cited by
- 7
- References
- 7
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Neural CRF-based Hierarchical Approach for Linear Text Segmentation. https://app.cochat.ai/papers/10.18653/v1/2023.findings-eacl.65
- The paper
- Nair, I., Garimella, A., Srinivasan, B. V., Modani, N., Chhaya, N., Karanam, S., & Shekhar, S. (2023). A Neural CRF-based Hierarchical Approach for Linear Text Segmentation. Findings of the Association for Computational Linguistics: EACL 2023. https://doi.org/10.18653/v1/2023.findings-eacl.65
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.