The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index; none of the 2 sources its claims cite is checked for support yet.
Word Segmentation for Classical Chinese Buddhist Literature
Open accessAbstract
With the growth of digital humanities, information technologies take on more important roles in humanities research, including the study of religion. To analyze text for further processing, many text analysis tools treat a word as a unit. However, in Chinese, there are no word boundary markers. Word segmentation is required for processing Chinese texts. Although several word segmentation tools are available for modern Chinese, there is still no practical word segmentation tool for Classical Chinese, especially for Classical Chinese Buddhist literature. In this paper, we adopt unsupervised and supervised learning techniques to build Classical Chinese word segmentation approaches for processing Buddhist literature. Normalized variation of branching entropy (nVBE) is adopted for unsupervised word segmentation. Conditional random fields (CRF) are used to generate supervised models for Classical Chinese word segmentation. The performance of our word segmentation approach achieves an F-score of up to 0.9396. The experimental results show that our proposed method is effective for correctly segmenting most Classical Chinese sentences in Buddhist literature. Our word segmentation method can be a fundamental tool for further text analysis and processing research, such as word embedding, syntactic parsing, and semantic labeling.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
11 Own result · 5 Own interpretation · 4 Own method · 1 Sources not found · 1 No full text of the sources · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall average F-score is 0.7303.”
- Also stated in: § 5 Conclusion · plain
“The F-scores of our unsupervised and supervised approaches average 0.7307 and 0.9396, respectively.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“most of these errors can be divided into two types.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.5 Error Cases of Model 2
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we constructed a dataset with wider and more diverse coverage”
- Also stated in: § 5 Conclusion · plain
“we constructed a training set from annotated corpora created by Academia Sinica and Beijing Longquan Temple”
§ Word Segmentation for Classical Chinese Buddhist Literature > 1 Introduction
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our supervised model is more suitable and effective for the Classical Chinese word segmentation.”
- Also stated in: § 5 Conclusion · plain
“outperforms the nVBE approach, as well as the widely used modern Chinese word segmentation methods CKIP and Jieba, which can be regarded as baseline tests.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.2 Evaluation Metrics
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 Conclusion · plain
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, current Chinese word segmentation tools do not achieve good results on Classical Chinese texts.”
- Also stated in: § 4.2 Evaluation Metrics · plain
“The results show that the general modern Chinese word segmentation tools cannot achieve good results on Classical Chinese Buddhist texts.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.2 Evaluation Metrics · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The CRF model clearly outperforms the nVBE model: its average F-score is up to 0.9396.”
- Also stated in: § 5 Conclusion · plain
“The results show that the supervised approach based on the CRF model is more effective for segmenting words in Classical Chinese Buddhist texts and outperforms the nVBE approach”
§ Word Segmentation for Classical Chinese Buddhist Literature > Additional information
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The first type occurs because segmentation standards in our training dataset were not consistent.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.5 Error Cases of Model 2
plainstates a causen = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“if some terms, especially transliterations, share the same prefix or suffix, the nVBE model usually partitions the prefix and the following characters.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.4 Comparison between Model 1 and Model 2
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of our word segmentation approach achieves an F-score of up to 0.9396.”
- Also stated in: § Additional information · plain
“its average F-score is up to 0.9396.”
- Also stated in: § 5 Conclusion · plain
“The F-scores of our unsupervised and supervised approaches average 0.7307 and 0.9396, respectively.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Additional information · plain
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although our model 2 achieves good performance in all three metrics”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.5 Error Cases of Model 2
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The quote only partly states this claim. 1
- Probably not a substantial claim. 6
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the nVBE model tends to combine some fixed expressions into a word.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 4.4 Comparison between Model 1 and Model 2
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“human experts from our team at the Dharma Drum Institute of Liberal Arts (DILA) recheck and unify both to assemble one final dataset.”
§ Word Segmentation for Classical Chinese Buddhist Literature > 3.2.2 CRF Model Training
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the unsupervised model does not require training data, to compare its performance with the supervised model, we also used ten segments for the unsupervised model.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“there is still no practical word segmentation tool for Classical Chinese, especially for Classical Chinese Buddhist literature.”
- Also stated in: § 1 Introduction · plain
“Classical Chinese word segmentation has received much less attention.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- Also stated in: § 1 Introduction · plain
In the paper
“CRF has been shown to be the optimal algorithm for sequence classification (Rosenfeld, Feldman, and Fresko 2006).”
§ Word Segmentation for Classical Chinese Buddhist Literature > 1 Introduction
What the sources say
- Not matchedcites Rosenfeld, Feldman, and Fresko 2006
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The whole dataset is randomly partitioned into ten subsets of equal size.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For evaluating the supervised model, we adopt the commonly used method of tenfold crossvalidation.”
§ Word Segmentation for Classical Chinese Buddhist Literature > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The majority of Chinese Buddhist scriptures use Classical Chinese.
“Most of the Chinese Buddhist scriptures are written in Classical Chinese.”
The paper relies on this without support.
Classical and modern Chinese differ greatly in vocabulary and syntax.
“The vocabulary and syntax of Classical and modern Chinese are very different.”
The paper relies on this without support.
Advanced analysis of Chinese Buddhist literature requires word segmentation as a foundational step.
“Word segmentation is the necessary and fundamental basis for advanced analysis of Chinese Buddhist literature, which underlies other forms of analysis such as word embedding, dependency parsing, syntactic parsing, and semantic role labeling.”
The paper relies on this without support.
Compared with typical Classical Chinese, Buddhist texts in Classical Chinese have more multi-syllable words, particularly transliterations from Indic languages.
“Classical Chinese Buddhist literature contains many longer words with multiple syllables, especially transliterations from Indic languages.”
The paper relies on this without support.
The two source corpora used inconsistent word segmentation criteria.
“Since the word segmentation criteria used by these two resources are not consistent”
The authors state this as an assumption.
The branching entropy method assumes that if human language sequences were random, branching entropy would decrease as sequence length increases.
“The major idea of branching entropy is based on the hypothesis that if sequences produced by human language were random, we would expect the branching entropy of a sequence (n-grams in a corpus) to decrease as we increase the length of the sequence.”
The authors state this as an assumption.
An increase in entropy of successive tokens marks a word boundary.
“If the entropy of successive tokens increases, the location is at a word border.”
The paper relies on this without support.
The VBEs of strings of different lengths are not directly comparable and require normalization.
“The VBEs are not directly comparable for strings of different lengths and need to be normalized.”
Citations
Cites
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Chinese segmentation and new word detection using conditional random fields
Peng, F., Feng, F., & McCallum, A. (2004) · International Conference on Computational Linguistics · doi:10.3115/1220355.1220436 · Cited by 468
Chinese Word Segmentation as Character Tagging
Xue, N. (2003) · doi:10.30019/ijclclp.200302.0002 · Cited by 373
A Conditional Random Field Word Segmenter for Sighan Bakeoff 2005
Tseng, H., Chang, P.-C., Andrew, G., Jurafsky, D. S., & Manning, C. D. (2005) · Cited by 329
Deep Learning for Chinese Word Segmentation and POS Tagging
Zheng, X., Chen, H., & Xu, T. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1061 · Cited by 295
Long Short-Term Memory Neural Networks for Chinese Word Segmentation
Chen, X., Qiu, X., Zhu, C., Liu, P., & Huang, X. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1141 · Cited by 283
The first international Chinese word segmentation Bakeoff
Sproat, R., & Emerson, T. (2003) · doi:10.3115/1119250.1119269 · Cited by 214
Chinese Word Segmentation: A Decade Review
Zhao, H. Y. (2007) · Zhongwen xinxi xuebao · Cited by 141
Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field Modeling
Zhao, H., Huang, C., Li, M., et al. (2006) · Institutional Repositories DataBase (IRDB) · Cited by 112
Chinese word segmentation based on maximum matching and word binding force
Wong, P.-K., & Chan, C. (1996) · International Conference on Computational Linguistics · doi:10.3115/992628.992665 · Cited by 93
Cited by
Sorted by citations
Shieh, C.-J., & Na-Songkhla, J. (2023) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/14060 · Cited by 12
The Impact of High-Tech Policies on Corporate ESG Performance: Evidence from China
Qin, F., & Zhang, M. (2024) · Sustainability · doi:10.3390/su162310321 · Cited by 5
Crosslinguistic Semantic Textual Similarity of Buddhist Chinese and Classical Tibetan
Felbur, R., Meelen, M., & Vierthaler, P. (2022) · Journal of Open Humanities Data · doi:10.5334/johd.86 · Cited by 4
Untung, S. H., & Sakti, M. D. A. B. (2023) · Jurnal Nyanadassana Jurnal Penelitian Pendidikan Sosial dan Keagamaan · doi:10.59291/jnd.v2i1.24 · Cited by 2
“Agama ini menekankan aspek kemanusiaan yang cukup tinggi dengan beberapa aturan serta norma-norma yang berkaitan dengan aspek-aspek kemanusiaan tersebut (Wang, 2020).”
Background
Metadata
- Authors
- Wang, Y.-C.
- Venue
- Journal of the Japanese Association for Digital Humanities
- Published
- 2020-12-25
- Volume
- 5
- Issue
- 2
- Pages
- 154-172
- ISSN
- 2188-7276 (ISSN-L)
- OpenAlex
- W3116629257
- Cited by
- 6
- References
- 24
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 1 · 2023: 2 · 2022: 1
- FWCI
- 0.25
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- other-oa
- Locations
- Journal of the Japanese Association for Digital Humanities (publishedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- word segmentation, classical Chinese, Buddhist scriptures, conditional random fields, unsupervised segmentation, supervised segmentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
