Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Automated information extraction model enhancing traditional Chinese medicine RCT evidence extraction (Evi-BERT): algorithm development and validation.

Open access
Li, Yizhen, Luan, Zhongzhi, Liu, Yixing, Liu, Heyuan, Qi, Jiaxing, Han, Dongran · 2024 · Frontiers in Artificial Intelligence

Abstract

Background: In the field of evidence-based medicine, randomized controlled trials (RCTs) are of critical importance for writing clinical guidelines and providing guidance to practicing physicians. Currently, RCTs rely heavily on manual extraction, but this method has data breadth limitations and is less efficient. Objectives: To expand the breadth of data and improve the efficiency of obtaining clinical evidence, here, we introduce an automated information extraction model for traditional Chinese medicine (TCM) RCT evidence extraction. Methods: We adopt the Evidence-Bidirectional Encoder Representation from Transformers (Evi-BERT) for automated information extraction, which is combined with rule extraction. Eleven disease types and 48,523 research articles from the China National Knowledge Infrastructure (CNKI), WanFang Data, and VIP databases were selected as the data source for extraction. We then constructed a manually annotated dataset of TCM clinical literature to train the model, including ten evidence elements and 24,244 datapoints. We chose two models, BERT-CRF and BiLSTM-CRF, as the baseline, and compared the training effects with Evi-BERT and Evi-BERT combined with rule expression (RE). Results: We found that Evi-BERT combined with RE achieved the best performance (precision score = 0.926, Recall = 0.952, F1 score = 0.938) and had the best robustness. We totally summarized 113 pieces of rule datasets in the regulation extraction procedure. Our model dramatically expands the amount of data that can be searched and greatly improves efficiency without losing accuracy. Conclusion: Our work provided an intelligent approach to extracting clinical evidence for TCM RCT data. Our model can help physicians reduce the time spent reading journals and rapidly speed up the screening of clinical trial evidence to help generate accurate clinical reference guidelines. Additionally, we hope the structured clinical evidence and structured knowledge extracted from this study will help other researchers build large language models in TCM.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frai.2024.1454945

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Evidence element output types. We represented the element output states through four different levels: A-levels were long sentences, B-levels were short sentences, C-level were phrases, and D-levels were numbers.
Figure 1 Evidence element output types. We represented the element output states through four different levels: A-levels were long sentences, B-levels were short sentences, C-level were phrases, and D-levels were numbers.CC BY · PubMed Central
Figure 2 Number of disease types included in the corpus.
Figure 2 Number of disease types included in the corpus.CC BY · PubMed Central
Figure 3 A demonstration of the input and output of the Evi-BERT model combined with rule extraction.
Figure 3 A demonstration of the input and output of the Evi-BERT model combined with rule extraction.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

1 retracted

Show all 36 references

Research works printed but not linked: 1

  1. Liu, 2022, Research on SikuBERT-enhanced entity recognition of historical records from the perspective of digital humanity, Library Tribune, 42, 61

Cited by

See all 12

Metadata

Authors
Li, Y., Luan, Z., Liu, Y., Liu, H., Qi, J., & Han, D.
Affiliations
  • Beihang University (CN)
  • Beijing University of Chinese Medicine (CN)
Corresponding author
Luan, Zhongzhi
Venue
Frontiers in Artificial Intelligence
Published
2024-08-15
Volume
7
Pages
1454945-1454945
ISSN
2624-8212 (ISSN-L)
OpenAlex
W4401622044
Cited by
9
References
36
Citations by year
2026: 4 · 2025: 5
FWCI
1.22
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Artificial Intelligence (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
Keywords
automatic information extraction, traditional Chinese medicine, randomized controlled trials, rule extraction, clinical trial screening, clinical guideline development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automated information extraction model enhancing traditional Chinese medicine RCT evidence extraction (Evi-BERT): algorithm development and validation. https://app.cochat.ai/papers/10.3389/frai.2024.1454945
The paper
Li, Y., Luan, Z., Liu, Y., Liu, H., Qi, J., & Han, D. (2024). Automated information extraction model enhancing traditional Chinese medicine RCT evidence extraction (Evi-BERT): algorithm development and validation. Frontiers in Artificial Intelligence. https://doi.org/10.3389/frai.2024.1454945

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.