Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials.

Open access
Witte, Christian, Schmidt, David M., Cimiano, Philipp · 2024 · Journal of Biomedical Semantics

Abstract

Background: Systematic reviews of Randomized Controlled Trials (RCTs) are an important part of the evidence-based medicine paradigm. However, the creation of such systematic reviews by clinical experts is costly as well as time-consuming, and results can get quickly outdated after publication. Most RCTs are structured based on the Patient, Intervention, Comparison, Outcomes (PICO) framework and there exist many approaches which aim to extract PICO elements automatically. The automatic extraction of PICO information from RCTs has the potential to significantly speed up the creation process of systematic reviews and this way also benefit the field of evidence-based medicine. Results: Previous work has addressed the extraction of PICO elements as the task of identifying relevant text spans or sentences, but without populating a structured representation of a trial. In contrast, in this work, we treat PICO elements as structured templates with slots to do justice to the complex nature of the information they represent. We present two different approaches to extract this structured information from the abstracts of RCTs. The first approach is an extractive approach based on our previous work that is extended to capture full document representations as well as by a clustering step to infer the number of instances of each template type. The second approach is a generative approach based on a seq2seq model that encodes the abstract describing the RCT and uses a decoder to infer a structured representation of a trial including its arms, treatments, endpoints and outcomes. Both approaches are evaluated with different base models on a manually annotated dataset consisting of RCT abstracts on an existing dataset comprising 211 annotated clinical trial abstracts for Type 2 Diabetes and Glaucoma. For both diseases, the extractive approach (with ) reached the best $$F_1$$ F 1 score, i.e. 0.547 ( $$\pm 0.006$$ ± 0.006 ) for type 2 diabetes and 0.636 ( $$\pm 0.006$$ ± 0.006 ) for glaucoma. Generally, the $$F_1$$ F 1 scores were higher for glaucoma than for type 2 diabetes and the standard deviation was higher for the generative approach. Conclusion: In our experiments, both approaches show promising performance extracting structured PICO information from RCTs, especially considering that most related work focuses on the far easier task of predicting less structured objects. In our experimental results, the extractive approach performs best in both cases, although the lead is greater for glaucoma than for type 2 diabetes. For future work, it remains to be investigated how the base model size affects the performance of both approaches in comparison. Although the extractive approach currently leaves more room for direct improvements, the generative approach might benefit from larger models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13326-024-00305-2

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Illustration of both described approaches starting with the tokenized input and ending with the generated template instances
Fig. 1 Illustration of both described approaches starting with the tokenized input and ending with the generated template instancesCC BY · PubMed Central
Fig. 2 Schema of the PICO data model used in the experiments
Fig. 2 Schema of the PICO data model used in the experimentsCC BY · PubMed Central
Fig. 3 Illustration of linearization of an intervention template instance
Fig. 3 Illustration of linearization of an intervention template instanceCC BY · PubMed Central
Listing 1 Grammar of the data model used for decoding in our experiments
Listing 1 Grammar of the data model used for decoding in our experimentsCC BY · PubMed Central
Listing 2 Linearization of Glaucoma Publication Template Instance
Listing 2 Linearization of Glaucoma Publication Template InstanceCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 62 references

Research works printed but not linked: 2

  1. Sanchez-Graillet O, Cimiano P, Witte C, Ell B. C-TrO: An Ontology for Summarization and Aggregation of the Level of Evidence in Clinical Trials. In: Proc. of the 5th Joint Ontology Workshops (JOWO): Ontologies and Data in the Life Sciences. 2019. https://ceur-ws.org/Vol-2518/paper-ODLS7.pdf.
  2. Jin D, Szolovits P. PICO Element Detection in Medical Text via Long Short-Term Memory Neural Networks. In: Proceedings of the BioNLP 2018 Workshop. Melbourne: Association for Computational Linguistics; 2018. p. 67–75. https://doi.org/10.18653/v1/W18-2308. · DOI · Paper page

Cited by

See all 12

Metadata

Authors
Witte, C., Schmidt, D. M., & Cimiano, P.
Affiliations
Bielefeld University (DE)
Corresponding author
Schmidt, David M.
Venue
Journal of Biomedical Semantics
Published
2024-04-23
Volume
15
Issue
1
Pages
3-3
ISSN
2041-1480 (ISSN-L)
OpenAlex
W4395039840
Cited by
7
References
62
Citations by year
2026: 3 · 2025: 3 · 2024: 1
FWCI
0.95
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Biomedical Semantics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion, cc-by)
Funders
  • Federal Ministry of Health (BMG), Germany (KINBIOTICS)
  • Ministry of Culture and Science of the State of North Rhine-Westphalia, Germany (NW21-059A (SAIL))
  • Deutsche Forschungsgemeinschaft (Transregional Collaborative Research Centre TRR 318 "Constructing Explainability" (Projects B1 and C5))
  • Universität Bielefeld
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
Keywords
randomized controlled trials, clinical trial abstracts, structured data extraction, sequence-to-sequence models, template-based extraction, systematic reviews, type 2 diabetes, glaucoma, clinical text mining, document clustering, PICO framework, evidence-based medicine, clinical trial comparison
MeSH terms
Abstracting and Indexing, Humans, Natural Language Processing, Randomized Controlled Trials as Topic, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials. https://app.cochat.ai/papers/10.1186/s13326-024-00305-2
The paper
Witte, C., Schmidt, D. M., & Cimiano, P. (2024). Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials. Journal of Biomedical Semantics. https://doi.org/10.1186/s13326-024-00305-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.