CoChat
Sign Up
Cited by 101Biomedical Text Mining and OntologiesMachine Learning in HealthcareTopic Modeling

The DOI matches its Crossref record; 158 of the 158 references it cites were found in a registry or an index.

A survey on clinical natural language processing in the United Kingdom from 2007 to 2022

Open access
Wu, Honghan, Wang, Minhong, Wu, Jinge, Francis, Farah, Chang, Yun-Hsuan, Shavick, Alex, Dong, Hang, Poon, Michael T. C., Fitzpatrick, Natalie, Levine, Adam P., Slater, Karin T., Handy, Alex, Karwath, Andreas, Gkoutos, Georgios V., Chelala, Claude, Shah, Anoop Dinesh, Stewart, Robert, Collier, Nigel, Alex, Beatrice, Whiteley, William, Sudlow, Cathie, Roberts, Angus, Dobson, Richard J. B.·2022·npj Digital Medicine
DOI 10.1038/s41746-022-00730-6PMC PMC9770568
Open this paper in CoChat — free accountOpen PDF

Abstract

Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects ( n = 94; £ = 41.97 m) funded by UK funders or the European Union’s funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019–2022 was 80 times that of 2007–2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP’s great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-022-00730-6

Cites

158 of 158 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 The scope of this study is composed of two main parts. a A UK community survey (the lower oval); and b a literature review of the community’s research outputs (the upper oval). *NHS—National Health Service in the UK; RL/ML/LLM—NLP technologies of rule-based, machine learning and large language models.
Fig. 1 The scope of this study is composed of two main parts. a A UK community survey (the lower oval); and b a literature review of the community’s research outputs (the upper oval). *NHS—National Health Service in the UK; RL/ML/LLM—NLP technologies of rule-based, machine learning and large language models.CC BY · PubMed Central
Fig. 2 Snapshots (force-directed visualisations) of the community from 2012 to 2022. The graphs contain four types of entities: projects, persons, organisations and funders. Each graph is constructed using data from projects with a start date earlier than or in the given year. Graph data is cumulative, meaning a later year’s data is a superset of its previous years. The size of organisation nodes indicates the number of total amounts in pound sterling they received in funding.
Fig. 2 Snapshots (force-directed visualisations) of the community from 2012 to 2022. The graphs contain four types of entities: projects, persons, organisations and funders. Each graph is constructed using data from projects with a start date earlier than or in the given year. Graph data is cumulative, meaning a later year’s data is a superset of its previous years. The size of organisation nodes indicates the number of total amounts in pound sterling they received in funding.CC BY · PubMed Central
Fig. 3 Histogram of person nodes Eigenvector centrality scores. The x-axis is the eigenvector centrality score and the y-axis (log scale) is the number of people with certain scores.
Fig. 3 Histogram of person nodes Eigenvector centrality scores. The x-axis is the eigenvector centrality score and the y-axis (log scale) is the number of people with certain scores.CC BY · PubMed Central
Fig. 4 Trends in the last 15 years on budgets of all clinical NLP projects, those involving NHS and those involving industry organisations. Each tick on the x-axis is a 3-year period. The y-axis shows the total budget. The sums of NHS involved and industry involved project budgets are plotted alongside the budget of all projects across five 3-year periods.
Fig. 4 Trends in the last 15 years on budgets of all clinical NLP projects, those involving NHS and those involving industry organisations. Each tick on the x-axis is a 3-year period. The y-axis shows the total budget. The sums of NHS involved and industry involved project budgets are plotted alongside the budget of all projects across five 3-year periods.CC BY · PubMed Central
Fig. 5 The development of studentship projects in clinical NLP from 2016 to 2021. The three figures (from left to right) show the networks of studentship projects and their associated entities (funders, organisations and persons) for 2016, 2017 and 2021 respectively. The 2021 entire network is too big to be shown fully using the same scale. Therefore, a low-resolution overview is shown at the top right and a snapshot of it is displayed using the same scale as other years.
Fig. 5 The development of studentship projects in clinical NLP from 2016 to 2021. The three figures (from left to right) show the networks of studentship projects and their associated entities (funders, organisations and persons) for 2016, 2017 and 2021 respectively. The 2021 entire network is too big to be shown fully using the same scale. Therefore, a low-resolution overview is shown at the top right and a snapshot of it is displayed using the same scale as other years.CC BY · PubMed Central
Fig. 6 NLP tasks versus technical objectives. The x-axis is the categories of NLP tasks and the y-axis is the technical objectives. The size of the circles denotes the number of publications.
Fig. 6 NLP tasks versus technical objectives. The x-axis is the categories of NLP tasks and the y-axis is the technical objectives. The size of the circles denotes the number of publications.CC BY · PubMed Central
Fig. 7 NLP algorithm type breakdown and their development trends over the last 15 years. The main bar chart shows the changes of different NLP algorithms in the last 15 years. The pie chart at the top left corner depicts the overall breakdown of algorithms of all research work analysed.
Fig. 7 NLP algorithm type breakdown and their development trends over the last 15 years. The main bar chart shows the changes of different NLP algorithms in the last 15 years. The pie chart at the top left corner depicts the overall breakdown of algorithms of all research work analysed.CC BY · PubMed Central
Fig. 8 Knowledge representation and distributed representations. The pie chart at the left shows the breakdown of representation techniques. For ontologies, the bar chart on the right depicts the top five frequently used ontologies in clinical NLP applications.
Fig. 8 Knowledge representation and distributed representations. The pie chart at the left shows the breakdown of representation techniques. For ontologies, the bar chart on the right depicts the top five frequently used ontologies in clinical NLP applications.CC BY · PubMed Central
Fig. 9 Information collection and data extraction. Step 1: Data were collected for funded clinical NLP projects by querying three searchable datasets from UK and EU funding bodies and downloading project data from UK charities such as British Heart Foundation and Cancer Research UK. Step 2: Data were extracted to obtain metadata of projects and their associated entities.
Fig. 9 Information collection and data extraction. Step 1: Data were collected for funded clinical NLP projects by querying three searchable datasets from UK and EU funding bodies and downloading project data from UK charities such as British Heart Foundation and Cancer Research UK. Step 2: Data were extracted to obtain metadata of projects and their associated entities.CC BY · PubMed Central
Fig. 10 Flow chart describing publication identification for clinical NLP literature review. We started with 431 extracted publications, out of which 361 have sufficient information suitable for screening. The title/abstract screen further removed 202 papers which were deemed irrelevant. This left us 159 publications for an eligibility assessment using inclusion/exclusion criteria on their full text. After this final check step, 107 publications were included for the final review.
Fig. 10 Flow chart describing publication identification for clinical NLP literature review. We started with 431 extracted publications, out of which 361 have sufficient information suitable for screening. The title/abstract screen further removed 202 papers which were deemed irrelevant. This left us 159 publications for an eligibility assessment using inclusion/exclusion criteria on their full text. After this final check step, 107 publications were included for the final review.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 157 references

Research works printed but not linked: 9

  1. Lewin, I., Silins, I., Korhonen, A., Hogberg, J. & Stenius, U. A new challenge for text mining: cancer risk assessment. Proc. ISMB BioLINK Spec. Interest Group Text. Data Min. 20, 1–4 (2008).
  2. Baker, S., Korhonen, A. & Pyysalo, S. Cancer hallmark text classification using convolutional neural networks. In Proc. Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016), 1–9 (2016).
  3. Limsopatham, N. & Collier, N. Learning orthographic features in bi-directional lstm for biomedical named entity recognition. In Proc. Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016), 10–19 (2016).
  4. Wu, H. et al. SemEHR: a general-purpose semantic search system to surface semantic data from clinical notes for tailored care, trial recruitment, and clinical research. J. Am. Med. Inform. Assoc. 25, 530–537 (2018). · DOI
  5. Patel, R. et al. Impact of the covid-19 pandemic on remote mental healthcare and prescribing in psychiatry: an electronic health record study. BMJ Open 11, e046365 (2021). · DOI
  6. Congress, U. HR 3590: Patient Protection and Affordable Care Act. In 111th Congress, vol. 2010 (2009).
  7. Topol, E. et al. The topol review. Preparing the healthcare workforce to deliver the digital future. 1–48 (2019).
  8. Fruchterman, T. M. & Reingold, E. M. Graph drawing by force-directed placement. Software 21, 1129–1164 (1991).
  9. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186 (Association for Computational Linguistics, Minneapolis, Minnesota, 2019).

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Wu, H., Wang, M., Wu, J., Francis, F., Chang, Y.-H., Shavick, A., Dong, H., Poon, M. T. C., Fitzpatrick, N., Levine, A. P., Slater, K. T., Handy, A., Karwath, A., Gkoutos, G. V., Chelala, C., Shah, A. D., Stewart, R., Collier, N., Alex, B., Whiteley, W., Sudlow, C., Roberts, A., & Dobson, R. J. B.
Affiliations
  • University College London (GB)
  • University of Edinburgh (GB)
  • University of Oxford (GB)
  • University of Birmingham (GB)
  • Queen Mary University of London (GB)
  • King's College London (GB)
  • University of Cambridge (GB)
Corresponding author
Wu, Honghan
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2022-12-21
Volume
5
Issue
1
Pages
186-186
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-022-00730-6
PMC
PMC9770568
OpenAlex
W4312113143
Cited by
101
References
157
Citations by year
2026: 29 · 2025: 40 · 2024: 18 · 2023: 13 · 2022: 1
FWCI
1.87
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • Apollo
  • Apollo (cc-by)
Funders
  • DH | National Institute for Health Research (NIHR202639)
  • British Council (UCL-NMU-SEU International Collaboration)
  • University of Edinburgh (The Advanced Care Research Centre Programme)
  • RCUK | Medical Research Council (MR/W014386, National Text Analytics Project)
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
clinical natural language processing, clinical research funding, research funding trends, disease phenotyping, clinical text mining, research collaboration networks, academia-industry collaboration, United Kingdom

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review