Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.

Deidentification of free-text medical records using pre-trained bidirectional transformers

Open access
Johnson, Alistair E. W., Bulgarelli, Lucas, Pollard, Tom J. · 2020 · Proceedings of the ACM Conference on Health, Inference, and Learning

Abstract

The ability of caregivers and investigators to share patient data is fundamental to many areas of clinical practice and biomedical research. Prior to sharing, it is often necessary to remove identifiers such as names, contact details, and dates in order to protect patient privacy. Deidentification, the process of removing identifiers, is challenging, however. High-quality annotated data for developing models is scarce; many target identifiers are highly heterogenous (for example, there are uncountable variations of patient names); and in practice anything less than perfect sensitivity may be considered a failure. As a result, patient data is often withheld when sharing would be beneficial, and identifiable patient data is often divulged when a deidentified version would suffice. In recent years, advances in machine learning methods have led to rapid performance improvements in natural language processing tasks, in particular with the advent of large-scale pretrained language models. In this paper we develop and evaluate an approach for deidentification of clinical notes based on a bidirectional transformer model. We propose human interpretable evaluation measures and demonstrate state of the art performance against modern baseline models. Finally, we highlight current challenges in deidentification, including the absence of clear annotation guidelines, lack of portability of models, and paucity of training data. Code to develop our model is open source, allowing for broad reuse.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3368555.3384455

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1: Architecture of the model with example text and predictions. Text is tokenized and fed into 12 identically constructed transformer blocks. Weights within the transformer blocks are initialized using various publicly available pretrained models. The final output of the transformer blocks is fed into a linear classification layer. Note the use of sub-word tokenization (represented by two hashes before the sub-word), and the class of the intermediate punctuation tokens.
Figure 1: Architecture of the model with example text and predictions. Text is tokenized and fed into 12 identically constructed transformer blocks. Weights within the transformer blocks are initialized using various publicly available pretrained models. The final output of the transformer blocks is fed into a linear classification layer. Note the use of sub-word tokenization (represented by two hashes before the sub-word), and the class of the intermediate punctuation tokens.CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 33 references

Research works printed but not linked: 13

  1. 10.5555/3327144.3327272 · doi:10.5555/3327144.3327272
  2. Halevy Alon, 2009, The Unreasonable Effectiveness of Data. Intelligent Systems, IEEE, 24, 8
  3. Johnson Alistair EW, 2016, Leo Anthony Celi, and Roger G Mark
  4. 10.13026/0757-0y85 · doi:10.13026/0757-0y85
  5. Lafferty John D., Proceedings of the Eighteenth International Conference on Machine Learning (ICML '01)
  6. 10.1093/bioinformatics/btz682 · DOI · Paper page
  7. Paszke Adam, 2017, Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration.
  8. Paszke Adam, PyTorch: An Imperative Style
  9. Peters Matthew E, 2018, Deep contextualized word representations. arXiv preprint arXiv:1802.05365 · arXiv
  10. Sacha, 2019, Auditable Hypothesis Testing. arXiv
  11. 10.5555/3169406.3169752 · doi:10.5555/3169406.3169752
  12. Employee Benefits Security Administration U.S. Dept., 2004, of Labor
  13. Yang Zhilin, 2019, Le

Cited by

See all 55

Metadata

Authors
Johnson, A. E. W., Bulgarelli, L., & Pollard, T. J.
Affiliations
Massachusetts Institute of Technology (US)
Venue
Proceedings of the ACM Conference on Health, Inference, and Learning
Publisher
ACM
Published
2020-04-02
Volume
2020
Pages
214-221
OpenAlex
W3012137864
Cited by
55
References
33
Citations by year
2026: 7 · 2025: 16 · 2024: 7 · 2023: 8 · 2022: 5 · 2021: 10 · 2020: 2
FWCI
2.39
Type
conference paper
Language
en
Open access
Locations
  • ACM Conference on Health, Inference, and Learning (CHIL) (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, cc-by-nc)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
Funders
NIH (NIH-R01-EB017205)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Thoreau and American Literature
Keywords
de-identification, clinical notes, patient identifiers, bidirectional transformer, pre-trained language models, patient privacy, data sharing, annotation guidelines, model portability, limited training data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Deidentification of free-text medical records using pre-trained bidirectional transformers. https://app.cochat.ai/papers/10.1145/3368555.3384455
The paper
Johnson, A. E. W., Bulgarelli, L., & Pollard, T. J. (2020). Deidentification of free-text medical records using pre-trained bidirectional transformers. Proceedings of the ACM Conference on Health, Inference, and Learning. https://doi.org/10.1145/3368555.3384455

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.