A Systematic Review of Re-Identification Attacks on Health Data
Open accessThe DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Privacy legislation in most jurisdictions allows the disclosure of health data for secondary purposes without patient consent if it is de-identified. Some recent articles in the medical, legal, and computer science literature have argued that de-identification methods do not provide sufficient protection because they are easy to reverse. Should this be the case, it would have significant and important implications on how health information is disclosed, including: (a) potentially limiting its availability for secondary purposes such as research, and (b) resulting in more identifiable health information being disclosed. Our objectives in this systematic review were to: (a) characterize known re-identification attacks on health data and contrast that to re-identification attacks on other kinds of data, (b) compute the overall proportion of records that have been correctly re-identified in these attacks, and (c) assess whether these demonstrate weaknesses in current de-identification methods. METHODS AND FINDINGS: Searches were conducted in IEEE Xplore, ACM Digital Library, and PubMed. After screening, fourteen eligible articles representing distinct attacks were identified. On average, approximately a quarter of the records were re-identified across all studies (0.26 with 95% CI 0.046-0.478) and 0.34 for attacks on health data (95% CI 0-0.744). There was considerable uncertainty around the proportions as evidenced by the wide confidence intervals, and the mean proportion of records re-identified was sensitive to unpublished studies. Two of fourteen attacks were performed with data that was de-identified using existing standards. Only one of these attacks was on health data, which resulted in a success rate of 0.00013. CONCLUSIONS: The current evidence shows a high re-identification rate but is dominated by small-scale studies on data that was not de-identified according to existing standards. This evidence is insufficient to draw conclusions about the efficacy of de-identification methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even if the data collected by ChatGPT are deidentified, there is still the potential for individuals to be reidentified by combining them with other available data sources [31].”
From Ethical Considerations of Using ChatGPT in Health Care (2023) · cites this paper as [31]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Such re-identification attacks 39 have been shown to yield high success rates both with tabular data 40,41 (such as patient records) and medical imaging data 42 .”
From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“To evaluate the accuracy of the screening, we performed an inter-rater reliability analysis with two independent raters. After the first rater completed his screening (KEE), a second rater not involved in the study in any way was recruited (KA). We went through the study objectives and screening criteria with the seco…”
MethodPreferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“Figure 1 depicts a PRISMA diagram (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) for this review [50,51], and Checklist S1 contains the completed PRISMA checklist.”
MethodDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 2×
“identified to a tolerance value based on the commonly used ruleof-thumb provided in the literature of 5|kz10, where k is the number of studies included in the analysis [46,47].”
Methodk-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571 · cited 2×
“C [67] 2002 Yes Researcher 1 of 135,000 USA Removal of names and addresses Yes”
Background
Research works printed but not linked: 4
- A Robeznieks, 2005, Privacy fear factor arises., Modern Healthcare, 35, 6
- E Bartlett, 1993, RMS need to safeguard computerized patient records to protect hospitals., Hospital Risk Management, 15
- 2007, Mike Gordon v. the Minister of Health and the Privacy Commissioner of Canada: Memorandum of Fact and Law of the Privacy Commissioner of Canada., Federal Court
- L Sweeney, Data sharing under HIPAA: 12 years later. Workshop on the HIPAA Privacy Rule's De-Identification Standard, March 8–9, 2010
Cited by
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,676
“Such re-identification attacks 39 have been shown to yield high success rates both with tabular data 40,41 (such as patient records) and medical imaging data 42 .”
BackgroundPreparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164
“Note, however, that even the most accurate automatic de-identification techniques fail on occasion to remove all possible instances of personally identifiable information (PII).(16) Because we planned to make these studies publicly available, we wanted to verify that the automatic de-identification processes removed a…”
BackgroundPrivacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001
“A number of previous studies [21], [22], [47], [48], [49], [50] have looked into membership attacks from statistics commonly published in genome-wide association studies (GWAS).”
ResultPrivacy in the Internet of Things: threats and challenges
Ziegeldorf, J. H., Morchon, O. G., & Wehrle, K. (2014) · Security and Communication Networks · doi:10.1002/sec.795 · Cited by 610
“Those approaches are difficult to fit to the IoT: Most data anonymization techniques can be broken using auxiliary data [78, 79, 80], that is likely to become available at some point during the IoT evolution.”
BackgroundTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
Metadata
- Authors
- El Emam, K., Jonker, E., Arbuckle, L., & Malin, B.
- Affiliations
- Children's Hospital of Eastern Ontario (CA)
- Vanderbilt University (US)
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-12-02
- Volume
- 6
- Issue
- 12
- Pages
- e28071
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3229505
- OpenAlex
- W2083384763
- Semantic Scholar
- 3803595fac5818f0fce6296a6e3afc414c280a5a
- Cited by
- 427
- References
- 64
- Citations by year
- 2026: 36 · 2025: 42 · 2024: 48 · 2023: 34 · 2022: 33 · 2021: 29 · 2020: 46 · 2019: 36 · 2018: 27 · 2017: 19 · 2016: 20 · 2015: 20 · 2014: 15 · 2013: 13 · 2012: 9
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-04-16)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Vanderbilt University
- Canada Research Chairs
- National Institutes of Health
- National Human Genome Research Institute (U01HG00460301, U01HG004603)
- U.S. National Library of Medicine (R01LM009989)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Health Sciences › Health Professions › Health Information Management › Thoreau and American Literature
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Keywords
- re-identification attacks, health data privacy, de-identification, secondary use of health data, privacy law, health information disclosure
- MeSH terms
- Confidentiality, Humans, Software, United States, Reproducibility of Results, Models, Statistical, Databases, Factual, Medical Records Systems, Computerized, Computer Security, Privacy, Health Insurance Portability and Accountability Act
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Systematic Review of Re-Identification Attacks on Health Data. https://app.cochat.ai/papers/10.1371/journal.pone.0028071
- The paper
- El Emam, K., Jonker, E., Arbuckle, L., & Malin, B. (2011). A Systematic Review of Re-Identification Attacks on Health Data. PLoS ONE. https://doi.org/10.1371/journal.pone.0028071
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.