A Multifaceted benchmarking of synthetic electronic health record generation models
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Synthetic health data have the potential to mitigate privacy concerns in supporting biomedical research and healthcare applications. Modern approaches for data generation continue to evolve and demonstrate remarkable potential. Yet there is a lack of a systematic assessment framework to benchmark methods as they emerge and determine which methods are most appropriate for which use cases. In this work, we introduce a systematic benchmarking framework to appraise key characteristics with respect to utility and privacy metrics. We apply the framework to evaluate synthetic data generation methods for electronic health records data from two large academic medical centers with respect to several use cases. The results illustrate that there is a utility-privacy tradeoff for sharing synthetic health data and further indicate that no method is unequivocally the best on all criteria in each use case, which makes it evident why synthetic data generation methods need to be assessed in context.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 1×
“DPGAN was a differentially private (DP) version of WGAN 36 , which achieved a theoretically guaranteed privacy protection on synthetic health data via employing the differential privacy principle 58 in GAN training.”
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · cited 1×
“We focus speci fi cally on structured EHR data 24 , as this type of data has supported numerous clinical association and outcome prediction studies.”
Estimating the success of re-identifications in incomplete datasets using generative models
Rocher, L., Hendrickx, J. M., & de Montjoye, Y.-A. (2019) · Nature Communications · doi:10.1038/s41467-019-10933-3 · Cited by 816
The National COVID Cohort Collaborative (N3C): Rationale, design, infrastructure, and deployment
Haendel, M. A., Chute, C. G., Bennett, T. D., Eichmann, D. A., Guinney, J., Kibbe, W. A., Payne, P. R. O., Pfaff, E. R., Robinson, P. N., Saltz, J. H., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa196 · Cited by 553
Self-citationSynthetic data generation for tabular health records: A systematic review
Hernandez, M., Epelde, G., Alberdi, A., Cilla, R., & Rankin, D. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.053 · Cited by 278 · cited 1×
“In earlier investigations 8,14 , the term utility was de fi ned in parallel with resemblance (i.e., the statisticalsimilarity of two datasets) and was speci fi cally used to refer to the value of real or synthetic data to support predictions.”
Generative Adversarial Networks and Its Applications in Biomedical Informatics
Lan, L., You, L., Zhang, Z., Fan, Z., Zhao, W., Zeng, N., Chen, Y., & Zhou, X. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00164 · Cited by 237 · cited 2×
“This is problematic because GAN models are often associated with unstable training trajectories, which can lead to quite different models and inconsistencies in the quality of the generated data 15,22,23 .”
Self-citationRankin, D., Black, M., Bond, R., Wallace, J., Mulvenna, M., & Epelde, G. (2020) · JMIR Medical Informatics · doi:10.2196/18910 · Cited by 169
Data augmentation for enhancing EEG-based emotion recognition with deep generative models.
Luo, Y., Zhu, L.-Z., Wan, Z.-Y., & Lu, B.-L. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abb580 · Cited by 162
Self-citationA Survey of Generative Adversarial Networks for Synthesizing Structured Electronic Health Records
Ghosheh, G. O., Li, J., & Zhu, T. (2024) · ACM Computing Surveys · doi:10.1145/3636424 · Cited by 95
Wang, Z., Myles, P., & Tucker, A. (2021) · Computational Intelligence · doi:10.1111/coin.12427 · Cited by 65
Cited by
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Ning, Y., Teixayavong, S., Shang, Y., Savulescu, J., Nagaraj, V., Miao, D., Mertens, M., Ting, D. S. W., Ong, J. C. L., Liu, M., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00143-2 · Cited by 212
Metadata
- Authors
- Yan, C., Yan, Y., Wan, Z., Zhang, Z., Omberg, L., Guinney, J., Mooney, S. D., & Malin, B. A.
- Venue
- Nature Communications
- Published
- 2022-12-09
- Volume
- 13
- Issue
- 1
- Pages
- 7609-7609
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W4310917402
- Cited by
- 95
- References
- 11
- FWCI
- 5.98
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Multifaceted benchmarking of synthetic electronic health record generation models. https://app.cochat.ai/papers/10.1038/s41467-022-35295-1
- The paper
- Yan, C., Yan, Y., Wan, Z., Zhang, Z., Omberg, L., Guinney, J., Mooney, S. D., & Malin, B. A. (2022). A Multifaceted benchmarking of synthetic electronic health record generation models. Nature Communications. https://doi.org/10.1038/s41467-022-35295-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.