Cited by 30Domain Adaptation and Few-Shot LearningMachine Learning in HealthcareArtificial Intelligence in Healthcare and Education

Pre-training in Medical Data: A Survey

Open access
Qiu, Yixuan, Lin, Feng, Chen, Weitong, Xu, Miao·2023·Machine Intelligence Research

Abstract

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods′ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11633-022-1382-8

Cites

232 of 235 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Pretrained models followed by fine-tuning on target clinical data enable efficient adaptation to specialized clinical tasks even with limited data from the specific patient cohort.

    “This approach enables efficient adaptation to specialized clinical tasks, even with limited data from the specific patient cohort [77].”

    From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [77]

    The paper’s opening passage (section and page not provided) says pre-training improves performance in data-limited scenarios and is used across medical settings. It does not establish the more specific claim that fine-tuning enables efficient adaptation with limited data from a particular patient cohort. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. [28][28] V. Jain, E. Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of IEEE Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 577–584, 2011. DOI: 10.1109/CV-PR.2011.5995317. · DOI
  2. [47][47] P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, C. W. Fu, X. Han, P. A. Heng, J. Hesser, S. Kadoury, T. K. Konopczynski, M. Le, C. M. Li, X. M. Li, J. Lipková, J. S. Lowengrub, H. Meine, J. H. Moltz, C. Pal, M. Piraud, X. J. Qi, J. Qi, M. Rempfler, K. Roth, A. Schenk, A. Sekuboyina, P. Zhou, C. Hülsemeyer, M. Beetz, F. Ettlinger, F. Grün, G. Kaissis, F. Lohöfer, R. Braren, J. Holch, F. Hofmann, W. H. Sommer, V. Heinemann, C. Jacobs, G. E. H. Mamani, B. van Ginneken, G. Chartrand, A. Tang, M. Drozdzal, A. Ben-Cohen, E. Klang, M. M. Amitai, E. Konen, H. Greenspan, J. Moreau, A. Hostettler, L. Soler, R. Vivanti, A. Szeskin, N. Lev-Cohain, J. Sosna, L. Joskowicz, B. H. Menze. The liver tumor segmentation benchmark (LiTS). [Online], Available: https://arxiv.org/abs/1901.04056, 2019. · arXiv
  3. [98][98] K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Vegas, USA, pp. 770–778, 2016. DOI: 10.1109/CV-PR.2016.90. · DOI
  4. [108][108] Health Information Science and Systems 2020 8 1 10.1007/s13755-019-0089-x PMC6898694 31867102 · DOI
  5. [121][121] He K M Chen X L Xie S N Li Y H Dollár P Girshick R Masked autoencoders are scalable vision learners 2022 New Orleans, USA IEEE 15979 15988
  6. [129][129] Journal of Medical Systems 2018 42 11
  7. [204][204] Journal of Neural Engineering 2021 18 4
  8. [209][209] N. Wagh, J. H. Wei, S. Rawal, B. M. Berry, L. Barnard, B. Brinkmann, G. Worrell, D. Jones, Y. Varatharajah. Domain-guided self-supervision of EEG data improves downstream classification performance and generalizability. In Proceedings of Machine Learning for Health, pp. 130–142, 2021.
  9. [218][218] ACM Transactions on Knowledge Discovery from Data 2022 1 1
  10. [222][222] K. Hur, J. Lee, J. Oh, W. Price, Y. Kim, E. Choi. Unifying heterogeneous electronic health records systems via text-based code embedding. In Proceedings of Conference on Health, Inference, and Learning, pp.183–203, 2022.
  11. [224][224] Journal of Medical Internet Research 2021 23 1
  12. [236][236] L. H. Li, M. Yatskar, D. Yin, C. J. Hsieh, K. W. Chang. VisualBERT: A simple and performant baseline or vision and language. [Online], Available: https://arxiv.org/abs/1908.03557, 2019. · arXiv

Cited by

Sorted by citations

Authors
Qiu, Y., Lin, F., Chen, W., & Xu, M.
Affiliations
  • The University of Queensland (AU)
  • The University of Adelaide (AU)
Corresponding author
Chen, Weitong
Venue
Machine Intelligence Research
Published
2023-02-21
Volume
20
Issue
2
Pages
147-179
ISSN
  • 1476-8186 (ISSN-L)
  • 2731-538X
  • 2731-5398
OpenAlex
W4321437302
Cited by
30
References
235
Citations by year
2026: 6 · 2025: 10 · 2024: 8 · 2023: 6
FWCI
2.83
Type
journal article
Language
en
Open access
Locations
  • Machine Intelligence Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
Funders
University of Adelaide
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
pre-training, health data, deep neural network, electronic health records, multimodal medical data, benchmarking, transfer learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review