The DOI matches its Crossref record; 232 of the 235 references it cites were found in a registry or an index, and 3 are not checked yet.
Pre-training in Medical Data: A Survey
Open accessAbstract
Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods′ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This approach enables efficient adaptation to specialized clinical tasks, even with limited data from the specific patient cohort [77].”
From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [77]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Research works printed but not linked: 10
- B. Ramsundar, P. Eastman, P. Walters, V. Pande. Deep Learning for the Life Sciences: Applying Deep Learning to Genomics, Microscopy, Drug Discovery, and More, Sebastopol, USA: O’Reilly Media, 2019.
- V. Jain, E. Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of IEEE Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 577–584, 2011. DOI: https://doi.org/10.1109/CV-PR.2011.5995317. · DOI
- P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, C. W. Fu, X. Han, P. A. Heng, J. Hesser, S. Kadoury, T. K. Konopczynski, M. Le, C. M. Li, X. M. Li, J. Lipková, J. S. Lowengrub, H. Meine, J. H. Moltz, C. Pal, M. Piraud, X. J. Qi, J. Qi, M. Rempfler, K. Roth, A. Schenk, A. Sekuboyina, P. Zhou, C. Hülsemeyer, M. Beetz, F. Ettlinger, F. Grün, G. Kaissis, F. Lohöfer, R. Braren, J. Holch, F. Hofmann, W. H. Sommer, V. Heinemann, C. Jacobs, G. E. H. Mamani, B. van Ginneken, G. Chartrand, A. Tang, M. Drozdzal, A. Ben-Cohen, E. Klang, M. M. Amitai, E. Konen, H. Greenspan, J. Moreau, A. Hostettler, L. Soler, R. Vivanti, A. Szeskin, N. Lev-Cohain, J. Sosna, L. Joskowicz, B. H. Menze. The liver tumor segmentation benchmark (LiTS). [Online], Available: https://arxiv.org/abs/1901.04056, 2019. · arXiv
- G. B. Moody, W. K. Muldrow, R. G. Mark. A noise stress test for arrhythmia detectors. Computers in Cardiology, vol. 11, no. 3, pp. 381–384, 1984.
- G. B. Moody, R. G. Mark. Development and evaluation of a 2-lead ECG analysis program. Computers in Cardiology, vol. 1982, pp. 39–44, 1982.
- K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CV-PR.2016.90. · DOI
- K. M. He, X. L. Chen, S. N. Xie, Y. H. Li, P. Dollár, R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 15979–15988, 2022. DOI: https://doi.org/10.1109/CV-PR52688.2022.01553. · DOI
- A. Fernandez-Quilez. Deep Learning for an Improved Diagnostic Pathway of Prostate Cancer in a Small Multi-Parametric Magnetic Resonance Data Regime, Ph.D. dissertation, University of Stavanger, Stavanger, Norway, 2022.
- N. Wagh, J. H. Wei, S. Rawal, B. M. Berry, L. Barnard, B. Brinkmann, G. Worrell, D. Jones, Y. Varatharajah. Domain-guided self-supervision of EEG data improves downstream classification performance and generalizability. In Proceedings of Machine Learning for Health, pp. 130–142, 2021.
- L. H. Li, M. Yatskar, D. Yin, C. J. Hsieh, K. W. Chang. VisualBERT: A simple and performant baseline or vision and language. [Online], Available: https://arxiv.org/abs/1908.03557, 2019. · arXiv
Cited by
Sorted by citations
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 158
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 162
A systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95
Revisiting Computer-Aided Tuberculosis Diagnosis
Liu, Y., Wu, Y.-H., Zhang, S., Liu, L., Wu, M., & Cheng, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330825 · Cited by 51
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58
Self-citationLessons learned from RadiologyNET foundation models for transfer learning in medical radiology
Napravnik, M., Hržić, F., Urschler, M., Miletić, D., & Štajduhar, I. (2025) · Scientific Reports · doi:10.1038/s41598-025-05009-w · Cited by 15
Zakaria, T. M., Langi, A. Z. R., Nazaruddin, M. S., & Anshori, I. (2025) · IEEE Access · doi:10.1109/access.2025.3582805 · Cited by 17
Generative artificial intelligence in diabetes healthcare
Vehi, J., Mujahid, O., Beneyto, A., & Contreras, I. (2025) · iScience · doi:10.1016/j.isci.2025.113051 · Cited by 12
A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical ApplicationsPreprint
Guarrasi, V., Aksu, F., Caruso, C. M., Feola, F. D., Rofena, A., Ruffini, F., & Soda, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4952813 · Cited by 13
Hierarchical Pretraining on Multimodal Electronic Health Records
Wang, X., Luo, J., Wang, J., Yin, Z., Cui, S., Zhong, Y., Wang, Y., & Ma, F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.171 · Cited by 17
Complementary to Multiple Labels: A Correlation-Aware Correction Approach
Gao, Y. D., Xu, M., & Zhang, M.-L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3416384 · Cited by 8
Self-citationIrregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamics
Zheng, L. N., Li, Z., Dong, C. G., Zhang, W. E., Yue, L., Xu, M., Maennel, O., & Chen, W. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679716 · Cited by 8
Self-citationSWAP: Exploiting Second-Ranked Logits for Adversarial Attacks on Time Series
Dong, C. G., Zheng, L. N., Chen, W., Zhang, W. E., & Yue, L. (2023) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg59574.2023.00020 · Cited by 8
Self-citationZheng, L. N., Dong, C. G., Zhang, W. E., Chen, X., Yue, L., & Chen, W. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679719 · Cited by 6
Self-citationFeature extraction and deep learning
Mahyar, A., Motamednia, H., Cheraaqee, P., & Mansouri, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-332818-3.00019-8 · Cited by 7
Buser, M. A. D., Simons, D. C., Fitski, M., Wijnen, M. H. W. A., Littooij, A. S., ter Brugge, A. H., Vos, I. N., Janse, M. H. A., de Boer, M., ter Maat, L. S., et al. (2025) · Bioengineering · doi:10.3390/bioengineering12111157 · Cited by 3
Foundation models for brain imaging: A systematic review
Ghamizi, S., Kanli, G., Deng, Y., Palissot, V., Perquin, M., & Keunen, O. (2026) · NeuroImage · doi:10.1016/j.neuroimage.2026.121877 · Cited by 5
Neural Reconstruction through Scattering Media with Forward and Backward Losses
Wang, Y., Shen, S., Xia, S., Li, R., Peng, X., Yu, Y., Li, S., & Jingyi Yu, A. (2023) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccp56744.2023.10233796 · Cited by 3
Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review
Yang, M., Shi, N., Chen, H., Po, E. Y. P., Ng, P. Y., Lai, P. C. K., Wu, Y.-C., & Fong, D. Y. T. (2026) · Health Data Science · doi:10.34133/hds.0356 · Cited by 3
“This approach enables efficient adaptation to specialized clinical tasks, even with limited data from the specific patient cohort [ 77 ].”
Data-Efficient Sleep Staging with Synthetic Time Series Pretraining
Grieger, N., Mehrkanoon, S., & Bialonski, S. (2025) · Algorithms · doi:10.3390/a18090580 · Cited by 2
Cardiac Dynamic Characteristics Classification on Cine MRI Using Semi-supervised Imaging Approach
Ahmad, F., Xiong, J., Wu, J., Xia, J., & Xia, Z. (2025) · Machine Intelligence Research · doi:10.1007/s11633-024-1534-0 · Cited by 2
Answer Semantics-enhanced Medical Visual Question Answering
Liang, Y., Yang, E., Guo, G., Cai, W., Jiang, L., Zhao, J., & Wang, X. (2025) · Machine Intelligence Research · doi:10.1007/s11633-025-1564-2 · Cited by 2
BucketAugment: Reinforced Domain Generalisation in Abdominal CT Segmentation
Hresko, D. J., & Drotár, P. (2024) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2024.3397623 · Cited by 2
A Progressive Sampling Method for Dual-Node Imbalanced Learning with Restricted Data Access
Qiu, Y., Chen, W., & Xu, M. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00060 · Cited by 2
Self-citationTransforming Raw CKD Data into a Fixed Dataset: Evaluating Preprocessing Methods
Subashini., N. J., Pushparaj., E., & Venkatesh, K. (2025) · International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE) · doi:10.1109/rmkmate64874.2025.11042790 · Cited by 1
Monteleone, M., & Postiglione, A. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia241455
AI strategies for drug discovery and bioactivity prediction: opportunities and challenges
Azad, A. K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-34059-8.00013-0
Medical data foundations: Key concepts and definitions for clinicians and researchers
Boubaker, O. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-27638-5.00003-1
Wang, L., Zhou, Y., Zhou, S., & Qiao, L. (2026) · World Conference on Computing and Communication Technologies (WCCCT) · doi:10.1109/wccct69960.2026.11549832
Cross-variability decoding for motor imagery EEG signals: a comprehensive review
Wang, L., Zhou, Y., Wang, P., Ma, X., & Zhou, S. (2026) · Frontiers in Neuroscience · doi:10.3389/fnins.2026.1899645
Metadata
- Authors
- Qiu, Y., Lin, F., Chen, W., & Xu, M.
- Affiliations
- The University of Queensland (AU)
- The University of Adelaide (AU)
- Corresponding author
- Chen, Weitong
- Venue
- Machine Intelligence Research
- Published
- 2023-02-21
- Volume
- 20
- Issue
- 2
- Pages
- 147-179
- ISSN
- 1476-8186 (ISSN-L)
- 2731-538X
- 2731-5398
- PMC
- PMC9942039
- OpenAlex
- W4321437302
- Cited by
- 30
- References
- 235
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 8 · 2023: 6
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Machine Intelligence Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- Funders
- University of Adelaide
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- pre-training, health data, deep neural network, electronic health records, multimodal medical data, benchmarking, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
