The DOI matches its Crossref record; 232 of the 235 references it cites were found in a registry or an index, and 3 are not checked yet.
Pre-training in Medical Data: A Survey
Open accessAbstract
Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods′ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This approach enables efficient adaptation to specialized clinical tasks, even with limited data from the specific patient cohort [77].”
From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [77]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–235 of 235
Self-Supervised Adversarial Distribution Regularization for Medication Recommendation
Wang, Y., Chen, W., Pi, D., Yue, L., Wang, S., & Xu, M. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/431 · Cited by 44
Self-citationRAPT: Pre-training of Time-Aware Transformer for Learning Robust Healthcare Representation
Ren, H., Wang, J., Zhao, W. X., & Wu, N. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467069 · Cited by 41
Chen, Y.-P., Lo, Y., Lai, F., & Huang, C. (2021) · Journal of Medical Internet Research · doi:10.2196/25113 · Cited by 40
Self-supervised ECG pre-training
Liu, H., Zhao, Z., & She, Q. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103010 · Cited by 39
Prakash, R. M., & Kumari, R. S. S. (2019) · International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) · doi:10.1109/wispnet45539.2019.9032811 · Cited by 39
Double-Shot Transfer Learning for Breast Cancer Classification from X-Ray Images
Alkhaleefah, M., Ma, S., Chang, Y.-L., Huang, B., Chittem, P. K., & Achhannagari, V. P. (2020) · Applied Sciences · doi:10.3390/app10113999 · Cited by 37
CLECG: A Novel Contrastive Learning Framework for Electrocardiogram Arrhythmia Classification
Chen, H., Wang, G., Zhang, G., Zhang, P., & Yang, H. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3114119 · Cited by 35
Adversarially regularized medication recommendation model with multi-hop memory network
Wang, Y., Chen, W., Pi, D., & Yue, L. (2020) · Knowledge and Information Systems · doi:10.1007/s10115-020-01513-9 · Cited by 33
Self-citationAnomaly Detection on Electroencephalography with Self-supervised Learning
Xu, J., Zheng, Y., Mao, Y., Wang, R., & Zheng, W.-S. (2020) · IEEE International Conference on Bioinformatics and Biomedicine · doi:10.1109/bibm49941.2020.9313163 · Cited by 31
Classification of Atrial Fibrillation with Pre-Trained Convolutional Neural Network Models
Qayyum, A., Mériaudeau, F., & Chan, G. C. Y. (2018) · IEEE-EMBS Conference on Biomedical Engineering and Sciences · doi:10.1109/iecbes.2018.8626624 · Cited by 30
Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations
Konwer, A., Xu, X., Bae, J., Chen, C., & Prasanna, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01826 · Cited by 30
Self-supervised Image-text Pre-training With Mixed Data In Chest X-raysPreprint
Wang, X., Xu, Z., Tam, L., Yang, D., & Xu, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16022 · Cited by 27
Distilling Knowledge from Publicly Available Online EMR Data to Emerging Epidemic for Prognosis
Ma, L., Ma, X., Gao, J., Jiao, X., Yu, Z., Zhang, C., Ruan, W., Wang, Y., Tang, W., & Wang, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449855 · Cited by 27
Self-supervised Longitudinal Neighbourhood Embedding
Ouyang, J., Zhao, Q., Adeli, E., Sullivan, E. V., Pfefferbaum, A., Zaharchuk, G., & Pohl, K. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87196-3_8 · Cited by 26
Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis
Ren, M., Dey, N., Styner, M., Botteron, K., & Gerig, G. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2206.04281 · Cited by 26
An improved self-supervised learning for EEG classification
Ou, Y., Sun, S., Gan, H., Zhou, R., & Yang, Z. Q. (2022) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2022325 · Cited by 26
A comprehensive EHR timeseries pre-training benchmark
McDermott, M., Nestor, B., Kim, E., Zhang, W., Goldenberg, A., Szolovits, P., & Ghassemi, M. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451877 · Cited by 24
Transfer Learning for Subject-Independent Stress Detection using Physiological Signals
Radhika, K. R., & Murthy, O. V. R. (2020) · IEEE India Conference · doi:10.1109/indicon49873.2020.9342505 · Cited by 22
Xiao, X., Xu, X., Sun, Y., Liu, X., Li, X., Xie, G., & Wang, F. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00089 · Cited by 21
Doctor2Vec: Dynamic Doctor Representation Learning for Clinical Trial Recruitment
Biswal, S., Xiao, C., Glass, L. M., Milkovits, E., & Sun, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5394 · Cited by 19
Encoding-decoding Network With Pyramid Self-attention Module For Retinal Vessel Segmentation
Wu, C., Sun, J., Wang, J., Xu, L., & Zhan, S. (2021) · Machine Intelligence Research · doi:10.1007/s11633-020-1277-0 · Cited by 18
TAHDNet: Time-aware hierarchical dependency network for medication recommendation
Su, Y., Shi, Y., Lee, W., Cheng, L., & Guo, H. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104069 · Cited by 18
First Steps Towards Self-Supervised Pretraining of the 12-Lead ECG
Gedon, D., Ribeiro, A. H., Wahlström, N., & Schön, T. B. (2021) · Computing in cardiology · doi:10.23919/cinc53138.2021.9662748 · Cited by 16
Deep Autoencoder Networks Optimized with Genetic Algorithms for Efficient ECG Clustering
Thinsungnoen, T., Kerdprasop, K., & Kerdprasop, N. (2018) · International Journal of Machine Learning and Computing · doi:10.18178/ijmlc.2018.8.2.672 · Cited by 14
STCT: Spatial-Temporal Conv-Transformer Network for Cardiac Arrhythmias Recognition
Qiu, Y., Chen, W., Yue, L., Xu, M., & Zhu, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-95405-5_7 · Cited by 14
Self-citationDeep Learning Applied to Attractor Images Derived from ECG Signals for Detection of Genetic Mutation
Aston, P. J., Lyle, J. V., Bonet-Luz, E., Huang, C. L.-H., Zhang, Y., Jeevaratnam, K., & Nandi, M. (2019) · Computing in cardiology · doi:10.22489/cinc.2019.097 · Cited by 14
Pham, T. D. (2021) · Scientific Reports · doi:10.1038/s41598-021-93160-5 · Cited by 12
Almalchy, M. T., Al-Gayar, S. M. S., & Popescu, N. (2020) · International Conference on Communications (COMM) · doi:10.1109/comm48946.2020.9141994 · Cited by 12
Sun, Z., Peng, S., Yang, Y., Wang, X., & Li, F. (2019) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983098 · Cited by 12
Semi-Supervised Learning for ECG Classification
Rodrigues, R. A., & Couto, P. (2021) · Computing in cardiology · doi:10.23919/cinc53138.2021.9662693 · Cited by 12
A Generative Adversarial Approach To ECG Synthesis And DenoisingPreprint
Antczak, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.02700 · Cited by 11
Hur, K., Lee, J., Oh, J., Price, W., Kim, Y., & Choi, E. (2021) · doi:10.2196/preprints.32523 · Cited by 7
Representation Learning for Electronic Health Records: A Survey
Chen, P. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1487/1/012015 · Cited by 6
Learning to Select Best Forecast Tasks for Clinical Outcome Prediction
Xue, Y., Du, N., Mottram, A., Seneviratne, M. G., & Dai, A. M. (2020) · neural information processing systems · Cited by 5
Semi-self-supervised Automated ICD CodingPreprint
Hlynsson, H. D., Ellertsson, S., Daðason, J. F., Sigurdsson, E. L., & Loftsson, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.10088 · Cited by 2
Flagged first, then by citations
Research works printed but not linked: 10
- B. Ramsundar, P. Eastman, P. Walters, V. Pande. Deep Learning for the Life Sciences: Applying Deep Learning to Genomics, Microscopy, Drug Discovery, and More, Sebastopol, USA: O’Reilly Media, 2019.
- V. Jain, E. Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of IEEE Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 577–584, 2011. DOI: https://doi.org/10.1109/CV-PR.2011.5995317. · DOI
- P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, C. W. Fu, X. Han, P. A. Heng, J. Hesser, S. Kadoury, T. K. Konopczynski, M. Le, C. M. Li, X. M. Li, J. Lipková, J. S. Lowengrub, H. Meine, J. H. Moltz, C. Pal, M. Piraud, X. J. Qi, J. Qi, M. Rempfler, K. Roth, A. Schenk, A. Sekuboyina, P. Zhou, C. Hülsemeyer, M. Beetz, F. Ettlinger, F. Grün, G. Kaissis, F. Lohöfer, R. Braren, J. Holch, F. Hofmann, W. H. Sommer, V. Heinemann, C. Jacobs, G. E. H. Mamani, B. van Ginneken, G. Chartrand, A. Tang, M. Drozdzal, A. Ben-Cohen, E. Klang, M. M. Amitai, E. Konen, H. Greenspan, J. Moreau, A. Hostettler, L. Soler, R. Vivanti, A. Szeskin, N. Lev-Cohain, J. Sosna, L. Joskowicz, B. H. Menze. The liver tumor segmentation benchmark (LiTS). [Online], Available: https://arxiv.org/abs/1901.04056, 2019. · arXiv
- G. B. Moody, W. K. Muldrow, R. G. Mark. A noise stress test for arrhythmia detectors. Computers in Cardiology, vol. 11, no. 3, pp. 381–384, 1984.
- G. B. Moody, R. G. Mark. Development and evaluation of a 2-lead ECG analysis program. Computers in Cardiology, vol. 1982, pp. 39–44, 1982.
- K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CV-PR.2016.90. · DOI
- K. M. He, X. L. Chen, S. N. Xie, Y. H. Li, P. Dollár, R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 15979–15988, 2022. DOI: https://doi.org/10.1109/CV-PR52688.2022.01553. · DOI
- A. Fernandez-Quilez. Deep Learning for an Improved Diagnostic Pathway of Prostate Cancer in a Small Multi-Parametric Magnetic Resonance Data Regime, Ph.D. dissertation, University of Stavanger, Stavanger, Norway, 2022.
- N. Wagh, J. H. Wei, S. Rawal, B. M. Berry, L. Barnard, B. Brinkmann, G. Worrell, D. Jones, Y. Varatharajah. Domain-guided self-supervision of EEG data improves downstream classification performance and generalizability. In Proceedings of Machine Learning for Health, pp. 130–142, 2021.
- L. H. Li, M. Yatskar, D. Yin, C. J. Hsieh, K. W. Chang. VisualBERT: A simple and performant baseline or vision and language. [Online], Available: https://arxiv.org/abs/1908.03557, 2019. · arXiv
Cited by
Sorted by citations
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 158
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 162
A systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95
Revisiting Computer-Aided Tuberculosis Diagnosis
Liu, Y., Wu, Y.-H., Zhang, S., Liu, L., Wu, M., & Cheng, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330825 · Cited by 51
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58
Self-citation
Metadata
- Authors
- Qiu, Y., Lin, F., Chen, W., & Xu, M.
- Affiliations
- The University of Queensland (AU)
- The University of Adelaide (AU)
- Corresponding author
- Chen, Weitong
- Venue
- Machine Intelligence Research
- Published
- 2023-02-21
- Volume
- 20
- Issue
- 2
- Pages
- 147-179
- ISSN
- 1476-8186 (ISSN-L)
- 2731-538X
- 2731-5398
- PMC
- PMC9942039
- OpenAlex
- W4321437302
- Cited by
- 30
- References
- 235
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 8 · 2023: 6
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Machine Intelligence Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- Funders
- University of Adelaide
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- pre-training, health data, deep neural network, electronic health records, multimodal medical data, benchmarking, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
