Breaking away from labels: The promise of self-supervised machine learning in intelligent health
Open accessAbstract
Medicine is undergoing an unprecedented digital transformation, as massive amounts of health data are being produced, gathered, and curated, ranging from in-hospital (e.g., intensive care unit [ICU]) to person-generated data (wearables). Annotating all these data for training purposes in order to feed to deep learning models for pattern recognition is impractical. Here, we discuss some exciting recent results of self-supervised learning (SSL) applications to high-resolution health signals. These examples leverage unlabeled data to learn meaningful representations that can generalize to situations where the ground truth is inadequate or simply infeasible to collect due to the high burden or associated costs. The most prominent bottleneck of deep learning today is access to labeled, carefully curated datasets, and self-supervision on health signals opens up new possibilities to eliminate data silos through general-purpose models that can transfer to low-resource environments and tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the context of health signals, self-supervised tasks can range from predicting the arrow of time, to contrastive learning and masking.
“In the context of health signals, these tasks can range from predicting the arrow of time, to contrastive learning and masking [145,146].”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [145]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · see citing works
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · see citing works
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · see citing works
Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890 · see citing works
Self-supervised Learning: Generative or Contrastive
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3090866 · Cited by 1,890 · see citing works
Large-scale physical activity data reveal worldwide activity inequality
Althoff, T., Sosič, R., Hicks, J. L., King, A. C., Delp, S. L., & Leskovec, J. (2017) · Nature · doi:10.1038/nature23018 · Cited by 1,000 · see citing works
Mandsager, K., Harb, S., Cremer, P., Phelan, D., Nissen, S. E., & Jaber, W. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.3605 · Cited by 501 · cited 1× · see citing works
“Additionally, low fitness is a stronger predictor of mortality than diabetes, hypertension, and smoking.(19) Overall, this is one of the first multi-modal self-supervisedmodels for behavioral and physiological data, with implications for large-scale health and lifestyle monitoring (open source code and pre-trained mod…”
BackgroundAn Efficient Learning Procedure for Deep Boltzmann Machines
Salakhutdinov, R., & Hinton, G. E. (2012) · Neural Computation · doi:10.1162/neco_a_00311 · Cited by 477 · see citing works
Learning and Using the Arrow of Time
Wei, D., Lim, J. J., Zisserman, A., & Freeman, W. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00840 · Cited by 318 · see citing works
Best practices for analyzing large-scale health data from wearables and smartphone apps
Hicks, J. L., Althoff, T., Sosic, R., Kuhar, P., Bostjancic, B., King, A. C., Leskovec, J., & Delp, S. L. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0121-1 · Cited by 183 · see citing works
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Newell, A., & Deng, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00737 · Cited by 122 · see citing works
Chowdhury, A., Rosenthal, J., Waring, J., & Umeton, R. (2021) · Informatics · doi:10.3390/informatics8030059 · Cited by 85 · see citing works
Subject-Aware Contrastive Learning for Biosignals
Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020) · arXiv · doi:10.48550/arxiv.2007.04871 · Cited by 74 · see citing works
Chen, H., Lundberg, S., Erion, G., Kim, J. H., & Lee, S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00536-y · Cited by 48 · see citing works
Self-supervised transfer learning of physiological representations from free-living wearable data
Spathis, D., Perez-Pozuelo, I., Brage, S., Wareham, N. J., & Mascolo, C. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451863 · Cited by 40 · see citing works
Self-citationNeighborhood Contrastive Learning Applied to Online Patient MonitoringPreprint
Yèche, H., Dresdner, G., Locatello, F., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05142 · Cited by 17 · see citing works
Adversarial Unsupervised Representation Learning for Activity Time-Series
Aggarwal, K., Joty, S., Fernandez-Luque, L., & Srivastava, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301834
Flagged first, then by citations
Research works printed but not linked: 5
- [5]Brown T.B. Mann B. Ryder N. Subbiah M. Kaplan J. Dhariwal P. Language Models Are Few-Shot Learners 2020 NeurIPS
- [13]Tang C.I. Perez-Pozuelo I. Spathis D. Mascolo C. Exploring Contrastive Learning in Human Activity Recognition for Healthcare 2020 NeurIPS Machine Learning for Mobile Health workshop
- [14]Kiyasseh D. Zhu T. Clifton D.A. Clocs: Contrastive Learning of Cardiac Signals across Space, Time, and Patients 2021 ICML
- [20]Tang C.I. Perez-Pozuelo I. Spathis D. Brage S. Wareham N. Mascolo C. SelfHAR: Improving Human Activity Recognition through Self-Training with Unlabeled Data 2021 ACM IMWUT
- [22]Tonekaboni S. Eytan D. Goldenberg A. Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding 2021 ICLR
Cited by
Sorted by citations
Self-supervised learning: A succinct review
Rani, V., Nabi, S. T., Kumar, M., Mittal, A., & Kumar, K. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09884-2 · Cited by 295 · see citing works
Self-Supervised Contrastive Learning for Medical Time Series: A Systematic Review
Liu, Z., Alavi, A., Li, M., & Zhang, X. (2023) · Sensors · doi:10.3390/s23094221 · Cited by 90 · see citing works
Human-centred artificial intelligence for mobile health sensing: challenges and opportunities
Dang, T., Spathis, D., Ghosh, A., & Mascolo, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230806 · Cited by 30 · see citing works
Self-citation
- Authors
- Spathis, D., Perez-Pozuelo, I., Marques-Fernandez, L., & Mascolo, C.
- Affiliations
- University of Cambridge (GB)
- Cambridge University Hospitals NHS Foundation Trust (GB)
- Corresponding author
- Spathis, Dimitris
- Venue
- Patterns
- Publisher
- Elsevier BV
- Published
- 2022-02
- Volume
- 3
- Issue
- 2
- Pages
- 100410
- ISSN
- 2666-3899 (ISSN-L)
- PMC
- PMC8848012
- OpenAlex
- W4211062518
- Cited by
- 39
- References
- 19
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 10 · 2023: 10 · 2022: 3
- FWCI
- 2.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Patterns (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- EPSRC (EP/N509620/1)
- GlaxoSmithKline (17100053)
- European Research Council
- University of Cambridge Jesus College
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
- Keywords
- self-supervised learning, unlabeled data, representation learning, deep learning, wearable device data, intensive care unit data, resource-limited settings, transfer learning, digital health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
