CoChat
Sign Up
PreprintCited by 5Time Series Analysis and ForecastingMachine Learning in HealthcareContext-Aware Activity Recognition Systems

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

Open access
Merrill, Mike A., Althoff, Tim·2022·arXiv
DOI 10.48550/arxiv.2205.13607arXiv 2205.13607
Open this paper in CoChat — free accountOpen PDF

Abstract

Detailed mobile sensing data from phones, watches, and fitness trackers offer an unparalleled opportunity to quantify and act upon previously unmeasurable behavioral changes in order to improve individual health and accelerate responses to emerging diseases. Unlike in natural language processing and computer vision, deep representation learning has yet to broadly impact this domain, in which the vast majority of research and clinical applications still rely on manually defined features and boosted tree models or even forgo predictive modeling altogether due to insufficient accuracy. This is due to unique challenges in the behavioral health domain, including very small datasets (~10^1 participants), which frequently contain missing data, consist of long time series with critical long-range dependencies (length>10^4), and extreme class imbalances (>10^3:1). Here, we introduce a neural architecture for multivariate time series classification designed to address these unique domain challenges. Our proposed behavioral representation learning approach combines novel tasks for self-supervised pretraining and transfer learning to address data scarcity, and captures long-range dependencies across long-history time series through transformer self-attention following convolutional neural network-based dimensionality reduction. We propose an evaluation framework aimed at reflecting expected real-world performance in plausible deployment scenarios. Concretely, we demonstrate (1) performance improvements over baselines of up to 0.15 ROC AUC across five prediction tasks, (2) transfer learning-induced performance improvements of 16% PR AUC in small data scenarios, and (3) the potential of transfer learning in novel disease scenarios through an exploratory case study of zero-shot COVID-19 prediction in an independent data set. Finally, we discuss potential implications for medical surveillance testing.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2205.13607

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Foundation models learn general-purpose representations from large-scale data that can be adapted to diverse downstream tasks.

    “Foundation Models learn general-purpose representations from large-scale data that can be adapted to diverse downstream tasks [6,8,13,19,25,28,30,35,45,60,62,69].”

    From Towards Customizable Foundation Models for Human Activity Recognition with Wearable Devices (2025) · cites this paper as [28]

    The Methods (Section 3.2, p. 3) say self-supervised pretraining learns generalizable representations from unlabeled data, and the Introduction (p. 2) reports transfer across flu and COVID-19 prediction. The paper does not characterize its model as a foundation model or establish the broader claim that foundation models adapt to diverse downstream tasks. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 33

  1. [4][4] Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The LongDocument Transformer. arXiv:2004.05150 [cs] (2020). - · arXiv
  2. [5][5] Pavel B. Brazdil and Carlos Soares. 2000. A Comparison of Ranking Methods for Classification Algorithm Selection. In Machine Learning: ECML 2000 , Jaime G. Carbonell, Jörg Siekmann, G. Goos, J. Hartmanis, J. van Leeuwen, et al. (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. -
  3. [7][7] Teodora Sandra Buda, Mohammed Khwaja, and Aleksandar Matic. 2021. Outliers in Smartphone Sensor Data Reveal Outliers in Daily Happiness. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (2021). -
  4. [8][8] Bokai Cao, Lei Zheng, Chenwei Zhang, Philip S Yu, Andrea Piscitello, et al. 2017. Deepmood: modeling mobile phone typing dynamics for mood detection. In KDD . -
  5. [9][9] Hugh Chen, Scott Lundberg, Gabe Erion, Jerry H Kim, and Su-In Lee. 2020. Forecasting adverse surgical events using self-supervised transfer learning for physiological signals. arXiv:2002.04770 (2020). - · arXiv
  6. [11][11] Helen Y. Chu, Janet A. Englund, Lea M. Starita, Michael Famulare, Elisabeth Brandstetter, et al. 2020. Early Detection of Covid-19 through a Citywide Pandemic Surveillance Platform. New England Journal of Medicine (2020). -
  7. [12][12] Elizabeth R. DeLong, David M. DeLong, and Daniel L. Clarke-Pearson. 1988. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics (1988). -
  8. [13][13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1) . -
  9. [14][14] Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. 2019. Deep learning for time series classification: a review. 33, 4 (July 2019), 917-963. arXiv: 1809.04356. - · arXiv
  10. [16][16] F. M. Fusco, M. Pisaturo, V. Iodice, R. Bellopede, O. Tambaro, et al. 2020. COVID19 among healthcare workers in a specialist infectious diseases setting in Naples, - Southern Italy: results of a cross-sectional surveillance study. Journal of Hospital Infection (2020). -
  11. [18][18] Haibo He and E.A. Garcia. 2009. Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering (2009), 1263-1284. -
  12. [19][19] Haraldur T Hallgrímsson, Filip Jankovic, Tim Althoff, and Luca Foschini. 2018. Learning individualized cardiovascular responses from large-scale wearable sensors data. NeurIPS ML4H (2018). -
  13. [20][20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770-778. -
  14. [21][21] Yosuke Hirotsu, Makoto Maejima, Masahiro Shibusawa, Yuki Nagakubo, Kazuhiro Hosaka, et al. 2020. Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients. International Journal of Infectious Diseases 99 (Oct. 2020), 397-402. -
  15. [25][25] Natasha Jaques, Ognjen (Oggi) Rudovic, Sara Taylor, Akane Sano, and Rosalind Picard. 2017. Predicting Tomorrow's Mood, Health, and Stress Level using Personalized Multitask Learning and Domain Adaptation. In Proceedings of IJCAI 2017 Workshop on Artificial Intelligence in Affective Computing . -
  16. [28][28] Arinbjörn Kolbeinsson, Piyusha Gade, Raghu Kainkaryam, Filip Jankovic, and Luca Foschini. 2021. Self-supervision of wearable sensors time-series data for influenza detection. arXiv:2112.13755 [cs] (Dec. 2021). - · arXiv
  17. [29][29] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. NeurIPS (2012). -
  18. [30][30] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, et al. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019). - · arXiv
  19. [31][31] Francisco Laport-López, Emilio Serrano, Javier Bajo, and Andrew T. Campbell. 2020. A review of mobile sensing systems, applications, and opportunities. Knowledge and Information Systems (2020). -
  20. [32][32] Daniel B. Larremore, Bryan Wilder, Evan Lester, Soraya Shehata, James M. Burke, et al. 2020. Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance. medRxiv (Sept. 2020). -
  21. [37][37] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, et al. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692 (2019). - · arXiv
  22. [41][41] Momin M. Malik, Afsaneh Doryab, Michael Merrill, Jürgen Pfeffer, and Anind K. Dey. 2020. Can Smartphone Co-locations Detect Friendship? It Depends How You Model It. arXiv:2008.02919 [cs] (2020). - · arXiv
  23. [45][45] Aziz Mezlini, Allison Shapiro, Eric J Daza, Eamon Caddigan, Ernesto Ramirez, et al. 2021. Estimating the Burden of Influenza on Daily Activity at Population Scale Using Commercial Wearable Sensors. medRxiv (2021). -
  24. [46][46] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems . 3111-3119. -
  25. [50][50] Bret Nestor, Jaryd Hunter, Raghu Kainkaryam, Erik Drysdale, Jeffrey B Inglis, et al. 2021. Dear Watch, Should I Get a COVID-19 Test? Designing deployable machine learning for wearables. medRxiv (2021). -
  26. [56][56] Yichen Shen, Maxime Voisin, Alireza Aliamiri, Anand Avati, Awni Hannun, et al. 2019. Ambulatory Atrial Fibrillation Monitoring Using Wearable Photoplethysmography with Deep Learning. In KDD . -
  27. [57][57] Huan Song, Deepta Rajan, Jayaraman J Thiagarajan, and Andreas Spanias. 2018. Attend and diagnose: Clinical time series analysis using attention models. In AAAI . -
  28. [58][58] Dimitris Spathis, Sandra Servia-Rodriguez, Katayoun Farrahi, Cecilia Mascolo, and Jason Rentfrow. 2019. Sequence multi-task learning to forecast mental wellbeing from sparse self-reported data. In KDD . -
  29. [60][60] Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, Soren Brage, Nick Wareham, et al. 2021. SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data. arXiv:2102.06073 (2021). - · arXiv
  30. [61][61] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, et al. 2017. Attention is all you need. In Advances in neural information processing systems . 5998-6008. -
  31. [62][62] Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2019. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters (2019). -
  32. [68][68] George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A Transformer-based Framework for Multivariate Time Series Representation Learning. In KDD . -
  33. [69][69] Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, et al. 2019. A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. AAAI (July 2019). -

Cited by

Sorted by citations

Authors
Merrill, M. A., & Althoff, T.
Affiliations
University of Washington (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2205.13607
arXiv
2205.13607
OpenAlex
W4281786927
Cited by
5
References
0
Citations by year
2024: 2 · 2023: 3
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
Keywords
mobile sensing, self-supervised pre-training, transfer learning, multivariate time series classification, COVID-19 prediction, wearable sensors, wearable activity trackers, long-range dependencies, class imbalance, small datasets, zero-shot prediction, medical surveillance, transformer self-attention, convolutional neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review