COCOA: Cross Modality Contrastive Learning for Sensor Data
Open accessThe DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index.
Abstract
Self-Supervised Learning (SSL) is a new paradigm for learning discriminative representations without labeled data, and has reached comparable or even state-of-the-art results in comparison to supervised counterparts. Contrastive Learning (CL) is one of the most well-known approaches in SSL that attempts to learn general, informative representations of data. CL methods have been mostly developed for applications in computer vision and natural language processing where only a single sensor modality is used. A majority of pervasive computing applications, however, exploit data from a range of different sensor modalities. While existing CL methods are limited to learning from one or two data sources, we propose COCOA (Cross mOdality COntrastive leArning), a self-supervised model that employs a novel objective function to learn quality representations from multisensor data by computing the cross-correlation between different data modalities and minimizing the similarity between irrelevant instances. We evaluate the effectiveness of COCOA against eight recently introduced state-of-the-art self-supervised models, and two supervised baselines across five public datasets. We show that COCOA achieves superior classification performance to all other approaches. Also, COCOA is far more label-efficient than the other baselines including the fully supervised model using only one-tenth of available labeled data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Due to the differences in signal patterns across modalities, modality-specific encoders are used, which improve the model's robustness to missing modalities and its computational efficiency.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [40]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
(2008) · Journal of Machine Learning Research · Cited by 33,051
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 4×
“, images and videos) for purposes such as extracting captions for the images and videos [58, 93], contrasting text against audio [40] for speech recognition, or vision against audio [83].”
BackgroundInfluentialMomentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 4×
“During the last two years there was a rapid progress in contrastive learning methods in the eld of computer vision [11, 12, 32, 38, 85], audio and speech processing [32, 50, 54, 62, 82, 90], and natural language processing domain [9, 20, 23, 25, 28, 54, 89, 91].”
BackgroundMethodInfluentialEmerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 3×
“During the last two years there was a rapid progress in contrastive learning methods in the eld of computer vision [11, 12, 32, 38, 85], audio and speech processing [32, 50, 54, 62, 82, 90], and natural language processing domain [9, 20, 23, 25, 28, 54, 89, 91].”
BackgroundMethodResultUnsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., & Saul, L. K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1577078 · Cited by 4,044
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 3×
“During the last two years there was a rapid progress in contrastive learning methods in the eld of computer vision [11, 12, 32, 38, 85], audio and speech processing [32, 50, 54, 62, 82, 90], and natural language processing domain [9, 20, 23, 25, 28, 54, 89, 91].”
BackgroundNoise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,992
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
SimCSE: Simple Contrastive Learning of Sentence Embeddings
Gao, T., Yao, X., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.552 · Cited by 2,851
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2×
“For the sake of completeness and providing a supervised reference, we compare the performance of COCOA to the supervised baseline publicly accepted model, DeepConvLSTM [55] which we call it DeepCL, and a network with the same architecture as COCOA which we call it “sup.” for short.”
MethodUnsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1×
“In addition, in contrast to hand-crafted auxiliary tasks [27, 65], which may require extensive domain knowledge and hyper-parameter tuning, contrastive methods are task and domain-agnostic.”
BackgroundDistance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., Blitzer, J. C., & Saul, L. K. (2005) · neural information processing systems · Cited by 2,132
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
(2016) · neural information processing systems · Cited by 1,572
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“We evaluate COCOA on ve dierent well-known public datasets: UCIHAR [4], SLEEPEDF [29, 44], Opportunity [61], PAMAP2 [59], and WESAD [67].”
MethodDeepSleepNet: a model for automatic sleep stage scoring based on raw single-channel EEG
Supratak, A., Dong, H., Wu, C., & Guo, Y. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2017.2721116 · Cited by 1,376
Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection
Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., & Van Laerhoven, K. (2018) · Proceedings of the 20th ACM International Conference on Multimodal Interaction · doi:10.1145/3242969.3242985 · Cited by 1,277
(2011) · Cited by 1,258
Contrastive Learning with Hard Negative SamplesPreprint
Robinson, J., Chuang, C.-Y., Sra, S., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04592 · Cited by 1,225
Data-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014
Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG
Kemp, B., Zwinderman, A. H., Tuk, B., Kamphuisen, H. A. C., & Oberye, J. J. L. (2000) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.867928 · Cited by 953
Sampling Matters in Deep Embedding Learning
Wu, C.-Y., Manmatha, R., Smola, A. J., & Krahenbuhl, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.309 · Cited by 836
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Arandjelovic, R., & Zisserman, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 782
What makes for good views for contrastive learning
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10243 · Cited by 773
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Žbontar, J., Jing, L., Misra, I., LeCun, Y., & Deny, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03230 · Cited by 751
Time Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/631 · Cited by 727
Time Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2020) · arXiv · doi:10.48550/arxiv.2002.12478 · Cited by 727
Time-Series Representation Learning via Temporal and Contextual Contrasting
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., & Guan, C. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/324 · Cited by 696 · cited 16×
“customized and extended the similar CL framework on the ubiquitous sensor and time-series data[18, 22, 63, 70, 73, 78].”
BackgroundMethodInfluentialSplit-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.76 · Cited by 616
Time-Contrastive Networks: Self-Supervised Learning from Video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Schaal, S., Levine, S., & Brain, G. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462891 · Cited by 546
Debiased Contrastive LearningPreprint
Chuang, C.-Y., Robinson, J., Lin, Y.-C., Torralba, A. B., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00224 · Cited by 534
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer
Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., & Xu, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.393 · Cited by 474
Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv · doi:10.48550/arxiv.1906.05849 · Cited by 465
Self-Supervised ECG Representation Learning for Emotion Recognition
Sarkar, P., & Etemad, A. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3014842 · Cited by 383
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
Giorgi, J., Nitski, O., Wang, B., & Bader, G. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.72 · Cited by 350
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · cited 3×
“For UCIHAR dataset, as suggested by [63] and [73], we used windows with 400 samples.”
BackgroundMethodSelf-citationAudioclip: Extending Clip to Image, Text and Audio
Guzhov, A., Raue, F., Hees, J., & Dengel, A. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747631 · Cited by 330
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
Akbari, H., Yuan, L., Qian, R., Chuang, W.-H., Chang, S., Cui, Y., & Gong, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11178 · Cited by 326
CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients
Kiyasseh, D., Zhu, T., & Clifton, D. A. (2020) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2005.13249 · Cited by 319
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
Liu, A. T., Li, S.-W., & Lee, H.-Y. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3095662 · Cited by 312
Generative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 311
Unsupervised Scalable Representation Learning for Multivariate Time Series
Franceschi, J.-Y., Dieuleveut, A., & Jaggi, M. (2019) · arXiv · doi:10.48550/arxiv.1901.10738 · Cited by 305
Understanding self-supervised Learning Dynamics without Contrastive PairsPreprint
Tian, Y., Chen, X., & Ganguli, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06810 · Cited by 300
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298
Hard Negative Mixing for Contrastive Learning
Kalantidis, Y., Sarıyıldız, M. B., Pion, N., Weinzaepfel, P., & Larlus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01028 · Cited by 257
CERT: Contrastive Self-supervised Learning for Language UnderstandingPreprint
Fang, H., Wang, S., Zhou, M., Ding, J., & Xie, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.12766 · Cited by 254
Uncovering the structure of clinical EEG signals with self-supervised learning
Banville, H., Chehab, O., Hyvärinen, A., Engemann, D.-A., & Gramfort, A. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abca18 · Cited by 245
Ranked List Loss for Deep Metric Learning
Wang, X., Hua, Y., Kodirov, E., Hu, G., Garnier, R., & Robertson, N. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00535 · Cited by 242
Contrastive Learning of General-Purpose Audio Representations
Saeed, A., Grangier, D., & Zeghidour, N. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413528 · Cited by 219
Self-citationCLEAR: Contrastive Learning for Sentence Representation
Wu, Z., Wang, S., Gu, J., Khabsa, M., Sun, F., & Ma, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15466 · Cited by 214
Audio-Visual Instance Discrimination with Cross-Modal Agreement
Morgado, P., Vasconcelos, N., & Misra, I. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01229 · Cited by 211
Self-Supervised MultiModal Versatile Networks
Alayrac, J.-B., Recasens, A., Schneider, R. G., Arandjelović, R., Ramapuram, J., Fauw, J. D., Smaira, L., Dieleman, S., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16228 · Cited by 189
Multimodal Contrastive Training for Visual Representation Learning
Yuan, X., Lin, Z., Kuen, J., Zhang, J., Wang, Y., Maire, M., Kale, A., & Faieta, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00692 · Cited by 179
Semantic Re-tuning with Contrastive Tension
Carlsson, F., Sahlgren, M., Gogoulou, E., Gyllensten, A. C., & Hellqvist, E. Y. (2021) · DiVA (Digitala Vetenskapliga Arkivet) · Cited by 175
Unsupervised representation learning for time series with temporal neighborhood coding
Tonekaboni, S., Eytan, D., & Goldenberg, A. (2021) · arXiv · doi:10.48550/arxiv.2106.00750 · Cited by 170
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · cited 19×
“human activity recognition, sleep stage detection and emotion recognition) while existing rival methods only consider single type of sensors (mostly accelerometer data [36, 42]).”
BackgroundMethodInfluential(2020) · Cited by 147
Masked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146
Self-supervised Pretraining of Visual Features in the Wild
Goyal, P., Caron, M., Lefaudeux, B., Xu, M., Wang, P., Pai, V., Singh, M., Liptchinsky, V., Misra, I., Joulin, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.01988 · Cited by 141 · cited 1×
“Recent works in the eld of Computer Vision (CV), such as SEER [31] and Dino [11], and eld of Natural Language Processing (NLP), such as BERT [20] and XLM-R [16], conrm self-supervise pre-training can yield noticeable improvement toward various downstream tasks compared to the fully-supervised trained models [48].”
ResultSelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., Brage, S., Wareham, N., & Mascolo, C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448112 · Cited by 140
Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding
Deldari, S., Smith, D. V., Xue, H., & Salim, F. D. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449903 · Cited by 136 · cited 10×
“Contrastive learning methods mostly have been proposed for single modality data across a variety of applications including computer vision [10, 12, 32, 37], audio processing [32, 62, 90], natural language processing [23, 28], sensor data analytics [13, 18, 22, 24, 47, 64, 65, 78].”
BackgroundMethodInfluentialSelf-citationCERT: Contrastive Self-supervised Learning for Language Understanding
Fang, H., & Xie, P. (2020) · doi:10.36227/techrxiv.12308378 · Cited by 129
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113
Time-Contrastive Networks: Self-Supervised Learning from Multi-view Observation
Sermanet, P., Lynch, C., Hsu, J., & Levine, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.69 · Cited by 111
Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Ozcelebi, T., & Lukkien, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3009358 · Cited by 106
Self-citationContrastive Self-supervised Learning for Sensor-based Human Activity Recognition
Khaertdinov, B., Ghaleb, E., & Asteriadis, S. (2021) · 2021 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb52358.2021.9484410 · Cited by 95 · cited 2×
“CSSHAR(2021)[45] No NT-Xent[12] Augmentation HAR -(2020)[13] No Info-NCE Augmentation Biosignals (EEG, ECG) TCN(2020) [78] No Triplet Loss temporal General time-series TS-CP2(2020) [18] No Info-NCE temporal General time-series CPC(HAR)(2021) [36] No Info-NCE temporal HAR CPC(EEG)(2021)[8] No Info-NCE temporal EEG sign…”
BackgroundMethodFederated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Özçelebi, T., & Lukkien, J. J. (2024) · Figshare · Cited by 92
Self-citationWenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
Huo, Y., Zhang, M., Liu, G., Lu, H., Gao, Y., Yang, G., Wen, J., Zhang, H., Xu, B., Zheng, W., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06561 · Cited by 79
Valverde, F. R., Valeria Hurtado, J., & Valada, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01144 · Cited by 76
Subject-Aware Contrastive Learning for Biosignals
Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020) · arXiv · doi:10.48550/arxiv.2007.04871 · Cited by 74 · cited 4×
“This visualisation shows the importance of training subject-based models along with training the core model to capture user-specic features of biosignals as discussed in [13].”
BackgroundInfluentialAudioCLIP: Extending CLIP to Image, Text and AudioPreprint
Guzhov, A., Raue, F., Hees, J., & Dengel, A. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13043 · Cited by 69
What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?
Qian, H., Tian, T., & Miao, C. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539134 · Cited by 67
Exploring Contrastive Learning in Human Activity Recognition for Healthcare
Tang, C. I., Perez-Pozuelo, I., Spathis, D., & Mascolo, C. (2020) · arXiv · doi:10.48550/arxiv.2011.11542 · Cited by 67
Multimodal Self-Supervised Learning of General Audio RepresentationsPreprint
Wang, L., Luc, P., Recasens, A., Alayrac, J.-B., & van den Oord, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.12807 · Cited by 59
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework
Yao, S., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., & Abdelzaher, T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264954 · Cited by 54
Robust Contrastive Learning Using Negative Samples with Diminished SemanticsPreprint
Ge, S., Mishra, S., Wang, H., Li, C.-L., & Jacobs, D. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14189 · Cited by 53
ESPRESSO: Entropy and ShaPe awaRe timE-Series SegmentatiOn for Processing Heterogeneous Sensor Data
Deldari, S., Smith, D. V., Sadri, A., & Salim, F. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411832 · Cited by 48
Self-citationA Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive LearningPreprint
Tsai, Y.-H. H., Bai, S., Morency, L., & Salakhutdinov, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13712 · Cited by 46
Active Contrastive Learning of Audio-Visual Video Representations
Ma, S., Zeng, Z., McDuff, D., & Song, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09805 · Cited by 41
Deep Embedding Learning With Discriminative Sampling Policy
Duan, Y., Chen, L., Lu, J., & Zhou, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00510 · Cited by 40
Exploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification
Xue, H., & Salim, F. D. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467263 · Cited by 40
Self-citationUnderstanding Chinese Video and Language via Contrastive Multimodal Pre-Training
Lei, C., Luo, S., Liu, Y., He, W., Wang, J., Wang, G., Tang, H., Miao, C., & Li, H. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475431 · Cited by 35
Sense and Learn: Self-supervision for omnipresent sensors
Saeed, A., Ungureanu, V., & Gfeller, B. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100152 · Cited by 34
Self-citationWeakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables
Sheng, T., & Huber, M. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397330 · Cited by 28
CLEAR: Contrastive-Prototype Learning with Drift Estimation for Resource Constrained Stream Mining
Wang, Z., Chen, Y., Zhao, C., Lin, Y., Zhao, X., Tao, H., Wang, Y., & Khan, L. R. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449820 · Cited by 26
Deldari, S., Xue, H., Saeed, A., He, J., Smith, D. V., & Salim, F. D. (2022) · arXiv · doi:10.48550/arxiv.2206.02353 · Cited by 26
Self-citationDigging deeper: towards a better understanding of transfer learning for human activity recognition
Hoelzemann, A., & Van Laerhoven, K. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414311 · Cited by 20
Opportunistic Activity Recognition in IoT Sensor Ecosystems via Multimodal Transfer Learning
Baños, O., Calatroni, A., Damas, M., Pomares, H., Roggen, D., Rojas, I., & Villalonga, C. (2021) · Neural Processing Letters · doi:10.1007/s11063-021-10468-z · Cited by 18
CLOCS: Contrastive Learning of Cardiac Signals.Preprint
Kiyasseh, D., Zhu, T., & Clifton, D. A. (2020) · arXiv (Cornell University) · Cited by 12
Device or User: Rethinking Federated Learning in Personal-Scale Multi-Device Environments
Cho, H., Mathur, A., & Kawsar, F. (2021) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3485730.3493449 · Cited by 9
Understanding Chinese Video and Language via Contrastive Multimodal Pre-TrainingPreprint
Lei, C., Luo, S., Liu, Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09411 · Cited by 4
Contrastive Separative Coding for Self-Supervised Representation Learning
Wang, J., Lam, M. W. Y., Su, D., & Yu, D. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414352 · Cited by 2
What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?Preprint
Qian, H., Tian, T., & Miao, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05998
Research works printed but not linked: 1
- Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. 2000. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101, 23 (2000).
Cited by
Deep Learning for Time-Series Prediction in IIoT: Progress, Challenges, and Prospects
Ren, L., Jia, Z., Laili, Y., & Huang, D. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3291371 · Cited by 184
CALDA: Improving Multi-Source Time Series Domain Adaptation With Contrastive Adversarial Learning
Wilson, G., Doppa, J. R., & Cook, D. J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298346 · Cited by 83
Applications of Self-Supervised Learning to Biomedical Signals: A Survey
Pup, F. D., & Atzori, M. (2023) · IEEE Access · doi:10.1109/access.2023.3344531 · Cited by 43
PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor Data
Jeong, D., & Han, K. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3659594 · Cited by 53
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33
Metadata
- Authors
- Deldari, S., Xue, H., Saeed, A., Smith, D. V., & Salim, F. D.
- Affiliations
- RMIT University (AU)
- UNSW Sydney (AU)
- Philips (Netherlands) (NL)
- Commonwealth Scientific and Industrial Research Organisation (AU)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2022-09-06
- Volume
- 6
- Issue
- 3
- Pages
- 1-28
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4289550913
- Cited by
- 92
- References
- 95
- Citations by year
- 2026: 14 · 2025: 28 · 2024: 26 · 2023: 23
- FWCI
- 6.33
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- self-supervised learning, contrastive learning, cross-modal contrastive learning, multi-sensor data, cross correlation function, representation learning, label efficiency, Theobroma cacao, wearable sensors, ubiquitous computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.