Siamese Networks for Weakly Supervised Human Activity Recognition
Open accessAbstract
Deep learning has been successfully applied to human activity recognition. However, training deep neural networks requires explicitly labeled data which is difficult to acquire. In this paper, we present a model with multiple siamese networks that are trained by using only the information about the similarity between pairs of data samples without knowing the explicit labels. The trained model maps the activity data samples into fixed size representation vectors such that the distance between the vectors in the representation space approximates the similarity of the data samples in the input space. Thus, the trained model can work as a metric for a wide range of different clustering algorithms. The training process minimizes a similarity loss function that forces the distance metric to be small for pairs of samples from the same kind of activity, and large for pairs of samples from different kinds of activities. We evaluate the model on three datasets to verify its effectiveness in segmentation and recognition of continuous human activity sequences.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · see citing works
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · see citing works
Everitt, B. J., Landau, S., Leese, M., & Stahl, D. R. (2011) · Wiley series in probability and statistics · doi:10.1002/9780470977811 · Cited by 2,926 · see citing works
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 3× · see citing works
“Hand designed features are widely used in HAR [1], [9].”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1× · see citing works
“been developed and applied in HAR systems [2], [3], [4].”
MethodSignature Verification Using A "Siamese" Time Delay Neural Network
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1993) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001493000339 · Cited by 2,077 · see citing works
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959 · see citing works
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861 · see citing works
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · see citing works
Siamese Recurrent Architectures for Learning Sentence Similarity
Mueller, J., & Thyagarajan, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10350 · Cited by 935 · see citing works
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · see citing works
Transition-Aware Human Activity Recognition Using Smartphones
Reyes-Ortiz, J.-L., Oneto, L., Samà, A., Parra, X., & Anguita, D. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.07.085 · Cited by 740 · see citing works
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · see citing works
Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2016) · Sensors · doi:10.3390/s16040426 · Cited by 423 · see citing works
Learning Text Similarity with Siamese Recurrent Networks
Neculoiu, P., Versteegh, M., & Rotaru, M. (2016) · Proceedings of the 1st Workshop on Representation Learning for NLP · doi:10.18653/v1/w16-1617 · Cited by 341 · cited 1× · see citing works
“[18] used a siamese RNN for job title normalization.”
MethodBeyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video
Pigou, L., van den Oord, A., Dieleman, S., Van Herreweghe, M., & Dambre, J. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-016-0957-7 · Cited by 290 · see citing works
Activity recognition from acceleration data based on discrete consine transform and SVM
He, Z., & Jin, L. (2009) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2009.5346042 · Cited by 270 · see citing works
On the use of ensemble of classifiers for accelerometer-based activity recognition
Catal, C., Tufekci, S., Pirmit, E., & Kocabag, G. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.01.025 · Cited by 240 · see citing works
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D. T., Doyle, L., Lin, S., & Tan, H.-P. (2015) · DR-NTU (Nanyang Technological University) · Cited by 191 · see citing works
Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · cited 2× · see citing works
“The results in Table II show that the proposed weakly supervised method outperforms the state-of-the-art, LSTM with continuous temporal and sensor attention [28] , on the DG dataset.”
BackgroundMethodDeep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.04664 · Cited by 126 · see citing works
Bächlin, M., Roggen, D., Tröster, G., Plotnik, M., Inbar, N., Meidan, I., Herman, T., Brozgol, M., Shaviv, E., Giladi, N., et al. (2009) · International Symposium on Wearable Computers · doi:10.1109/iswc.2009.14 · Cited by 100 · see citing works
Classification of Acceleration Waveforms during Walking by Wavelet Transform
Sekine, M., Ogawa, M., Togawa, T., Fukui, Y., & TAMURA, T. (1997) · Methods of Information in Medicine · doi:10.1055/s-0038-1636855 · Cited by 51 · see citing works
Joint Learning of Speaker and Phonetic Similarities with Siamese Networks
Zeghidour, N., Synnaeve, G., Usunier, N., & Emmanuel Dupoux, A. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-811 · Cited by 41 · see citing works
An Automatic Segmentation Technique in Body Sensor Networks based on Signal Energy
Guenterberg, E., Ostadabbas, S., Ghasemzadeh, H. G., & Jafari, R. (2009) · International Conference on Body Area Networks · doi:10.4108/icst.bodynets2009.6036 · Cited by 31 · see citing works
Evaluating the effects of signal segmentation on activity recognition.
Baños, O., Gálvez, J. M., Damas, M., Guillén, A., Herrera, L. J., Pomares, H., & Rojas, I. (2014) · IWBBIO · Cited by 14 · see citing works
Human Activity Recognition using Deep Learning and Sensor Fusion
Berggren, K. (2018) · Lund University Publications Student Papers (Lund University) · Cited by 3 · see citing works
Flagged first, then by citations
1 of the 33 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- T. Tamura, M. Sekine, M. Ogawa, T. Togawa, and Y. Fukui, "Classi- fication of acceleration waveforms during walking by wavelet trans- form," Methods of information in medicine, vol. 36, pp. 356-9, 01 1998.
- M. Zeng, L. T. Nguyen, B. Yu, O. J. Mengshoel, J. Zhu, P. Wu, and J. Zhang, "Convolutional neural networks for human activity recognition using mobile sensors," in 6th International Conference on Mobile Computing, Applications and Services, pp. 197-205, Nov 2014.
Cited by
Sorted by citations
What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?
Qian, H., Tian, T., & Miao, C. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539134 · Cited by 67 · see citing works
Recent use of deep learning techniques in clinical applications based on gait: a survey
Matsushita, Y., Tran, D. T., Yamazoe, H., & Lee, J. (2021) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwab054 · Cited by 44 · see citing works
Deep Triplet Networks with Attention for Sensor-based Human Activity Recognition
Khaertdinov, B., Ghaleb, E., & Asteriadis, S. (2021) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom50583.2021.9439116 · Cited by 43 · see citing works
Activity, Plan, and Goal Recognition: A Review
Van-Horenbeke, F. A., & Peer, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.643010 · Cited by 42 · see citing works
Wu, J., Huang, J., Wu, X., & Dai, H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120968 · Cited by 27 · see citing works
Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition
Guo, C., Zhang, Y., Chen, Y., Xu, C., & Wang, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3379019 · Cited by 18 · see citing works
Modality-wise relational reasoning for one-shot sensor-based activity recognition
Kasnesis, P., Chatzigeorgiou, C., Patrikakis, C. Z., & Rangoussi, M. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.03.003 · Cited by 14 · see citing works
CogAx: Early Assessment of Cognitive and Functional Impairment from Accelerometry
Ramamurthy, S. R., Chatterjee, S., Galik, E. M., Gangopadhyay, A., Roy, N., Mitra, B., & Chakraborty, S. (2022) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom53586.2022.9762401 · Cited by 7 · see citing works
Cha, B., & Vaidya, B. (2024) · Electronics · doi:10.3390/electronics13091739 · Cited by 7 · see citing works
Amjad, A., Qaiser, S., Błaszczyszyn, M., & Szczęsna, A. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1557 · Cited by 7 · see citing works
Sheng, T., & Huber, M. (2025) · Sensors · doi:10.3390/s25134032 · Cited by 5 · see citing works
Self-citationGuo, C., Zhang, Y., Chen, Y., Yang, W., Wu, Q., & Wang, Z. (2023) · IEEE Smart World Congress (SWC) · doi:10.1109/swc57546.2023.10449131 · Cited by 2 · see citing works
Skeleton Based Human Activity Prediction in Gait Thermal images using Siamese Networks
Srihari, P., & Harikiran, J. (2022) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca55336.2022.10009412 · Cited by 1 · see citing works
Towards Generalizing Temporal Action Segmentation to Unseen Views
Bahrami, E., Zatsarynna, O., Francesca, G., & Gall, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02703-0 · Cited by 1 · see citing works
Auto-Augmentation Contrastive Learning for Wearable-based Human Activity Recognition
Wu, Q., Shen, J., Fan, F., Gu, Y., Xu, C., & Chen, Y. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385468 · Cited by 1 · see citing works
LEARNING EMBEDDINGS FOR WEARABLE-BASED HUMAN ACTIVITY ANALYSIS
Sheng, T. (2020)
Self-citationSarkar, S., & Huber, M. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36.133318
Self-citationFeature representation learning for human activity recognition
Khaertdinov, B. (2024) · doi:10.26481/dis.20240618bk
Stable graph based decision route explanation in siamese neural networks
Saleem, R., Anjum, A., Yuan, B., & Liu, L. (2025) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-025-01154-y
An anomaly detection based approach for continuous authentication with smartwatch inertial sensors
Gholami, A., Alaca, F., & Zulkernine, M. (2025) · Computers & Security · doi:10.1016/j.cose.2025.104656
Sheng, T., & Huber, M. (2025) · 2025 International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla66185.2025.00049
Self-citation
- Authors
- Sheng, T., & Huber, M.
- Affiliations
- The University of Texas at Arlington (US)
- Venue
- 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 4069-4075
- ISSN
- 1062-922X (ISSN-L)
- OpenAlex
- W2991288262
- Cited by
- 21
- References
- 33
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 4 · 2023: 4 · 2022: 3 · 2021: 4 · 2020: 1
- FWCI
- 0.65
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- Siamese network, weakly supervised learning, human activity recognition, similarity loss, metric learning, clustering, activity segmentation, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
