Learning to Recognize Human Activities Using Soft Labels
Open accessThe DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Abstract
Human activity recognition system is of great importance in robot-care scenarios. Typically, training such a system requires activity labels to be both completely and accurately annotated. In this paper, we go beyond such restriction and propose a learning method that allow labels to be incomplete and uncertain. We introduce the idea of soft labels which allows annotators to assign multiple, and weighted labels to data segments. This is very useful in many situations, e.g., when the labels are uncertain, when part of the labels are missing, or when multiple annotators assign inconsistent labels. We formulate the activity recognition task as a sequential labeling problem. Latent variables are embedded in the model in order to exploit sub-level semantics for better estimation. We propose a max-margin framework which incorporate soft labels for learning the model parameters. The model is evaluated on two challenging datasets. To simulate the uncertainty in data annotation, we randomly change the labels for transition segments. The results show significant improvement over the state-of-the-art approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
Graph Implementations for Nonsmooth Convex Programs
Grant, M. C., & Boyd, S. P. (2007) · Lecture Notes in Control and Information Sciences · doi:10.1007/978-1-84800-155-8_7 · Cited by 2,666
Support vector machine learning for interdependent and structured output spaces
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015341 · Cited by 1,243
Cutting-plane training of structural SVMs
Joachims, T., Finley, T., & Yu, C.-N. J. (2009) · Machine Learning · doi:10.1007/s10994-009-5108-8 · Cited by 958
Accurate activity recognition in a home setting
van Kasteren, T., Noulas, A., Englebienne, G., & Kröse, B. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409637 · Cited by 874
Self-citationLearning human activities and object affordances from RGB-D videos
Koppula, H. S., Gupta, R., & Saxena, A. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913478446 · Cited by 683
Learning structural SVMs with latent variables
Yu, C.-N. J., & Joachims, T. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553523 · Cited by 610
Hidden Conditional Random Fields for Gesture Recognition
Wang, S. B., Quattoni, A., Morency, L.-P., Demirdjian, D., & Darrell, T. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.132 · Cited by 451
Posterior Regularization for Structured Latent Variable Models
Ganchev, K., Graça, J., Gillenwater, J., & Taskar, B. (2010) · Cited by 440
The Concave-Convex Procedure (CCCP)
Yuille, A., & Rangarajan, A. (2001) · Cited by 397
Conditional random fields for activity recognition
Vail, D. L., Veloso, M. M., & Lafferty, J. D. (2007) · Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems · doi:10.1145/1329125.1329409 · Cited by 357
Learning latent temporal structure for complex event detection
Tang, K., Fei-Fei, L., & Koller, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247808 · Cited by 355
Human Activity Detection from RGBD ImagesPreprint
Sung, J., Ponce, C., Selman, B., & Saxena, A. (2011) · arXiv (Cornell University) · Cited by 282
Unsupervised activity recognition using automatically mined common sense
Wyatt, D., Philipose, M., & Choudhury, T. (2005) · National Conference on Artificial Intelligence · Cited by 241
Joint segmentation and classification of human actions in video
Hoai, M., Lan, Z.-Z., & De la Torre, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995470 · Cited by 234
libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models
Mooij, J. M. (2010) · Max Planck Institute for Plasma Physics · doi:10.5555/1756006.1859925 · Cited by 227
Posterior Regularization for Structured Latent Variable Models
GanchevKuzman, GraçaJoão, GillenwaterJennifer, & TaskarBen (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859918 · Cited by 201
Hidden Part Models for Human Action Recognition: Probabilistic versus Max Margin
Wang, Y., & Mori, G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.214 · Cited by 190
Max-margin hidden conditional random fields for human action recognition
Wang, Y., & Mori, G. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206709 · Cited by 179
Learning Spatio-Temporal Structure from RGB-D Videos for Human Activity Detection and Anticipation
Koppula, H. S., & Saxena, A. (2013) · International Conference on Machine Learning · Cited by 169
Anticipating human activities for reactive robotic response
Koppula, H. S., & Saxena, A. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696634 · Cited by 101
Learning latent structure for activity recognition
Hu, N., Englebienne, G., Lou, Z., & Kröse, B. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906983 · Cited by 41
Self-citationInfinite Latent Conditional Random Fields for Modeling Environments through Humans
Yun, J., & Saxena, A. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.034 · Cited by 32
Structured Learning from Partial AnnotationsPreprint
Lou, X., & Hamprecht, F. A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6421 · Cited by 30
Handling Uncertain Tags in Visual Recognition
Vahdat, A., & Mori, G. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.462 · Cited by 20
A two-layered approach to recognize high-level human activities
Hu, N., Englebienne, G., & Kröse, B. (2014) · IEEE RO-MAN · doi:10.1109/roman.2014.6926260 · Cited by 7
Self-citation
Cited by
Facial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_14 · Cited by 341
Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data
Wilson, G., Doppa, J. R., & Cook, D. J. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403228 · Cited by 141
CALDA: Improving Multi-Source Time Series Domain Adaptation With Contrastive Adversarial Learning
Wilson, G., Doppa, J. R., & Cook, D. J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298346 · Cited by 83
Handling annotation uncertainty in human activity recognition
Kwon, H., Abowd, G. D., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347744 · Cited by 52
Switching Structured Prediction for Simple and Complex Human Activity Recognition
Arzani, M. M., Fathy, M., Azirani, A. A., & Adeli, E. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2960481 · Cited by 44
Metadata
- Authors
- Hu, N., Englebienne, G., Lou, Z., & Krose, B.
- Affiliations
- University of Amsterdam (NL)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-10-01
- Volume
- 39
- Issue
- 10
- Pages
- 1973-1984
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2541965240
- Cited by
- 29
- References
- 27
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 2 · 2023: 4 · 2022: 1 · 2021: 4 · 2020: 6 · 2019: 4 · 2018: 1 · 2017: 1
- FWCI
- 1.36
- Type
- journal article
- Language
- en
- Open access
- green
- research.utwente.nl
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- University of Twente Research Information (publishedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- University of Twente Research Information (submittedVersion, other-oa)
- Funders
- European project ACCOMPANY (287624)
- European project MONARCH (601033)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- human activity recognition, soft labels, incomplete labels, multiple annotators, label inconsistency, sequence labeling, latent variables, max-margin learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.