Layered representations for human activity recognition
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
We present the use of layered probabilistic representations using hidden Markov models for performing sensing, learning, and inference at multiple levels of temporal granularity We describe the use of representation in a system that diagnoses states of a user's activity based on real-time streams of evidence from video, acoustic, and computer interactions. We review the representation, present an implementation, and report on experiments with the layered representation in an office-awareness application.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The S-HSMM thus can be viewed as a member of the exponential family [5] with parameter θ.”
From Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model (2005) · cites this paper as [5]
Citations
Cites
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
Event structure in perception and conception.
Zacks, J. M., & Tversky, B. (2001) · Psychological Bulletin · doi:10.1037//0033-2909.127.1.3 · Cited by 1,198 · cited 1×
“Psychologists have, in fact, found that many human behaviors are hierarchically structured [ 20 ].”
BackgroundCoupled hidden Markov models for complex action recognition
Brand, M., Oliver, N., & Pentland, A. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609450 · Cited by 1,052 · cited 2×
“More complex models, such as Parameterized-HMM (PHMM) [19], Entropic-HMM [1], Variable-length HMM (VHMM) [8] and Coupled-HMM (CHMM) [2], have been used to recognize more complex activities such as the interaction between two people.”
MethodSelf-citationReal-time American Sign Language recognition from video using hidden Markov models
Starner, T., & Pentland, A. (2002) · Proceedings of International Symposium on Computer Vision - ISCV · doi:10.1109/iscv.1995.477012 · Cited by 773 · cited 1×
“Starner and Pentland in [18] use an HMM for recognizing hand movements used to relay symbols in American Sign Language.”
MethodRecognition of visual activities and interactions by stochastic parsing
Ivanov, Y. A., & Bobick, A. F. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868686 · Cited by 659 · cited 3×
“[12] Y. Ivanov and A. Bobick.”
BackgroundInfluentialModels of attention in computing and communication: from principles to applications
Horvitz, E., Kadie, C., Paek, T., & Hovel, D. (2003) · Communications of the ACM · doi:10.1145/636772.636798 · Cited by 313
Self-citationHorvitz, E., Jacobs, A., & Hovel, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6707 · Cited by 311 · cited 1×
“Recent work on probabilistic models for reasoning about a user’s location, intentions, and focus of attention have highlighted opportunities for building new kinds of applications and services [10].”
BackgroundSelf-citationDiscovery and segmentation of activities in video
Brand, M., & Kettnaker, V. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868685 · Cited by 301 · cited 3×
“Brand and Kettnaker in [1] propose an entropicHMM approach to organize the observed video activities (office activity and outdoor traffic) into meaningful states.”
BackgroundMethodA practical methodology for speech source localization with microphone arrays
Brandstein, M. S., & Silverman, H. F. (1997) · Computer Speech & Language · doi:10.1006/csla.1996.0024 · Cited by 291 · cited 4×
“Typically, TDOA-based approaches have two steps: the time delay estimation and the sound source localization.”
BackgroundInfluentialLearning Variable-Length Markov Models of Behavior
Galata, A., Johnson, N., & Hogg, D. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0894 · Cited by 265 · cited 2×
“More complex models, such as Parameterized-HMM (PHMM) [19], Entropic-HMM [1], Variable-length HMM (VHMM) [8] and Coupled-HMM (CHMM) [2], have been used to recognize more complex activities such as the interaction between two people.”
MethodA framework for recognizing multi-agent action from visual evidence
Intille, S. S., & Bobick, A. (1999) · Cited by 172
The BATmobile: towards a Bayesian automated taxi
Forbes, J., Huang, T. S., Kanazawa, K., & Russell, S. (1995) · Cited by 170
Unsupervised clustering of ambulatory audio and video
Clarkson, B., & Pentland, A. (1999) · 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258) · doi:10.1109/icassp.1999.757481 · Cited by 144 · cited 1×
“Clarkson and Pentland model events and scenes from audiovisual information in [5].”
BackgroundRecognition and interpretation of parametric gesture
Wilson, A. D., & Bobick, A. (2002) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1998.710739 · Cited by 93 · cited 2×
“More complex models, such as Parameterized-HMM (PHMM) [19], Entropic-HMM [1], Variable-length HMM (VHMM) [8] and Coupled-HMM (CHMM) [2], have been used to recognize more complex activities such as the interaction between two people.”
MethodA Bayesian approach to human activity recognition
Madabhushi, A., & Aggarwal, J. K. (2003) · IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS) · doi:10.1109/vs.1999.780265 · Cited by 91 · cited 1×
“recent years, more complex Bayesian networks have also been adopted for the modeling and recognition of human activities [15, 9, 6, 4, 11, 7].”
BackgroundRepresentation and optimal recognition of human activities
Hongeng, S., Brémond, F., & Nevatia, R. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.855905 · Cited by 81 · cited 1×
“In [17], a probabilistic finite-state automaton is used for recognizing different scenarios, such as monitoring pedestrians or cars on a freeway.”
MethodReal-time multi-view face detection
Zhang, Z., Zhu, L., Li, S. Z., & Zhang, H. (2003) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2002.1004147 · Cited by 75 · cited 1×
“... signal: the density of skin color in the image (obtained by discriminating between skin and non-skin models, consisting of histograms in HSV color space), the density of motion in the image (obtained by image differences), the density of foreground pixels in the image (obtained by background subtraction, after hav…”
MethodAdvanced visual surveillance using Bayesian networks
Buxton, H. (1997) · doi:10.1049/ic:19970385 · Cited by 74 · cited 1×
“recent years, more complex Bayesian networks have also been adopted for the modeling and recognition of human activities [15, 9, 6, 4, 11, 7].”
BackgroundBuilding qualitative event models automatically from visual input
Fernyhough, J. H., Cohn, A. G., & Hogg, D. (2002) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1998.710742 · Cited by 30 · cited 1×
“recent years, more complex Bayesian networks have also been adopted for the modeling and recognition of human activities [15, 9, 6, 4, 11, 7].”
BackgroundJudging people's availability for interaction from video snapshots
Johnson, B., & Greenberg, S. (1999) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.1999.772710 · Cited by 23 · cited 1×
“Some of these activities have been proposed in the past as indicators of a person’s availability [13].”
Background
Cited by
Human activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model
Duong, T. V., Bui, H. H., Phung, D. Q., & Venkatesh, S. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.61 · Cited by 535
“Examples of these models include the Abstract HMM [2], the Hierarchical HMM [3, 8, 1], and the Layered HMM [18].”
MethodSurvey on Emotional Body Gesture Recognition
Noroozi, F., Corneanu, C. A., Kaminska, D., Sapinski, T., Escalera, S., & Anbarjafari, G. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2874986 · Cited by 455
Pareek, P., & Thakkar, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09904-8 · Cited by 455
“LHMM segments into different layers with different temporal information.”
MethodInfluentialDiscovery of activity patterns using topic models
Huynh, T., Fritz, M., & Schiele, B. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409638 · Cited by 442
“[16] use a layered HMM representation to infer office activities such as giving a presentation, having a conversation or making a phone call, based on lowlevel information from audio and visual sensors as well as from the user’s keyboard and mouse activity.”
Method
Metadata
- Authors
- Oliver, N., Horvitz, E., & Garg, A.
- Affiliations
- Microsoft (United States) (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings. Fourth IEEE International Conference on Multimodal Interfaces
- Publisher
- IEEE Comput. Soc
- Published
- 2003-06-26
- Pages
- 3-8
- OpenAlex
- W2134067070
- Cited by
- 312
- References
- 20
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 6 · 2022: 2 · 2021: 4 · 2020: 12 · 2019: 6 · 2018: 14 · 2017: 17 · 2016: 12 · 2015: 18 · 2014: 13 · 2013: 26 · 2012: 23
- FWCI
- 14.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, hidden Markov models, temporal granularity, video analysis, acoustic sensing, computer interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.