Layered representations for human activity recognition
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
We present the use of layered probabilistic representations using hidden Markov models for performing sensing, learning, and inference at multiple levels of temporal granularity We describe the use of representation in a system that diagnoses states of a user's activity based on real-time streams of evidence from video, acoustic, and computer interactions. We review the representation, present an implementation, and report on experiments with the layered representation in an office-awareness application.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The S-HSMM thus can be viewed as a member of the exponential family [5] with parameter θ.”
From Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model (2005) · cites this paper as [5]
Citations
Cites
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
Event structure in perception and conception.
Zacks, J. M., & Tversky, B. (2001) · Psychological Bulletin · doi:10.1037//0033-2909.127.1.3 · Cited by 1,198 · cited 1×
“Psychologists have, in fact, found that many human behaviors are hierarchically structured [ 20 ].”
BackgroundCoupled hidden Markov models for complex action recognition
Brand, M., Oliver, N., & Pentland, A. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609450 · Cited by 1,052 · cited 2×
“More complex models, such as Parameterized-HMM (PHMM) [19], Entropic-HMM [1], Variable-length HMM (VHMM) [8] and Coupled-HMM (CHMM) [2], have been used to recognize more complex activities such as the interaction between two people.”
MethodSelf-citationReal-time American Sign Language recognition from video using hidden Markov models
Starner, T., & Pentland, A. (2002) · Proceedings of International Symposium on Computer Vision - ISCV · doi:10.1109/iscv.1995.477012 · Cited by 773 · cited 1×
“Starner and Pentland in [18] use an HMM for recognizing hand movements used to relay symbols in American Sign Language.”
MethodRecognition of visual activities and interactions by stochastic parsing
Ivanov, Y. A., & Bobick, A. F. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868686 · Cited by 659 · cited 3×
“[12] Y. Ivanov and A. Bobick.”
BackgroundInfluentialModels of attention in computing and communication: from principles to applications
Horvitz, E., Kadie, C., Paek, T., & Hovel, D. (2003) · Communications of the ACM · doi:10.1145/636772.636798 · Cited by 313
Self-citationHorvitz, E., Jacobs, A., & Hovel, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6707 · Cited by 311 · cited 1×
“Recent work on probabilistic models for reasoning about a user’s location, intentions, and focus of attention have highlighted opportunities for building new kinds of applications and services [10].”
BackgroundSelf-citationDiscovery and segmentation of activities in video
Brand, M., & Kettnaker, V. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868685 · Cited by 301 · cited 3×
“Brand and Kettnaker in [1] propose an entropicHMM approach to organize the observed video activities (office activity and outdoor traffic) into meaningful states.”
BackgroundMethodA practical methodology for speech source localization with microphone arrays
Brandstein, M. S., & Silverman, H. F. (1997) · Computer Speech & Language · doi:10.1006/csla.1996.0024 · Cited by 291 · cited 4×
“Typically, TDOA-based approaches have two steps: the time delay estimation and the sound source localization.”
BackgroundInfluentialLearning Variable-Length Markov Models of Behavior
Galata, A., Johnson, N., & Hogg, D. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0894 · Cited by 265 · cited 2×
“More complex models, such as Parameterized-HMM (PHMM) [19], Entropic-HMM [1], Variable-length HMM (VHMM) [8] and Coupled-HMM (CHMM) [2], have been used to recognize more complex activities such as the interaction between two people.”
Method
Cited by
Human activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model
Duong, T. V., Bui, H. H., Phung, D. Q., & Venkatesh, S. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.61 · Cited by 535
“Examples of these models include the Abstract HMM [2], the Hierarchical HMM [3, 8, 1], and the Layered HMM [18].”
MethodSurvey on Emotional Body Gesture Recognition
Noroozi, F., Corneanu, C. A., Kaminska, D., Sapinski, T., Escalera, S., & Anbarjafari, G. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2874986 · Cited by 455
Pareek, P., & Thakkar, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09904-8 · Cited by 455
“LHMM segments into different layers with different temporal information.”
MethodInfluentialDiscovery of activity patterns using topic models
Huynh, T., Fritz, M., & Schiele, B. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409638 · Cited by 442
“[16] use a layered HMM representation to infer office activities such as giving a presentation, having a conversation or making a phone call, based on lowlevel information from audio and visual sensors as well as from the user’s keyboard and mouse activity.”
MethodA survey on activity recognition and behavior understanding in video surveillance
Vishwakarma, S., & Agrawal, A. (2013) · The Visual Computer · doi:10.1007/s00371-012-0752-6 · Cited by 418
“[128] developed layered hidden Markov models (LHMMs), to model a real-time office activities.”
BackgroundAdvanced internet of things for personalised healthcare systems: A survey
Qi, J., Yang, P., Min, G., Amft, O., Dong, F., & Xu, L. (2017) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2017.06.018 · Cited by 406
“Typical image sensor related IoT enabled PHS cases[100] include SenseCam, Sony Xperia eye, etc.”
MethodA Hybrid Discriminative/Generative Approach for Modeling Human Activities
Lester, J., Choudhury, T., Kern, N., Borriello, G., & Hannaford, B. (2005) · TUbilio (Technical University of Darmstadt) · Cited by 468
Segmenting motion capture data into distinct behaviors
Barbič, J., Safonova, A., Pan, J.-Y., Faloutsos, C., Hodgins, J. K., & Pollard, N. S. (2004) · Graphics Interface · Cited by 441
Automatic analysis of multimodal group actions in meetings
McCowan, L., Gatica-Perez, D., Bengio, S., Lathoud, G., Barnard, M., & Zhang, D. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.49 · Cited by 350
A Systematic Review of Hidden Markov Models and Their Applications
Mor, B., Garhwal, S., & Kumar, A. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09422-4 · Cited by 343
“Oliver et al. [120] represented humans activity from real-time streams of video, computer interaction and acoustic with LHMM.”
BackgroundSimultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors
Wilson, D. H., & Atkeson, C. (2005) · Lecture Notes in Computer Science · doi:10.1007/11428572_5 · Cited by 314
“In particular, researchers have used GPS readings to infer walking, driving and bus riding behaviors [23], laser range finders to learn motion paths in a home [8], and combinations of audio and video to recognize behavior in an office environment [22] and interactions between individuals [12].”
Nguyen, N. T., Phung, D. Q., Venkatesh, S., & Bui, H. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.203 · Cited by 303
“In [14], Oliver et al. use the LHMM for recognizing highlevel activities based on real-time streams of signals from multiple sources of sensors.”
BackgroundMethodVideo-based event recognition: activity representation and probabilistic recognition methods
Hongeng, S., Nevatia, R., & Bremond, F. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.02.005 · Cited by 287
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., Li, Z., & Metaxas, D. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.59 · Cited by 282
“and match them to those of unseen events [18,19,17,14]; (ii) Methods based on temporal logic represent critical ordering constraints on activities [20,21], and (iii) Algorithms based on stochastic grammars or hidden Markov models (HMMs) represent the observed data distribution and model the dynamic, temporal state con…”
BackgroundCOSAR: hybrid reasoning for context-aware activity recognition
Riboni, D., & Bettini, C. (2011) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0331-7 · Cited by 281
“A research direction consists in the use of cameras with the help of sound, image and scene recognition software (see, e.g., [5, 21 , 28]).”
BackgroundCohen, I., Cozman, F. G., Sebe, N., Cirelo, M. C., & Huang, T. S. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.127 · Cited by 251
a CAPpella: programming by demonstration of context-aware applications
Dey, A. K., Hamid, R., Beckmann, C., Li, I., & Hsu, D. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985697 · Cited by 235
“Recognizing human activity is a rapidly growing subfield and has become more prominent in the HCI community of late [10, 19 ,] While most research focuses on techniques for programmers to build useful recognizers, there are a number of systems that investigate the idea of putting a human “in the loop” to build pattern…”
BackgroundLavee, G., Rivlin, E., & Rudzsky, M. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2023380 · Cited by 218
“[85] use a layered HMM (LHMM) event model in the event domain of office surveillance.”
MethodReview of fall detection techniques: A data availability perspective
Khan, S. S., & Hoey, J. (2017) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2016.10.014 · Cited by 210
“In the past decade, there has been considerable research carried out in the field of activity recognition using Hidden Markov Models (HMMs) [34, 35, 36].”
BackgroundSensor-Based Datasets for Human Activity Recognition – A Systematic Review of Literature
De-La-Hoz-Franco, E., Ariza-Colpas, P., Quero, J. M., & Espinilla, M. (2018) · IEEE Access · doi:10.1109/access.2018.2873502 · Cited by 209
“In this way, different modalities of data collection have been proposed: video [9], [10], audio [11], [12] and binary sensors [6], [13] or portable sensors deployed on the body such as accelerometers and gyroscopes [14], [15], among others.”
BackgroundMining models of human activities from the web
Perkowitz, M., Philipose, M., Fishkin, K., & Patterson, D. J. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988750 · Cited by 183
“In some cases [2][13][14] the developers define the structure of the possible models, but the system tunes the parameters of the model based on examples from the user.”
BackgroundActivity Recognition and Abnormal Behaviour Detection with Recurrent Neural Networks
Arifoglu, D., & Bouchachia, A. (2017) · Procedia Computer Science · doi:10.1016/j.procs.2017.06.121 · Cited by 181
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.014 · Cited by 178
Recent advances in video-based human action recognition using deep learning: A review
Wu, D., Sharma, N., & Blumenstein, M. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966210 · Cited by 167
Scalable Recognition of Daily Activities with Wearable Sensors
Huỳnh, T., Blanke, U., & Schiele, B. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75160-1_4 · Cited by 158
“[20, 10, 21]).”
Plan, Activity, and Intent Recognition
Sukthankar, G., Geib, C., Bui, H., Pynadath, D. V., & Goldman, R. P. (2014) · Elsevier eBooks · doi:10.1016/c2012-0-01169-6 · Cited by 181
Advances in Human Action Recognition: A SurveyPreprint
Cheng, G., Wan, Y., Saudagar, A. N., Namuduri, K. R., & Buckles, B. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.05964 · Cited by 169
“Previous research efforts on statistical approaches mainly dwell on applications of extended HMMs and dynamic Bayesian networks: 2-layered hierarchical hidden Markov models (HMMs) [59, 92, 88] and dynamic probabilistic networks (DPNs) also known as dynamic Bayesian networks (DBNs) [19, 12].”
The use of pervasive sensing for behaviour profiling — a survey
Atallah, L., & Yang, G. (2009) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2009.06.009 · Cited by 170
“According to [70], layered HMMs provide higher accuracy than HMMs in terms of recognising office activities; they are also more robust to changes in the environment.”
BackgroundSocial signal processing: state-of-the-art and future perspectives of an emerging domain
Vinciarelli, A., Pantic, M., Bourlard, H., & Pentland, A. (2008) · Proceedings of the 16th ACM international conference on Multimedia · doi:10.1145/1459359.1459573 · Cited by 136
“The combination of the information extracted from different modalities is performed with different algorithms including Dynamic Bayesian Networks [54] and layered hidden Markov models [57].”
MethodA new method for violence detection in surveillance scenes
Zhang, T., Yang, Z., Jia, W., Yang, B., Yang, J., & He, X. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-015-2648-8 · Cited by 149
“[34] developed layered HMMs (LHMMs) to model a real-time activity with sequential structures.”
BackgroundExamining the robustness of sensor-based statistical models of human interruptibility
Fogarty, J., Hudson, S. E., & Lai, J. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985719 · Cited by 134
Multi-resolution RASTA filtering for TANDEM-based ASR
Heřmanský, H., & Fousek, P. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-184 · Cited by 131
A daily behavior enabled hidden Markov model for human behavior understanding
Chung, P.-C., & Liu, C.-D. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2007.10.022 · Cited by 130
Audiovisual Information Fusion in Human–Computer Interfaces and Intelligent Environments: A Survey
Shivappa, S. T., Trivedi, M. M., & Rao, B. D. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2057231 · Cited by 128
“In [146], the authors propose a hierarchial HMM framework for modeling human activity.”
BackgroundA Real-Time, Multiview Fall Detection System: A LHMM-Based Approach
Thome, N., Miguet, S., & Ambellouis, S. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2008.2005606 · Cited by 137
“With LHMM, there are two kinds of strategies for providing the observations when going up in the hierarchy (see [10]): the maxbelief and the distributional approach.”
MethodResultInfluentialLeveraging temporal, contextual and ordering constraints for recognizing complex activities in video
Laxton, B., Lim, J., & Kriegman, D. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383074 · Cited by 121
Multi-Person Visual Focus of Attention from Head Pose and Meeting Contextual Cues
Ba, S., Ba, S., & Odobez, J. (n.d.) · Cited by 123
Efficient duration and hierarchical modeling for human activity recognition
Duong, T., Phung, D., Bui, H., & Venkatesh, S. (2009) · Artificial Intelligence · doi:10.1016/j.artint.2008.12.005 · Cited by 115
“Another extension to the HMM is the incorporation of hierarchical knowledge such as the hierarchical HMM (HHMM) [10], the abstract HMM (AHMM) [4], and the layered HMMs [30].”
BackgroundMethodInfluentialYao, R., Lin, G., Shi, Q., & Ranasinghe, D. C. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.12.024 · Cited by 105
Luo, X., Li, H., Yang, X., Yu, Y., & Cao, D. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12419 · Cited by 111
“Hidden Markov models (HMMs) (Oliver et al., 2002; Nguyen et al., 2005; Zhang et al., 2006), dynamic Bayesian networks (Gong and Xiang, 2003; Dai et al., 2008), and conditional random fields (Liao et al., 2007; Sun and Nevatia, 2013; Shu et al., 2015) are frequently used in earlier studies.”
MethodA novel sequence representation for unsupervised analysis of human activities
Hamid, R., Maddi, S., Johnson, A. Y., Bobick, A., Essa, I. A., & Isbell, C. L. (2009) · Artificial Intelligence · doi:10.1016/j.artint.2009.05.002 · Cited by 119
“Events in human activities depend on preceding events over variable durations [38].”
BackgroundKanade, T., & Hebert, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2200554 · Cited by 126
Situation awareness within the context of connected cars: A comprehensive review and recent trends
Golestan, K., Soua, R., Karray, F., & Kamel, M. S. (2015) · Information Fusion · doi:10.1016/j.inffus.2015.08.001 · Cited by 116
Survey of Context Provisioning Middleware
Knappmeyer, M., Kiani, S. L., Reetz, E. S., Baker, N., & Tönjes, R. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2013.010413.00207 · Cited by 106
Recognizing and monitoring high-level behaviors in complex spatial environments
Nguyen, N., Bui, H., Venkatsh, S., & WEST, G. A. W. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211524 · Cited by 101
“Oliver [14] proposed a Layered Hidden Markov Model (LHMM) where the classification results of the low layer are used as inputs to the higher layer.”
BackgroundA Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities
Oliver, N., & Horvitz, E. (2005) · Lecture notes in computer science · doi:10.1007/11527886_26 · Cited by 99
Self-citationComplex activity recognition using context-driven activity theory and activity signatures
Saguna, S., Zaslavsky, A., & Chakraborty, D. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2490832 · Cited by 101
Context-Aware Activity Recognition through a Combination of Ontological and Statistical Reasoning
Riboni, D., & Bettini, C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02830-4_5 · Cited by 103
“A research direction consists in devising techniques to recognize activities using cameras with the help of sound, image and scene recognition software (see, e.g., [ 4 ,5]), but this is limited to very confined environments and often subject to serious privacy concerns, clearly perceived by the monitored users.”
Social signals, their function, and automatic analysis: a survey
Vinciarelli, A., Pantić, M., Bourlard, H. A., & Pentland, A. (2008) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1452392.1452405 · Cited by 96
“The combination of the information extracted from di.erent modalities is performed with di.erent algorithms including Dynamic Bayesian Networks [44] and layered Hidden Markov Models [46].”
MethodDaily Routine Recognition through Activity Spotting
Blanke, U., & Schiele, B. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01721-6_12 · Cited by 76
“In [7] the authors show that Hidden Markov Models (HMM) have the ability to capture different levels of abstraction with respect to time granularity.”
BackgroundActivity Modeling Using Event Probability Sequences
Cuntoor, N. P., Yegnanarayana, B., & Chellappa, R. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.916991 · Cited by 73
“In [25], the factorization approach was used to compute a set of shapes that represent normal activities.”
MethodRadio-based device-free activity recognition with radio frequency interference
Wei, B. W., Hu, W. S., Yang, M., & Chou, C. T. (2015) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/2737095.2737117 · Cited by 81
“Cameras are also widely used for activity recognition, localisation and tracking [19, 7, 41, 4, 16, 13, 25].”
BackgroundBlanke, U., & Schiele, B. (2010) · International Symposium on Wearable Computers (ISWC) 2010 · doi:10.1109/iswc.2010.5665869 · Cited by 65
Assessing Human Activity in Elderly People Using Non-Intrusive Load Monitoring
Alcalá, J., Ureña, J., Hernández, Á., & Gualda, D. (2017) · Sensors · doi:10.3390/s17020351 · Cited by 79
“In [16] probabilistic models, such as naïve Bayesian (NB), hidden Markov model (HMM), and hidden semi-Markov model (HSMM), are evaluated to avoid overlapping; whereas in [17] a layered hidden Markov model is used to model the different durations of activities.”
MethodLin, W., Sun, M., Poovendran, R., & Zhang, Z. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2008.927111 · Cited by 70
“Models such as hidden Markov model (HMMs) or dynamic Bayesian network (DBN) [12]–[14], state machine [4], [15], Adaboost [16], [17], and SVM [26], [27] are widely used in these works for activity recognition.”
Detecting and discriminating behavioural anomalies
Loy, C. C., Xiang, T., & Gong, S. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.07.023 · Cited by 75
“It is worth pointing out that cascade structure of DBNs has been considered for activity analysis [31] [32].”
BackgroundView-Invariant Human Activity Recognition Based on Shape and Motion Features
Niu, F., & Abdel-Mottaleb, M. S. (2005) · IEEE International Symposium on Multimedia Software Engineering (MSE) · doi:10.1109/mmse.2004.88 · Cited by 67
Visualizing non-speech sounds for the deaf
Matthews, T. L., Fong, J., & Mankoff, J. C. (2005) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1090785.1090797 · Cited by 66
Human modeling for human–robot collaboration
Hiatt, L. M., Narber, C., Bekele, E., Khemlani, S., & Trafton, J. G. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917690592 · Cited by 75
“Other structured, hierarchical-like formulations have also been used, such as coupled HMMs for modeling interactions (Natarajan and Nevatia, 2007), layered HMMs for analysis at varying temporal granularity (Oliver et al., 2002), and factorial HMMs to provide better generalization capabilities (Kulić et al.”
MethodHMM-Based Segmentation and Recognition of Human Activities from Video Sequences
Niu, F., & Abdel-Mottaleb, M. S. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521545 · Cited by 65
“Several human activity recognition methods [2][5][ 6 ][7][8][10] were proposed in the past few years to classify single human activities such as walking, skipping, sitting down, etc.”
BackgroundEvaluating non-speech sound visualizations for the deaf
Matthews, T., Fong, J., Ho-Ching, F. W.-L., & Mankoff, J. (2006) · Behaviour & Information Technology · doi:10.1080/01449290600636488 · Cited by 60
Towards automatic analysis of social interaction patterns in a nursing home environment from video
Chen, D., Yang, J., & Wactlar, H. D. (2004) · Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval · doi:10.1145/1026711.1026757 · Cited by 59
An opportunistic activity-sensing approach to save energy in office buildings
Milenkovic, M., & Amft, O. (2013) · ACM International Conference on Future Energy Systems (e-Energy) · doi:10.1145/2487166.2487194 · Cited by 60
Luo, X., Li, H., Yu, Y., Zhou, C., & Cao, D. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12538 · Cited by 60
“Earlier studies on statistical methods used hidden Markov models (HMMs) (Nguyen, Phung, Venkatesh, & Bui, 2005; Oliver, Horvitz, & Garg, 2002; Zhang, Gaticaperez, Bengio, & McCowan, 2006), dynamic Bayesian networks (Dai, Di, Dong, Tao, & Xu, 2008; S. Gong & Xiang, 2003), and conditional random…”
Problems people with dementia have with kitchen tasks: The challenge for pervasive computing
Wherton, J. P., & Monk, A. (2010) · Interacting with computers · doi:10.1016/j.intcom.2010.03.004 · Cited by 62
Ontology and Taxonomy Collaborated Framework for Meeting Classification
Hakeem, A., & Shah, M. (2004) · Cited by 53
Recognizing Energy-related Activities Using Sensors Commonly Installed in Office Buildings
Milenkovic, M., & Amft, O. (2013) · Procedia Computer Science · doi:10.1016/j.procs.2013.06.089 · Cited by 52
Structure from Statistics - Unsupervised Activity Analysis using Suffix Trees
Hamid, R., Maddi, S., Bobick, A., & Essa, I. A. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408894 · Cited by 58
“Finally, exploiting properties of Suffix Trees, we present a novel perspective on anomalous subsequences of activities, and propose an algorithm to detect them in linear-time.”
BackgroundKnowledge Based Activity Recognition with Dynamic Bayesian Network
Zeng, Z. Y., & Ji, Q. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_39 · Cited by 57
“In comparison, layered HMM [ 6 ], switching hidden semi-Markov models [7] and Hierarchical HMM [8] try to model activity at multiple levels, with the upper layers encoding the transitions among the high-level states (such as the constituent actions) and the bottom layer encoding the transitions among the low-level sta…”
BackgroundActivity Recognition and Room-Level Tracking in an Office Environment
Wojek, C., Nickel, K., & Stiefelhagen, R. (2006) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi.2006.265608 · Cited by 58
“[6] who exploit multi-layer HMMs for a one person office activity recognition system.”
BackgroundAn integrated framework for human activity classification
Cao, H., Nguyen, M. N., Phua, C., Krishnaswamy, S. P., & Li, X. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370268 · Cited by 54
“neuro-fuzzy approaches [7]), Bayesian/topic modeling [8] and varied machine learning approaches [9-10, 12-15, 17-22] for RHA from sensory data.”
MethodActivity, Plan, and Goal Recognition: A Review
Van-Horenbeke, F. A., & Peer, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.643010 · Cited by 66
“…Markov models exist that where designed with hierarchical structure in mind and that have been successfully applied to the problem of activity, plan and goal recognition, such as layered hidden Markov models (LHMM) Oliver et al. (2002) or hierarchical hidden Markov models (HHMM) (Bui et al., 2004).”
BackgroundSustainable Homecare Monitoring System by Sensing Electricity Data
Alcalá, J., Ureña, J., Hernández, Á., & Gualda, D. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2713645 · Cited by 51
Ontology and taxonomy collaborated framework for meeting classification
Hakeem, A., & Shah, M. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1333743 · Cited by 44
X-CHAR: A Concept-based Explainable Complex Human Activity Recognition Model
Jeyakumar, J. V., Sarker, A., Garcia, L. A., & Srivastava, M. B. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580804 · Cited by 58
“Several existing works [27, 28, 60] show that the combinations of convolutional and Long-Short Term Memory (LSTM) layers achieve better performance in complex activity recognition tasks when compared to the traditional statistical-based methods [14, 24, 39, 62].”
BackgroundMethodPark, S., & Trivedi, M. M. (2007) · Machine Vision and Applications · doi:10.1007/s00138-006-0055-x · Cited by 49
“HMM-based approaches to activity recognition have been presented in [8,14] with different feature sets.”
BackgroundMethodViewpoint Integration for Hand-Based Recognition of Social Interactions from a First-Person View
Bambach, S., Crandall, D., & Yu, C. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2820771 · Cited by 45
“Personal activity recognition in everyday contexts has a wide range of applications for intelligent human-computer interfaces [2, 5, 7, 11,12].”
BackgroundFisherposes for Human Action Recognition Using Kinect Sensor Data
Ghojogh, B., Mohammadzade, H., & Mokari, M. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2784425 · Cited by 45
“References [11] and [16], Layered Hidden Markov Model (LHMM) [45], variable length Markov model [46], hierarchical maximum entropy Markov model (MEMM) [47], Switching Hidden Semi-Markov Model (S-HSMM) [48], and Abstract Hidden Markov mEmory Model (AHMEM) [49] can be mentioned as examples in this category.”
BackgroundA Framework for Activity Recognition and Detection of Unusual Activities.
Mahajan, D. K., Kwatra, N., Jain, S., Kalra, P. K., & Banerjee, S. (2004) · Cited by 44
Using a minimal action grammar for activity understanding in the real world
Summers-Stay, D. A., Teo, C. L., Yang, Y., Fermuller, C. M., & Aloimonos, Y. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385483 · Cited by 44
“specific objects and actions [13], [24].”
Towards the detection of unusual temporal events during activities using HMMs
Khan, S. S., Karg, M. E., Hoey, J., & Kulić, D. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370444 · Cited by 43
“It applies the X-factor introduced in [23] for SLDS to HMMs.”
BackgroundInfluentialSensCare: Semi-automatic Activity Summarization System for Elderly Care
Wu, P., Peng, H.-K., Zhu, J., & Zhang, Y. (2012) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-642-32320-1_1 · Cited by 47
“To overcome some of the shortages of HMM, like performance degrades when the range of activities become more complex or long-term temporal dependency exists between activities that makes Markov assumption difficult to deal with [6], variations of HMM were developed such as Layered HMM [19], Switching Hidden Semi-Marko…”
MethodTemporal ICA for classification of acoustic events i a kitchen environment
Kraft, F., Malkin, R. E., Schaaf, T., & Waibel, A. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-261 · Cited by 41
Human Activity Recognition Using Cell Phone-Based Accelerometer and Convolutional Neural Network
Prasad, A., Tyagi, A. K., Althobaiti, M. M., Almulihi, A. H., Mansour, R. F., & Mahmoud, A. M. (2021) · Applied Sciences · doi:10.3390/app112412099 · Cited by 44
A Frequent Pattern Mining Approach for ADLs Recognition in Smart Environments
Chikhaoui, B., Wang, S., & Pigot, H. (2011) · International Conference on Advanced Information Networking and Applications (AINA) · doi:10.1109/aina.2011.13 · Cited by 45
“The experimental results showed that both TMM and HMM achieved a high accuracy in predicting tasks using simulated data, whereas, on real data, HMM performed better than TMM. N. Oliver et al. [21] proposed a layered HMMs for human activity recognition at multiple level of temporal granularity.”
MethodAssistive intelligent environments for automatic health monitoring
Atkeson, C. G., & Wilson, D. H. (2005) · Cited by 45
Ranganathan, G. (2020) · Journal of Innovative Image Processing · doi:10.36548/jiip.2020.2.004 · Cited by 40
“Oliver et al [6] performed the "Understanding and modeling of wifi signal based human activity recognition."”
BackgroundGeneric Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real
Li, J., Ma, H., Zhan, W., & Tomizuka, M. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569780 · Cited by 40
A Presence-Based Context-Aware Chronic Stress Recognition System
Peternel, K., Pogačnik, M., Tavčar, R., & Kos, A. (2012) · Sensors · doi:10.3390/s121115888 · Cited by 40
“A number of interesting algorithms exist in the literature, for example the Layered HMM (LHMM) [37] which can facilitate the learning procedure.”
MethodSVAS: Surveillance Video Analysis System
Kardas, K., & Cicekli, N. K. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.07.051 · Cited by 40
Discovering the Top-k Unexplained Sequences in Time-Stamped Observation Data
Albanese, M., Molinaro, C., Persia, F., Picariello, A., & Subrahmanian, V. S. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.33 · Cited by 39
“However, specific work in the domains of video and cyber security have focused on anomalous activity detection.”
BackgroundSAMMPLE: Detecting Semantic Indoor Activities in Practical Settings Using Locomotive Signatures
Yan, Z., Chakraborty, D., Misra, A., Jeung, H., & Aberer, K. (2012) · International Symposium on Wearable Computers · doi:10.1109/iswc.2012.22 · Cited by 37
“Prior investigations on building hierarchical representations of complex activities from low-level sensors [6, 3, 2] have mostly operated using multiple wearable accelerometers (e.”
BackgroundHaider, W., Hu, J., Xie, Y., Yu, X., & Wu, Q. (2017) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2017.2736555 · Cited by 37
“has been applied in a number of areas, such as network traffic modeling [22], information mining [23], subject recognition [24], [25], video analysis [26] and image classifica-”
BackgroundFrom Real to Complex: Enhancing Radio-based Activity Recognition Using Complex-Valued CSI
Wei, B. W., Hu, W. S., Yang, M., & Chou, C. T. (2019) · ACM Transactions on Sensor Networks · doi:10.1145/3338026 · Cited by 42
“In terms of system components, there are three broad approaches to the activity recognition problem: camera-based [4, 7, 18, 21, 59], sensor-based [3, 6, 8, 12, 24, 48, 54] and device-free [32, 38, 41].”
BackgroundLanguage-Motivated Approaches to Action Recognition
Malgireddy, M. R., Nwogu, I., & Govindaraju, V. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_5 · Cited by 36
Feature Set Selection and Optimal Classifier for Human Activity Recognition
Lösch, M., Schmidt-Rohr, S. R., Knoop, S., Vacek, S., & Dillmann, R. (2007) · IEEE RO-MAN · doi:10.1109/roman.2007.4415232 · Cited by 36
“Exemplarily, usage of hidden markov models in the context of human activity recognition can be found in [5] and [6].”
BackgroundA survey on intelligent human action recognition techniques
Kumar, R., & Kumar, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17529-6 · Cited by 46
Xie, Y., Hu, J., Xiang, Y., Yu, S., Tang, S., & Wang, Y. (2012) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2012.268 · Cited by 35
“Index Terms—Network traffic, traffic behavior, modeling, synthetic, stationary, nonstationary Ç”
Recognition of repetitive sequential human activity
Fan, Q., Bobbitt, R., Zhai, Y., Yanagawa, A., Pankanti, S., & Hampapur, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206644 · Cited by 34
Metadata
- Authors
- Oliver, N., Horvitz, E., & Garg, A.
- Affiliations
- Microsoft (United States) (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings. Fourth IEEE International Conference on Multimodal Interfaces
- Publisher
- IEEE Comput. Soc
- Published
- 2003-06-26
- Pages
- 3-8
- OpenAlex
- W2134067070
- Cited by
- 312
- References
- 20
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 6 · 2022: 2 · 2021: 4 · 2020: 12 · 2019: 6 · 2018: 14 · 2017: 17 · 2016: 12 · 2015: 18 · 2014: 13 · 2013: 26 · 2012: 23
- FWCI
- 14.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, hidden Markov models, temporal granularity, video analysis, acoustic sensing, computer interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.