The DOI matches its Crossref record.
Modeling individual and group actions in meetings with layered HMMs
Open accessAbstract
We address the problem of recognizing sequences of human interaction patterns in meetings, with the goal of structuring them in semantic terms. The investigated patterns are inherently group-based (defined by the individual activities of meeting participants, and their interplay), and multimodal (as captured by cameras and microphones). By defining a proper set of individual actions, group actions can be modeled as a two-layer process, one that models basic individual activities from low-level audio-visual (AV) features,and another one that models the interactions. We propose a two-layer hidden Markov model (HMM) framework that implements such concept in a principled manner, and that has advantages over previous works. First, by decomposing the problem hierarchically, learning is performed on low-dimensional observation spaces, which results in simpler models. Second, our framework is easier to interpret, as both individual and group actions have a clear meaning, and thus easier to improve. Third, different HMMs can be used in each layer, to better reflect the nature of each subproblem. Our framework is general and extensible, and we illustrate it with a set of eight group actions, using a public 5-hour meeting corpus. Experiments and comparison with a single-layer HMM baseline system show its validity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Human activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
A survey on activity recognition and behavior understanding in video surveillance
Vishwakarma, S., & Agrawal, A. (2013) · The Visual Computer · doi:10.1007/s00371-012-0752-6 · Cited by 418
A Systematic Review of Hidden Markov Models and Their Applications
Mor, B., Garhwal, S., & Kumar, A. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09422-4 · Cited by 343
Automatic nonverbal analysis of social interaction in small groups: A review
Gatica-Perez, D. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.01.004 · Cited by 309
Self-citationDiscriminative Latent Models for Recognizing Contextual Group Activities
Lan, T., Wang, Y., Yang, W., Robinovitch, S. N., & Mori, G. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.228 · Cited by 297
Modeling dominance in group conversations using nonverbal activity cues
Jayagopi, D. B., Hung, H., Yeo, C., & Gatica-Perez, D. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.2008238 · Cited by 217
Self-citationVideo based activity recognition in trauma resuscitation
Chakraborty, I., Elgammal, A., & Burd, R. S. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553758 · Cited by 29
Metadata
- Authors
- Zhang, D., Gatica-Perez, D., Bengio, S., & McCowan, I.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2006-06
- Volume
- 8
- Issue
- 3
- Pages
- 509-520
- ISSN
- 1520-9210 (ISSN-L)
- OpenAlex
- W2134282114
- Cited by
- 168
- References
- 0
- FWCI
- 15.29
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.