Multi-agent event recognition in structured scenarios
The DOI matches its Crossref record.
Abstract
We present a framework for the automatic recognition of complex multi-agent events in settings where structure is imposed by rules that agents must follow while performing activities. Given semantic spatio-temporal descriptions of what generally happens (i.e., rules, event descriptions, physical constraints), and based on video analysis, we determine the events that occurred. Knowledge about spatio-temporal structure is encoded using first-order logic using an approach based on Allen's Interval Logic, and robustness to low-level observation uncertainty is provided by Markov Logic Networks (MLN). Our main contribution is that we integrate interval-based temporal reasoning with probabilistic logical inference, relying on an efficient bottom-up grounding scheme to avoid combinatorial explosion. Applied to one-on-one basketball, our framework detects and tracks players, their hands and feet, and the ball, generates event observations from the resulting trajectories, and performs probabilistic logical inference to determine the most consistent sequence of events. We demonstrate our approach on 1hr (100,000 frames) of outdoor videos.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A Review of Human Activity Recognition Methods
Vrigkas, M., Nikou, C., & Kakadiaris, I. A. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00028 · Cited by 566
A Hierarchical Deep Temporal Model for Group Activity Recognition
Ibrahim, M. S., Muralidharan, S., Deng, Z., Vahdat, A., & Mori, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.217 · Cited by 523
Social interactions: A first-person perspective
Fathi, A., Hodgins, J. K., & Rehg, J. M. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247805 · Cited by 288
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_3 · Cited by 246
“There are many applications for this task including in industrial manufacturing [7,8], surgical training [9,10,11], and general human activity analysis (e.g. cooking, sports) [4,6,12,13,14,15].”
BackgroundContext-Aware Activity Recognition and Anomaly Detection in Video
Zhu, Y., Nayak, N. M., & Roy-Chowdhury, A. K. (2013) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2012.2234722 · Cited by 121
“Human-object interaction has been frequently used as context in many past works [4], [5].”
BackgroundMethodIntroducing uncertainty in complex event processing: Model, implementation, and validation
Cugola, G., Margara, A., Matteucci, M., & Tamburrelli, G. (2015) · Computing · doi:10.1007/s00607-014-0404-y · Cited by 72
Video based activity recognition in trauma resuscitation
Chakraborty, I., Elgammal, A., & Burd, R. S. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553758 · Cited by 29
Metadata
- Authors
- Morariu, V. I., & Davis, L. S.
- Venue
- CVPR 2011
- Publisher
- IEEE
- Published
- 2011-06
- Pages
- 3289-3296
- OpenAlex
- W2098106399
- Cited by
- 130
- References
- 0
- FWCI
- 14.83
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.