Understanding egocentric activities
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
We present a method to analyze daily activities, such as meal preparation, using video from an egocentric camera. Our method performs inference about activities, actions, hands, and objects. Daily activities are a challenging domain for activity recognition which are well-suited to an egocentric approach. In contrast to previous activity recognition methods, our approach does not require pre-trained detectors for objects and hands. Instead we demonstrate the ability to learn a hierarchical model of an activity by exploiting the consistent appearance of objects, hands, and actions that results from the egocentric context. We show that joint modeling of activities, actions, and objects leads to superior performance in comparison to the case where they are considered independently. We introduce a novel representation of actions based on object-hand interactions and experimentally demonstrate the superior performance of our representation in comparison to standard activity representations such as bag of words.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Clustering by Passing Messages Between Data Points
Frey, B. J., & Dueck, D. (2007) · Science · doi:10.1126/science.1136800 · Cited by 6,758
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Recognizing human actions: a local SVM approach
Schuldt, C., Laptev, I., & Caputo, B. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1334462 · Cited by 3,010
Recognizing human actions: A local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Cited by 2,906
The recognition of human movement using temporal templates
Bobick, A. F., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,811
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Towards a general theory of action and time
Allen, J. F. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90008-0 · Cited by 2,559
Improved boosting algorithms using confidence-rated predictions
Schapire, R. E., & Singer, Y. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279960 · Cited by 2,547
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,541
Maximum likelihood estimation via the ECM algorithm: A general framework
MENG, X.-L., & RUBIN, D. B. (1993) · Biometrika · doi:10.1093/biomet/80.2.267 · Cited by 1,861
Real-time American sign language recognition using desk and wearable computer based video
Starner, T., Weaver, J., & Pentland, A. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735811 · Cited by 1,335
Machine Recognition of Human Activities: A Survey
Turaga, P., Chellappa, R., Subrahmanian, V. S., & Udrea, O. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2008.2005594 · Cited by 1,333
Recognizing action at a distance
Efros, Berg, Mori, & Malik (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238420 · Cited by 1,241
Marszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
What, where and who? Classifying events by scene and object recognition
Li, L.-J., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408872 · Cited by 781
Modeling mutual context of object and human pose in human-object interaction activities
Yao, B., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540235 · Cited by 573
Learning to recognize objects in egocentric activities
Fathi, A., Ren, X., & Rehg, J. M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995444 · Cited by 557 · cited 6×
“We use semi-supervised learning framework from [7] to build object classifiers given these weak informations.”
BackgroundMethodInfluentialSelf-citationObserving Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 502
Activity recognition using the velocity histories of tracked keypoints
Messing, R., Pal, C., & Kautz, H. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459154 · Cited by 455
Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words
Niebles, J. C., Wang, H., Wang, H., & Fei-Fei, L. (2006) · Procedings of the British Machine Vision Conference 2006 · doi:10.5244/c.20.127 · Cited by 385
A Scalable Approach to Activity Recognition based on Object Use
Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., & Rehg, J. M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408865 · Cited by 324
Self-citationExploiting human actions and object context for recognition tasks
Moore, D. J., Essa, I. A., & Hayes, M. H. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.791201 · Cited by 281
Gupta, A., Srinivasan, P., Shi, J., & Davis, L. S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206492 · Cited by 231
Fast unsupervised ego-action learning for first-person sports videos
Kitani, K. M., Okabe, T., Sato, Y., & Sugimoto, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995406 · Cited by 220
Temporal segmentation and activity classification from first-person sensing
Spriggs, E. H., De La Torre, F., & Hebert, M. (2009) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2009.5204354 · Cited by 219
Recognizing human actions from still images with latent poses
Yang, W., Wang, Y., & Mori, G. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539879 · Cited by 201
Figure-ground segmentation improves handled object recognition in egocentric video
Ren, X., & Gu, C. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540074 · Cited by 163
Tracklet Descriptors for Action Modeling and Video Analysis
Raptis, M., & Soatto, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_42 · Cited by 147
Action recognition using probabilistic parsing
Bobick, A., & Ivanov, Y. A. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1998.698609 · Cited by 146
Combining Self Training and Active Learning for Video Segmentation
Fathi, A., Balcan, M.-F., Ren, X., & Rehg, J. M. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.78 · Cited by 92
Self-citationNovelty detection from an ego-centric perspective
Aghazadeh, O., Sullivan, J., & Carlsson, S. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995731 · Cited by 64
Hierarchical Recognition of Human Activities Interacting with Objects
Ryoo, M. S., & Aggarwal, J. K. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383487 · Cited by 57
Computational perception of scene dynamics
Mann, R., Jepson, A. S., & Siskind, J. M. (1996) · Lecture notes in computer science · doi:10.1007/3-540-61123-1_167 · Cited by 42
Research works printed but not linked: 1
- spriggs, 2009, Temporal segmentation and activity classification from firstperson sensing, Egovision Workshop
Cited by
Temporal Convolutional Networks for Action Segmentation and Detection
Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.113 · Cited by 2,238
“In the literature, our task goes by the name action segmentation [7, 8, 6, 15, 29, 16, 10] or action detection [28, 20, 21, 25].”
MethodTemporal Convolutional Networks: A Unified Approach to Action Segmentation
Lea, C., Vidal, R., Reiter, A., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_7 · Cited by 924
MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation
Farha, Y. A., & Gall, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00369 · Cited by 673
“In [6], actions are represented based on the interactions between hands and objects.”
BackgroundDiscovering important people and objects for egocentric video summarization
Lee, Y. J., Ghosh, J., & Grauman, K. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247820 · Cited by 658
Detecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248010 · Cited by 649
Metadata
- Authors
- Fathi, A., Farhadi, A., & Rehg, J. M.
- Affiliations
- Georgia Institute of Technology (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- 2011 International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2011-11
- Pages
- 407-414
- OpenAlex
- W2149276562
- Cited by
- 384
- References
- 36
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 14 · 2023: 15 · 2022: 18 · 2021: 35 · 2020: 31 · 2019: 32 · 2018: 36 · 2017: 41 · 2016: 60 · 2015: 49 · 2014: 25 · 2013: 11 · 2012: 8
- FWCI
- 10.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- SMARTech Repository (Georgia Institute of Technology) (submittedVersion)
- Funders
- National Science Foundation
- Multidisciplinary University Research Initiative
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- egocentric video, activity recognition, daily activities, meal preparation, hand detection, object detection, action representation, bag-of-words
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.