Learning to recognize objects in egocentric activities
Open accessAbstract
This paper addresses the problem of learning object models from egocentric video of household activities, using extremely weak supervision. For each activity sequence, we know only the names of the objects which are present within it, and have no other knowledge regarding the appearance or location of objects. The key to our approach is a robust, unsupervised bottom up segmentation method, which exploits the structure of the egocentric domain to partition each frame into hand, object, and background categories. By using Multiple Instance Learning to match object instances across sequences, we discover and localize object occurrences. Object representations are refined through transduction and object-level classifiers are trained. We demonstrate encouraging results in detecting novel object instances using models produced by weakly-supervised learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The GTEA dataset contains 28 videos corresponding to 7 different activities performed by 4 subjects.
“The GTEA dataset contains 28 videos corresponding to 7 different activities, like preparing coffee or cheese sandwich, performed by 4 subjects.”
From MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (2023) · cites this paper as [8]
Section 5.1 (Dataset; page not available) says the dataset has seven activities performed by four subjects and lists coffee and cheese-sandwich activities. It does not state that the dataset contains 28 videos or establish that each subject contributed exactly one video per activity. (LLM, conf=0.86)
Experimental — accuracy not yet measuredGTEA is originally 15 fps videos, and 50 Salads is 30 fps videos.
“GTEA is originally 15 fps (frames per second) videos, and 50 Salads is 30 fps videos.”
From Stacked encoder–decoder transformer with boundary smoothing for action segmentation (2022) · cites this paper as [8]
Section 5.1 (Dataset) says the GTEA videos were recorded at 30 fps and frames were then extracted at 15 fps. Thus, 15 fps describes the extracted frames, not the original videos. The cited paper does not address 50 Salads or its frame rate. (LLM, conf=0.87)
Experimental — accuracy not yet measuredThe proposed model achieves state-of-the-art results on the 50Salads, GTEA, and Breakfast datasets.
“Our model achieves state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.”
From MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (2023) · cites this paper as [8]
SEDT yields state-of-the-art results on 50 Salads and GTEA.
“Our framework yields stateof-the-art results on two challenging datasets: 50 Salads [7] and GTEA [8].”
From Stacked encoder–decoder transformer with boundary smoothing for action segmentation (2022) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Adaptive background mixture models for real-time tracking
Stauffer, C., & Grimson, W. E. L. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1999.784637 · Cited by 6,925 · see citing works
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509 · see citing works
Learning realistic human actions from movies
Ivan Laptev, A., Marszałek, M., Schmid, C., & Rozenfeld, B. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587756 · Cited by 3,481 · see citing works
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,698 · see citing works
Support Vector Machines for Multiple-Instance Learning
Andrews, S. J., Tsochantaridis, I., & Hofmann, T. F. (2002) · Cited by 1,370 · see citing works
Marszałek, M., Laptev, I., & Schmid, C. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206557 · Cited by 1,197 · see citing works
A Statistical Approach to Texture Classification from Single Images
Varma, M., & Zisserman, A. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-4635-4 · Cited by 1,144 · see citing works
Discovering objects and their location in images
Šivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.77 · Cited by 973 · see citing works
A probabilistic framework for semi-supervised clustering
Basu, S., Bilenko, M., & Mooney, R. J. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014062 · Cited by 823 · see citing works
What, where and who? Classifying events by scene and object recognition
Li, L.-J., & Fei-Fei, L. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408872 · Cited by 790 · see citing works
MILES: Multiple-Instance Learning via Embedded Instance Selection
Chen, Y., Bi, J., & Wang, J. Z. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.248 · Cited by 748 · see citing works
Yan, J., & Pollefeys, M. (2006) · Lecture notes in computer science · doi:10.1007/11744085_8 · Cited by 507 · see citing works
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A. K., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 494 · see citing works
From contours to regions: An empirical evaluation
Arbeláez, P., Maire, M., Fowlkes, C. C., & Malik, J. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206707 · Cited by 447 · see citing works
Looking at people: sensing for ubiquitous and wearable computing
Pentland, A. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824823 · Cited by 403 · see citing works
A Scalable Approach to Activity Recognition based on Object Use
Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., & Rehg, J. M. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408865 · Cited by 328 · see citing works
Self-citationObject, Scene and Actions: Combining Multiple Features for Human Action Recognition
İkizler-Cinbiş, N., & Sclaroff, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_36 · Cited by 288 · see citing works
Temporal segmentation and activity classification from first-person sensing
Spriggs, E., la Torre, F. D., & Hebert, M. (2009) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2009.5204354 · Cited by 234 · see citing works
Berg, T. L., & Forsyth, D. A. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.57 · Cited by 181 · see citing works
Figure-ground segmentation improves handled object recognition in egocentric video
Ren, X., & Gu, C. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540074 · Cited by 167 · see citing works
Self-citationLearning sign language by watching TV (using weakly aligned subtitles)
Buehler, P., Zisserman, A., & Everingham, M. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206523 · Cited by 157 · see citing works
Keywords to visual categories: Multiple-instance learning forweakly supervised object categorization
Vijayanarasimhan, S., & Grauman, K. L. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587632 · Cited by 123 · see citing works
Semantic Label Sharing for Learning with Many Categories
Fergus, R., Bernal, H. J., Weiss, Y., & Torralba, A. B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_55 · Cited by 118 · see citing works
Hierarchical Recognition of Human Activities Interacting with Objects
Ryoo, M. S., & Aggarwal, J. K. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383487 · Cited by 57 · see citing works
An Interactive Computer Vision System DyPERS: Dynamic Personal Enhanced Reality System
Schiele, B., Oliver, N., Jebara, T., & Pentland, A. (1999) · Lecture notes in computer science · doi:10.1007/3-540-49256-9_4 · Cited by 32 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [6]6. R. Fergus, H. Bernal, Y. Weiss, and A. Torralba. Se- mantic label sharing for learning with many cate- gories. In ECCV, 2010. 3282
Cited by
Sorted by citations
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · see citing works
“ADE20K[117] BBBC038v1[12] Cityscapes[25] DOORS[80] DRAM[24] EgoHOS[113] GTEA[34,63] Hypersim[86] IBD[17] iShape[111] LVIS[44] NDD20[100] NDISPark[22,23] OVIS[81] PPDLS[74] Plittersdorf[46] STREETS[91] TimberSeg[38] TrashCan[52] VISOR[28,27] WoodScape[112] PIDRay[104] ZeroWaste-f[6] Figure 8: Samples from the 23 divers…”
BackgroundInfluentialTemporal Convolutional Networks for Action Segmentation and Detection
Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.113 · Cited by 2,238 · see citing works
“We use MERL Shopping which was designed for action detection, Georgia Tech Egocentric Activities which was designed for action segmentation, and 50 Salads which has been used for both.”
BackgroundMethodInfluentialA Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Perazzi, F., Pont-Tuset, J., McWilliams, B., Gool, L. V., Groß, M. H., & Sorkine‐Hornung, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.85 · Cited by 2,039 · see citing works
Temporal Convolutional Networks: A Unified Approach to Action Segmentation
Lea, C., Vidal, R., Reiter, A., & Hager, G. D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_7 · Cited by 1,059 · see citing works
MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation
Farha, Y. A., & Gall, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00369 · Cited by 673 · see citing works
“We evaluate the proposed model on three challenging datasets: 50Salads [25], Georgia Tech Egocentric Activities (GTEA) [8], and the Breakfast dataset [12].”
MethodDetecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248010 · Cited by 666 · see citing works
First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
Garcia-Hernando, G., Yuan, S., Baek, S., & Kim, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00050 · Cited by 520 · see citing works
A Survey on Activity Detection and Classification Using Wearable Sensors
Cornacchia, M., Ozcan, K., Zheng, Y., & Velipasalar, S. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2628346 · Cited by 427 · see citing works
HOnnotate: A Method for 3D Annotation of Hand and Object Poses
Hampali, S., Rad, M., Oberweger, M., & Lepetit, V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00326 · Cited by 425 · see citing works
Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions
Bambach, S., Lee, S., Crandall, D., & Yu, C. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.226 · Cited by 420 · see citing works
Understanding egocentric activities
Fathi, A., Farhadi, A., & Rehg, J. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126269 · Cited by 384 · see citing works
Self-citationLearning to Recognize Daily Actions Using Gaze
Fathi, A., Li, Y., & Rehg, J. M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_23 · Cited by 396 · see citing works
Self-citationMS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation
Li, S., Farha, Y. A., Liu, Y., Cheng, M.-M., & Gall, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3021756 · Cited by 349 · see citing works
“Egocentric Activities (GTEA) [16], and the Breakfast dataset [17].”
BackgroundMethodThe developing infant creates a curriculum for statistical learning
Smith, L. B., Jayaraman, S., Clerkin, E. M., & Chen, Y. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.02.004 · Cited by 259 · see citing works
Social interactions: A first-person perspective
Fathi, A., Hodgins, J. K., & Rehg, J. M. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247805 · Cited by 298 · see citing works
Self-citationGoing Deeper into First-Person Activity Recognition
Ma, M., Fan, H., & Kitani, K. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.209 · Cited by 281 · see citing works
From faces to hands: Changing visual input in the first two years
Fausey, C. M., Jayaraman, S., & Smith, L. B. (2016) · Cognition · doi:10.1016/j.cognition.2016.03.005 · Cited by 278 · see citing works
Advancing High Fidelity Identity Swapping for Forgery Detection
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00512 · Cited by 288 · see citing works
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions
Tekin, B., Bogo, F., & Pollefeys, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00464 · Cited by 268 · see citing works
Real-world visual statistics and infants' first-learned object names
Clerkin, E. M., Hart, E., Rehg, J. M., Yu, C., & Smith, L. B. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2016.0055 · Cited by 268 · see citing works
Self-citationPixel-Level Hand Detection in Ego-centric Videos
Li, C., & Kitani, K. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.458 · Cited by 247 · see citing works
Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
Şener, F., Chatterjee, D., Shelepov, D., He, K., Singhania, D., Wang, R. Y., & Yao, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02042 · Cited by 217 · see citing works
Video-Based Human Behavior Understanding: A Survey
Borges, P. V. K., Conci, N., & Cavallaro, A. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2270402 · Cited by 208 · see citing works
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition
Kwon, T., Tekin, B., Stühmer, J., Bogo, F., & Pollefeys, M. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.00998 · Cited by 356 · see citing works
The Evolution of First Person Vision Methods: A Survey
Betancourt, A., Morerio, P., Regazzoni, C. S., & Matthias Rauterberg, A. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2409731 · Cited by 176 · see citing works
Alleviating Over-segmentation Errors by Detecting Action Boundaries
Ishikawa, Y., Kasai, S., Aoki, Y., & Kataoka, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00237 · Cited by 171 · see citing works
HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
Liu, Y., Liu, Y., Jiang, C., Lyu, K., Wan, W., Shen, H., Liang, B., Fu, Z., Wang, H., & Yi, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02034 · Cited by 161 · see citing works
Temporal Deformable Residual Networks for Action Segmentation in Videos
Lei, P., & Todorovic, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00705 · Cited by 202 · see citing works
Improving Action Segmentation via Graph-Based Temporal Reasoning
Huang, Y., Sugano, Y., & Sato, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01404 · Cited by 160 · see citing works
Understanding Everyday Hands in Action from RGB-D Images
Rogez, G., Supančič, J. S., & Ramanan, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.443 · Cited by 147 · see citing works
Boundary-Aware Cascade Networks for Temporal Action Segmentation
Wang, Z., Gao, Z., Wang, L., Li, Z., & Wu, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58595-2_3 · Cited by 146 · see citing works
Modeling Actions through State Changes
Fathi, A., & Rehg, J. M. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.333 · Cited by 145 · see citing works
Self-citationActor and Observer: Joint Modeling of First and Third-Person Videos
Sigurdsson, G. A., Gupta, A., Schmid, C., Farhadi, A., & Alahari, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00772 · Cited by 146 · see citing works
First Person Action Recognition Using Deep Learned Descriptors
Singh, S., Arora, C., & Jawahar, C. V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.287 · Cited by 192 · see citing works
Toward Storytelling From Visual Lifelogging: An Overview
Bolaños, M., Dimiccoli, M., & Radeva, P. I. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2616296 · Cited by 136 · see citing works
A Survey on Video Diffusion Models
Xing, Z., Feng, Q., Chen, H., Dai, Q., Hu, H., Xu, H., Wu, Z., & Jiang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696415 · Cited by 160 · see citing works
Action Segmentation With Joint Self-Supervised Temporal Domain Adaptation
Chen, M.-H., Li, B., Bao, Y., AlRegib, G., & Kira, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00947 · Cited by 118 · see citing works
Detecting eye contact using wearable eye-tracking glasses
Ye, Z., Li, Y., Fathi, A., Han, Y., Rozga, A., Abowd, G. D., & Rehg, J. M. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370368 · Cited by 123 · see citing works
Self-citationRecognition of Activities of Daily Living with Egocentric Vision: A Review
Nguyen, T.-H.-C., Nebel, J., & Flórez‐Revuelta, F. (2016) · Sensors · doi:10.3390/s16010072 · Cited by 118 · see citing works
Why are faces denser in the visual experiences of younger than older infants?
Jayaraman, S., Fausey, C. M., & Smith, L. B. (2016) · Developmental Psychology · doi:10.1037/dev0000230 · Cited by 116 · see citing works
Sequential Modeling Enables Scalable Learning for Large Vision Models
Bai, Y., Geng, X., Mangalam, K., Bar, A., Yuille, A., Darrell, T., Malik, J., & Efros, A. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02157 · Cited by 126 · see citing works
Summarization of Egocentric Videos: A Comprehensive Survey
del Molino, A. G., Tan, C., Lim, J., & Tan, A. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2623480 · Cited by 111 · see citing works
Analysis of the Hands in Egocentric Vision: A Survey
Bandini, A., & Zariffa, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2986648 · Cited by 107 · see citing works
PlaceAvoider: Steering First-Person Cameras away from Sensitive Spaces
Templeman, R., Korayem, M., Crandall, D., & Kapadia, A. C. (2014) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2014.23014 · Cited by 110 · see citing works
Temporal Action Segmentation: An Analysis of Modern Techniques
Ding, G., Şener, F., & Yao, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327284 · Cited by 124 · see citing works
A Developmental Approach to Machine Learning?
Smith, L. B., & Slone, L. K. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02124 · Cited by 107 · see citing works
Cao, C., Zhang, Y., Wu, Y., Lu, H., & Cheng, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.406 · Cited by 104 · see citing works
Takeo Kanade, A., & Hebert, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2200554 · Cited by 105 · see citing works
Compact CNN for indexing egocentric videos
Poleg, Y., Ephrat, A., Peleg, S., & Arora, C. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477708 · Cited by 105 · see citing works
Charades-Ego: A Large-Scale Dataset of Paired Third and First Person VideosPreprint
Sigurdsson, G. A., Gupta, A. K., Schmid, C., Farhadi, A., & Alahari, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09626 · Cited by 99 · see citing works
ASFormer: Transformer for Action Segmentation
Yi, F., Wen, H., & Jiang, T. (2021) · Proceedings of the British Machine Vision Conference 2021 · doi:10.5244/c.35.49 · Cited by 170 · see citing works
Gesture Recognition in Ego-centric Videos Using Dense Trajectories and Hand Segmentation
Baraldi, L., Paci, F., Serra, G., Luca Benini, C., & Cucchiara, R. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.107 · Cited by 94 · see citing works
Scene Semantics from Long-Term Observation of People
Delaitre, V., Fouhey, D. F., Ivan Laptev, A., Šivic, J., Gupta, A., & Efros, A. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_21 · Cited by 93 · see citing works
Am I a Baller? Basketball Performance Assessment from First-Person Videos
Bertasius, G., Park, H. S., Yu, S. X., & Shi, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.239 · Cited by 93 · see citing works
Behrmann, N., Golestaneh, S. A., Kolter, J. Z., Gall, J., & Noroozi, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_4 · Cited by 93 · see citing works
Challenges in Multi-modal Gesture Recognition
Escalera, S., Athitsos, V., & Guyon, I. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_1 · Cited by 92 · see citing works
Temporal Action Segmentation from Timestamp Supervision
Li, Z., Farha, Y. A., & Gall, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00826 · Cited by 90 · see citing works
Multi-shot Temporal Event Localization: a Benchmark
Liu, X., Hu, Y. G., Bai, S., Ding, F., Bai, X., & Torr, P. H. S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01241 · Cited by 92 · see citing works
Multimodal Multi-Stream Deep Learning for Egocentric Activity Recognition
Song, S., Chandrasekhar, V., Mandal, B., Li, L., Lim, J., Babu, G. S. D., San, P. P., & Cheung, N. M. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.54 · Cited by 82 · see citing works
Recognizing Micro-Actions and Reactions from Paired Egocentric Videos
Yonetani, R., Kitani, K. M., & Sato, Y. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2016.288 · Cited by 95 · see citing works
Depth-Based Hand Pose Estimation: Methods, Data, and Challenges
Supančič, J. S., Rogez, G., Yang, Y., Shotton, J., & Ramanan, D. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1081-7 · Cited by 82 · see citing works
3D Hand Pose Detection in Egocentric RGB-D Images
Rogez, G., Khademi, M., Supančič, J. S., Montiel, J. M. M., & Ramanan, D. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16178-5_25 · Cited by 81 · see citing works
Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos
Li, M., Chen, L., Duarr, Y., Hu, Z., Feng, J., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01926 · Cited by 80 · see citing works
Analysis of Hand Segmentation in the Wild
Khan, A. U., & Borji, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00495 · Cited by 79 · see citing works
A review of vision-based indoor HAR: state-of-the-art, challenges, and future prospects
Bhola, G., & Vishwakarma, D. K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15443-5 · Cited by 78 · see citing works
Action Anticipation: Reading the Intentions of Humans and Robots
Duarte, N. F., Raković, M., Tasevski, J., Coco, M. I., Billard, A., & Santos-Victor, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2861569 · Cited by 76 · see citing works
Deep Temporal Multimodal Fusion for Medical Procedure Monitoring Using Wearable Sensors
Bernal, E. A., Yang, X., Li, Q., Kumar, J., Madhvanath, S., Ramesh, P. S., & Bala, R. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2726187 · Cited by 75 · see citing works
Model Recommendation with Virtual Probes for Egocentric Hand Detection
Li, C., & Kitani, K. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.326 · Cited by 72 · see citing works
Refining Action Segmentation with Hierarchical Video Representations
Ahn, H., & Lee, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01599 · Cited by 75 · see citing works
Global2Local: Efficient Structure Search for Video Action Segmentation
Gao, S., Han, Q., Li, Z., Peng, P., Wang, L., & Cheng, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01653 · Cited by 74 · see citing works
Multi-Stream Deep Neural Networks for RGB-D Egocentric Action Recognition
Tang, Y., Wang, Z., Lu, J., Feng, J., & Zhou, J. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2875441 · Cited by 71 · see citing works
Parent-driven use of wearable cameras for autism support
Marcu, G. M., Dey, A. K., & Kiesler, S. B. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370277 · Cited by 70 · see citing works
Coupling eye-motion and ego-motion features for first-person activity recognition
Ogaki, K., Kitani, K., Sugano, Y., & Sato, Y. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239188 · Cited by 70 · see citing works
Intra- and Inter-Action Understanding via Temporal Action Parsing
Shao, D., Zhao, Y., Dai, B., & Lin, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00081 · Cited by 66 · see citing works
Temporal action segmentation for video encryption
Gao, S., Iu, H. H., Mou, J., Erkan, U., Liu, J., Wu, R., & Tang, X. (2024) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2024.114958 · Cited by 69 · see citing works
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals
Nakamura, K., Yeung, S. Y., Alahi, A., & Fei-Fei, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.721 · Cited by 68 · see citing works
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications
Zhang, L., Zhou, S., Stent, S., & Shi, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19818-2_8 · Cited by 62 · see citing works
BlendFace: Re-designing Identity Encoders for Face-Swapping
Shiohara, K., Yang, X., & Taketomi, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00702 · Cited by 63 · see citing works
Learning Semantics-Preserving Attention and Contextual Interaction for Group Activity Recognition
Tang, Y., Lu, J., Wang, Z., Yang, M., & Zhou, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2914577 · Cited by 62 · see citing works
A review on video-based active and assisted living technologies for automated lifelogging
Climent-Pérez, P., Spinsante, S., Mihailidis, A., & Flórez‐Revuelta, F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112847 · Cited by 61 · see citing works
C2F-TCN: A Framework for Semi- and Fully-Supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284080 · Cited by 60 · see citing works
Recognizing Activities of Daily Living with a Wrist-Mounted Camera
Ohnishi, K., Kanehira, A., Kanezaki, A., & Harada, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.338 · Cited by 60 · see citing works
An Attention-Based Activity Recognition for Egocentric Video
Matsuo, K., Yamada, K., Ueno, S., & Naito, S. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.87 · Cited by 59 · see citing works
Hand segmentation for gesture recognition in EGO-vision
Serra, G., Camurri, M., Baraldi, L., Benedetti, M., & Cucchiara, R. (2013) · doi:10.1145/2505483.2505490 · Cited by 59 · see citing works
Interactive Prototype Learning for Egocentric Action Recognition
Wang, X., Zhu, L., Wang, H., & Yang, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00806 · Cited by 59 · see citing works
Dimiccoli, M., Marín, J., & Thomaz, E. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3161190 · Cited by 59 · see citing works
Predicting the future from first person (egocentric) vision: A survey
Rodin, I., Furnari, A., Mavroeidis, D., & Farinella, G. M. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103252 · Cited by 57 · see citing works
A Sequential Classifier for Hand Detection in the Framework of Egocentric Vision
Betancourt, A. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.92 · Cited by 57 · see citing works
RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks
Gao, S., Li, Z., Qi, H., Cheng, M., & Wang, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183829 · Cited by 57 · see citing works
First-Person Action-Object Detection with EgoNet
Bertasius, G., Park, H. S., Yu, S. X., & Shi, J. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.012 · Cited by 57 · see citing works
ASFormer: Transformer for Action SegmentationPreprint
Yi, F., Wen, H., & Jiang, T. (2021) · British Machine Vision Conference · doi:10.48550/arxiv.2110.08568 · Cited by 319 · see citing works
Continuous Human Action Recognition for Human-machine Interaction: A Review
Gammulle, H., Ahmedt‐Aristizabal, D., Denman, S., Tychsen-Smith, L., Petersson, L., & Fookes, C. (2023) · ACM Computing Surveys · doi:10.1145/3587931 · Cited by 59 · see citing works
Trajectory aligned features for first person action recognition
Singh, S., Arora, C., & Jawahar, C. V. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.031 · Cited by 56 · see citing works
Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home
Likitlersuang, J., Sumitro, E. R., Cao, T., Visée, R. J., Kalsi‐Ryan, S., & Zariffa, J. (2019) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-019-0557-1 · Cited by 56 · see citing works
Object-Centric Spatio-Temporal Pyramids for Egocentric Activity Recognition
McCandless, T., & Grauman, K. L. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.30 · Cited by 55 · see citing works
Temporal Perception and Prediction in Ego-Centric Video
Zhou, Y., & Berg, T. L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.511 · Cited by 55 · see citing works
Damen, D., Leelasawassuk, T., & Mayol-Cuevas, W. W. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.02.016 · Cited by 56 · see citing works
Perceptually-guided deep neural networks for ego-action prediction: Object grasping
González-Díaz, I., Benois‐Pineau, J., Domenger, J., Cattaert, D., & de Rugy, A. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.11.013 · Cited by 55 · see citing works
The KIT Bimanual Manipulation Dataset
Krebs, F., Meixner, A., Patzer, I., & Asfour, T. (2021) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids47582.2021.9555788 · Cited by 53 · see citing works
Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity Detection
Dai, R., Das, S., Sharma, S., Minciullo, L., Garattoni, L., Brémond, F., & Francesca, G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3169976 · Cited by 54 · see citing works
- Authors
- Fathi, A., Ren, X., & Rehg, J. M.
- Affiliations
- Georgia Institute of Technology (US)
- Intel (United States) (US)
- Venue
- CVPR 2011
- Publisher
- IEEE
- Published
- 2011-06
- Pages
- 3281-3288
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2031688197
- Cited by
- 557
- References
- 26
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 47 · 2023: 41 · 2022: 43 · 2021: 57 · 2020: 63 · 2019: 44 · 2018: 36 · 2017: 32 · 2016: 57 · 2015: 31 · 2014: 21 · 2013: 12 · 2012: 10
- FWCI
- 10.47
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- SMARTech Repository (Georgia Institute of Technology) (submittedVersion)
- Funders
- National Science Foundation (0916687)
- Multidisciplinary University Research Initiative
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- egocentric video, housework, object recognition, weakly supervised learning, unsupervised segmentation, hand segmentation, object localization, multiple instance learning, transductive learning, novel object detection, activity recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
