What Would You Expect? Anticipating Egocentric Actions With Rolling-Unrolling LSTMs and Modality Attention
Open accessAbstract
Egocentric action anticipation consists in understanding which objects the camera wearer will interact with in the near future and which actions they will perform. We tackle the problem proposing an architecture able to anticipate actions at multiple temporal scales using two LSTMs to 1) summarize the past, and 2) formulate predictions about the future. The input video is processed considering three complimentary modalities: appearance (RGB), motion (optical flow) and objects (object-based features). Modality-specific predictions are fused using a novel Modality ATTention (MATT) mechanism which learns to weigh modalities in an adaptive fashion. Extensive evaluations on two large-scale benchmark datasets show that our method outperforms prior art by up to +7% on the challenging EPIC-Kitchens dataset including more than 2500 actions, and generalizes to EGTEA Gaze+. Our approach is also shown to generalize to the tasks of early action recognition and action recognition. Our method is ranked first in the public leaderboard of the EPIC-Kitchens egocentric action anticipation challenge 2019. Please see the project web page for code and additional details: http://iplab.dmi.unict.it/rulstm.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1× · see citing works
“For the object branch, we use a Faster R-CNN object detector [20] with a ResNet-101 backbone [22], as implemented in [21].”
MethodGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 4× · see citing works
“The representation function φ1 of the spatial branch computes the feature vector f1,t by extracting features from the last frame It,5 of the video snippet using a Batch Normalized Inception CNN [26] trained for action recognition.”
MethodInfluentialNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1× · see citing works
“Inspired by previous work on attention [3, 62] and multi-modal fusion [41], we introduce a Modality ATTention (MATT) module which computes a set of attention scores indicating the relative importance of each modality for the final prediction.”
Method(2017) · Cited by 11,625 · see citing works
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 1× · see citing works
“Inspired by previous work on attention [3, 62] and multi-modal fusion [41], we introduce a Modality ATTention (MATT) module which computes a set of attention scores indicating the relative importance of each modality for the final prediction.”
MethodTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · cited 4× · see citing works
“However, in contrast with the aforementioned works, we address the problem of egocentric action anticipation and show that approaches designed for action recognition, such as TSN [61] and early/late fusion to merge multi-modal predictions [49] are not directly applicable to the problem of egocentric action anticipatio…”
BackgroundMethodResultInfluentialLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundTemporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,164 · cited 13× · see citing works
“It is worth noting that second best Top-5 verb and noun accuracy are achieved by different methods (i.e., ATSN in the case of verbs and RL in the case of nouns), while both are outperformed by the proposed RU.”
BackgroundMethodResultInfluentialLearning realistic human actions from movies
Ivan Laptev, A., Marszałek, M., Schmid, C., & Rozenfeld, B. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587756 · Cited by 3,481 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundAction Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407 · see citing works
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.213 · Cited by 2,842 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundIvan Laptev, A. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,631 · see citing works
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundA Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658 · cited 1× · see citing works
“Optical flows are extracted using the TVL1 algorithm [63].”
MethodTemporal Relational Reasoning in Videos
Zhou, B., Andonian, A., Oliva, A., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_49 · Cited by 1,104 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundScaling Egocentric Vision: The "Equation missing" Dataset
Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_44 · Cited by 875 · see citing works
Self-citationKitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701 · see citing works
Spatiotemporal Multiplier Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.787 · Cited by 692 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundDetecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248010 · Cited by 666 · cited 3× · see citing works
“Specifically, we build on previous ideas investigated in the context of action recognition such as the use of multiple modalities for video analysis [49], the use of Temporal Segment Networks [61] as a principled way to train CNNs for action recognition, as well as the explicit encoding of object-based features [11, 3…”
BackgroundMethodAnticipating Human Activities Using Object Affordances for Reactive Robotic Response
Koppula, H. S., & Saxena, A. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2430335 · Cited by 596 · see citing works
Human activity prediction: Early recognition of ongoing activities from streaming videos
Ryoo, M. S. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126349 · Cited by 565 · cited 1× · see citing works
“The problem has been widely investigated in the domain of third person vision [2, 4, 6, 9, 19, 23, 24, 39, 46].”
BackgroundAnticipating Visual Representations from Unlabeled Video
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.18 · Cited by 538 · cited 10× · see citing works
“While previous approaches attempted to address these two sub-tasks jointly [2, 8, 39, 58], our method disentangles them using two separate LSTMs.”
BackgroundMethodInfluentialLearning to Recognize Daily Actions Using Gaze
Fathi, A., Li, Y., & Rehg, J. M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_23 · Cited by 396 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundUnderstanding egocentric activities
Fathi, A., Farhadi, A., & Rehg, J. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126269 · Cited by 384 · cited 2× · see citing works
“Specifically, we build on previous ideas investigated in the context of action recognition such as the use of multiple modalities for video analysis [49], the use of Temporal Segment Networks [61] as a principled way to train CNNs for action recognition, as well as the explicit encoding of object-based features [11, 3…”
BackgroundMethodLearning Activity Progression in LSTMs for Activity Detection and Early Detection
Ma, S., Sigal, L., & Sclaroff, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.214 · Cited by 382 · cited 6× · see citing works
“While previous approaches attempted to address these two sub-tasks jointly [2, 8, 39, 58], our method disentangles them using two separate LSTMs.”
BackgroundMethodInfluentialIn the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video
Li, Y., Liu, M., & Rehg, J. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_38 · Cited by 317 · cited 4× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundMethodInfluentialA Hierarchical Representation for Future Action Prediction
Lan, T., Chen, T.-C., & Savarese, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_45 · Cited by 298 · cited 1× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundDelving into egocentric actions
Li, Y., Ye, Z., & Rehg, J. M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298625 · Cited by 285 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundGoing Deeper into First-Person Activity Recognition
Ma, M., Fan, H., & Kitani, K. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.209 · Cited by 281 · cited 2× · see citing works
“Specifically, we build on previous ideas investigated in the context of action recognition such as the use of multiple modalities for video analysis [49], the use of Temporal Segment Networks [61] as a principled way to train CNNs for action recognition, as well as the explicit encoding of object-based features [11, 3…”
BackgroundMethodCar that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models
Jain, A., Koppula, H. S., Raghavan, B., Soh, S., & Saxena, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.364 · Cited by 274 · cited 1× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundRecurrent Neural Networks for driver activity anticipation via sensory-fusion architecture
Jain, A., Singh, A., Koppula, H. S., Soh, S., & Saxena, A. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487478 · Cited by 271 · cited 5× · see citing works
“We also consider baselines obtained adapting approaches proposed for early action recognition to the problem of egocentric action anticipation: the Feedback Network LSTM (FN) proposed in [19], a single LSTM architecture (we use the same parameters of our R-LSTM) trained using the Ranking Loss on Detection Score propos…”
BackgroundMethodInfluentialTemporal segmentation and activity classification from first-person sensing
Spriggs, E., la Torre, F. D., & Hebert, M. (2009) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2009.5204354 · Cited by 234 · see citing works
Max-Margin Early Event Detectors
Hoai, M., & la Torre, F. D. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0683-3 · Cited by 220 · cited 2× · see citing works
“To assess the timeliness of anticipations, we design a new evaluation measure inspired by the AMOC curve [23].”
BackgroundMethodGeest, R. D., Gavves, E., Ghodrati, A., Li, Z., Snoek, C. G. M., & Tuytelaars, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_17 · Cited by 201 · see citing works
Recognize Human Activities from Partially Observed Videos
Cao, Y., Barrett, D. P., Barbu, A., Siddharth, N., Yu, H., Michaux, A., Lin, Y., Dickinson, S., Siskind, J. M., & Wang, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.343 · Cited by 199 · cited 1× · see citing works
“The problem has been widely investigated in the domain of third person vision [2, 4, 6, 9, 19, 23, 24, 39, 46].”
BackgroundRED: Reinforced Encoder-Decoder Networks for Action Anticipation
Gao, J., Yang, Z., & Nevatia, R. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.92 · Cited by 194 · cited 9× · see citing works
“Additionally, we show that other approaches, such as the direct regression of future representations [18, 58], do not achieve satisfactory performance in the considered scenario.”
BackgroundMethodInfluentialFirst Person Action Recognition Using Deep Learned Descriptors
Singh, S., Arora, C., & Jawahar, C. V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.287 · Cited by 192 · cited 1× · see citing works
“Specifically, we build on previous ideas investigated in the context of action recognition such as the use of multiple modalities for video analysis [49], the use of Temporal Segment Networks [61] as a principled way to train CNNs for action recognition, as well as the explicit encoding of object-based features [11, 3…”
MethodEncouraging LSTMs to Anticipate Actions Very Early
Aliakbarian, M. S., Saleh, F., Salzmann, M., Fernando, B., Petersson, L., & Andersson, L. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.39 · Cited by 183 · cited 4× · see citing works
“Additionally, tasks such as action anticipation [18, 32, 58] and early action recognition [2, 9, 39] pose a series of key challenges from a computational perspective.”
BackgroundMethodInfluentialWhen will you do what? - Anticipating Temporal Occurrences of Activities
Farha, Y. A., Richard, A., & Gall, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00560 · Cited by 180 · cited 3× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundMethodLSTA: Long Short-Term Attention for Egocentric Action Recognition
Sudhakaran, S., Escalera, S., & Lanz, O. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01019 · Cited by 165 · cited 3× · see citing works
“01% - reported from [55]), and obtaining performances comparable to state-of-the-art approaches such as Sudhakaran and Lanz [56] (−0.”
BackgroundPooled motion features for first-person videos
Ryoo, M. S., Rothrock, B., & Matthies, L. H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298691 · Cited by 161 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundFirst-Person Activity Forecasting with Online Inverse Reinforcement Learning
Rhinehart, N., & Kitani, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.399 · Cited by 120 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundAction-Reaction: Forecasting the Dynamics of Human Interaction
Huang, D.-A., & Kitani, K. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_32 · Cited by 115 · cited 2× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundTakeo Kanade, A., & Hebert, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2200554 · Cited by 105 · cited 5× · see citing works
“First Person (Egocentric) Vision [29], in particular, offers an interesting scenario to study anticipation problems.”
BackgroundInfluentialDeep Future Gaze: Gaze Anticipation on Egocentric Videos Using Adversarial Networks
Zhang, M., Ma, K. T., Lim, J., Zhao, Q., & Feng, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.377 · Cited by 104 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundAdding Attentiveness to the Neurons in Recurrent Neural Networks
Zhang, P., Xue, J., Lan, C., Zeng, W., Gao, Z., & Zheng, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_9 · Cited by 103 · cited 2× · see citing works
“27 eleGAtt [66] 57.”
Robot-Centric Activity Prediction from First-Person Videos
Ryoo, M. S., Fuchs, T. J., Xia, L., Aggarwal, J. K., & Matthies, L. H. (2015) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2696454.2696462 · Cited by 99 · see citing works
Choosing smartly: Adaptive multimodal fusion for object detection in changing environments
Mees, O., Eitel, A., & Burgard, W. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759048 · Cited by 96 · cited 1× · see citing works
“Inspired by previous work on attention [3, 62] and multi-modal fusion [41], we introduce a Modality ATTention (MATT) module which computes a set of attention scores indicating the relative importance of each modality for the final prediction.”
MethodEgocentric Future Localization
Park, H. S., Hwang, J.-J., Niu, Y., & Shi, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.508 · Cited by 94 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundNext-active-object prediction from egocentric videos
Furnari, A., Battiato, S., Grauman, K. L., & Farinella, G. M. (2017) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2017.10.004 · Cited by 92 · cited 2× · see citing works
“While a direct comparison of the proposed approach with the aforementioned works is unfeasible due to the lack of a common framework, our method incorporates some ideas from past approaches, such as the analysis of past actions [47] and the detection of the objects present in the scene to infer future actions [17].”
BackgroundMethodSelf-citationModeling Temporal Structure with LSTM for Online Action Detection
Geest, R. D., & Tuytelaars, T. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00173 · Cited by 92 · cited 5× · see citing works
“The method proposed in [19] has been implemented considering the most performing architecture investigated by the authors, which comprises the “optional” LSTM layer and performs fusion by concatenation.”
BackgroundMethodResultInfluentialSequential Max-Margin Event Detectors
Huang, D., Yao, S., Wang, Y., & la Torre, F. D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_27 · Cited by 91 · cited 1× · see citing works
“The problem has been widely investigated in the domain of third person vision [2, 4, 6, 9, 19, 23, 24, 39, 46].”
BackgroundJoint Prediction of Activity Labels and Starting Times in Untrimmed Videos
Mahmud, T., Hasan, M., & Roy–Chowdhury, A. K. (2017) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.616 · Cited by 84 · cited 2× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundMethodWhat will Happen Next? Forecasting Player Moves in Sports Videos
Felsen, P., Agrawal, P., & Malik, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.362 · Cited by 76 · cited 1× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 25, 27, 28, 31, 32, 33, 40, 58, 64].”
BackgroundFurnari, A., Battiato, S., & Farinella, G. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_24 · Cited by 63 · cited 8× · see citing works
“Differently from previous works which considered a fixed anticipation time [8, 16, 58], the proposed method can anticipate an action at multiple anticipation times, e.”
BackgroundMethodResultInfluentialSelf-citationGenerating Notifications for Missing Actions: Don't Forget to Turn the Lights Off!
Soran, B., Farhadi, A., & Shapiro, L. G. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.530 · Cited by 59 · cited 2× · see citing works
“, to notify a user who is performing the wrong action in a known workflow [53]).”
BackgroundTrajectory aligned features for first person action recognition
Singh, S., Arora, C., & Jawahar, C. V. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.031 · Cited by 56 · cited 1× · see citing works
“Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
BackgroundTemporal Perception and Prediction in Ego-Centric Video
Zhou, Y., & Berg, T. L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.511 · Cited by 55 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundVisual Forecasting by Imitating Dynamics in Natural Sequences
Zeng, K.-H., Shen, W. B., Huang, D.-A., Sun, M., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.326 · Cited by 53 · cited 1× · see citing works
“Previous works investigated different forms of action and activity anticipation from third person video [1, 15, 18, 22, 24, 25, 28, 29, 30, 37, 54, 59].”
BackgroundKrishnaCam: Using a longitudinal, single-person, egocentric dataset for scene understanding tasks
Singh, K. K., Fatahalian, K., & Efros, A. A. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477717 · Cited by 46 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundAm I Done? Predicting Action Progress in Videos
Becattini, F., Uricchio, T., Seidenari, L., Ballan, L., & Bimbo, A. D. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3402447 · Cited by 39 · see citing works
Attention is All We Need: Nailing Down Object-centric Attention for Egocentric Activity RecognitionPreprint
Sudhakaran, S., & Lanz, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11794 · Cited by 38 · see citing works
Long-Term Activity Forecasting Using First-Person Vision
Bokhari, S. Z., & Kitani, K. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54193-8_22 · Cited by 21 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
BackgroundForecasting Hand and Object Locations in Future Frames.Preprint
Fan, C., Lee, J., & Ryoo, M. S. (2017) · arXiv (Cornell University) · Cited by 13 · see citing works
Forecasting Hands and Objects in Future FramesPreprint
Fan, C., Lee, J., & Ryoo, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07328 · Cited by 1 · cited 1× · see citing works
“Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 39, 41, 43, 46, 49, 60, 63].”
Background
Flagged first, then by citations
Research works printed but not linked: 6
- D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray. Scaling egocentric vision: The epic- kitchens dataset. In European Conference on Computer Vi- sion, pages 720-736, 2018.
- Failure action anticipation example qualitative results (best seen on screen). centric activities. In International Conference on Computer Vision, pages 407-414, 2011.
- R. Girshick. Fast R-CNN. In International Conference on Computer Vision, pages 1440-1448, 2015.
- Y. Li, Z. Ye, and J. M. Rehg. Delving into egocentric ac- tions. In Computer Vision and Pattern Recognition, pages 287-295, 2015.
- E. H. Spriggs, F. De La Torre, and M. Hebert. Temporal seg- mentation and activity classification from first-person sens- ing. In Computer Vision and Pattern Recognition Workshops, pages 17-24, 2009.
- H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Dense trajec- tories and motion boundary descriptors for action recogni- tion. International Journal of Computer Vision, 103(1):60- 79, 2013.
Cited by
Sorted by citations
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · see citing works
“[76] Antonino Furnari and Giovanni Maria Farinella.”
BackgroundSelf-citationTemporal Pyramid Network for Action Recognition
Yang, C., Xu, Y., Shi, J., Dai, B., & Zhou, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00067 · Cited by 475 · see citing works
“Following [8], we randomly select 232 videos (23439 segments) for training and 40 videos (4979 segments) for validation.”
MethodYadav, S. K., Tiwari, K., Pandey, H. M., & Akbar, S. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106970 · Cited by 328 · see citing works
VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation
Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00169 · Cited by 264 · see citing works
“However, we believe that it has a huge potential in addressing other tasks which require sequential interactions/decisions, such as language and visual dialog [5, 8, 21, 25, 45], dialog navigation [9, 40, 41] and action anticipation for reactive robot response [1, 13, 20].”
BackgroundAnticipative Video Transformer
Girdhar, R., & Grauman, K. L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01325 · Cited by 214 · see citing works
“First, we compare to the state-of-theart RULSTM method using only our AVT (head) model applied to the exact same features from TSN [88] trained for classification on EK100.”
BackgroundMethodInfluentialGate-Shift Networks for Video Action Recognition
Sudhakaran, S., Escalera, S., & Lanz, O. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00118 · Cited by 197 · see citing works
“63 RU-LSTM [10] ImageNet 56.”
Benchmark for Evaluating Pedestrian Action Prediction
Kotseruba, I., Rasouli, A., & Tsotsos, J. K. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00130 · Cited by 163 · see citing works
Audiovisual SlowFast Networks for Video RecognitionPreprint
Xiao, F., Lee, Y. J., Grauman, K. L., Malik, J., & Feichtenhofer, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08740 · Cited by 195 · see citing works
Temporal Aggregate Representations for Long-Range Video Understanding
Şener, F., Singhania, D., & Yao, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_10 · Cited by 139 · see citing works
“We experiment on Breakfast [18], 50Salads [34] and EPIC-Kitchens [5].”
BackgroundMethodInfluentialLiu, M., Tang, S., Li, Y., & Rehg, J. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_41 · Cited by 125 · see citing works
“We report Top1/Mean Class accuracy on EGTEA as in [35] and Top1/Top5 accuracy as on EPIC-Kitchens following [15].”
InfluentialAffordances from Human Videos as a Versatile Representation for Robotics
Bahl, S., Mendonca, R., Chen, L., Jain, U., & Pathak, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01324 · Cited by 119 · see citing works
“Similarly, in action anticipation and human motion forecasting, high-level semantic or low level actions are predicted using visual history [1, 11,19,22,31–33,37,40,46–48,55,58,72,75,100,119,120].”
BackgroundA survey on human-aware robot navigation
Möller, R., Furnari, A., Battiato, S., Härmä, A., & Farinella, G. M. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103837 · Cited by 108 · see citing works
“[255] utilizes LSTMs to predict future actions in a domestic setting.”
MethodSelf-citationSymbiotic Attention for Egocentric Action Recognition With Object-Centric Alignment
Wang, X., Zhu, L., Wu, Y., & Yang, Y. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3015894 · Cited by 104 · see citing works
“Besides, some researchers [35], [36] focus on the egocentric action prediction task, which predicts the near future egocentric action before it occurs.”
BackgroundEgo-Topo: Environment Affordances From Egocentric Video
Nagarajan, T., Li, Y., Feichtenhofer, C., & Grauman, K. L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00024 · Cited by 100 · see citing works
EPIC-KITCHENS-100Preprint
Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Kazakos, E., Ma, J., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13256 · Cited by 111 · see citing works
“We use our prior work [81] as a baseline.”
MethodSelf-citationJoint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos
Liu, S., Tripathi, S., Majumdar, S., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00328 · Cited by 82 · see citing works
“What space should the model predict on? Recent approaches [25,26,28,72] have been proposed to predict the discrete future action category given a sequence of frames as inputs, namely action anticipation.”
BackgroundMethodInfluentialIn the Eye of the Beholder: Gaze and Actions in First Person Video
Li, Y., Liu, M., & Rehg, J. M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051319 · Cited by 91 · see citing works
“Ourdataset is publicly available(1) andhas been considered by several recentworks [22], [23], [24], [25], [26], [27].”
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos
Li, Y., Nagarajan, T., Xiong, B., & Grauman, K. L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00687 · Cited by 73 · see citing works
“Model architectures proposed for these tasks include multi-stream networks [46, 38, 35, 68], recurrent networks [22, 21, 65], 3D conv nets [53, 44] and spatially grounded topological graph models [50].”
BackgroundAnticipating human actions by correlating past with the future with Jaccard similarity measures
Fernando, B., & Herath, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01302 · Cited by 72 · see citing works
“Indeed, encoder-decoder type of architecture is a natural choice for action anticipation and shown to be very effective for action anticipation [4, 6].”
BackgroundMethodInfluentialProcedure Planning in Instructional Videos
Chang, C.-Y., Huang, D.-A., Xu, D., Adeli, E., Fei-Fei, L., & Niebles, J. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_20 · Cited by 70 · see citing works
“The others ask the question “What is going to happen?”, such as early action recognition [9, 35], action label prediction [1, 6, 9, 14, 22, 27, 29, 36], video prediction [18–20, 24, 26, 34], etc.”
BackgroundLearning to Anticipate Egocentric Actions by Imagination
Wu, Y., Zhu, L., Wang, X., Yang, Y., & Wu, F. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3040521 · Cited by 68 · see citing works
A Survey of Human Action Recognition and Posture Prediction
Ma, N., Wu, Z., Cheung, Y., Guo, Y., Gao, Y., Li, J., & Jiang, B. (2022) · Tsinghua Science & Technology · doi:10.26599/tst.2021.9010068 · Cited by 70 · see citing works
Future Transformer for Long-term Action Anticipation
Gong, D., Lee, J., Kim, M., Ha, S. J., & Cho, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00306 · Cited by 67 · see citing works
Memory-and-Anticipation Transformer for Online Action Understanding
Wang, J., Chen, G., Huang, Y., Wang, L., & Lu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01271 · Cited by 62 · see citing works
“On EK100, following [14], we resample the videos at 30 FPS and then fine-tune the two-stream TSN on the classification task.”
BackgroundMethodInfluentialEgocentric Vision-based Action Recognition: A survey
Núñez-Marcos, A., Azkune, G., & Arganda‐Carreras, I. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.081 · Cited by 61 · see citing works
Predicting the future from first person (egocentric) vision: A survey
Rodin, I., Furnari, A., Mavroeidis, D., & Farinella, G. M. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103252 · Cited by 57 · see citing works
“Several papers used simple bag-of-words representations of objects present in the frame [10, 49, 53], while others encoded objects in bounding-box representations [109], or even”
BackgroundMethodInfluentialSelf-citationForecasting Action Through Contact Representations From First Person Video
Dessalene, E., Devaraj, C., Maynord, M., Fermuller, C. M., & Aloimonos, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3055233 · Cited by 53 · see citing works
“RULSTM outperforms our approach after p ¼ 50%.”
BackgroundMethodResultInfluentialAnticipative Feature Fusion Transformer for Multi-Modal Action Anticipation
Zhong, Z., Schneider, D., Voit, M., Stiefelhagen, R., & Beyerer, J. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00601 · Cited by 52 · see citing works
“We use the existing object and optical flow features provided by [16].”
BackgroundMethodInfluentialAn Outlook into the Future of Egocentric Vision
Plizzari, C., Goletto, G., Furnari, A., Bansal, S., Ragusa, F., Farinella, G. M., Damen, D., & Tommasi, T. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02095-7 · Cited by 52 · see citing works
“The dataset has been used to address different tasks such as anticipation (Furnari and Farinella 2019; Girdhar and Grauman 2021; Zhong et al 2023), action recognition (Kazakos et al 2021; Fathi et al 2012b), procedural learning (Bansal et al 2022), and future hand masks prediction (Jia et al 2022b).”
BackgroundMethodInfluentialSelf-citationSelf-Regulated Learning for Egocentric Video Activity Anticipation
Qi, Z., Wang, S., Su, C., Su, L., Huang, Q., & Tian, Q. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059923 · Cited by 57 · see citing works
“Activity anticipation for egocentric vision has been extensively studied in [4], [9], [10], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36].”
BackgroundMethodInfluentialVideo Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
Chen, G., Huang, Y., Xu, J., Pei, B., Wang, J., Chen, Z., Li, Z., Lu, T., & Wang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02597-y · Cited by 90 · see citing works
“…action detection [12, 51, 67], temporal action segmentation [25, 50, 121], dense video captioning [108, 119], and temporal action prediction [33, 102]; and (2) acting as long-term memory mechanisms [114, 115] to enhance short-term models in action recognition and spatiotemporal detection…”
Visual Object Tracking in First Person Vision
Dunnhofer, M., Furnari, A., Farinella, G. M., & Micheloni, C. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01694-6 · Cited by 47 · see citing works
“ject interaction [21] and early action recognition [40], or action anticipation [19], require real-time computation, we evaluate trackers in such a setting by following the instruc-”
BackgroundMethodSelf-citationHuang, Y., Guo, C., Xu, J., Zhang, M., Yang, L., Pei, B., Zhang, H., Lu, D., Wang, Y., Wang, L., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02084 · Cited by 43 · see citing works
“The unique recording perspective of egocentric videos presents a series of challenges including but not limited to action understanding [13, 26, 27, 40, 50, 93, 96, 113, 120, 121, 125], hand detection [12, 29, 107, 153], and video-language understanding [41, 48].”
Long-Term Anticipation of Activities with Cycle Consistency
Farha, Y. A., Ke, Q., Schiele, B., & Gall, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-71278-5_12 · Cited by 42 · see citing works
“There is a parallel line of research addressing the anticipation task in egocentric videos [8,9].”
BackgroundTSANet: Forecasting traffic congestion patterns from aerial videos using graphs and transformers
Kumar, K. N., Roy, D., Suman, T. A., Vishnu, C., & Mohan, C. K. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110721 · Cited by 40 · see citing works
Multitask Learning to Improve Egocentric Action Recognition
Kapidis, G., Poppe, R. W., van Dam, E., Noldus, L. P. J. J., & Veltkamp, R. C. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00540 · Cited by 38 · see citing works
“In [11], the attention mechanism weighs the importance of the input modalities to select optimal features for action anticipation and recognition.”
BackgroundA Recurrent Vision-and-Language BERT for NavigationPreprint
Hong, Y., Wu, Q., Qi, Y., Rodríguez-Opazo, C., & Gould, S. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13922 · Cited by 38 · see citing works
“However, we believe that it has a huge potential in addressing other tasks which require sequential interactions/decisions, such as language and visual dialog [5, 8, 21, 25, 45], dialog navigation [9, 40, 41] and action anticipation for reactive robot response [1, 13, 20].”
BackgroundOnline human action detection and anticipation in videos: A survey
Hu, X., Dai, J., Li, M., Peng, C., Li, Y., & Du, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.03.069 · Cited by 36 · see citing works
Ragusa, F., Furnari, A., & Farinella, G. M. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103764 · Cited by 36 · see citing works
Self-citation4D Human Body Capture from Egocentric Video via 3D Scene Grounding
Liu, M., Yang, D., Zhang, Y., Cui, Z., Rehg, J. M., & Tang, S. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00101 · Cited by 39 · see citing works
“Substantial progress has been made in understanding human-object interaction [41, 11, 9, 30, 28, 13, 29, 32, 36] from egocentric videos.”
Multi-Dataset, Multitask Learning of Egocentric Vision Tasks
Kapidis, G., Poppe, R. W., & Veltkamp, R. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3061479 · Cited by 35 · see citing works
“Explicit attention modeling mechanisms are increasingly common in egocentric video action recognition [14], [54], [55], [65], [66], [70], [71], [72], [73], [74], [75], [76], [77] to influence the network towards focusing on the significant spatio-temporal features of videos.”
BackgroundHutchinson, M., & Gadepally, V. N. (2021) · IEEE Access · doi:10.1109/access.2021.3115476 · Cited by 42 · see citing works
“Conv3D [222] which uses a C3D to generate unseen features for prediction, RGN [223] which uses a recursive generation and prediction scheme with a Kalman filter during training, and RU-LSTM [115], [224] which uses a multimodal rolling-unrolling encoder-decoder with modality attention.”
MethodDeep Learning for Vision-based Prediction: A SurveyPreprint
Rasouli, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00095 · Cited by 44 · see citing works
“Trajectory prediction applications like many other sequence prediction tasks heavily rely on recurrent architectures such as LSTMs [87], [101], [137], [138], [139], [140], [141], [142], [143], [144], [145], [146], [147], [148], [149], [150], [151], [152], [153], [154], [155], [156], [157], [158], [159], [160], and GRU…”
BackgroundInfluentialAction Anticipation Using Pairwise Human-Object Interactions and Transformers
Roy, D., & Fernando, B. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3113114 · Cited by 34 · see citing works
“Multi-modal transformer performs poorly for verb anticipation compared to temporal aggregation [52] or RU-LSTM [16] in-terms of top-1 accuracy.”
BackgroundMethodInfluentialMultimodal Distillation for Egocentric Action Recognition
Radevski, G., Grujicic, D., Blaschko, M. B., Moens, M., & Tuytelaars, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00481 · Cited by 32 · see citing works
“Its applications are numerous, ranging from recognizing [61] or anticipating [14] actions, to more complex tasks such as recognizing egocentric object-state changes, localizing action instances of a particular video moment [21], etc.”
BackgroundRethinking Learning Approaches for Long-Term Action Anticipation
Nawhal, M., Jyothi, A. A., & Mori, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_32 · Cited by 32 · see citing works
Deep Multimodal Complementarity Learning
Wang, D., Zhao, T., Yu, W., Chawla, N. V., & Jiang, M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3165180 · Cited by 31 · see citing works
“Modality refers to the form in which the data object is presented [12], [13].”
With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition
Kazakos, E., Huh, J., Nagrani, A., Zisserman, A., & Damen, D. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.171 · Cited by 31 · see citing works
“A few works have considered temporal context to improve the models’ performance for action anticipation [19, 55], action recognition [10, 51, 69, 74], and object detection [3, 6].”
BackgroundEgocentric Early Action Prediction via Multimodal Transformer-Based Dual Action Prediction
Guan, W., Song, X., Wang, K., Wen, H., Ni, H., Wang, Y., & Chang, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3248271 · Cited by 29 · see citing works
ExtDM: Distribution Extrapolation Diffusion Model for Video Prediction
Zhang, Z., Hu, J., Cheng, W., Paudel, D. P., & Yang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01827 · Cited by 28 · see citing works
“Advanced works in video prediction [20,22,64,66] pro-pose to capture the dynamic change in the video [2, 18, 36, 60].”
BackgroundMulti-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videos
Zatsarynna, O., Farha, Y. A., & Gall, J. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00254 · Cited by 28 · see citing works
“While some works try to anticipate several actions into the future (long-term anticipation) [1,11,15,25], others aim at anticipating only the next action at a fixed anticipation time based on the recent observations preceding it (short-term anticipation) [14, 20, 23, 45].”
BackgroundMethodInfluentialLiu, T., & Lam, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01353 · Cited by 26 · see citing works
Fall Detection With a Nonintrusive and First-Person Vision Approach
Wang, X., Talavera, E., Karastoyanova, D., & Azzopardi, G. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3314828 · Cited by 24 · see citing works
“D body pose estimation [9], sentiment recognition [10], and anticipation [11], showcasing their potential impact.”
BackgroundEgocentric Prediction of Action Target in 3D
Li, Y., Cao, Z., Liang, A. S., Liang, B., Chen, L., Zhao, H., & Feng, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02033 · Cited by 24 · see citing works
“Thanks to these well-built datasets, various egocentric vision tasks have been proposed and studied such as action recognition [25, 34, 51], action anticipation [16, 20, 36], video summarization [30, 38, 55], hand-object interaction parsing [3, 6, 41], social interaction analysis [13, 42, 56], and egocentric object de…”
BackgroundWeakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
Ghoddoosian, R., Dwivedi, I., Agarwal, N., Choi, C., & Dariush, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01341 · Cited by 24 · see citing works
“Online action understanding has been studied in various problems such as online action detection [10, 55, 59], start of action detection [15,44] or anticipation [1,13,14,23,30].”
BackgroundTogether Recognizing, Localizing and Summarizing Actions in Egocentric Videos
Sahu, A., & Chowdhury, A. S. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3070732 · Cited by 23 · see citing works
Is First Person Vision Challenging for Object Tracking?
Dunnhofer, M., Furnari, A., Farinella, G. M., & Micheloni, C. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00304 · Cited by 21 · see citing works
“Since many FPV tasks such as object interaction [20] and early action recognition [34], or action anticipation [19], require real-time computation, we evaluated the ability of trackers to provide their object localization in such a setting.”
InfluentialSelf-citationJaramillo, I. E., Chola, C., Jeong, J.-G., Oh, J.-H., Jung, H., Lee, J.-H., Lee, W. H., & Kim, T. (2023) · Sensors · doi:10.3390/s23146491 · Cited by 22 · see citing works
“Meanwhile, in [16], a Long Short-Term Memory Networks (LSTM) model was used to obtain an anticipated action prediction using an egocentric camera mounted on the head of the subjects.”
MethodBridging Requirements, Planning, and Evaluation: A Review of Social Robot Navigation
Karwowski, J., Szynkiewicz, W., & Niewiadomska‐Szynkiewicz, E. (2024) · Sensors · doi:10.3390/s24092794 · Cited by 21 · see citing works
An Egocentric Action Anticipation Framework via Fusing Intuition and Analysis
Zhang, T., Min, W., Zhu, Y., Rui, Y., & Jiang, S. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413964 · Cited by 21 · see citing works
“This kind of ability to act as the first person is closely related to the task of egocentric (i.e., First-Person Vision [4]) action anticipation, which has attracted increasing attention in recent years [5, 9, 10, 20].”
BackgroundMethodInfluentialTaniguchi, T., Hafi, L. E., Hagiwara, Y., Taniguchi, A., Shimada, N., & Nishiura, T. (2021) · Advanced Robotics · doi:10.1080/01691864.2021.1928552 · Cited by 20 · see citing works
Action anticipation using latent goal learning
Roy, D., & Fernando, B. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00088 · Cited by 20 · see citing works
“The training set is divided into 232 videos for training (23,493 segments), and 40 videos for validation (4,979 segments) based on the splits provided by [9].”
BackgroundMethodLearning to Recognize Actions on Objects in Egocentric Video With Attention Dictionaries
Sudhakaran, S., Escalera, S., & Lanz, O. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3058649 · Cited by 20 · see citing works
“ory (LSTM) [4], [14], [66] and its convolutional variant (ConvLSTM) [56], [57] are proposed to encode the temporal order of features extracted from a sequence of frames.”
BackgroundTransferring Knowledge From Text to Video: Zero-Shot Anticipation for Procedural Actions
Sener, F., Saraf, R., & Yao, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3218596 · Cited by 20 · see citing works
“Methods for anticipation include RNNs for encoding the observations [41], [44], predicting future features and performing future action classification”
BackgroundMethodForecasting Human Object Interaction: Joint Prediction of Motor Attention and Egocentric Activity.
Liu, M., Tang, S., Li, Y., & Rehg, J. M. (2019) · Cited by 18 · see citing works
Egocentric Action Recognition by Automatic Relation Modeling
Li, H., Zheng, W., Zhang, J., Hu, H., Lu, J., & Lai, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148790 · Cited by 18 · see citing works
“Researchers have explored different topics within the field of egocentric vision, including cross-view matching and synthesis [67], [68], [69], person localization [70], video summarization [71] and action forecasting [72], [73].”
BackgroundEgocentric Object Manipulation GraphsPreprint
Dessalene, E., Maynord, M., Devaraj, C., Fermuller, C. M., & Aloimonos, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03201 · Cited by 18 · see citing works
“For the experimental analysis of our approach, we use the training videos to create our validation set, randomly splitting the public EPIC-Kitchens training set into 232 videos for training and 40 videos for validation as in [3].”
BackgroundMethodPretrained Language Models as Visual Planners for Human Assistance
Patel, D., Eghbal-zadeh, H., Kamra, N., Iuzzolino, M. L., Jain, U., & Desai, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01404 · Cited by 18 · see citing works
“…entails future action sequence prediction for a video and is closely related to anticipation in video understanding, including future localization [22, 40, 78], future frame prediction [77, 46, 53], next active object estimation [21, 5, 25, 67, 4, 24, 44], as well as short-and long-term action…”
BackgroundPrediction and Description of Near-Future Activities in Video
Mahmud, T., Billah, M., Hasan, M., & Roy–Chowdhury, A. K. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103230 · Cited by 17 · see citing works
“For EPIC-KITCHENS, we use the pre-extracted RGB and flow features for the sequential activity context and the faster R-CNN-based object features for the scene context provided by [17, 99].”
MethodTTPP: Temporal Transformer with Progressive Prediction for efficient action anticipation
Wang, W., Peng, X., Su, Y., Qiao, Y., & Cheng, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.087 · Cited by 17 · see citing works
“In deep learning era, recent works turn to predict future actions in practical untrimmed video streams [11], [13], [56], [62], and mainly repurpose sequential models from the natural language processing (NLP) domain, like long short-term memory (LSTM) [15] and gated recurrent neural networks [6].”
BackgroundTAMformer: Multi-Modal Transformer with Learned Attention Mask for Early Intent Prediction
Osman, N., Camporese, G., & Ballan, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095781 · Cited by 17 · see citing works
“Recently, anticipating actions on videos gained attention given the development of new methods [1, 25, 3, 26, 27], datasets [28, 29, 30], and applications such as autonomous driving, human-robot interaction and virtual reality.”
BackgroundMethodStillFast: An End-to-End Approach for Short-Term Object Interaction Anticipation
Ragusa, F., Farinella, G. M., & Furnari, A. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00371 · Cited by 16 · see citing works
“[10, 11] proposed a model based on LSTM networks to encode past features and predict future actions from egocentric videos.”
BackgroundSelf-citationEgocentric Activity Recognition and Localization on a 3D Map
Liu, M., Ma, L., Somasundaram, K. K., Li, Y., Grauman, K. L., Rehg, J. M., & Li, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_36 · Cited by 16 · see citing works
“• I3D+Obj uses object detection results from a pre-trained object detector [62] as contextual cues as in [10].”
Predicting short-term next-active-object through visual attention and hand position
Jiang, J., Nan, Z., Chen, H., Chen, S., & Zheng, N. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.12.069 · Cited by 15 · see citing works
Multi-label affordance mapping from egocentric vision
Mur-Labadia, L., Guerrero, J. J., & Martínez-Cantín, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00483 · Cited by 15 · see citing works
“Recent advances include anticipating future actions [23, 26, 1], model the hands-object manipulation [22, 77, 27, 16], detect the change in an object state [2], identify interaction hotspots [20, 54] or create topological maps [55].”
BackgroundLeveraging Next-Active Objects for Context-Aware Anticipation in Egocentric Videos
Thakur, S. K., Beyan, C., Morerio, P., Murino, V., & Bue, A. D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00846 · Cited by 15 · see citing works
“In the case of EpicKitchen-100 dataset [4], we compare our proposed method against SOTA for action anticipation task, as described in [5, 12].”
BackgroundMethodInfluentialWang, X., Fathaliyan, A. H., & Santos, V. J. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.567571 · Cited by 16 · see citing works
“Whether using hand-crafted features (Fathi et al., 2011, 2012; Fathi and Rehg, 2013; Matsuo et al., 2014; Soran et al., 2015; Ma et al., 2016; Furnari and Farinella, 2019) or learning end-to-end models (Li et al.”
MethodGenerative Adversarial Network for Future Hand Segmentation from Egocentric Video
Jia, W., Liu, M., & Rehg, J. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_37 · Cited by 15 · see citing works
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 15 · see citing works
Moniruzzaman, M., Yin, Z., He, Z., Leu, M. C., & Qin, R. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232021 · Cited by 14 · see citing works
“Recently, some works [29], [30], [31], [32], and [33] investigated the anticipation of the next action immediately following the current one, but such short-term near-future anticipation is not sufficient for many real-world applications.”
BackgroundMethodInfluentialContext-Enhanced Memory-Refined Transformer for Online Action Detection
Pang, Z., Şener, F., & Yao, A. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00813 · Cited by 13 · see citing works
Dance self-learning application and its dance pose evaluations
Choi, J.-H., Lee, J., & Nasridinov, A. (2021) · ACM Symposium on Applied Computing · doi:10.1145/3412841.3441980 · Cited by 13 · see citing works
“These studies mainly deal with methods associated with the classification of actions [8, 15, 17, 30], prediction of future actions [2, 3, 7, 11, 16, 29], and the application of actions [14, 19, 24] pertaining to human posture information.”
BackgroundThapar, D., Nigam, A., & Arora, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00232 · Cited by 13 · see citing works
“To validate the performance of activity recognition on FPSI dataset we have used Poleg et al. [31] (referred to as “Po-leg”), and for EPIC-Kitchens, we have used [44], [11], and [10] and refer to them as “Verma”, “Ghadiyaram”, and “Furnari” respectively.”
BackgroundMethodLearning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognition
Wang, Q., Zhao, L., Yuan, L., Liu, T., & Peng, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00306 · Cited by 12 · see citing works
“To verify the quality of model learning, there are multiple egocentricrelated tasks, including action recognition [29, 42, 46], video summarization [26, 20], engagement detection [41], scene affordance [30], gaze prediction [14], video question answering [7], and activity anticipation [31, 9].”
BackgroundWhat If We Could Not See? Counterfactual Analysis for Egocentric Action Anticipation
Zhang, T., Min, W., Yang, J., Liu, T., Jiang, S., & Rui, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/182 · Cited by 12 · see citing works
“As for visual features, we use the RGB frame features, optical flow frame features and object-based features provided by [Furnari and Farinella, 2019] whose dimensions are 1,024, 1,024 and 352 respectively.”
BackgroundMethodResultInfluentialMultimodal Global Relation Knowledge Distillation for Egocentric Action Anticipation
Huang, Y., Yang, X., & Xu, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475327 · Cited by 12 · see citing works
“Similar to the work [9], we employ three kinds of features (i.”
BackgroundMethodInfluentialLearning Temporal Dynamics from Cycles in Narrated Video
Epstein, D., Wu, J., Schmid, C., & Sun, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00151 · Cited by 12 · see citing works
“Recent work extends this, using contrastive learning or other techniques to predict future representations [13, 15, 16, 47, 60].”
BackgroundRodin, I., Furnari, A., Mavroeidis, D., & Farinella, G. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06433-3_29 · Cited by 11 · see citing works
“, [6]) achieve limited performance in the untrimmed anticipation scenario.”
BackgroundMethodResultInfluentialSelf-citationRecognition of Grasping Patterns Using Deep Learning for Human–Robot Collaboration
Amaral, P., da Silva, F. M. T. P., & Santos, V. M. F. (2023) · Sensors · doi:10.3390/s23218989 · Cited by 10 · see citing works
Towards Understanding Future: Consistency Guided Probabilistic Modeling for Action Anticipation
Xie, Z., Shi, Y., Wu, K., Cheng, Y., & Guo, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28442 · Cited by 9 · see citing works
“The temporal feature can be enhanced by considering UnRolling model (Furnari and Farinella 2019), and high-order frames (Tai et al. 2021).”
BackgroundMethodUncertainty-Boosted Robust Video Activity Anticipation
Qi, Z., Wang, S., Zhang, W., & Huang, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3393730 · Cited by 10 · see citing works
“Concretely, for EK55 and EK100, we use the RGB feature from TSN [79], the Flow featurefromTSN,andtheOBJfeaturefromFRCNN[80],which areprovidedby[5].”
BackgroundMethodInfluentialInstance Tracking in 3D Scenes from Egocentric Videos
Zhao, Y., Ma, H., Kong, S., & Fowlkes, C. C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02071 · Cited by 9 · see citing works
“Recently, more tasks have emerged due to the increasing interest in ego-centric videos, such as action anticipation [18, 19, 51], privacy protection [13, 53, 61], and estimating social interactions [33, 42, 74].”
BackgroundFirst-Person Hand Action Recognition Using Multimodal Data
Li, R., Wang, H., Liu, Z., Cheng, N., & Xie, H. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3108136 · Cited by 9 · see citing works
“Examples of this approach can be found in [28], [44], [46], and [47].”
BackgroundMethodInfluentialLearning to Discriminate Information for Online Action Detection: Analysis and Application
Lee, S., Eun, H., Moon, J., Choi, S., Kim, Y., Jung, C., & Kim, C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204808 · Cited by 9 · see citing works
VLMAH: Visual-Linguistic Modeling of Action History for Effective Action Anticipation
Manousaki, V., Bacharidis, K., Papoutsakis, K. E., & Argyros, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00206 · Cited by 9 · see citing works
“The former type of features are computed based on the occurrences of the objects in each frame following the work of [12, 13].”
InfluentialUnified Recurrence Modeling for Video Action Anticipation
Tai, T., Fiameni, G., Lee, C., See, S. C. W., & Lanz, O. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956467 · Cited by 8 · see citing works
“The data split for the action anticipation tasks is inherited from [19], containing 23492 segments for training and 4979 for validation.”
BackgroundMethodInfluentialRamazanova, M., Escorcia, V. A., Heilbron, F. C., Zhao, C., & Ghanem, B. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00516 · Cited by 8 · see citing works
“For instance, at being unedited and continuous, actions unfold, with a more often than not, predictable sequence [21,22,25].”
BackgroundCross-view action recognition understanding from exocentric to egocentric perspective
Truong, T.-D., & Luu, K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128731 · Cited by 7 · see citing works
Joint-Based Action Progress Prediction
Pucci, D., Becattini, F., & Bimbo, A. D. (2023) · Sensors · doi:10.3390/s23010520 · Cited by 7 · see citing works
“There are several different techniques that can be used to perform action recognition in images or videos, exploiting different input modalities [4,9,10].”
MethodRazali, H., & Fernando, B. (2021) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw52041.2021.00027 · Cited by 6 · see citing works
“Future frame prediction is useful for various application such as early action prediction[23, 27, 24] and action anticipation [8, 18, 20].”
- Authors
- Furnari, A., & Farinella, G.
- Affiliations
- University of Catania (IT)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 6251-6260
- OpenAlex
- W2945026633
- Cited by
- 182
- References
- 68
- Citations by year
- 2026: 11 · 2025: 20 · 2024: 24 · 2023: 27 · 2022: 32 · 2021: 45 · 2020: 21 · 2019: 2
- FWCI
- 8.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- multimodal fusion, optical flow, EPIC-KITCHENS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
