The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Modeling Actions through State Changes
Abstract
In this paper we present a model of action based on the change in the state of the environment. Many actions involve similar dynamics and hand-object relationships, but differ in their purpose and meaning. The key to differentiating these actions is the ability to identify how they change the state of objects and materials in the environment. We propose a weakly supervised method for learning the object and material states that are necessary for recognizing daily actions. Once these state detectors are learned, we can apply them to input videos and pool their outputs to detect actions. We further demonstrate that our method can be used to segment discrete actions from a continuous video of an activity. Our results outperform state-of-the-art action recognition and activity segmentation results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,730
Learning realistic human actions from movies
Ivan Laptev, A., Marszałek, M., Schmid, C., & Rozenfeld, B. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587756 · Cited by 3,481
Recognizing human actions: a local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1334462 · Cited by 3,248
The recognition of human movement using temporal templates
Bobick, A., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,837
A survey of advances in vision-based human motion capture and analysis
Moeslund, T. B., Hilton, A., & Krüger, V. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.08.002 · Cited by 2,598
Machine Recognition of Human Activities: A Survey
Turaga, P. K., Chellappa, R., Subrahmanian, V. S., & Udrea, O. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2008.2005594 · Cited by 1,357
Recognizing action at a distance
Efros, A. A., Berg, Mori, & Malik (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238420 · Cited by 1,242
Marszałek, M., Laptev, I., & Schmid, C. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206557 · Cited by 1,197
Cited by
Sorted by citations
Items 101–145 of 145
Wan, S., & Aggarwal, J. K. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301346 · Cited by 8
Learning Multiple Object States from Actions via Large Language Models
Tateno, M., Yagi, T., Furuta, R., & Sato, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00925 · Cited by 9
Manual assembly actions segmentation system using temporal-spatial-contact features
Kang, Z., Cui, J., & Chu, Z. (2023) · Robotic Intelligence and Automation · doi:10.1108/ria-01-2023-0008 · Cited by 7
VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks
Gao, X., Gong, R., Shu, T., Xie, X., Wang, S., & Zhu, S. C. (2019) · Cited by 6
Sequential Interval Network for parsing complex structured activity
Vo, N., & Bobick, A. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.07.006 · Cited by 6
Unsupervised understanding of location and illumination changes in egocentric videos
Betancourt, A., Díaz-Rodríguez, N., Barakova, E. I., Marcenaro, L., Matthias Rauterberg, A., & Regazzoni, C. S. (2017) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2017.03.016 · Cited by 6
Modeling Long-Term Interactions to Enhance Action Recognition
Cartas, A., Radeva, P. I., & Dimiccoli, M. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412148 · Cited by 6
Learning Temporal Video Procedure Segmentation from an Automatically Collected Large Dataset
Ji, L., Wu, C., Zhou, D., Yan, K., Cui, E., Chen, X., & Duan, N. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00279 · Cited by 6
Egocentric Vision for Visual Market Basket Analysis
Santarcangelo, V., Farinella, G. M., & Battiato, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46604-0_37 · Cited by 5
Anticipating human activities from object interaction cues
Aranda, J., & Vinagre, M. (2016) · IEEE RO-MAN · doi:10.1109/roman.2016.7745091 · Cited by 5
MOSCATO: Predicting Multiple Object State Change through Actions
Zameni, P., Shen, Y., & Elhamifar, E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01079 · Cited by 4
Spatio-Temporal Triangular-Chain CRF for Activity Recognition
Cao, C., Zhang, Y., & Lu, H. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806304 · Cited by 4
Multimodal Attribute and Feature Embedding for Activity Recognition
Zhang, W., Huang, Y., Yu, W., Yang, X., Wang, W., & Sang, J. (2019) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3338533.3366592 · Cited by 4
Li, X., Yang, X., Wang, X., & Deng, C. D. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3367957 · Cited by 3
Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs
Gouidis, F., Papoutsakis, K. E., Patkos, T., Argyros, A., & Plexousakis, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00838 · Cited by 3
Cluster Assumption-Guided Timestamp-Supervised Temporal Action Segmentation
Ren, Z., Li, G., Cheng, N., Wu, A., Wang, N., & Gao, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535331 · Cited by 2
A variational causal inference-based method for recognizing object state changes in videos
Wang, Z., Ge, W., Huang, S., & Li, M. (2026) · Multimedia Systems · doi:10.1007/s00530-025-02185-7 · Cited by 2
Target State Classification by Attention-Based Branch Expansion Network
Zhang, Y., Sun, S., Liu, H., Lei, L., Liu, G., & Lu, D. (2021) · Applied Sciences · doi:10.3390/app112110208 · Cited by 2
Action Modifiers: Learning From Adverbs in Instructional Videos
Doughty, H., Laptev, I., Mayol-Cuevas, W. W., & Damen, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00095 · Cited by 2
Inferring Tasks and Fluents in Videos by Learning Causal Relations
Tang, H., Wei, P., Li, H., & Zheng, N. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412115 · Cited by 2
Towards a generative approach to activity recognition and segmentation.Preprint
Kuehne, H., & Serre, T. (2015) · arXiv (Cornell University) · Cited by 2
Wray, M., Moltisanti, D., Mayol-Cuevas, W. W., & Damen, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08338 · Cited by 2
Frontal Low-rank Random Tensors for Fine-grained Action SegmentationPreprint
Zhang, Y., Muandet, K., Ma, Q., Heiko Neumann, B., & Tang, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01004 · Cited by 2
Badminton Shot Recognition with LSTM Network
Jain, A., Agrawal, S., Chauhan, G., Shruti, I. S., & Singh, P. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-60935-0_28 · Cited by 1
Xiong, B., & Grauman, K. L. (2015) · Springer eBooks · doi:10.1007/978-3-319-24702-1_4 · Cited by 1
Multi-level radio tomographic imaging based three-dimensional static body posture sensing
Liu, T., Liang, Z., Liu, J., & Luo, X. (2016) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2016.7554699 · Cited by 1
How Shall We Evaluate Egocentric Action Recognition?
Furnari, A., Battiato, S., & Farinella, G. M. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.280 · Cited by 1
InterTracker: Discovering and Tracking General Objects Interacting with Hands in the Wild
Shao, Y., Ye, Q., Luo, W., Zhang, K., & Chen, J. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341690 · Cited by 1
Extraction of object-action and object-state associations from Knowledge Graphs
Vassiliades, A., Patkos, T., Efthymiou, V., Bikakis, A., Bassiliades, N., & Plexousakis, D. (2024) · Journal of Web Semantics · doi:10.1016/j.websem.2024.100816 · Cited by 1
An Efficient Micro Control Unit VLSI Design for Wearable Electronics and Sensor Networks
Tuan, M., Chen, S., Lin, T.-L., & Lee, H.-Y. (2016) · Pan Pacific Symposium · doi:10.1109/panpacific.2016.7428419 · Cited by 1
cvpaper.challenge in 2016: Futuristic Computer Vision through 1,600 Papers SurveyPreprint
Kataoka, H., Shirakabe, S., He, Y., Ueta, S., Suzuki, T., Abe, K., Kanezaki, A., Morita, S., Yabe, T., Kanehara, Y., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06436 · Cited by 1
Parsa, B., Samani, E. U., Hendrix, R., Singh, S. M., Devasia, S., & Banerjee, A. G. (2019) · arXiv (Cornell University) · Cited by 1
Segmental Spatiotemporal CNNs for Fine-grained Action SegmentationPreprint
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02995 · Cited by 1
Future Localization from an Egocentric Depth ImagePreprint
Park, H. S., Niu, Y., & Shi, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.02094
GMM-based Spatial Change Detection from Bimanual Tracking and Point Cloud Differences.
Monica, R., Zinelli, A., & Aleotti, J. (2016)
SEMBED: Semantic Embedding of Egocentric Action VideosPreprint
Wray, M., Moltisanti, D., Mayol-Cuevas, W. W., & Damen, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08414
Effective and Efficient Hand Detection in Egocentric Videos
Visée, R. J. (2019) · TSpace (University of Toronto)
Exploiting Egocentric Cues for Action Recognition for Ambient Assisted Living Applications
Núñez-Marcos, A., Azkune, G., & Arganda‐Carreras, I. (2021) · Advances in Science, Technology & Innovation/Advances in science, technology & innovation · doi:10.1007/978-3-030-14647-4_10
MS-MH-TCN: Multi-Stage and Multi-Head Temporal Convolutional Network for Action Segmentation
Kang, Z., & Chu, Z. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0885-7_5
Visualization of the Proficiency Level of Assembly Work Using Inertial and Sound Sensors
Hirade, T., Nakano, T., Aoki, T., Yamamoto, Y., & Nishimura, M. (2024) · Journal of Signal Processing · doi:10.2299/jsp.28.137
Cci: A Causal Counterfactual Inference-Based Method for Object State ChangesPreprint
Wang, Z., Huang, S., Zhang, Z., & Li, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5202372
Recognition and Modeling of Manipulation Actions
Bakr, N. A. (2020) · theses.fr (ABES) · doi:10.70675/165d1ebez10d8z4384zb307zb6d622a552a0
Object State Change Anticipation in Procedural Videos: The OSCA Task, Benchmark, and Baseline
Manousaki, V., Bacharidis, K., Gouidis, F., Papoutsakis, K. E., Plexousakis, D., & Argyros, A. (2026) · Sensors · doi:10.3390/s26144611
EgoAfford: Affordance-Aware Zero-Shot Open-Vocabulary Egocentric Action Recognition
Gesualdi, D., Santambrogio, R., Palermo, F., Plizzari, C., Mentasti, S., & Matteucci, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31936-4_7
Wang, C., Zheng, X., & Rao, G. (2026) · Measurement · doi:10.1016/j.measurement.2026.122819
Metadata
- Authors
- Fathi, A., & Rehg, J. M.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- 2013 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2013-06
- Pages
- 2579-2586
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1967686239
- Cited by
- 145
- References
- 29
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 11 · 2023: 9 · 2022: 10 · 2021: 12 · 2020: 11 · 2019: 12 · 2018: 11 · 2017: 12 · 2016: 23 · 2015: 18 · 2014: 5
- FWCI
- 9.78
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- SMARTech Repository (Georgia Institute of Technology) (submittedVersion)
- Funders
- National Science Foundation
- Multidisciplinary University Research Initiative
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- human action recognition, weakly supervised learning, activity segmentation, hand-object interaction, video action detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review