The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Combining embedded accelerometers with computer vision for recognizing food preparation activities
Abstract
This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing information captured by these sensor types can considerably improve recognition performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The 50Salads dataset contains 50 videos with 17 action classes.”
From MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (2023) · cites this paper as [25]
In the citing paper
“Our model achieves state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.”
From MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (2023) · cites this paper as [25]
Citations
Cites
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,290 · cited 2×
“For classifier fusion we tried the sum-rule and the product-rule [11] as well as Random Decision Forests as a non-linear combination method.”
MethodRecognizing human actions: A local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Cited by 2,906
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154 · cited 3×
“In addition to yielding low recognition performance, extracting local descriptors around densely sampled locations as proposed in [26] is computationally demanding in real-time, e.”
BackgroundMethodHuman activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
Marszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168 · cited 3×
“, KTH [23], YouTube [14], Hollywood2 [15] and URADL [17]).”
BackgroundRecognizing realistic actions from videos “in the wild”
Liu, J., Luo, J., & Shah, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206744 · Cited by 1,075
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Criminisi, A. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985415 · Cited by 703
Research works printed but not linked: 6
- 10.5555/2029024 · doi:10.5555/2029024
- 10.5555/2283516.2283683 · doi:10.5555/2283516.2283683
- M. E. Pollack . Intelligent technology for an aging population: The use of AI to assist elders with cognitive impairment . AI Magazine , 26 ( 2 ), 2005 . M. E. Pollack. Intelligent technology for an aging population: The use of AI to assist elders with cognitive impairment. AI Magazine, 26(2), 2005.
- 10.5555/2354409.2354909 · doi:10.5555/2354409.2354909
- 10.5555/1018429.1020906 · doi:10.5555/1018429.1020906
- 10.5555/1786014.1786017 · doi:10.5555/1786014.1786017
Cited by
Items 101–200 of 447
Efficient Two-Step Networks for Temporal Action Segmentation
Li, Y., Dong, Z., Liu, K., Feng, L., Hu, L., Zhu, J., Li, X., Wang, Y., & Liu, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.121 · Cited by 51
Action Shuffle Alternating Learning for Unsupervised Action Segmentation
Li, J., & Todorovic, S. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr46437.2021.01244 · Cited by 55
Park, J., Kim, D., Huh, S., & Jo, S. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108764 · Cited by 48
“To evaluate the proposed model, five-fold cross validation was used as in Stein and McKenna [26] .”
MethodCoarse to Fine Multi-Resolution Temporal Convolutional NetworkPreprint
Singhania, D., Rahaman, R., & Yao, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10859 · Cited by 66
“We evaluate on three standard action segmentation benchmarks: Breakfast Actions[25], 50Salads[42] and GTEA[10].”
MethodLong-Term Anticipation of Activities with Cycle Consistency
Farha, Y. A., Ke, Q., Schiele, B., & Gall, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-71278-5_12 · Cited by 46
“We evaluate the proposed model on two datasets: the Breakfast dataset [20] and 50Salads [41].”
MethodHutchinson, M., & Gadepally, V. N. (2021) · IEEE Access · doi:10.1109/access.2021.3115476 · Cited by 55
Μεδίτσκος, Γ., Plans, P.-M., Stavropoulos, T. G., Benois‐Pineau, J., Buso, V., & Kompatsiaris, I. Y. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.01.009 · Cited by 45
“The authors in [55] have used accelerometers attached to objects that individuals were using to prepare food, such as knifes and spoons.”
Fine-grained action segmentation using the semi-supervised action GAN
Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107039 · Cited by 49
“Frame wise metrics are widely used in many works [43, 19, 40], however, as explained in [19], models that gain similar frame wise accuracies still can show large variations when visualising their performance due to different segmentation behaviour.”
BackgroundMethodFiltjens, B., Ginis, P., Nieuwboer, A., Slaets, P., & Vanrumste, B. (2022) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-022-01025-3 · Cited by 52
“These action segmentation methods have historically been validated on video-based datasets [47, 48] and thus employ videobased features [49].”
MethodComprehensive Instructional Video Analysis: The COIN Dataset and Performance Evaluation
Tang, Y., Lu, J., & Zhou, J. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2980824 · Cited by 49
“In the meantime, a number of datasets for instructional video analysis [6], [15], [36], [56], [65], [70], [82] have been collected in the community.”
BackgroundUnsupervised Learning of Deep Feature Representation for Clustering Egocentric Actions
Bhatnagar, B. L., Singh, S., Arora, C., & Jawahar, C. V. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/200 · Cited by 42
“Though not the focus of this paper, we have also experimented in a third person setup,50 SALAD [Stein and McKenna, 2013], to establish Sequence Optical Flow Frame Flow+Frame the generality of our approach.”
BackgroundMethodInfluentialChao, X., Hou, Z., & Mo, Y. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3150225 · Cited by 49
Ragusa, F., Furnari, A., & Farinella, G. M. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103764 · Cited by 62
Transfer learning and its extensive appositeness in human activity recognition: A survey
Ray, A., & Kolekar, M. H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122538 · Cited by 39
Ng, Y. B., & Fernando, B. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3021497 · Cited by 38
“Similarly, now we provide results on 50Salads dataset using the five fold cross validation with provided splits in [43] which is the protocol used in [3], [21].”
MethodLEMMA: A Multi-view Dataset for L Earning Multi-agent Multi-task Activities
Jia, B., Chen, Y., Huang, S., Zhu, Y., & Zhu, S. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_46 · Cited by 86
Multistage Temporal Convolution Transformer for Action Segmentation
Aziere, N., & Todorović, S. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104567 · Cited by 44
AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection
Zhu, W., Huang, Y.-F., Xie, X., Liu, W., Deng, J., Zhang, D., Wang, Z., & Liu, J. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00218 · Cited by 74
DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition
Perrett, T., & Damen, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00804 · Cited by 36
“50 Salads [40]: The 50 Salads dataset contains 50 videos, by 25 participants.”
BackgroundMethodAction Anticipation Using Pairwise Human-Object Interactions and Transformers
Roy, D., & Fernando, B. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3113114 · Cited by 35
“Our experiments are performed on two procedural cooking datasets-Breakfast [33], 50Salads [55] where the action sequences are regularly ordered and one unscripted regular activities dataset - EPIC-KITCHENS55 [9] where the actions can be of somewhat arbitrary order.”
BackgroundMethodError Detection in Egocentric Procedural Task Videos
Lee, S., Lu, Z., Zhang, Z., Hoai, M., & Elhamifar, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01765 · Cited by 33
Mavroudi, E., Bhaskara, D., Sefati, S., Ali, H., & Vidal, R. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00174 · Cited by 35
Action Segmentation with Mixed Temporal Domain Adaptation
Chen, M.-H., Li, B., Bao, Y., & Alregib, G. (2020) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093535 · Cited by 40
“To evaluate how our approaches can diminish spatialtemporal discrepancy for action segmentation, we choose three challenging datasets: Georgia Tech Egocentric Activities (GTEA) [8], 50Salads [35], and the Breakfast dataset [14], which separate the training and testing sets by different people (noted as subjects), resu…”
MethodSSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentation
Wang, Z., Chen, H., Li, X., Liu, C., Xiong, Y., Tighe, J., & Fowlkes, C. C. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00025 · Cited by 41
Rethinking Learning Approaches for Long-Term Action Anticipation
Nawhal, M., Jyothi, A. A., & Mori, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_32 · Cited by 51
Analysing Cooking Behaviour in Home Settings: Towards Health Monitoring
Yordanova, K., Lüdtke, S., Whitehouse, S., Krüger, F., Paiement, A., Mirmehdi, M., Craddock, I. J., & Kirste, T. (2019) · Sensors · doi:10.3390/s19030646 · Cited by 34
“Typical approaches include decision trees [20], support vector machines [21,22], or random forests [23,24].”
BackgroundLi, Y., Liu, K., Liu, S., Feng, L., & Qiao, H. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3285416 · Cited by 29
“We evaluate IDT-GCN on skeleton-based datasets: including MCFS-22 [38] and MCFS-130 [38] datasets; TSR on RGB-based datasets: e.g., Breakfast [12] and 50Salads [13] datasets.”
MethodAdaptive receptive field U-shaped temporal convolutional network for vulgar action segmentation
Cao, J., Xu, R., Lin, X., Qin, F., Peng, Y. K., & Shao, Y. (2023) · Neural Computing and Applications · doi:10.1007/s00521-022-08190-5 · Cited by 35
“We compared our model with other methods on two challenging benchmarks: 50Salads [15] and Georgia Tech Egocentric Activities (GTEA) [16], which verified the feasibility of the model.”
MethodFrom pose to activity: Surveying datasets and introducing CONVERSE
Edwards, M., Deng, J., & Xie, X. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.10.010 · Cited by 29
“Numerous Human Action Recognition (HAR) datasets have been produced and publicly released in the last decade for the purpose of detecting and identifying action events in an observed scene.”
BackgroundOpenPack: A Large-Scale Dataset for Recognizing Packaging Works in IoT-Enabled Logistic Environments
Yoshimura, N., Morales, J., Maekawa, T., & Hara, T. (2024) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom59722.2024.10494448 · Cited by 31
“Specifically, many multimodal/vision-based cooking activity datasets are available such as CMUMMAC (Spriggs, De La Torre, and Hebert 2009), 50 Salads dataset (Stein and McKenna 2013), Breakfast Actions Dataset (Kuehne, Arslan, and Serre 2014), EPICKITCHENS (Damen et al.”
MethodProgress-Aware Online Action Segmentation for Egocentric Procedural Task Videos
Shen, Y., & Elhamifar, E. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01722 · Cited by 54
Temporally Consistent Unbalanced Optimal Transport for Unsupervised Action Segmentation
Xu, M., & Gould, S. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01385 · Cited by 60
“• The 50 Salads (FS) dataset [35] contains 50 videos of actors performing a cooking activity, totalling 4.5 hours in length.”
InfluentialMa, H., Yang, Z., & Liu, H. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3132016 · Cited by 25
“In daily activity recognition, a cooking action instance may be parsed into multiple low-level steps, including cut, place, mix, and serve [18].”
BackgroundMethodIterative Contrast-Classify for Semi-supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20124 · Cited by 44
Rivoir, D., Funke, I., & Speidel, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103126 · Cited by 31
“More training and evaluation details can be found in Appendix E & Appendix F. GTEA & 50Salads: To reproduce our claims on natural videos, we retrain phase recognition models for online action segmentation on GTEA (Fathi et al., 2011) and 50Salads (Stein and McKenna, 2013).”
MethodA Generalized and Robust Framework for Timestamp Supervision in Temporal Action Segmentation
Rahaman, R., Singhania, D., Thiéry, A. H., & Yao, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_17 · Cited by 26
Sensor substitution for video-based action recognition
Rupprecht, C., Lea, C., Tombari, F., Navab, N., & Hager, G. D. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759769 · Cited by 31
Local Temporal Bilinear Pooling for Fine-Grained Action Parsing
Zhang, Y., Tang, S., Muandet, K., Jarvers, C., & Neumann, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01228 · Cited by 29
“her methods, we downsample the input sequence to achieve the same temporal resolution as [6]. Figure1shows several examplar video frames from the three datasets introduced in the following. 50 Salads [50]. This multi-modal dataset collects 50 recordings from 25 people preparing 2 mixed salads, and each recording lasts…”
MethodAdversarial Generative Grammars for Human Activity Prediction
Piergiovanni, A., Angelova, A., Toshev, A., & Ryoo, M. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_30 · Cited by 39
A Contextual Inquiry of People with Vision Impairments in Cooking
Li, F. M., Liu, M. X., Kane, S. K., & Carrington, P. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642233 · Cited by 35
“Prior research has explored various ways of adopting AI-powered systems to identify objects, such as using computer vision with overhead RGB-D cameras [56] or recognizing objects in mobile device images [2, 47].”
Temporal Segmentation of Fine-gained Semantic Action: A Motion-Centered Figure Skating Dataset
Liu, S., Zhang, A., Li, Y., Zhou, J., Xu, L., Dong, Z., & Zhang, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16314 · Cited by 41
YouMakeup: A Large-Scale Domain-Specific Multimodal Dataset for Fine-Grained Semantic Comprehension
Wang, W., Wang, Y., Chen, S., & Jin, Q. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1517 · Cited by 25
“(Rohrbach et al., 2012; Stein and McKenna, 2013; Kuehne et al., 2014) contain videos about simple cooking activities.”
BackgroundCoupled Generative Adversarial Network for Continuous Fine-Grained Action Segmentation
Gammulle, H., Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00027 · Cited by 27
“Depth information is already available in the 50 Salads [47] dataset.”
BackgroundMethodInfluentialChange-point detection with deep learning: A review
Xu, R., Song, Z., Wu, J., Wang, C., & Zhou, S. (2025) · Frontiers of Engineering Management · doi:10.1007/s42524-025-4109-z · Cited by 32
“Data sets like the Breakfast Data set (Kuehne et al., 2014), the 50Salads Data set (Stein and McKenna 2013), the ActivityNet Data set (Caba Heilbron et al., 2015), and the Georgia Tech Egocentric Activities (GTEA) Data set (Fathi et al., 2011) provide extensive examples of actions, facilitating the…”
Local–Global Transformer Neural Network for temporal action segmentation
Tian, X., Jin, Y., & Tang, X. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00998-4 · Cited by 23
“For example, on the 50Salads dataset [22], LGTNN improves the segmental edit distance from 67.9% to 74.5% for MS-TCN [13], from 78.8% to 80.2% for ETSN [23], and from 70.1 to 76.2% for the F1 score of MS-TCN + + [24].”
BackgroundMethodInfluentialObject-centric Video Representation for Long-term Action Anticipation
Zhang, C., Fu, C., Wang, S., Agarwal, N., Lee, K., Choi, C., & Sun, C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00661 · Cited by 45
“We conduct thorough experiments on the Ego4D [21] and the 50Salads [55] long-term action anticipation benchmarks, and the anticipation benchmark EGTEA Gaze+ [33].”
BackgroundMethodInfluentialProgress Estimation and Phase Detection for Sequential Processes
Li, X., Zhang, Y., Zhang, J., Zhou, M., Chen, S., Gu, Y., Chen, Y., Marsic, I., Farneth, R. A., & Burd, R. S. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130936 · Cited by 24
“Using additional network layers to replace the GMM, our system slightly outperformed existing process-phase detection systems on the trauma resuscitation dataset [28, 30], while simultaneously providing progress and the remaining time estimation.”
BackgroundMethodTimestamp-Supervised Action Segmentation with Graph Convolutional Networks
Khan, H., Haresh, S., Ahmed, A., Siddiqui, S., Konin, A., Zia, M. Z., & Tran, Q.-H. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981351 · Cited by 34
“Next, temporal window size of 31 has the best performance on both 50 Salads and GTEA datasets.”
MethodInfluentialDo We Really Need Temporal Convolutions in Action Segmentation?
Du, D., Su, B., Li, Y., Qi, Z., Si, L., & Shan, Y. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00178 · Cited by 26
“We empirically perform experiments on two public benchmark datasets: 50Salads [28] and Breakfast [29].”
MethodManipulation action tree bank: A knowledge resource for humanoids
Yang, Y., Guha, A., Fermuller, C. M., & Aloimonos, Y. (2014) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2014.7041483 · Cited by 22
“To test manipulation action tree banks, we use two state-of-the-art fully annotated cooking datasets, namely the 50 Salad Dataset [22] and the TACoS Cooking Dataset [23].”
MethodSign Segmentation with Changepoint-Modulated Pseudo-Labelling
Renz, K., Stache, N. C., Fox, N., Varol, G., & Albanie, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00379 · Cited by 38
“in contrast to problem settings such as human action segmentation which have profited from a diverse collection of densely annotated datasets [16, 20, 22, 36, 55, 63], existing sign language datasets provide a relatively small quantity of data with precise temporal boundaries [27, 52], limiting their ability to train…”
BackgroundCross-enhancement transformer for action segmentation
Wang, J., Wang, Z., Zhuang, S., Hao, Y., & Wang, H. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16041-1 · Cited by 32
“Our approach performs state-of-the-art on three challenging datasets: 50Salads [28], GTEA[10],andBreakfast[17].”
Fine-Grained Action Recognition in Assembly Work Scenes by Drawing Attention to the Hands
Kobayashi, T., Aoki, Y., Shimizu, S., Kusano, K., & Okumura, S. (2019) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis.2019.00077 · Cited by 25
“For example, most of the common action segmentation datasets available refer to the action classes of cooking [6], [7], [13].”
BackgroundLearning to Anticipate Future with Dynamic Context Removal
Xu, X., Li, Y., & Lu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01240 · Cited by 31
Sensors-Based Ambient Assistant Living via E-Monitoring Technology
Hafeez, S., Ghadi, Y. Y., Alarfaj, M., Shloul, T. A., Jalal, A., Kamal, S., & Kim, D. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.023841 · Cited by 20
Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation
Ding, G., & Yao, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_2 · Cited by 31
Weakly-Supervised Action Segmentation and Unseen Error Detection in Anomalous Instructional Videos
Ghoddoosian, R., Dwivedi, I., Agarwal, N., & Dariush, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00929 · Cited by 20
“Among untrimmed datasets, [29, 34] are small and lack sufficient training samples.”
BackgroundMethodPredicting Intentions from Motion: The Subject-Adversarial Adaptation Approach
Zunino, A., Cavazza, J., Volpi, R., Morerio, P., Cavallo, A., Becchio, C., & Murino, V. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01234-9 · Cited by 20
“For instance, MPII-CAD (Rohrbach et al. 2012) and Salad 50 (Stein and McKenna 2013) cover more articulated (cooking) actions, while UCF101 (Soomro et al. 2012) and HDMB51 (Kuehne et al. 2011) collect YouTube videos, thus guaranteeing a broad variability of backgrounds and context.”
BackgroundAction anticipation using latent goal learning
Roy, D., & Fernando, B. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00088 · Cited by 22
“We choose 3 datasets for our experiments - 50Sal-ads [25], Breakfast [15], and EPIC-KITCHENS55 [4].”
MethodUncertainty-Aware Representation Learning for Action Segmentation
Chen, L., Li, M., Duan, Y., Zhou, J., & Lu, J. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/115 · Cited by 22
“Datasets: We evaluate our proposed method on three challenging action segmentation datasets: 50Salads [ Stein and McKenna, 2013 ] , Georgia Tech Egocentric Activities (GTEA) [ Fathi et al. , 2011 ] , and the Breakfast dataset [ Hilde et al. , 2014 ] .”
MethodContext-Aware Query Selection for Active Learning in Event Recognition
Hasan, M., Paul, S., Mourikis, A. I., & Roy–Chowdhury, A. K. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2878696 · Cited by 21
“posed approach using 50Salads dataset [39].”
BackgroundMethodInfluentialGao, S., Wu, R., Liu, S., Erkan, U., Toktaş, A., Liu, J., & Tang, X. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02251-y · Cited by 23
“We evaluate MSLID-TCN on three commonly used action segmentation datasets: 50salads [23], Georgia Tech Egocentric Activities (GTEA) [24], and Breakfast [25].”
PREGO: Online Mistake Detection in PRocedural EGOcentric Videos
Flaborea, A., di Melendugno, G. M. D., Plini, L., Scofano, L., Matteis, E. D., Furnari, A., Farinella, G. M., & Galasso, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01749 · Cited by 59
“Procedural learning has seen significant advancements with the creation of diverse datasets [9, 18, 27, 33, 41] that provide insights into both structured [4, 26, 36] and unstructured [6, 14] tasks, covering a spectrum from industrial assembly [26–29] to daily cooking activities [6, 15, 31].”
BackgroundASPnet: Action Segmentation with Shared-Private Representation of Multiple Data Sources
van Amsterdam, B., Kadkhodamohammadi, A., Luengo, I., & Stoyanov, D. V. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00236 · Cited by 32
Efficient motion estimation methods for fast recognition of activities of daily living
Poularakis, S., Avgerinakis, K., Briassouli, A., & Kompatsiaris, I. Y. (2017) · Signal Processing Image Communication · doi:10.1016/j.image.2017.01.005 · Cited by 21
Energy expenditure estimation using visual and inertial sensors
Tao, L., Burghardt, T., Mirmehdi, M., Damen, D., Cooper, A. R., Camplani, M., Hannuna, S., Paiement, A., & Craddock, I. J. (2017) · IET Computer Vision · doi:10.1049/iet-cvi.2017.0112 · Cited by 21
“…receiving growing attention in various areas, including action recognition [5], gesture recognition [11], robotics [12], augmented/virtual reality [13], and assistive technologies applications, such as fall detection [14], food preparation [15] and in a general ambient assisted living system [16].”
BackgroundMethodUncovering Human Multimodal Activity Recognition with a Deep Learning Approach
Ranieri, C. M., Vargas, P. A., & Romero, R. A. F. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207255 · Cited by 21
“Datasets such as the UTD-MHAD [7], adopted in the experiments, and the 50 Salads [23] provide not only video and inertial measurements, but also positioning of skeleton joints, which are often used as an important input for the proposed methods [24].”
MethodMultimodal Large Models are Effective Action Anticipators
Wang, B., Tian, Y., Wang, S., & Yang, L. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557615 · Cited by 26
A survey of Semantic Reasoning frameworks for robotic systems
Liu, W., Daruna, A., Patel, M., Ramachandruni, K., & Chernova, S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104294 · Cited by 29
“Frequently, multiple modalities are used to capture the details of a 50 Salad Dataset [37] A video dataset containing 50 video sequences of 25 people preparing two different salad recipes, which focus on complex interactions between hands, tools, and manipulable objects.”
MethodTSRN: two-stage refinement network for temporal action segmentation
Tian, X., Jin, Y., & Tang, X. (2023) · Pattern Analysis and Applications · doi:10.1007/s10044-023-01166-8 · Cited by 18
“Datasets We evaluate our TSRN on three challenging data-sets: 50Salads [17], GTEA [18], and Breakfast [19].”
BackgroundMethodPermutation-Aware Activity Segmentation via Unsupervised Frame-to-Segment Alignment
Tran, Q.-H., Mehmood, A., Ahmed, M., Naufil, M., Zafar, A., Konin, A., & Zia, M. Z. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00630 · Cited by 34
Action representations in robotics: A taxonomy and systematic classification
Zech, P., Renaudo, E., Haller-Seeber, S., Zhang, X., & Piater, J. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919835020 · Cited by 27
“2011) 4 3D Action Pairs (Oreifej and Liu 2013) 2 50 Salads (Stein and McKenna 2013) 2 ADLs (Pirsiavash and Ramanan 2012) 2 CAD-60 (Sung et al.”
Gated forward refinement network for action segmentation
Wang, D., Yuan, Y., & Wang, Q. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.066 · Cited by 25
“We evaluate our approach on three datasets: University of Dundee 50 Salads (50Salads) [14], Georgia Tech Egocentric Activities (GTEA) [15], and the Breakfast dataset [16], which are widely used in action segmentation For evaluation, we use the standard 4 splits as proposed in [16] and report the…”
MethodUser-adaptive models for recognizing food preparation activities
Stein, S., & McKenna, S. J. (2013) · doi:10.1145/2506023.2506031 · Cited by 18
Self-citationTanaka, R., Suzuki, T., & Fujii, K. (2024) · Proceedings of the 7th ACM International Workshop on Multimedia Content Analysis in Sports · doi:10.1145/3689061.3689077 · Cited by 17
“The TAS datasets are mainly dealing with cooking procedures in the kitchen [10, 16, 37, 62] or assembly procedures of furniture and toys [4, 5, 55, 58], with actions such as “open” and “close” annotated for each frame, for example.”
BackgroundRecurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks
Perrett, T., & Damen, D. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.161 · Cited by 17
“(a) Breakfast [10] (b) 50 Salads [20] (c) MPII Cooking 2 [14]”
BackgroundMethodInfluentialADNet: Temporal Anomaly Detection in Surveillance Videos
Öztürk, H., & Can, A. B. (2021) · ICPR Workshops · doi:10.1007/978-3-030-68799-1_7 · Cited by 19
“Encoder-Decoder TCN (ED-TCN) achieves 68 F1@10 score, while Dilated TCN achieves 52.2 F1@10 score on 50 Salads [28] dataset.”
InfluentialOn Diverse Asynchronous Activity Anticipation
Zhao, H., & Wildes, R. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_46 · Cited by 19
“[6,47,22]) provide times in integer units; so, our discrete temporal representation is more consistent with that format than a continuous representation and also allows us to unify inference of activities and their times.”
BackgroundForecasting Future Action Sequences with Neural Memory NetworksPreprint
Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09278 · Cited by 21
“The ablation results on Breakfast and 50 Salads datasets are presented in Tab.”
BackgroundMethodInfluentialLoh, S. B., Roy, D., & Fernando, B. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00270 · Cited by 21
“The 50 Salads [28] dataset consists of 50 videos of 25 actors making salads based on recipes provided beforehand.”
MethodBezdek, M. A., Nguyen, T. T., Hall, C. S., Braver, T. S., Bobick, A., & Zacks, J. M. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01980-8 · Cited by 11
“One of our chapter types, “making breakfast,” is of the class of food preparation sequences that have been employed frequently in both event segmentation (Eisenberg & Zacks, 2016; Swal-low et al., 2018) and machine learning studies (Kuehne et al., 2014; Stein & McKenna, 2013; Zhou et al., 2018).”
BackgroundSF-TMN: SlowFast temporal modeling network for surgical phase recognition
Zhang, B., Sarhan, M. H., Goel, B., Petculescu, S., & Ghanem, A. (2023) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-024-03095-1 · Cited by 17
“The 50Salads dataset [19] contains 50 videos with 17 action classes and another two classes for “action start” and “action end.””
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 21
Male, J., & Martínez-Hernández, U. (2021) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit46573.2021.9453672 · Cited by 16
“These sensing types can complement each other, which is useful for activity recognition tasks in industrial and home settings [7][10].”
BackgroundUnsupervised Discriminative Embedding For Sub-Action Learning in Complex Activities
Swetha, S., Kuehne, H., Rawat, Y. S., & Shah, M. A. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506759 · Cited by 30
“Breakfast Dataset [9] comprises of 10 activities with 48 sub-actions in 1, 712 videos with variations in duration and ordering of sub-actions for activities.”
BackgroundInfluentialLearning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 26
MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation
Li, S., Farha, Y. A., Liu, Y., Cheng, M.-M., & Gall, J. (2020) · arXiv · Cited by 350
“Our approach achieves state-of-the-art results on three challenging benchmarks for action segmentation: 50Salads [15], Georgia Tech”
MethodMoniruzzaman, M., Yin, Z., He, Z., Leu, M. C., & Qin, R. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232021 · Cited by 15
“Our proposed approach outperforms other state-of-the-art methods on the Breakfast [34], 50Salads [35], and EPIC-Kiteches-55 [36] datasets.”
BackgroundMethodInfluentialTherbligs in Action: Video Understanding through Motion Primitives
Dessalene, E., Maynord, M., Fermuller, C. M., & Aloimonos, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01023 · Cited by 22
“Other datasets producing sub-action level annotations are of an instructional nature [14,17,21], providing annotations for steps of various cooking activities.”
BackgroundToward Long Form Audio-Visual Video Understanding
Hou, W., Li, G., Tian, Y., & Hu, D. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3672079 · Cited by 29
“58 50 Salads [45] Human UbiComp’13 ✗ ✓ 324 4.”
Bottom-up improved multistage temporal convolutional network for action segmentation
Chen, W., Chai, Y., Qi, M., Sun, H., Pu, Q., Kong, J., & Zheng, C. (2022) · Applied intelligence (Boston) · doi:10.1007/s10489-022-03382-x · Cited by 14
“Activities (GTEA) [35], 50Salads [36], and Breakfast [37] datasets.”
MethodChop & Learn: Recognizing and Generating Object-State Compositions
Saini, N., Wang, H., Swaminathan, A., Jayasundara, V., He, B., Gupta, K., & Shrivastava, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01852 · Cited by 26
“…with procedural and kitchen activities that capture object and state transformations, such as Epic-Kitchens [7] with object and hand bounding box annotation version VISOR [8], Youcook2 [72], Ego4D [13], COIN [61], HowTo100M [38], Breakfast [29], 50Sal-ads [59], CrossTask [73] and ChangeIt [57].”
BackgroundAtrous Temporal Convolutional Network for Video Action Segmentation
Wang, J., Du, Z., Li, A., & Wang, Y. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803088 · Cited by 14
“The proposed approach is assessed on three action segmentation benchmarks: University of Dundee 50 Salads (50 Salads) [19], JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) [20] and Georgia Tech Egocentric Activities (GTEA) [21].”
MethodInfluentialConcept drift adaptation in video surveillance: a systematic review
Gonçalves, V. P., Silva, L. P., Nunes, F. L. S., Ferreira, J. E., & de Araújo, L. V. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15855-3 · Cited by 13
Egocentric Human Activities Recognition With Multimodal Interaction Sensing
Hao, Y., Kanezaki, A., Sato, I., Kawakami, R., & Shinoda, K. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3349191 · Cited by 17
“Because kitchen activities have high diversity and high relevance to the subject’s field of vision, this set of activities is widely chosen as benchmark for ego-HAR, such as breakfast dataset [37], [38] and 50 Salad dataset [39].”
BackgroundDiffAct++: Diffusion Action Segmentation
Liu, D., Li, Q., Dinh, A.-D., Jiang, T., Shah, M., & Xu, C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3509434 · Cited by 24
“50Salads [104] includes 50 top-view videos of salad preparation, with 17 classes of actions.”
CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation
Wang, X., Liu, J., Mei, T., & Luo, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3263387 · Cited by 25
“Following the same protocol of previous work [11], [12], [13], [14], we evaluated CoSeg on the following public datasets: Breakfast [3], INRIA instructional [7], 50 Salads [21], and GEBD [20].”
MethodDilated Transformer with Feature Aggregation Module for Action Segmentation
Du, Z., & Wang, Q. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11133-9 · Cited by 23
“Our method achieves comparable performance compared to state-of-the-art methods on the task of temporal action segmentation on three challenging datasets, namely, 50Salads [29], Breakfast [17], and Georgia Tech Egocentric Activities (GTEA) [30] datasets.”
MethodSystem-Status-Aware Adaptive Network for Online Streaming Video Understanding
Foo, L. G., Gong, J. L., Fan, Z., & Liu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01013 · Cited by 21
Sun, S., Wang, F., Liang, Q., & He, L. (2017) · International Conference on Multimedia Retrieval · doi:10.1145/3078971.3079039 · Cited by 13
“The SALADS [11] and the JIGSAWS [12] are two datasets on fine-grained action which contain video data as well as the accelerometer data.”
Method
Metadata
- Authors
- Stein, S., & McKenna, S. J.
- Affiliations
- University of Dundee (GB)
- Venue
- Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing
- Publisher
- ACM
- Published
- 2013-09-08
- Pages
- 729-738
- OpenAlex
- W2109698606
- Cited by
- 448
- References
- 33
- Citations by year
- 2026: 26 · 2025: 41 · 2024: 56 · 2023: 56 · 2022: 54 · 2021: 50 · 2020: 37 · 2019: 49 · 2018: 26 · 2017: 15 · 2016: 24 · 2015: 7 · 2014: 5 · 2013: 2
- FWCI
- 8.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Discovery Research Portal (University of Dundee) (submittedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- Funders
- Research Councils UK
- Engineering and Physical Sciences Research Council (EP/G066019/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- activity recognition, embedded accelerometers, RGB-D video, multimodal fusion, cooking, salad preparation, node localization, feature-level fusion, annotated dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.