The DOI matches its Crossref record, with a small difference in title or year; 56 of the 60 references it cites were found in a registry or an index; 4 of the 14 sources its claims cite were checked for support.
MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation
Abstract
With the success of deep learning in classifying short trimmed videos, more attention has been focused on temporally segmenting and classifying activities in long untrimmed videos. State-of-the-art approaches for action segmentation utilize several layers of temporal convolution and temporal pooling. Despite the capabilities of these approaches in capturing temporal dependencies, their predictions suffer from over-segmentation errors. In this paper, we propose a multi-stage architecture for the temporal action segmentation task that overcomes the limitations of the previous approaches. The first stage generates an initial prediction that is refined by the next ones. In each stage we stack several layers of dilated temporal convolutions covering a large receptive field with few parameters. While this architecture already performs well, lower layers still suffer from a small receptive field. To address this limitation, we propose a dual dilated layer that combines both large and small receptive fields. We further decouple the design of the first stage from the refining stages to address the different requirements of these stages. Extensive evaluation shows the effectiveness of the proposed model in capturing long-range dependencies and recognizing action segments. Our models achieve state-of-the-art results on three datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.
Overview
CoChat Summary
TL;DR
The paper introduces MS-TCN, a multi-stage temporal convolutional network for fully supervised temporal action segmentation that operates at full temporal resolution and refines predictions stage by stage, trained with a combined cross-entropy and truncated-MSE smoothing loss. It reports state-of-the-art results on 50Salads, GTEA, and Breakfast.
Key findings
- The multi-stage architecture substantially outperforms a single-stage TCN of equal parameter count (48 layers) on 50Salads: F1@{10,25,50} of 76.3/74.0/64.5 vs 49.0/46.4/40.2, Edit 67.9 vs 40.7, Acc 80.7 vs 78.0 — a margin of up to 27%.
- The proposed smoothing loss improves over cross-entropy alone on 50Salads: L_cls gives F1@{10,25,50} 71.3/69.7/60.7, Edit 64.2, Acc 79.9, while L_cls + λL_T-MSE gives 76.3/74.0/64.5, Edit 67.9, Acc 80.7 (an absolute F1/edit improvement of ~5%); it also beats a KL-divergence smoothing loss (71.9/69.3/60.1, Edit 64.6, Acc 80.2).
- Four stages is optimal; adding a fifth degrades performance (F1@10 76.4 but F1@25/50 drop to 73.4/63.6 and Acc to 79.5), attributed to over-fitting from more parameters.
- Concatenating features to the input probabilities of higher stages causes a large drop (F1@10 56.2 vs 76.3, Edit 47.6 vs 67.9, ~20% drop), so only probabilities are passed.
- The model outperforms state-of-the-art methods on all three datasets, with up to 12.6% improvement in frame-wise accuracy on 50Salads; on GTEA with fine-tuning it reaches F1@{10,25,50} 87.5/85.4/74.6, Edit 81.4, Acc 79.2.
Methods
- Architecture: a single-stage TCN of dilated 1D convolutions (kernel size 3, acausal, dilation doubled per layer 1–512, residual connections, no pooling or fully connected layers), stacked into a multi-stage model where each stage refines the previous stage's frame-wise probabilities; implementation uses 4 stages, 10 dilated layers per stage, 64 filters, dropout, Adam with learning rate 0.0005.
- Loss: sum over stages of a per-stage loss combining cross-entropy classification loss and a truncated mean squared error smoothing loss over frame-wise log-probabilities, with τ = 4 and λ = 0.15.
- Data and evaluation: I3D features at 15 fps on 50Salads (50 videos, 17 classes, 5-fold cross-validation), GTEA (28 videos, 11 classes, leave-one-subject-out), and Breakfast (1,712 videos, 48 actions, standard 4 splits); metrics are frame-wise accuracy, segmental edit distance, and segmental F1 at IoU thresholds 10%, 25%, 50%.
Limitations
- Stated: the weakly supervised setting is explicitly out of scope ("the weakly supervised case is beyond the scope of this paper"); performance is slightly worse on longer videos due to the limited receptive field (GTEA ≥1.5 min: F1@10 81.2, Edit 71.8, Acc 75.9 vs <1 min: 89.6, 82.5, 76.6); a fifth stage degrades results, possibly from over-fitting.
- Evident from the text: evaluation is confined to cooking/egocentric activity datasets (50Salads, GTEA, Breakfast), so generalization to other domains is untested; the model is fully supervised and requires frame-level annotations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
3 Valid · 1 Not valid · 5 Not checked · 71 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Our model achieves state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.”
- Also stated in: § 1. Introduction · plain
“Our approach achieves state-of-the-art results on three challenging benchmarks for action segmentation: 50Salads [25], Georgia Tech Egocentric Activities (GTEA) [8], and the Breakfast dataset [12].”
What the sources say
- Not checked — no full text of the sourcecites [25] [25] Sebastian Stein and Stephen J McKenna. Combining embedded accelerometers with computer vision for recognizing food preparation activities. In ACM International Joint Conference on Pervasive and Ubiquitous Computing , pages 729-738, 2013. - source
- Supportedcites [8] [8] Alireza Fathi, Xiaofeng Ren, and James M Rehg. Learning to recognize objects in egocentric activities. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3281-3288, 2011. - source
The citation identifies the GTEA dataset, not the citing authors’ own model or its results. Section 5.1 describes the paper’s egocentric daily-activity dataset, and the Conclusion identifies its release as GTEA. No page numbers are available in our copy. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [12] [12] Hilde Kuehne, Ali Arslan, and Thomas Serre. The language of actions: Recovering the syntax and semantics of goaldirected human activities. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 780787, 2014. - source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § 1. Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in the table, all of these models achieve a comparable frame-wise accuracy.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Extensive evaluation on three datasets shows the effectiveness of our model in capturing long range dependencies between action classes and producing high quality predictions.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This architecture is trained using a combination of a classification loss and a proposed smoothing loss that penalizes over-segmentation errors.”
- Also stated in: § 1. Introduction · plain
“we further employ a smoothing loss during training which penalizes over-segmentation errors in the predictions.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1. Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 4, the impact of λ is very small on the performance.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 80 claims and every source verdict.
- Probably not a substantial claim. 8
- Borderline: one of the tests for a substantial claim is unclear. 1
- It may state two separate claims as one. 22
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 4×
“While the dual dilated layer combines local and global context from the input sequence, there are other techniques in the literature for fusing multi-scale features like feature pyramid networks (FPN) [47].”
BackgroundInfluentialQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1705.07750 · Cited by 10,485
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · cited 7×
“Most of the efforts, however, have been focused on classifying short trimmed videos [1], [2], [3], [4].”
BackgroundMethodInfluentialTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · cited 1×
“Most of the efforts, however, have been focused on classifying short trimmed videos [1], [2], [3], [4].”
BackgroundStacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 1×
“Stacking several predictors sequentially has shown significant improvements in many tasks like human pose estimation [44], [45], [46].”
BackgroundWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · cited 1×
“Most of the efforts, however, have been focused on classifying short trimmed videos [1], [2], [3], [4].”
BackgroundRes2Net: A New Multi-Scale Backbone Architecture
Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y., Yang, M.-H., & Torr, P. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2938758 · Cited by 3,385 · cited 1×
“Multi-scale information is important for speech synthesis [29] and image recognition [30].”
BackgroundSelf-citation
Research works printed but not linked: 1
- karaman, 2014, Fast saliency based pooling of fisher encoded dense trajectories, Proc Eur Conf Comput Vis THUMOS Workshop
Cited by
Multi-Stage Progressive Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01458 · Cited by 2,095
MAXIM: Multi-Axis MLP for Image Processing
Tu, Z., Talebi, H., Zhang, H., Yang, F., Milanfar, P., Bovik, A., & Li, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00568 · Cited by 616
Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
Sener, F., Chatterjee, D., Shelepov, D., He, K., Singhania, D., Wang, R., & Yao, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02042 · Cited by 213
Improving Action Segmentation via Graph-Based Temporal Reasoning
Huang, Y., Sugano, Y., & Sato, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01404 · Cited by 226
HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
Liu, Y., Liu, Y., Jiang, C., Lyu, K., Wan, W., Shen, H., Liang, B., Fu, Z., Wang, H., & Yi, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02034 · Cited by 476
Self-citationTemporal Action Segmentation: An Analysis of Modern Techniques
Ding, G., Şener, F., & Yao, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327284 · Cited by 213
Ramesh, S., Dall’Alba, D., Gonzalez, C. A., Yu, T., Mascagni, P., Mutter, D., Marescaux, J., Fiorini, P., & Padoy, N. (2021) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-021-02388-z · Cited by 119
Temporal Action Segmentation from Timestamp Supervision
Li, Z., Farha, Y. A., & Gall, J. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr46437.2021.00826 · Cited by 112
Self-citationBehrmann, N., Golestaneh, S. A., Kolter, J. Z., Gall, J., & Noroozi, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_4 · Cited by 155
Self-citationIntelligent Stress Monitoring Assistant for First Responders
Lai, K., Yanushkevich, S. N., & Shmerko, V. P. (2021) · IEEE Access · doi:10.1109/access.2021.3057578 · Cited by 83
Multi-shot Temporal Event Localization: a Benchmark
Liu, X., Hu, Y. G., Bai, S., Ding, F., Bai, X., & Torr, P. H. S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01241 · Cited by 118
Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos
Li, M., Chen, L., Duarr, Y., Hu, Z., Feng, J., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01926 · Cited by 115
Global2Local: Efficient Structure Search for Video Action Segmentation
Gao, S., Han, Q., Li, Z., Peng, P., Wang, L., & Cheng, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01653 · Cited by 103
Self-citationBackground-Click Supervision for Temporal Action Localization
Yang, L., Han, J., Zhao, T., Lin, T., Zhang, D., & Chen, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132058 · Cited by 100
Short-Term Load Forecasting Based on Improved TCN and DenseNet
Liu, M., Qin, H., Cao, R., & Deng, S. (2022) · IEEE Access · doi:10.1109/access.2022.3218374 · Cited by 71
Temporal action segmentation for video encryption
Gao, S., Iu, H. H., Mou, J., Erkan, U., Liu, J., Wu, R., & Tang, X. (2024) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2024.114958 · Cited by 69
High-Temperature-Resistance Flexible Piezoelectric Sensor via Cyclized PAN/BTO Nanofibers
Zhou, T., Wang, S.-L., Ao, Y., et al. (2025) · Nano Energy · doi:10.1016/j.nanoen.2025.110910 · Cited by 57
Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering
Kumar, S., Haresh, S., Ahmed, A., Konin, A., Zia, M. Z., & Tran, Q.-H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01954 · Cited by 104
Vision-based activity recognition in children with autism-related behaviors
Wei, P., Ahmedt‐Aristizabal, D., Gammulle, H., Denman, S., & Armin, M. A. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16763 · Cited by 70
Artificial intelligence for phase recognition in complex laparoscopic cholecystectomy
Golany, T., Aides, A., Freedman, D., Rabani, N., Liu, Y., Rivlin, E., Corrado, G. S., Matias, Y., Khoury, W., Kashtan, H., et al. (2022) · Surgical Endoscopy · doi:10.1007/s00464-022-09405-5 · Cited by 69
Self-citationFast Weakly Supervised Action Segmentation Using Mutual Consistency
Souri, Y., Fayyaz, M., Minciullo, L., Francesca, G., & Gall, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3089127 · Cited by 76
Self-citationC2F-TCN: A Framework for Semi- and Fully-Supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284080 · Cited by 74
Learning by Aligning Videos in Time
Haresh, S., Kumar, S., Coskun, H., Syed, S. N., Konin, A., Zia, M. Z., & Tran, Q.-H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00550 · Cited by 93
Pose Refinement Graph Convolutional Network for Skeleton-Based Action Recognition
Li, S., Yi, J., Farha, Y. A., & Gall, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3056361 · Cited by 63
“Although deep learning models are very popular for recognizing activities in videos [1]–[3], these approaches are computationally expensive and cannot be”
BackgroundSelf-citationRF-Next: Efficient Receptive Field Search for Convolutional Neural Networks
Gao, S., Li, Z., Qi, H., Cheng, M., & Wang, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183829 · Cited by 80
Self-citationPredicting the future from first person (egocentric) vision: A survey
Rodin, I., Furnari, A., Mavroeidis, D., & Farinella, G. M. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103252 · Cited by 72
Exploring Segment-Level Semantics for Online Phase Recognition From Surgical Videos
Ding, X., & Li, X. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3182995 · Cited by 65
Continuous Human Action Recognition for Human-machine Interaction: A Review
Gammulle, H., Ahmedt‐Aristizabal, D., Denman, S., Tychsen-Smith, L., Petersson, L., & Fookes, C. (2023) · ACM Computing Surveys · doi:10.1145/3587931 · Cited by 70
Multi-Stage Progressive Image RestorationPreprint
Zamir, S. W., Arora, A., Khan, S. H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02808 · Cited by 54
How Much Temporal Long-Term Context is Needed for Action Segmentation?
Bahrami, E., Francesca, G., & Gall, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00950 · Cited by 92
Self-citationTMMF: Temporal Multi-Modal Fusion for Single-Stage Continuous Gesture Recognition
Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3108349 · Cited by 52
Temporal Relational Modeling with Self-Supervision for Action Segmentation
Wang, D., Hu, D., Li, X., & Dou, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16377 · Cited by 64
Sehar, U., & Naseem, M. L. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12821-3 · Cited by 60
Transformer in Convolutional Neural NetworksPreprint
Liu, Y., Sun, G., Qiu, Y., Zhang, L., Chhatkuli, A., & Gool, L. V. (2021) · arXiv (Cornell University) · Cited by 66
Self-citationEfficient Two-Step Networks for Temporal Action Segmentation
Li, Y., Dong, Z., Liu, K., Feng, L., Hu, L., Zhu, J., Li, X., Wang, Y., & Liu, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.121 · Cited by 51
Nan, M., Trăscău, M., Florea, A., & Iacob, C. C. (2021) · Italian National Conference on Sensors · doi:10.3390/s21062051 · Cited by 44
Filtjens, B., Ginis, P., Nieuwboer, A., Slaets, P., & Vanrumste, B. (2022) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-022-01025-3 · Cited by 52
A unified modeling framework for lane change intention recognition and vehicle status prediction
Yuan, R., Abdel-Aty, M. A., Gu, X., Zheng, O., & Xiang, Q. (2023) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2023.129332 · Cited by 38
Chand, S., Zheng, H., & Lu, Y. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.10.002 · Cited by 41
Error Detection in Egocentric Procedural Task Videos
Lee, S., Lu, Z., Zhang, Z., Hoai, M., & Elhamifar, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01765 · Cited by 33
Semantic2Graph: graph-based multi-modal feature fusion for action segmentation in videos
Zhang, J., Tsai, P.-H., & Tsai, M. (2024) · Applied Intelligence · doi:10.1007/s10489-023-05259-z · Cited by 43
Yang, P.-K., Filtjens, B., Ginis, P., Goris, M., Nieuwboer, A., Gilat, M., Slaets, P., & Vanrumste, B. (2024) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-024-01320-1 · Cited by 34
Multistage Temporal Convolution Transformer for Action Segmentation
Aziere, N., & Todorović, S. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104567 · Cited by 38
Progressive convolutional transformer for image restoration
Wan, Y., Shao, M., Cheng, Y., Meng, D., & Zuo, W. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106755 · Cited by 35
Gaugel, S., & Reichert, M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106078 · Cited by 31
Li, Y., Liu, K., Liu, S., Feng, L., & Qiao, H. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3285416 · Cited by 29
Long Short View Feature Decomposition via Contrastive Video Representation Learning
Behrmann, N., Fayyaz, M., Gall, J., & Noroozi, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00911 · Cited by 40
Self-citationTemporally Consistent Unbalanced Optimal Transport for Unsupervised Action Segmentation
Xu, M., & Gould, S. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01385 · Cited by 27
Progress-Aware Online Action Segmentation for Egocentric Procedural Task Videos
Shen, Y., & Elhamifar, E. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01722 · Cited by 54
Iterative Contrast-Classify for Semi-supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20124 · Cited by 44
Learning to Segment Actions from Observation and Narration
Fried, D., Alayrac, J.-B., Blunsom, P., Dyer, C., Clark, S., & Nematzadeh, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.231 · Cited by 38
Surgical workflow recognition with temporal convolution and transformer for action segmentation
Zhang, B., Goel, B., Sarhan, M. H., Goel, V. K., Abukhalil, R., Kalesan, B., Stottler, N., & Petculescu, S. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02811-z · Cited by 37
Tang, B. Y., Yao, D., Yang, J., & Zhang, F. W. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3413142 · Cited by 26
Temporal Segmentation of Fine-gained Semantic Action: A Motion-Centered Figure Skating Dataset
Liu, S., Zhang, A., Li, Y., Zhou, J., Xu, L., Dong, Z., & Zhang, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16314 · Cited by 41
Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
Ghoddoosian, R., Dwivedi, I., Agarwal, N., Choi, C.-H., & Dariush, B. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01341 · Cited by 27
Attention-based adaptive feature selection for multi-stage image dehazing
Li, X., Hua, Z., & Li, J. (2022) · The Visual Computer · doi:10.1007/s00371-021-02365-2 · Cited by 24
“Due to the effectiveness of multi-stage methods in other fields (such as image deraining [32], scene parsing [33] and action segmentation [34,35], recent multi-stage methods have been introduced into the field of dehazing, focusing on iterative networks and multi-scale.”
MethodData-centric multi-task surgical phase estimation with sparse scene segmentation
Sánchez-Matilla, R., Robu, M., Grammatikopoulou, M., Luengo, I., & Stoyanov, D. V. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02616-0 · Cited by 22
Adaptive Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization
Zhai, Y., Wang, L., Tang, W., Zhang, Q., Zheng, N., Doermann, D., Yuan, J., & Hua, G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3189662 · Cited by 27
Local–Global Transformer Neural Network for temporal action segmentation
Tian, X., Jin, Y., & Tang, X. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00998-4 · Cited by 23
Spatial-Temporal Consistency Network for Low-Latency Trajectory Forecasting
Li, S.-J., Zhou, Y.-Y., Yi, J., & Gall, J. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.00195 · Cited by 27
Self-citationTimestamp-Supervised Action Segmentation with Graph Convolutional Networks
Khan, H., Haresh, S., Ahmed, A., Siddiqui, S., Konin, A., Zia, M. Z., & Tran, Q.-H. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981351 · Cited by 34
Cascade Multi-Level Transformer Network for Surgical Workflow Analysis
Yue, W., Liao, H., Xia, Y., Lam, V. W. T., Luo, J., & Wang, Z. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3265354 · Cited by 21
Aldarraji, M., Vega-Márquez, B., Pontes, B., Mahmood, B. M., & Riquelme, J. C. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e25821 · Cited by 34
Coarse to Fine Multi-Resolution Temporal Convolutional NetworkPreprint
Singhania, D., Rahaman, R., & Yao, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10859 · Cited by 66
Change-point detection with deep learning: A review
Xu, R., Song, Z., Wu, J., Wang, C., & Zhou, S. (2025) · Frontiers of Engineering Management · doi:10.1007/s42524-025-4109-z · Cited by 32
SVIP: Sequence VerIfication for Procedures in Videos
Qian, Y., Luo, W., Lian, D., Tang, X., Zhao, P., & Gao, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01927 · Cited by 31
Kim, D., Jin, Y., Cho, H., Jones, T., Zhou, Y. M., Fadaie, A., Popov, D., Swaminathan, K., & Walsh, C. J. (2025) · Nature Communications · doi:10.1038/s41467-025-58624-6 · Cited by 29
Actor-agnostic Multi-label Action Recognition with Multi-modal Query
Mondal, A., Nag, S., Prada, J. M., Zhu, X., & Dutta, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00086 · Cited by 27
Son, G., Eo, T., An, J., Oh, D. J., Shin, Y., Rha, H., Kim, Y. J., Lim, Y. J., & Hwang, D. (2022) · Diagnostics · doi:10.3390/diagnostics12081858 · Cited by 23
Sardari, S., Sharifzadeh, S., Daneshkhah, A., Loke, S. W., Palade, V., Duncan, M. J., & Nakisa, B. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108382 · Cited by 20
Wagner, L., Jourdan, S., Mayer, L., Müller, C., Bernhard, L., Kolb, S., Harb, F., Jell, A., Berlet, M., Feußner, H., et al. (2024) · Communications Medicine · doi:10.1038/s43856-024-00581-0 · Cited by 29
Temporal Action Segmentation With High-Level Complex Activity Labels
Ding, G., & Yao, A. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3231099 · Cited by 31
ASPnet: Action Segmentation with Shared-Private Representation of Multiple Data Sources
van Amsterdam, B., Kadkhodamohammadi, A., Luengo, I., & Stoyanov, D. V. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00236 · Cited by 31
Sign Segmentation with Changepoint-Modulated Pseudo-Labelling
Renz, K., Stache, N. C., Fox, N., Varol, G., & Albanie, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00379 · Cited by 38
Uncertainty-Aware Representation Learning for Action Segmentation
Chen, L., Li, M., Duan, Y., Zhou, J., & Lu, J. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/115 · Cited by 22
TSRN: two-stage refinement network for temporal action segmentation
Tian, X., Jin, Y., & Tang, X. (2023) · Pattern Analysis and Applications · doi:10.1007/s10044-023-01166-8 · Cited by 18
Do We Really Need Temporal Convolutions in Action Segmentation?
Du, D., Su, B., Li, Y., Qi, Z., Si, L., & Shan, Y. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00178 · Cited by 26
Digital-Twin-Assisted Skill Learning for 3C Assembly Tasks
Sun, F.-C., Liu, N., Wang, X.-Z., Sun, R.-Z., Miao, S., Kang, Z.-X., Fang, B., Liu, H.-P., Zhao, Y., & Huang, H. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3368148 · Cited by 17
Permutation-Aware Activity Segmentation via Unsupervised Frame-to-Segment Alignment
Tran, Q.-H., Mehmood, A., Ahmed, M., Naufil, M., Zafar, A., Konin, A., & Zia, M. Z. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00630 · Cited by 34
Gao, S., Wu, R., Liu, S., Erkan, U., Toktaş, A., Liu, J., & Tang, X. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02251-y · Cited by 23
“To capture long-term dependencies and improve accuracy, many researchers have proposed network structures based on TCN models [18–22].”
Automated Movement Assessment in Stroke Rehabilitation
Ahmed, T., Thopalli, K., Rikakis, T., Turaga, P. K., Kelliher, A., Huang, J., & Wolf, S. L. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.720650 · Cited by 17
“The HMM, Transformer, and MSTCN++ blocks generate per-frame state probabilities from the input video data.”
MethodInfluentialLoh, S. B., Roy, D., & Fernando, B. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00270 · Cited by 16
Prediction of dissolved gas content in transformer oil based on multi‐information fusion
Yang, T., Fang, Y., Zhang, C., et al. (2024) · High Voltage · doi:10.1049/hve2.12408 · Cited by 16
FIFA: Fast Inference Approximation for Action Segmentation
Souri, Y., Farha, Y. A., Despinoy, F., Francesca, G., & Gall, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92659-5_18 · Cited by 20
Self-citationSD-Net: joint surgical gesture recognition and skill assessment
Zhang, J., Nie, Y., Lyu, Y., Yang, X., Chang, J. M., & Zhang, J. (2021) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-021-02495-x · Cited by 22
Li, Q., Ou, B., Liang, Y., Wang, Y., Yang, X., & Li, L. (2023) · Journal of Advanced Transportation · doi:10.1155/2023/1286977 · Cited by 17
Gloss Prior Guided Visual Feature Learning for Continuous Sign Language Recognition
Guo, L., Xue, W., Лю, Б., Zhang, K., Yuan, T., & Metaxas, D. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3404869 · Cited by 19
Chen, Z., Bi, X., Zhang, Y. X., Yue, J., & Wang, H. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06700-5 · Cited by 15
Feng, G., Zhang, L., Ai, F., Zhang, Y., & Hou, Y. (2022) · Entropy · doi:10.3390/e24081111 · Cited by 15
Speed-Variable Gait Phase Estimation During Ambulation via Temporal Convolutional Network
Guo, Y. Q., Hutabarat, Y., Owaki, D., et al. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3343721 · Cited by 14
DiffAct++: Diffusion Action Segmentation
Liu, D., Li, Q., Dinh, A.-D., Jiang, T., Shah, M., & Xu, C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3509434 · Cited by 24
SF-TMN: SlowFast temporal modeling network for surgical phase recognition
Zhang, B., Sarhan, M. H., Goel, B., Petculescu, S., & Ghanem, A. (2023) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-024-03095-1 · Cited by 17
Benfarhat, I., Goh, V. T., Lim, S. C., Lee, I. E., Sheraz, M., Ngu, E. E., & Chuah, T. C. (2025) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2025.3559421 · Cited by 15
Leveraging triplet loss for unsupervised action segmentation
Bueno-Benito, E., Vecino, B. T., & Dimiccoli, M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00520 · Cited by 21
Unsupervised Anomaly Detection for Power Batteries: A Temporal Convolution Autoencoder Framework
Wang, J., Ye, Y., Wu, M., Zhang, F., Cao, Y., Zhang, Z., Chen, M., & Tang, J. (2024) · Journal of Electrochemical Energy Conversion and Storage · doi:10.1115/1.4065445 · Cited by 14
Tanaka, R., Suzuki, T., & Fujii, K. (2024) · Proceedings of the 7th ACM International Workshop on Multimedia Content Analysis in Sports · doi:10.1145/3689061.3689077 · Cited by 17
Deep learning prediction of error and skill in robotic prostatectomy suturing
Sirajudeen, N., Boal, M. W. E., Anastasiou, D., Xu, J., Stoyanov, D. V., Kelly, J. D., Collins, J. W., Sridhar, A. N., Mazomenos, E. B., & Francis, N. K. (2024) · Surgical Endoscopy · doi:10.1007/s00464-024-11341-5 · Cited by 23
Li, J., Chen, R., & Huang, X. (2022) · Measurement Science and Technology · doi:10.1088/1361-6501/ac632d · Cited by 14
Spatial-temporal graph transformer network for skeleton-based temporal action segmentation
Tian, X., Jin, Y., Zhang, Z., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17276-8 · Cited by 12
Xu, J., Sirajudeen, N., Boal, M. W. E., Francis, N. K., Stoyanov, D. V., & Mazomenos, E. B. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3505818 · Cited by 15
Metadata
- Authors
- Li, S., Farha, Y. A., Liu, Y., Cheng, M.-M., & Gall, J.
- Affiliations
- University of Bonn (DE)
- Nankai University (CN)
- University of Bonn, Bonn, Germany
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2023-06-01
- Volume
- 45
- Issue
- 6
- Pages
- 6647-6658
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3083550439
- Semantic Scholar
- 2db9d36748bf7058cbe72856021500794f711d37
- Cited by
- 349
- References
- 60
- Citations by year
- 2026: 45 · 2025: 69 · 2024: 74 · 2023: 49
- FWCI
- 15.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (GA 1927/4-1)
- ERC Starting (677650)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- temporal action segmentation, long-range temporal dependencies, over-segmentation, untrimmed video, activity recognition, receptive fields
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.