CoChat

Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition

Open access
Ding, Hao, Guo, Chen, Sun, Jing, Jiang, Xiaoping, Shi, Hongling, Li, Jianjin · 2025 · EURASIP Journal on Image and Video Processing

Preprint — published as 10.1186/s13640-025-00675-2 · Paper page

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

Frame selection is a fundamental issue in video action recognition. It aims to minimize temporal redundancy and reduce computational cost. Current frame sampling strategies either rely on uniform sampling based on motion, lacking emphasis on discriminative frames, or employ complex learning models or additional modal information, compromising generalizability. To address these challenges, this paper presents an adaptive frame selection strategy. It filters redundant frames through motion information and models relationships between each frame and others, thereby predicting the significance of each frame. This strategy combines the advantages of motion prior information and supervised learning. During training, frame importance-related constraints are integrated, guiding frames selection with strong discriminative features as inputs for the action recognition network. This frame selection method is integrated with backbone network structures such as TDN, GCTDN, AIM, and tested on three action datasets, Diving-48, UCF101 and HMDB51.The improvement on action recognition achieved is 4.4% on the Diving-48 dataset, 1.9% on the UCF101 dataset and 2.3% on HMDB51 dataset. Experimental results demonstrate that our selection strategy can be integrated with state-of-the-art action recognition models, leading to improved recognition performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.21203/rs.3.rs-5495634/v1

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 55

  1. F. Wu, Q. Wang, J. Bian et al., A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications. IEEE T. Multimedia. 25, 7943-7966, (2023). https://doi.org/10.1109/TMM.2022.3232034 · DOI
  2. N. Ejaz, S. Baik, H. Majeed, et al. Multi-scale contrast and relative motion-based key frame extraction. J Image Video Proc. 2018, 40 (2018). https://doi.org/10.1186/s13640-018-0280-z · DOI
  3. S. Zhang, J. Yin, Y. Dang et al., SiT-MLP: A Simple MLP with Point-wise Topology Feature Learning for Skeleton-based Action Recognition. IEEE T. Circ. Syst. Vid. 34(9), 8122-8134, (2024). https://doi.org/10.1109/TCSVT.2024.3386553 · DOI
  4. Y. Zhao, K.L. Man, J. Smith, et al. Improved two-stream model for human action recognition. J Image Video Proc. 2020, 24 (2020). https://doi.org/10.1186/s13640-020-00501-x · DOI
  5. M. Kim, P. H. Seo, C. Schmid, et al., Learning Correlation Structures for Vision Transformers. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Seattle, WA, USA, 18941-18951, (2024). https://doi.org/10.1109/CVPR52733.2024.01792 · DOI
  6. A. Kar, N. Rai, K. Sikka et al., Adascan: Adaptive scan pooling in deep convolutional neural networks for human action recognition in videos. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Venice, Italy, (2017). https://doi.org/10.1109/CVPR.2017.604 · DOI · Paper page
  7. S. Sun, Z. Kuang, L. Sheng, et al., Optical flow guided feature: A fast and robust motion representation for video action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake, Utah, USA, 1390--1399, (2018). https://doi.org/10.48550/arXiv.1711.11152 · DOI · Paper page
  8. D. Tran, L. Bourdev, R. Fergus, et al., Learning spatiotemporal features with 3d convolutional networks. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Santiago, Chile, 4489--4497, (2015). https://doi.org/10.1109/ICCV.2015.510 · DOI · Paper page
  9. C. Feichtenhofer, X3d: Expanding architectures for efficient video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Seattle, WA, USA, 200--210, (2020). https://doi.org/10.1109/CVPR42600.2020.00028 · DOI · Paper page
  10. Y. Zhang, J. Zhao, Z. Chen, et al., A Closer Look at Video Sampling for Sequential Action Recognition. IEEE T. Circ. syst. Vid. 33(12), 7503-7514, (2023). https://doi.org/10.1109/TCSVT.2023.3274108 · DOI
  11. K. Simonyan, A. Zisserman, Two-stream convolutional networks for action recognition in videos. in Proc. 27th Inter. Conf. Neu. Infor. Proc. Systems, Montreal, Canada, 568--576, (2014). https://doi.org/10.48550/arXiv.1406.2199 · DOI · Paper page
  12. C. Luo, A. L. Yuille, Grouped spatial-temporal aggregation for efficient action recognition. in Proc. IEEE Int. Conf. Comput. Vis., Seoul, Korea, 5512--5521, (2019). https://doi.org/10.1109/ICCV.2019.00561 · DOI · Paper page
  13. Z. Qiu, T. Yao, T. Mei, Learning spatio-temporal representation with pseudo-3D residual networks. in Proc. IEEE Int. Conf. Comput. Vis., Venice, Italy, 5533--5541, (2017). https://doi.org/10.1109/ICCV.2017.590 · DOI · Paper page
  14. Z. Wu, C. Xiong, C. Y. Ma, et al., AdaFrame: Adaptive frame selection for fast video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Los Angeles, USA, 1278--1287, (2019). https://doi.org/10.1109/CVPR.2019.00137 · DOI · Paper page
  15. B. Korbar, D. Tran, L. Torresani, SCSampler: Sampling salient clips from video for efficient action recognition. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Seoul, Korea, 6231--6241, (2019). https://doi.org/10.1109/ICCV.2019.00633 · DOI · Paper page
  16. Y. Zhi, Z. Tong, L. Wang, et al., MGSampler: An Explainable Sampling Strategy for Video Action Recognition. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Virtual Only, (2021). https://doi.org/10.48550/arXiv.2104.09952 · DOI
  17. D. Tran, H. Wang, L. Torresani, et al., A closer look at spatiotemporal convolutions for action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake, Utah, USA, 6450--6459, (2018). https://doi.org/10.48550/arXiv.1711.11248 · DOI · Paper page
  18. J. Lin, C. Gan, S. Han, TSM: Temporal shift module for efficient video understanding. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Seoul, Korea, 7082--7092, (2019). https://doi.org/10.1109/ICCV.2019.00718 · DOI · Paper page
  19. L. Wang, Z. Tong, B. Ji, et al., TDN: Temporal difference networks for efficient action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, 1895--1904, (2021). https://doi.org/10.1109/CVPR46437.2021.00193 · DOI · Paper page
  20. W. Xiang, C. Li, B. Wang, et al., Spatiotemporal self-attention modeling with temporal patch shift for action recognition. in Proc. Eur. Conf. Comput. Vis., Tel Aviv, Israel, 627--644, (2022). https://doi.org/10.48550/arXiv.2207.13259 · DOI
  21. Y. Hao, H. Zhang, C. W. Ngo, et al., Group contextualization for video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., New Orleans, Louisiana, USA, 928--938, (2022). https://doi.org/10.48550/arXiv.2203.09694 · DOI
  22. Y. H. Ng, M. Hausknecht, S. Vijayanarasimhan, et al., Beyond short snippets: Deep networks for video classification. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Boston, Mass, USA, 4694--4702, (2015). https://doi.org/10.1109/CVPR.2015.7299101 · DOI · Paper page
  23. S. N. Gowda, M. Rohrbach, L. Sevilla-Lara, Smart frame selection for action recognition. in Proc. 35th AAAI Conf. Artif. Intell., Vancouver, Canada, 35(2), 1451--1459, (2021). https://doi.org/10.48550/arXiv.2012.10671 · DOI
  24. C. Szegedy, W. Liu, Y. Jia, et al., Going deeper with convolutions. in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Boston, Mass, USA, 1--9, (2015). https://doi.org/10.1109/CVPR.2015.7298594 · DOI · Paper page
  25. Y. Li, Y. Li, N. Vasconcelos, Resound: Towards action recognition without representation bias. in Eur. Conf. Comput. Vis., Munich, Germany, 513--528, (2018). https://doi.org/10.1007/978-3-030-01231-1\_{32} · DOI · Paper page
  26. K. Soomro, A. Roshan Zamir, M. Shah, UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. in CRCV-TR-12-01., (2012). https://doi.org/10.48550/arXiv.1212.0402 · DOI · Paper page
  27. L. Wang, B. Huang, Z. Zhao, et al., VideoMAE v2: Scaling video masked autoencoders with dual masking. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Vancouver, Canada, 14549--14560, (2023). https://doi.org/10.48550/arXiv.2303.16727 · DOI · Paper page
  28. T. Yang, Y. Zhu, Y. Xie, et al., AIM: Adapting image models for efficient video action recognition. in Int. Conf. Learn. Represent., Kigali, Rwanda, (2023). https://doi.org/10.48550/arXiv.2302.03024 · arXiv · doi:10.48550/arxiv.2302.03024 · Paper page
  29. M. Zhao, Y. Yu, X. Wang, et al., Search-Map-Search: A frame selection paradigm for action recognition. in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Vancouver, Canada, 10627--10636, (2023). https://doi.org/10.48550/arXiv.2304.10316 · DOI
  30. W. Dong, Z. Zhang, T. Tan, Attention-aware sampling via deep reinforcement learning for action recognition. in Proc. AAAI Conf. Artif. Intell., Honolulu, Hawaii, USA, 33, 8247--8254, (2019). https://doi.org/10.1609/aaai.v33i01.33018247 · DOI
  31. L. Wang, Y. Xiong, Z. Wang, et al., Temporal segment networks for action recognition in videos. IEEE Trans. Pattern Anal. Mach. Intell., 41(11), 2740--2755, (2019). https://doi.org/10.1109/TPAMI.2018.2868668 · DOI · Paper page
  32. S. Zhang, Tfcnet: Temporal fully connected networks for static unbiased temporal reasoning. [Online]. https://arxiv.org/pdf/2203.05928.pdf · arXiv
  33. G. Bertasius, H. Wang, L. Torresani, Is space-time attention all you need for video understanding?. in Int. Conf. Mach. Learn., Virtual Only, 2(3), 2, 6--8, (2021). https://doi.org/10.48550/arXiv.2102.05095 · DOI · Paper page
  34. R. Wang, D. Chen, Z. Wu, et al., BEVT: BERT pretraining of video transformers. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., New Orleans, Louisiana, USA, 14713--14723, (2022). https://doi.org/10.48550/arXiv.2112.01529 · DOI · Paper page
  35. K. He, X. Zhang, S Ren, et al., Deep residual learning for image recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Las Vegas, NV, USA, 770--778, (2016). https://doi.org/10.48550/arXiv.1512.03385 · DOI · Paper page
  36. A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. in Int. Conf. Learn. Represent., Vienna, Austria, (2021). https://doi.org/10.48550/arXiv.2010.11929 · DOI · Paper page
  37. R. Qian, T. Meng, B. Gong, et al., Spatiotemporal contrastive video representation learning. in Proc.IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, (2021). https://doi.org/https://doi.org/10.48550/arXiv.2008.03800 · DOI · Paper page
  38. S. Kumawat, M. Verma, Y. Nakashima, et al., Depthwise spatio-temporal STFT convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 4839-4851, (2022). https://doi.org/10.1109/TPAMI.2021.3076522 · DOI
  39. J. T. Pan, Z. Y. Lin, X. T. Zhu, et al., St-adapter: Parameter-efficient image-to-video transfer learning. in Adv. Neural Inform. Process. Syst., 35, 26462--26477, (2022). https://doi.org/https://doi.org/10.48550/arXiv.2206.13559 · arXiv · doi:10.48550/arxiv.2206.13559
  40. M. Wang, J. Xing, J. Su, et al., Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition. IEEE T. Pattern Anal. 45(3), 3347-3362, (2023). https://doi.org/10.1109/TPAMI.2022.3173658 · DOI
  41. C. Zhang, A. Gupta, A. Zisserman, Temporal query networks for fine-grained video understanding. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, 4486--4496, (2021). https://doi.org/10.48550/arXiv.2104.09496 · DOI · Paper page
  42. S. Sudhakaran, S. Escalera, O. Lanz, Gate-shift-fuse for video action recognition. IEEE T. Pattern Anal. 45 (9), 10913-10928, (2023). https://doi.org/10.1109/TPAMI.2023.3268134 · DOI
  43. H. Kuehne, H. Jhuang, E. Garrote, et al., HMDB: a large video database for human motion recognition. In Proc. IEEE/CVF Int. Conf. Comput. Vis., 2556-2563, (2021). https://doi.org/10.1109/ICCV.2011.61265 · DOI · Paper page
  44. X. Wang, S. Zhang, Z. Qing, et al., MoLo: Motion-augmented long-short contrastive learning for few-shot action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Vancouver, 18011-18021, (2023). https://doi.org/10.1109/CVPR52729.2023.01727 · DOI
  45. Campbell, S. L. and Gear, C. W. (1995) The index of general nonlinear {D}{A}{E}{S}. Numer. {M}ath. 72(2): 173--196 · DOI
  46. Slifka, M. K. and Whitton, J. L. (2000) Clinical implications of dysregulated cytokine production. J. {M}ol. {M}ed. 78: 74--80 https://doi.org/10.1007/s001090000086 · DOI · Paper page
  47. Hamburger, C. (1995) Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations. Ann. Mat. Pura. Appl. 169(2): 321--354 · DOI
  48. Geddes, K. O. and Czapor, S. R. and Labahn, G. (1992) Algorithms for {C}omputer {A}lgebra. Kluwer, Boston · DOI · Paper page
  49. Broy, M. Software engineering---from auxiliary to key technologies. In: Broy, M. and Denert, E. (Eds.) Software Pioneers, 1992, Springer, New {Y}ork, 10--13 · DOI
  50. Smith, S. E. (1976) Neuromuscular blocking drugs in man. Springer, Heidelberg, 593--660, Neuromuscular junction. {H}andbook of experimental pharmacology, 42, Zaimis, E. · DOI
  51. Chung, S. T. and Morris, R. L.. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae. Paper presented at the 3rd international symposium on the genetics of industrial microorganisms, University of {W}isconsin, {M}adison, 4--9 June 1978. 1978
  52. Hao, Z. and AghaKouchak, A. and Nakhjiri, N. and Farahmand, A.. Global integrated drought monitoring and prediction system (GIDMaPS) data sets. figshare https://doi.org/10.6084/m9.figshare.853801. 2014 · DOI · Paper page
  53. Babichev, S. A. and Ries, J. and Lvovsky, A. I.. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon. Preprint at https://arxiv.org/abs/quant-ph/0208066v1. 2002 · DOI · Paper page
  54. Beneke, M. and Buchalla, G. and Dunietz, I. (1997) Mixing induced {CP} asymmetries in inclusive {B} decays. Phys. {L}ett. B393: 132-142 gr-gc, 0707.3168, arXiv · DOI
  55. Abbott, T. M. C. and others (2019) {Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing}. Phys. Rev. D 99(12): 123505 https://doi.org/10.1103/PhysRevD.99.123505, FERMILAB-PUB-18-507-PPD, astro-ph.CO, arXiv, 1810.02499, DES · DOI · Paper page

Metadata

Authors
Ding, H., Guo, C., Sun, J., Jiang, X., Shi, H., & Li, J.
Affiliations
  • South Central Minzu University (CN)
  • Chinese Academy of Sciences (CN)
  • University Clermont Auvergne: Universite Clermont Auvergne
Corresponding author
Jiang, Xiaoping
Venue
EURASIP Journal on Image and Video Processing
Published
2025-01-09
Volume
2025
Issue
1
Pages
12
ISSN
1687-5176 (ISSN-L)
OpenAlex
W4411006449
Cited by
9
References
1
Citations by year
2026: 8 · 2025: 1
FWCI
3.16
Type
journal article
Language
en
Open access
Locations
  • EURASIP Journal on Image and Video Processing (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • HAL (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Key Research and Development Program of China (2023YFC3321600)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
Keywords
video action recognition, frame selection, motion priors, temporal redundancy reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition. https://app.cochat.ai/papers/10.21203/rs.3.rs-5495634/v1
The paper
Ding, H., Guo, C., Sun, J., Jiang, X., Shi, H., & Li, J. (2025). Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition. EURASIP Journal on Image and Video Processing. https://doi.org/10.21203/rs.3.rs-5495634/v1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.