Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition
Open accessPreprint — published as 10.1186/s13640-025-00675-2 · Paper page
The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Frame selection is a fundamental issue in video action recognition. It aims to minimize temporal redundancy and reduce computational cost. Current frame sampling strategies either rely on uniform sampling based on motion, lacking emphasis on discriminative frames, or employ complex learning models or additional modal information, compromising generalizability. To address these challenges, this paper presents an adaptive frame selection strategy. It filters redundant frames through motion information and models relationships between each frame and others, thereby predicting the significance of each frame. This strategy combines the advantages of motion prior information and supervised learning. During training, frame importance-related constraints are integrated, guiding frames selection with strong discriminative features as inputs for the action recognition network. This frame selection method is integrated with backbone network structures such as TDN, GCTDN, AIM, and tested on three action datasets, Diving-48, UCF101 and HMDB51.The improvement on action recognition achieved is 4.4% on the Diving-48 dataset, 1.9% on the UCF101 dataset and 2.3% on HMDB51 dataset. Experimental results demonstrate that our selection strategy can be integrated with state-of-the-art action recognition models, leading to improved recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909
Research works printed but not linked: 55
- F. Wu, Q. Wang, J. Bian et al., A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications. IEEE T. Multimedia. 25, 7943-7966, (2023). https://doi.org/10.1109/TMM.2022.3232034 · DOI
- N. Ejaz, S. Baik, H. Majeed, et al. Multi-scale contrast and relative motion-based key frame extraction. J Image Video Proc. 2018, 40 (2018). https://doi.org/10.1186/s13640-018-0280-z · DOI
- S. Zhang, J. Yin, Y. Dang et al., SiT-MLP: A Simple MLP with Point-wise Topology Feature Learning for Skeleton-based Action Recognition. IEEE T. Circ. Syst. Vid. 34(9), 8122-8134, (2024). https://doi.org/10.1109/TCSVT.2024.3386553 · DOI
- Y. Zhao, K.L. Man, J. Smith, et al. Improved two-stream model for human action recognition. J Image Video Proc. 2020, 24 (2020). https://doi.org/10.1186/s13640-020-00501-x · DOI
- M. Kim, P. H. Seo, C. Schmid, et al., Learning Correlation Structures for Vision Transformers. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Seattle, WA, USA, 18941-18951, (2024). https://doi.org/10.1109/CVPR52733.2024.01792 · DOI
- A. Kar, N. Rai, K. Sikka et al., Adascan: Adaptive scan pooling in deep convolutional neural networks for human action recognition in videos. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Venice, Italy, (2017). https://doi.org/10.1109/CVPR.2017.604 · DOI · Paper page
- S. Sun, Z. Kuang, L. Sheng, et al., Optical flow guided feature: A fast and robust motion representation for video action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake, Utah, USA, 1390--1399, (2018). https://doi.org/10.48550/arXiv.1711.11152 · DOI · Paper page
- D. Tran, L. Bourdev, R. Fergus, et al., Learning spatiotemporal features with 3d convolutional networks. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Santiago, Chile, 4489--4497, (2015). https://doi.org/10.1109/ICCV.2015.510 · DOI · Paper page
- C. Feichtenhofer, X3d: Expanding architectures for efficient video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Seattle, WA, USA, 200--210, (2020). https://doi.org/10.1109/CVPR42600.2020.00028 · DOI · Paper page
- Y. Zhang, J. Zhao, Z. Chen, et al., A Closer Look at Video Sampling for Sequential Action Recognition. IEEE T. Circ. syst. Vid. 33(12), 7503-7514, (2023). https://doi.org/10.1109/TCSVT.2023.3274108 · DOI
- K. Simonyan, A. Zisserman, Two-stream convolutional networks for action recognition in videos. in Proc. 27th Inter. Conf. Neu. Infor. Proc. Systems, Montreal, Canada, 568--576, (2014). https://doi.org/10.48550/arXiv.1406.2199 · DOI · Paper page
- C. Luo, A. L. Yuille, Grouped spatial-temporal aggregation for efficient action recognition. in Proc. IEEE Int. Conf. Comput. Vis., Seoul, Korea, 5512--5521, (2019). https://doi.org/10.1109/ICCV.2019.00561 · DOI · Paper page
- Z. Qiu, T. Yao, T. Mei, Learning spatio-temporal representation with pseudo-3D residual networks. in Proc. IEEE Int. Conf. Comput. Vis., Venice, Italy, 5533--5541, (2017). https://doi.org/10.1109/ICCV.2017.590 · DOI · Paper page
- Z. Wu, C. Xiong, C. Y. Ma, et al., AdaFrame: Adaptive frame selection for fast video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Los Angeles, USA, 1278--1287, (2019). https://doi.org/10.1109/CVPR.2019.00137 · DOI · Paper page
- B. Korbar, D. Tran, L. Torresani, SCSampler: Sampling salient clips from video for efficient action recognition. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Seoul, Korea, 6231--6241, (2019). https://doi.org/10.1109/ICCV.2019.00633 · DOI · Paper page
- Y. Zhi, Z. Tong, L. Wang, et al., MGSampler: An Explainable Sampling Strategy for Video Action Recognition. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Virtual Only, (2021). https://doi.org/10.48550/arXiv.2104.09952 · DOI
- D. Tran, H. Wang, L. Torresani, et al., A closer look at spatiotemporal convolutions for action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake, Utah, USA, 6450--6459, (2018). https://doi.org/10.48550/arXiv.1711.11248 · DOI · Paper page
- J. Lin, C. Gan, S. Han, TSM: Temporal shift module for efficient video understanding. in Proc. IEEE/CVF Int. Conf. Comput. Vis., Seoul, Korea, 7082--7092, (2019). https://doi.org/10.1109/ICCV.2019.00718 · DOI · Paper page
- L. Wang, Z. Tong, B. Ji, et al., TDN: Temporal difference networks for efficient action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, 1895--1904, (2021). https://doi.org/10.1109/CVPR46437.2021.00193 · DOI · Paper page
- W. Xiang, C. Li, B. Wang, et al., Spatiotemporal self-attention modeling with temporal patch shift for action recognition. in Proc. Eur. Conf. Comput. Vis., Tel Aviv, Israel, 627--644, (2022). https://doi.org/10.48550/arXiv.2207.13259 · DOI
- Y. Hao, H. Zhang, C. W. Ngo, et al., Group contextualization for video recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., New Orleans, Louisiana, USA, 928--938, (2022). https://doi.org/10.48550/arXiv.2203.09694 · DOI
- Y. H. Ng, M. Hausknecht, S. Vijayanarasimhan, et al., Beyond short snippets: Deep networks for video classification. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Boston, Mass, USA, 4694--4702, (2015). https://doi.org/10.1109/CVPR.2015.7299101 · DOI · Paper page
- S. N. Gowda, M. Rohrbach, L. Sevilla-Lara, Smart frame selection for action recognition. in Proc. 35th AAAI Conf. Artif. Intell., Vancouver, Canada, 35(2), 1451--1459, (2021). https://doi.org/10.48550/arXiv.2012.10671 · DOI
- C. Szegedy, W. Liu, Y. Jia, et al., Going deeper with convolutions. in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Boston, Mass, USA, 1--9, (2015). https://doi.org/10.1109/CVPR.2015.7298594 · DOI · Paper page
- Y. Li, Y. Li, N. Vasconcelos, Resound: Towards action recognition without representation bias. in Eur. Conf. Comput. Vis., Munich, Germany, 513--528, (2018). https://doi.org/10.1007/978-3-030-01231-1\_{32} · DOI · Paper page
- K. Soomro, A. Roshan Zamir, M. Shah, UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. in CRCV-TR-12-01., (2012). https://doi.org/10.48550/arXiv.1212.0402 · DOI · Paper page
- L. Wang, B. Huang, Z. Zhao, et al., VideoMAE v2: Scaling video masked autoencoders with dual masking. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Vancouver, Canada, 14549--14560, (2023). https://doi.org/10.48550/arXiv.2303.16727 · DOI · Paper page
- T. Yang, Y. Zhu, Y. Xie, et al., AIM: Adapting image models for efficient video action recognition. in Int. Conf. Learn. Represent., Kigali, Rwanda, (2023). https://doi.org/10.48550/arXiv.2302.03024 · arXiv · doi:10.48550/arxiv.2302.03024 · Paper page
- M. Zhao, Y. Yu, X. Wang, et al., Search-Map-Search: A frame selection paradigm for action recognition. in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Vancouver, Canada, 10627--10636, (2023). https://doi.org/10.48550/arXiv.2304.10316 · DOI
- W. Dong, Z. Zhang, T. Tan, Attention-aware sampling via deep reinforcement learning for action recognition. in Proc. AAAI Conf. Artif. Intell., Honolulu, Hawaii, USA, 33, 8247--8254, (2019). https://doi.org/10.1609/aaai.v33i01.33018247 · DOI
- L. Wang, Y. Xiong, Z. Wang, et al., Temporal segment networks for action recognition in videos. IEEE Trans. Pattern Anal. Mach. Intell., 41(11), 2740--2755, (2019). https://doi.org/10.1109/TPAMI.2018.2868668 · DOI · Paper page
- S. Zhang, Tfcnet: Temporal fully connected networks for static unbiased temporal reasoning. [Online]. https://arxiv.org/pdf/2203.05928.pdf · arXiv
- G. Bertasius, H. Wang, L. Torresani, Is space-time attention all you need for video understanding?. in Int. Conf. Mach. Learn., Virtual Only, 2(3), 2, 6--8, (2021). https://doi.org/10.48550/arXiv.2102.05095 · DOI · Paper page
- R. Wang, D. Chen, Z. Wu, et al., BEVT: BERT pretraining of video transformers. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., New Orleans, Louisiana, USA, 14713--14723, (2022). https://doi.org/10.48550/arXiv.2112.01529 · DOI · Paper page
- K. He, X. Zhang, S Ren, et al., Deep residual learning for image recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Las Vegas, NV, USA, 770--778, (2016). https://doi.org/10.48550/arXiv.1512.03385 · DOI · Paper page
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. in Int. Conf. Learn. Represent., Vienna, Austria, (2021). https://doi.org/10.48550/arXiv.2010.11929 · DOI · Paper page
- R. Qian, T. Meng, B. Gong, et al., Spatiotemporal contrastive video representation learning. in Proc.IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, (2021). https://doi.org/https://doi.org/10.48550/arXiv.2008.03800 · DOI · Paper page
- S. Kumawat, M. Verma, Y. Nakashima, et al., Depthwise spatio-temporal STFT convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 4839-4851, (2022). https://doi.org/10.1109/TPAMI.2021.3076522 · DOI
- J. T. Pan, Z. Y. Lin, X. T. Zhu, et al., St-adapter: Parameter-efficient image-to-video transfer learning. in Adv. Neural Inform. Process. Syst., 35, 26462--26477, (2022). https://doi.org/https://doi.org/10.48550/arXiv.2206.13559 · arXiv · doi:10.48550/arxiv.2206.13559
- M. Wang, J. Xing, J. Su, et al., Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition. IEEE T. Pattern Anal. 45(3), 3347-3362, (2023). https://doi.org/10.1109/TPAMI.2022.3173658 · DOI
- C. Zhang, A. Gupta, A. Zisserman, Temporal query networks for fine-grained video understanding. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Virtual Only, 4486--4496, (2021). https://doi.org/10.48550/arXiv.2104.09496 · DOI · Paper page
- S. Sudhakaran, S. Escalera, O. Lanz, Gate-shift-fuse for video action recognition. IEEE T. Pattern Anal. 45 (9), 10913-10928, (2023). https://doi.org/10.1109/TPAMI.2023.3268134 · DOI
- H. Kuehne, H. Jhuang, E. Garrote, et al., HMDB: a large video database for human motion recognition. In Proc. IEEE/CVF Int. Conf. Comput. Vis., 2556-2563, (2021). https://doi.org/10.1109/ICCV.2011.61265 · DOI · Paper page
- X. Wang, S. Zhang, Z. Qing, et al., MoLo: Motion-augmented long-short contrastive learning for few-shot action recognition. in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Vancouver, 18011-18021, (2023). https://doi.org/10.1109/CVPR52729.2023.01727 · DOI
- Campbell, S. L. and Gear, C. W. (1995) The index of general nonlinear {D}{A}{E}{S}. Numer. {M}ath. 72(2): 173--196 · DOI
- Slifka, M. K. and Whitton, J. L. (2000) Clinical implications of dysregulated cytokine production. J. {M}ol. {M}ed. 78: 74--80 https://doi.org/10.1007/s001090000086 · DOI · Paper page
- Hamburger, C. (1995) Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations. Ann. Mat. Pura. Appl. 169(2): 321--354 · DOI
- Geddes, K. O. and Czapor, S. R. and Labahn, G. (1992) Algorithms for {C}omputer {A}lgebra. Kluwer, Boston · DOI · Paper page
- Broy, M. Software engineering---from auxiliary to key technologies. In: Broy, M. and Denert, E. (Eds.) Software Pioneers, 1992, Springer, New {Y}ork, 10--13 · DOI
- Smith, S. E. (1976) Neuromuscular blocking drugs in man. Springer, Heidelberg, 593--660, Neuromuscular junction. {H}andbook of experimental pharmacology, 42, Zaimis, E. · DOI
- Chung, S. T. and Morris, R. L.. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae. Paper presented at the 3rd international symposium on the genetics of industrial microorganisms, University of {W}isconsin, {M}adison, 4--9 June 1978. 1978
- Hao, Z. and AghaKouchak, A. and Nakhjiri, N. and Farahmand, A.. Global integrated drought monitoring and prediction system (GIDMaPS) data sets. figshare https://doi.org/10.6084/m9.figshare.853801. 2014 · DOI · Paper page
- Babichev, S. A. and Ries, J. and Lvovsky, A. I.. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon. Preprint at https://arxiv.org/abs/quant-ph/0208066v1. 2002 · DOI · Paper page
- Beneke, M. and Buchalla, G. and Dunietz, I. (1997) Mixing induced {CP} asymmetries in inclusive {B} decays. Phys. {L}ett. B393: 132-142 gr-gc, 0707.3168, arXiv · DOI
- Abbott, T. M. C. and others (2019) {Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing}. Phys. Rev. D 99(12): 123505 https://doi.org/10.1103/PhysRevD.99.123505, FERMILAB-PUB-18-507-PPD, astro-ph.CO, arXiv, 1810.02499, DES · DOI · Paper page
Metadata
- Authors
- Ding, H., Guo, C., Sun, J., Jiang, X., Shi, H., & Li, J.
- Affiliations
- South Central Minzu University (CN)
- Chinese Academy of Sciences (CN)
- University Clermont Auvergne: Universite Clermont Auvergne
- Corresponding author
- Jiang, Xiaoping
- Venue
- EURASIP Journal on Image and Video Processing
- Published
- 2025-01-09
- Volume
- 2025
- Issue
- 1
- Pages
- 12
- ISSN
- 1687-5176 (ISSN-L)
- OpenAlex
- W4411006449
- Cited by
- 9
- References
- 1
- Citations by year
- 2026: 8 · 2025: 1
- FWCI
- 3.16
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- EURASIP Journal on Image and Video Processing (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- HAL (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Key Research and Development Program of China (2023YFC3321600)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Keywords
- video action recognition, frame selection, motion priors, temporal redundancy reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition. https://app.cochat.ai/papers/10.21203/rs.3.rs-5495634/v1
- The paper
- Ding, H., Guo, C., Sun, J., Jiang, X., Shi, H., & Li, J. (2025). Motion-driven Adaptive Frame Selection Strategy for Video Action Recognition. EURASIP Journal on Image and Video Processing. https://doi.org/10.21203/rs.3.rs-5495634/v1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.