Motion-driven adaptive frame selection strategy for video action recognition
Open accessPublished version of a preprint
Abstract
Frame selection is a fundamental issue in video action recognition. It aims to minimize temporal redundancy and reduce computational cost. Current frame sampling strategies either rely on US based on motion, lacking emphasis on discriminative frames, or employ complex learning models or additional modal information, compromising generalizability. To address these challenges, this paper presents an adaptive frame selection strategy. It filters redundant frames through motion information and models relationships between each frame and others, thereby predicting the significance of each frame. This strategy combines the advantages of motion prior information and supervised learning. During training, frame importance-related constraints are integrated, guiding frames selection with strong discriminative features as inputs for the action recognition network. This frame selection method is integrated with backbone network structures such as TDN, GCTDN, AIM, and tested on three action datasets, Diving-48, UCF101 and HMDB51. The improvement on action recognition achieved is 4.4% on the Diving-48 dataset, 1.9% on the UCF101 dataset and 2.3% on HMDB51 dataset. Experimental results demonstrate that our selection strategy can be integrated with state-of-the-art action recognition models, leading to improved recognition performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
Self-citationHmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · see citing works
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672 · see citing works
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,152 · see citing works
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,121 · see citing works
Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
Qiu, Z., Yao, T., & Mei, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.590 · Cited by 1,944 · see citing works
Is Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663 · see citing works
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,262 · see citing works
Algorithms for Computer Algebra
Geddes, K. O., Czapor, S. R., & Labahn, G. (1992) · Springer eBooks · doi:10.1007/b102438 · Cited by 960 · see citing works
Temporal Segment Networks for Action Recognition in Videos
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868668 · Cited by 928 · see citing works
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909 · see citing works
Global integrated drought monitoring and prediction system
Hao, Z., AghaKouchak, A., Nakhjiri, N., & Farahmand, A. (2014) · Scientific Data · doi:10.1038/sdata.2014.1 · Cited by 815 · see citing works
Clinical implications of dysregulated cytokine production
Slifka, M. K., & Whitton, J. L. (2000) · Journal of Molecular Medicine · doi:10.1007/s001090000086 · Cited by 677 · see citing works
The index of general nonlinear DAEs
Campbell, S. L., & Gear, C. W. (1995) · Numerische Mathematik · doi:10.1007/s002110050165 · Cited by 601 · see citing works
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., & Qiao, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01398 · Cited by 536 · see citing works
TDN: Temporal Difference Networks for Efficient Action Recognition
Wang, L., Tong, Z., Ji, B., & Wu, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00193 · Cited by 513 · see citing works
Quantum scissors: Teleportation of single-mode optical states by means of a nonlocal single photon
Бабичев, С. А., Ries, J., & Lvovsky, A. I. (2003) · Europhysics Letters (EPL) · doi:10.1209/epl/i2003-00504-y · Cited by 471 · see citing works
Spatiotemporal Contrastive Video Representation Learning
Qian, R., Meng, T., Gong, B., Yang, M., Wang, H., Belongie, S., & Cui, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00689 · Cited by 462 · see citing works
Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations
Hamburger, C. (1995) · Annali di Matematica Pura ed Applicata (1923 -) · doi:10.1007/bf01759359 · Cited by 414 · see citing works
Mixing-induced CP asymmetries in inclusive B decays
Beneke, Μ., Gerhard Buchalla, B., & Dunietz, I. (1997) · Physics Letters B · doi:10.1016/s0370-2693(96)01648-6 · Cited by 387 · see citing works
Software Engineering From Auxiliary to Key Technology
Broy, M. (2002) · Springer eBooks · doi:10.1007/978-3-642-59412-0_1 · Cited by 380 · see citing works
Neuromuscular Blocking Drugs in Man
Smith, S. E. (1976) · Springer eBooks · doi:10.1007/978-3-642-45476-9_9 · Cited by 369 · see citing works
RESOUND: Towards Action Recognition Without Representation Bias
Li, Y., Li, Y., & Vasconcelos, N. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_32 · Cited by 349 · see citing works
Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition
Sun, S., Kuang, Z., Sheng, L., Ouyang, W., & Zhang, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00151 · Cited by 347 · see citing works
BEVT: BERT Pretraining of Video Transformers
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Jiang, Y.-G., Zhou, L., & Yuan, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01432 · Cited by 234 · see citing works
SCSampler: Sampling Salient Clips From Video for Efficient Action Recognition
Korbar, B., Tran, D., & Torresani, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00633 · Cited by 227 · see citing works
AdaFrame: Adaptive Frame Selection for Fast Video Recognition
Wu, Z., Xiong, C., Ma, C., Socher, R., & Davis, L. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00137 · Cited by 225 · see citing works
Grouped Spatial-Temporal Aggregation for Efficient Action Recognition
Luo, C., & Yuille, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00561 · Cited by 177 · see citing works
Kar, A., Rai, N., Sikka, K., & Sharma, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.604 · Cited by 170 · see citing works
A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications
Wu, F. L., Wang, Q., Bian, J., Ding, N., Lu, F., Cheng, J. S., Dou, D., & Xiong, H. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3232034 · Cited by 166 · see citing works
SMART Frame Selection for Action Recognition
Gowda, S. N., Rohrbach, M., & Sevilla-Lara, L. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i2.16235 · Cited by 159 · see citing works
Abbott, T., Abdalla, F., Ávila, S., Banerji, M., Baxter, E., Bechtol, K., Becker, M., Bertin, E., Blazek, J., Bridle, S., et al. (2018) · Physical Review D · doi:10.1103/physrevd.99.123505 · Cited by 149 · see citing works
ST-Adapter: Parameter-Efficient Image-To-Video Transfer Learning
Pan, J., Lin, Z., Zhu, X., Shao, J., & Li, H. (2022) · neural information processing systems · doi:10.52202/068431-1919 · Cited by 145 · see citing works
MoLo: Motion-Augmented Long-Short Contrastive Learning for Few-Shot Action Recognition
Wang, X., Zhang, S., Qing, Z., Gao, C., Zhang, Y., Zhao, D., & Sang, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01727 · Cited by 106 · see citing works
AIM: Adapting Image Models for Efficient Video Action RecognitionPreprint
Yang, T., Zhu, Y., Xie, Y., Zhang, A., Chen, C., & Li, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.03024 · Cited by 100 · see citing works
Temporal Query Networks for Fine-grained Video Understanding
Zhang, C., Gupta, A., & Zisserman, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00446 · Cited by 93 · see citing works
MGSampler: An Explainable Sampling Strategy for Video Action Recognition
Zhi, Y., Tong, Z., Wang, L., & Wu, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00154 · Cited by 80 · see citing works
Improved two-stream model for human action recognition
Zhao, Y., Man, K. L., Smith, J. S., Siddique, K., & Guan, S.-U. (2020) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-020-00501-x · Cited by 72 · see citing works
Depthwise Spatio-Temporal STFT Convolutional Neural Networks for Human Action Recognition
Kumawat, S., Verma, M., Nakashima, Y., & Raman, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3076522 · Cited by 56 · see citing works
Attention-Aware Sampling via Deep Reinforcement Learning for Action Recognition
Dong, W., Zhang, Z., & Tan, T. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018247 · Cited by 55 · see citing works
Group Contextualization for Video Recognition
Hao, Y., Zhang, H., Ngo, C., & He, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00100 · Cited by 54 · see citing works
Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition
Xiang, W., Li, C., Wang, B., Wei, X., Hua, X., & Lei Zhang, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20062-5_36 · Cited by 47 · see citing works
Gate-Shift-Fuse for Video Action Recognition
Sudhakaran, S., Escalera, S., & Lanz, O. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268134 · Cited by 35 · see citing works
Learning Correlation Structures for Vision Transformers
Kim, M., Seo, P. H., Schmid, C., & Cho, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01792 · Cited by 33 · see citing works
Zhang, S., Yin, J.-Q., Dang, Y.-H., & Fu, J. (2023) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2024.3386553 · Cited by 31 · see citing works
Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition
Wang, M., Xing, J., Su, J., Chen, J., & Yong, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3173658 · Cited by 29 · see citing works
Multi-scale contrast and relative motion-based key frame extraction
Ejaz, N., Baik, S. W., Majeed, H., Chang, H., & Mehmood, I. (2018) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-018-0280-z · Cited by 25 · see citing works
Search-Map-Search: A Frame Selection Paradigm for Action Recognition
Zhao, M., Yu, Y., Wang, X., Yang, L., & Niu, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01024 · Cited by 17 · see citing works
A Closer Look at Video Sampling for Sequential Action Recognition
Zhang, Y., Zhao, J., Chen, Z., Mi, S., Zhu, H., & Geng, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3274108 · Cited by 7 · see citing works
Flagged first, then by citations
1 of the 57 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Fine-tuning large vision model for multimodal fusion in asphalt pavement crack segmentation
Liu, Z., Wang, J., Teng, X., & Li, N. (2025) · International Journal of Pavement Engineering · doi:10.1080/10298436.2025.2526158 · Cited by 3 · see citing works
Chen, H., Zhou, K., Zheng, Z., Han, L., & Zheng, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131093 · Cited by 1 · see citing works
A collaborative optimization framework for efficient long-sequence Audio-Visual understanding
Ma, Y., Bai, H., Li, X., Shang, B., Zhang, L., & Sun, W. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132367
Action Classification and Recognition
Zhang, Y. (2026) · Springer eBooks · doi:10.1007/978-981-95-7130-7_4
REVQA: Resource-Efficient MLLM Video Question Answering via Redundant Frame Elimination
Zhang, J., Wu, S., Chen, D., Liang, J., Yu, Z., & Chen, Z. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11587018
Taj, N., & Bharathi, P. T. (2026) · Research Square · doi:10.21203/rs.3.rs-10016193/v1
LGMFN: A local–global multimodal fusion network for robust and efficient human action recognition
Li, L., Hu, Z., Ji, G., Qi, X., & Xiao, J. Y. (2026) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2026.104962
Ou, Y., Hu, B., Wu, Y., & He, R. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05805-5
Shirazi, F., & Sadeghi, M. T. (2026) · Electronics · doi:10.3390/electronics15214961
- Authors
- Ding, H., Guo, C., Sun, J., Jiang, X., Shi, H., & Li, J.
- Affiliations
- South Central Minzu University (CN)
- Chinese Academy of Sciences (CN)
- Corresponding author
- Jiang, Xiaoping
- Venue
- EURASIP Journal on Image and Video Processing
- Published
- 2025-06-03
- Volume
- 2025
- Issue
- 1
- Pages
- 12
- ISSN
- 1687-5176 (ISSN-L)
- 1687-5281
- OpenAlex
- W4411006449
- Cited by
- 9
- References
- 57
- Citations by year
- 2026: 8 · 2025: 1
- FWCI
- 3.16
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- EURASIP Journal on Image and Video Processing (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- HAL (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Key R&D Program of China (2023YFC3321600)
- Fundamental Research Funds for Central Universities of the South-Central Minzu University (CZY23026)
- Academic Innovation Teams of South-Central Minzu University (XTZ24006)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Keywords
- video action recognition, frame selection, motion priors, temporal redundancy reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
