Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
The DOI matches its Crossref record; 76 of the 78 references it cites were found in a registry or an index.
Abstract
Models based on deep convolutional networks have dominated recent image interpretation tasks; we investigate whether models which are also recurrent are effective for tasks involving sequences, visual and otherwise. We describe a class of recurrent convolutional architectures which is end-to-end trainable and suitable for large-scale visual understanding tasks, and demonstrate the value of these models for activity recognition, image captioning, and video description. In contrast to previous models which assume a fixed visual representation or perform simple temporal averaging for sequential processing, recurrent convolutional models are "doubly deep" in that they learn compositional representations in space and time. Learning long-term dependencies is possible when nonlinearities are incorporated into the network state updates. Differentiable recurrent models are appealing in that they can directly map variable-length inputs (e.g., videos) to variable-length outputs (e.g., natural language text) and can model complex temporal dynamics; yet they can be optimized with backpropagation. Our recurrent sequence models are directly connected to modern visual convolutional network models and can be jointly trained to learn temporal dynamics and convolutional perceptual representations. Our results show that such models have distinct advantages over state-of-the-art models for recognition or generation which are separately defined or optimized.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Cited by
Items 301–400 of 1,001
Visual Recognition Based on Deep Learning for Navigation Mark Classification
Pan, M., Liu, Y., Cao, J., Li, Y., Li, C., & Chen, C. (2020) · IEEE Access · doi:10.1109/access.2020.2973856 · Cited by 79
Machine Learning for Human Activity Recognition: State-of-the-Art Techniques and Emerging Trends
Hossen, M. A., & Abas, P. E. (2025) · Journal of Imaging · doi:10.3390/jimaging11030091 · Cited by 79
GAIM: Graph Attention Interaction Model for Collective Activity Recognition
Lu, L., Lu, Y., Yu, R., Di, H., Zhang, L., & Wang, S. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2930344 · Cited by 78
Fighting Fake News: Two Stream Network for Deepfake Detection via Learnable SRM
Han, B., Han, X., Zhang, H., Li, J., & Cao, X. (2021) · IEEE Transactions on Biometrics, Behavior, and Identity Science · doi:10.1109/tbiom.2021.3065735 · Cited by 77
Detecting Deepfake Videos with Temporal Dropout 3DCNN
Zhang, D., Li, C., Lin, F., Zeng, D., & Ge, S. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/178 · Cited by 77
Howard, J. P., Tan, J., Shun-Shin, M. J., Mahdi, D., Nowbar, A. N., Arnold, A. D., Ahmad, Y., McCartney, P., Zolgharni, M., Linton, N., et al. (2020) · Journal of Medical Artificial Intelligence · doi:10.21037/jmai.2019.10.03 · Cited by 77
Graph-based High-order Relation Modeling for Long-term Action Recognition
Zhou, J., Lin, K.-Y., Li, H., & Zheng, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00887 · Cited by 77
Neural attention for image captioning: review of outstanding methods
Zohourianshahzadi, Z., & Kalita, J. K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10092-2 · Cited by 77
Multimodal Convolutional Neural Networks for Matching Image and Sentence
Ma, L., Lu, Z., Shang, L., & Li, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.06063 · Cited by 41
Bhooshan, R. S., Kumaraswamy, S., & Sasidharan, S. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00693-9 · Cited by 77
Huang, B., Ji, Z., Zhai, R., Xiao, C., Yang, F., Yang, B., & Wang, Y. (2021) · GPS Solutions · doi:10.1007/s10291-021-01115-0 · Cited by 74
The future of bone regeneration: integrating AI into tissue engineering
MacKay, B., Marshall, K. M., Grant‐Jacob, J. A., Kanczler, J. M., Eason, R. W., Oreffo, R. O. C., & Mills, B. (2021) · Biomedical Physics & Engineering Express · doi:10.1088/2057-1976/ac154f · Cited by 74
Common Subspace for Model and Similarity: Phrase Learning for Caption Generation from Images
Ushiku, Y., Yamaguchi, M., Mukuta, Y., & Harada, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.306 · Cited by 76
Non-intrusive Driver Drowsiness Detection based on Face and Eye Tracking
Bamidele, A. A., Kamardin, K., Aziz, N. S. N. A., Mohd, S., Ahmed, I. S., Azizan, A., Aini, N., & Kaidi, H. M. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0100775 · Cited by 74
Deep Image-to-Video Adaptation and Fusion Networks for Action Recognition
Liu, Y., Lu, Z., Li, J., Yang, T., & Yao, C. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2957930 · Cited by 73
Chen, C., Xiao, R., Chen, H., Lv, N., & Chen, S. (2020) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2020.08.028 · Cited by 72
A Dataset for Movie DescriptionPreprint
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.02530 · Cited by 72
Self-citationQuestion Relevance in VQA: Identifying Non-Visual And False-Premise Questions
Ray, A., Christie, G., Bansal, M., Batra, D., & Parikh, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1090 · Cited by 48
Context Matters: Refining Object Detection in Video with Recurrent Neural Networks
Tripathi, S., Lipton, Z. C., Belongie, S., & Nguyen, T. Q. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.44 · Cited by 71
Aesthetic Attributes Assessment of Images
Jin, X., Wu, L., Zhao, G., Li, X., Zhang, X., Ge, S., Zou, D., Zhou, B., & Zhou, X. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350970 · Cited by 71
Automated detection of part quality during two-photon lithography via deep learning
Lee, X. Y., Saha, S. K., Sarkar, S., & Giera, B. (2020) · Additive manufacturing · doi:10.1016/j.addma.2020.101444 · Cited by 70
A Survey of Deep Learning Based Models for Human Activity Recognition
Khan, N. S., & Ghani, M. S. (2021) · Wireless Personal Communications · doi:10.1007/s11277-021-08525-w · Cited by 69
Hybrid CNN-LSTM models for river flow prediction
Li, X., Xu, W., Ren, M., Jiang, Y., & Fu, G. (2022) · Water Science & Technology Water Supply · doi:10.2166/ws.2022.170 · Cited by 69
Wang, G., Wei, W., Jiang, J., Ning, C., Chen, H., Huang, J., Liang, B., Zang, N., Liao, Y., Chen, R., et al. (2019) · Epidemiology and Infection · doi:10.1017/s095026881900075x · Cited by 69
Di, Y., & Wang, E. (2021) · Rock Mechanics and Rock Engineering · doi:10.1007/s00603-020-02314-w · Cited by 68
Marinho, F. P., Rocha, P. A. C., Neto, A. R. R., & Bezerra, F. D. V. (2022) · Journal of Solar Energy Engineering · doi:10.1115/1.4056122 · Cited by 68
Gupta, H., Imran, J., & Sharma, C. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-023-04585-x · Cited by 69
Automatic image caption generation using deep learning
Verma, A., Yadav, A. K., Kumar, M., & Yadav, D. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15555-y · Cited by 68
Meng, B., Liu, X., & Wang, X. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5893-9 · Cited by 68
Video Storytelling: Textual Summaries for Events
Li, J., Wong, Y., Zhao, Q., & Kankanhalli, M. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2930041 · Cited by 68
Scene Graph Parsing as Dependency Parsing
Wang, Y.-S., Liu, C., Zeng, X., & Yuille, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1037 · Cited by 68
Novel deep learning model for facial expression recognition based on maximum boosted CNN and LSTM
Rajan, S., Chenniappan, P., Devaraj, S., & Madian, N. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2019.1188 · Cited by 68
Li, Z., Jiang, H., Kong, L., Chen, Y., Lang, K., Fan, X., Zhang, L., & Pian, C. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008767 · Cited by 67
Video description: A comprehensive survey of deep learning approaches
Rafiq, G., Rafiq, M. T., & Choi, G. S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10414-6 · Cited by 67
Video deepfake detection using Particle Swarm Optimization improved deep neural networks
Cunha, L., Zhang, L., Sowan, B. I., Lim, C. P., & Kong, Y. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09536-x · Cited by 69
Deep Learning for Image Sequence Classification of Astronomical Events
Carrasco-Davis, R., Cabrera-Vives, G., Förster, F., Estévez, P. A., Huijse, P., Protopapas, P., Reyes, I., Martínez-Palomera, J., & Donoso, C. (2019) · Publications of the Astronomical Society of the Pacific · doi:10.1088/1538-3873/aaef12 · Cited by 47
Multi-Modal Deep Analysis for Multimedia
Zhu, W., Wang, X. Y., & Li, H. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2940647 · Cited by 67
Integrating Part of Speech Guidance for Image Captioning
Zhang, J., Mei, K., Zheng, Y., & Fan, J. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2976552 · Cited by 67
Serpush, F., & Rezaei, M. (2021) · SN Computer Science · doi:10.1007/s42979-021-00484-0 · Cited by 67
Kawka, M., Gall, T., Fang, C., Liu, R., & Jiao, L. R. (2021) · Intelligent Surgery · doi:10.1016/j.isurg.2021.03.001 · Cited by 67
COMISR: Compression-Informed Video Super-Resolution
Li, Y., Jin, P., Yang, F. Y., Liu, C., Yang, M., & Milanfar, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00254 · Cited by 67
Yan, J., & Wang, Z. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.02.009 · Cited by 66
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech
Kamper, H., Shakhnarovich, G., & Livescu, K. (2018) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2018.2872106 · Cited by 66
Yousif, A., Niu, Z., Chambua, J., & Khan, Z. Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.021 · Cited by 66
DeepFake on Face and Expression Swap: A Review
Waseem, S., Abu Bakar, S. A. R. S., Ahmed, B. A., Omar, Z., Eisa, T. A. E., & Dalam, M. E. E. (2023) · IEEE Access · doi:10.1109/access.2023.3324403 · Cited by 65
Chen, C., Zhu, W., Oczak, M., Maschat, K., Baumgartner, J., Larsen, M. L. V., & Norton, T. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105580 · Cited by 65
Ma, X., Zhang, K., Wang, J., Yao, C., Yang, Y., Sun, H. X., & Yao, J. P. (2021) · SPE Journal · doi:10.2118/208604-pa · Cited by 65
Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines
Singh, M., Hoque, X., Zeng, D., Wang, Y., Ikeda, K., & Dhall, A. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3577190.3614164 · Cited by 66
Automated detection of bird roosts using NEXRAD radar data and Convolutional Neural Networks
Chilson, C., Avery, K., McGovern, A., Bridge, E. S., Sheldon, D. R., & Kelly, J. F. (2018) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.92 · Cited by 65
A Sensor Network Approach for Violence Detection in Smart Cities Using Deep Learning
Baba, M., Gui, V., Cernăzanu-Glăvan, C., & Pescaru, D. (2019) · Sensors · doi:10.3390/s19071676 · Cited by 65
Yue, Z., Ding, S., Zhao, W., Wang, H., Ma, J., Zhang, Y., & Zhang, Y. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2922682 · Cited by 65
Cai, J., Zhang, Y., & Cai, H. (2019) · Automation in Construction · doi:10.1016/j.autcon.2019.102886 · Cited by 65
Obj2Text: Generating Visually Descriptive Language from Object Layouts
Yin, X., & Ordóñez, V. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1017 · Cited by 65
Multimodal Encoder-Decoder Attention Networks for Visual Question Answering
Chen, C., Han, D., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.2975093 · Cited by 57
Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos
Amerini, I., & Caldelli, R. (2020) · Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3369412.3395070 · Cited by 64
An Efficient Framework for Dense Video Captioning
Suin, M., & Rajagopalan, A. N. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6881 · Cited by 64
Optimizing CNN-LSTM neural networks with PSO for anomalous query access control
Kim, T.-Y., & Cho, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.07.154 · Cited by 64
Sharma, H., & Padha, D. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10488-2 · Cited by 64
Perceptually-guided deep neural networks for ego-action prediction: Object grasping
González-Díaz, I., Benois‐Pineau, J., Domenger, J., Cattaert, D., & de Rugy, A. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.11.013 · Cited by 64
A Deep Learning‐Based Approach to Enable Action Recognition for Construction Equipment
Zhang, J., Zi, L., Hou, Y., Wang, M., Jiang, W., & Deng, J. D. (2020) · Advances in Civil Engineering · doi:10.1155/2020/8812928 · Cited by 64
Udriștoiu, A. L., Cazacu, I. M., Gruionu, L. G., Gruionu, G., Iacob, A. V., Burtea, D. E., Ungureanu, B. S., Costache, M. I., Constantin, A., Popescu, C. F., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251701 · Cited by 63
Stoeve, M., Schuldhaus, D., Gamp, A., Zwick, C., & Eskofier, B. M. (2021) · Sensors · doi:10.3390/s21093071 · Cited by 63
Machine Learning Assisted Inertia Estimation Using Ambient Measurements
Tuo, M., & Li, X. (2023) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2023.3269732 · Cited by 63
An FPGA implementation of a long short-term memory neural network
Ferreira, J. C., & Fonseca, J. P. (2016) · International Conference on ReConFigurable Computing and FPGAs (ReConFig) · doi:10.1109/reconfig.2016.7857151 · Cited by 63
Shin, A., Ishii, M., & Narihira, T. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01547-8 · Cited by 64
Cross-Modality Compensation Convolutional Neural Networks for RGB-D Action Recognition
Cheng, J. S., Ren, Z., Zhang, Q., Gao, X., & Hao, F. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3076165 · Cited by 63
Natural Language Understanding with Distributed RepresentationPreprint
Cho, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07916 · Cited by 63
Liu, T., Wang, J., Huang, X., Lu, Y., & Bao, J. (2021) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2021.01.017 · Cited by 62
Attention-Aware Sampling via Deep Reinforcement Learning for Action Recognition
Dong, W., Zhang, Z., & Tan, T. N. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018247 · Cited by 62
Unsupervised and Semi-Supervised Image Classification With Weak Semantic Consistency
Zhang, C., Cheng, J., & Tian, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2903628 · Cited by 62
Vision-to-Language Tasks Based on Attributes and Attention Mechanism
Li, X., Yuan, A., & Lu, X. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2914351 · Cited by 62
Mu, L., Wang, L., Wang, Y., Chen, X., & Han, W. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2956318 · Cited by 62
Sparse Black-Box Video Attack with Reinforcement Learning
Wei, X., Yan, H., & Li, B. H. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01604-w · Cited by 62
A multimodal fusion approach for image captioning
Zhao, D., Chang, Z., & Guo, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.004 · Cited by 61
Attention based automated radiology report generation using CNN and LSTM
Sirshar, M., Paracha, M. F. K., Akram, M. U., Alghamdi, N. S., Zaidi, S. Z. Y., & Fatima, T. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262209 · Cited by 61
3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition
Arif, S., Wang, J., Hassan, T. U., & Fei, Z. (2019) · Future Internet · doi:10.3390/fi11020042 · Cited by 61
DeepSIC: Deep Semantic Image Compression
Luo, S., Yang, Y., Yin, Y., Shen, C., Zhao, Y., & Song, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04167-0_9 · Cited by 62
Deep Learning for Plasma Tomography and Disruption Prediction From Bolometer Data
Ferreira, D. R., Carvalho, P. J., & Fernandes, H. (2019) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2019.2947304 · Cited by 61
Prediction of Ocean Weather Based on Denoising AutoEncoder and Convolutional LSTM
Kim, K.-S., Lee, J.-B., Roh, M.-I., Han, K.-M., & Lee, G.-H. (2020) · Journal of Marine Science and Engineering · doi:10.3390/jmse8100805 · Cited by 61
Improving Video Captioning with Temporal Composition of a Visual-Syntactic Embedding
Perez-Martin, J., Bustos, B., & Pérez, J. E. P. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00308 · Cited by 61
Direct Estimation of Regional Wall Thicknesses via Residual Recurrent Neural Network
Xue, W., Nachum, I. B., Pandey, S. K., Warrington, J. C., Leung, S., & Li, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59050-9_40 · Cited by 60
Xiong, S., Zhou, L., Dai, Y., & Ji, X. (2022) · Chinese Journal of Chemical Engineering · doi:10.1016/j.cjche.2022.06.029 · Cited by 60
Prediction of severe thunderstorm events with ensemble deep learning and radar data
Guastavino, S., Piana, M., Tizzi, M., Cassola, F., Iengo, A., Sacchetti, D., Solazzo, E., & Benvenuto, F. (2022) · Scientific Reports · doi:10.1038/s41598-022-23306-6 · Cited by 60
A Semantics-Assisted Video Captioning Model Trained With Scheduled Sampling
Chen, H., Lin, K., Maye, A., Li, J., & Hu, X. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.475767 · Cited by 60
Multi-modality learning for human action recognition
Ren, Z., Zhang, Q., Gao, X., Hao, P., & Cheng, J. S. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08576-z · Cited by 60
Group Activity Detection from Trajectory and Video Data in Soccer
Sanford, R., Gorji, S., Hafemann, L. G., Pourbabaee, B., & Javan, M. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00457 · Cited by 60
Ye, M., Luo, J., Xiao, C., & Ma, F. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411864 · Cited by 60
Inferring Interaction Force from Visual Information without Using Physical Force Sensors
Hwang, W., & Lim, S. (2017) · Sensors · doi:10.3390/s17112455 · Cited by 59
Global and Local Knowledge-Aware Attention Network for Action Recognition
Zheng, Z., An, G., Wu, D. O., & Ruan, Q. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978613 · Cited by 59
A Deep Learning-Based Method for Forecasting Gold Price with Respect to Pandemics
Khani, M. M., Vahidnia, S., & Abbasi, A. (2021) · SN Computer Science · doi:10.1007/s42979-021-00724-3 · Cited by 59
Multimodal Deep Learning Crime Prediction Using Tweets
Sakirin, T., & Tanrıöver, Ö. Ö. (2023) · IEEE Access · doi:10.1109/access.2023.3308967 · Cited by 58
Kim, H., Lee, K. M., Kim, E. J., & Lee, J. S. (2019) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims.2019.05.15 · Cited by 58
Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition
Zhu, X., Zhu, Y., Wang, H., Wen, H., Yan, Y., & Liu, P. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3491228 · Cited by 58
Taming unbalanced training workloads in deep learning with partial collective operations
Li, S., Ben-Nun, T., Girolamo, S. D., Alistarh, D., & Hoefler, T. (2020) · Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3332466.3374528 · Cited by 57
Short-Term Forecasting Approach Based on Bilstm-Cnn for Regional Photovoltaic Power Plants
Li, G., Guo, S., Li, X., & Cheng, C. (2022) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2023.101019 · Cited by 57
A Comprehensive Methodological Survey of Human Activity Recognition Across Diverse Data Modalities
Shin, J., Hassan, N., Miah, A. S. M., & Nishimura, S. (2025) · Sensors · doi:10.3390/s25134028 · Cited by 58
Discriminative Bimodal Networks for Visual Localization and Detection with Natural Language Queries
Zhang, Y., Yuan, L., Guo, Y., He, Z., Huang, I.-A., & Lee, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.122 · Cited by 57
Attentive Spatial–Temporal Summary Networks for Feature Learning in Irregular Gait Recognition
Li, S., Liu, W., & Ma, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2900134 · Cited by 57
Identifying Object States in Cooking-Related ImagesPreprint
Jelodar, A. B., Salekin, M. S., & Sun, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06956 · Cited by 57
Eye gaze pattern analysis for fatigue detection based on GP-BCNN with ESM
Wang, Y., Huang, R., & Guo, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.03.013 · Cited by 56
Metadata
- Authors
- Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T.
- Affiliations
- University of California, Berkeley (US)
- International Computer Science Institute (US)
- The University of Texas at Austin (US)
- University of Massachusetts Lowell (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-04-01
- Volume
- 39
- Issue
- 4
- Pages
- 677-691
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2508429489
- Cited by
- 1,547
- References
- 78
- Citations by year
- 2026: 46 · 2025: 79 · 2024: 111 · 2023: 164 · 2022: 202 · 2021: 250 · 2020: 227 · 2019: 225 · 2018: 149 · 2017: 70
- FWCI
- 102.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Funders
- DARPA's
- MSEE
- SMISC
- NSF (IIS-1427425, IIS-1212798)
- Berkeley Vision and Learning Center
- NVIDIA
- FITweltweit-Program
- German Academic Exchange Service (DAAD)
- NDSEG
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- long-term recurrent convolutional network, activity recognition, image captioning, audio description, long-range dependencies, temporal dynamics, compositional representations, end-to-end learning, visual description generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.