Long-term recurrent convolutional networks for visual recognition and description
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Models based on deep convolutional networks have dominated recent image interpretation tasks; we investigate whether models which are also recurrent, or “temporally deep”, are effective for tasks involving sequences, visual and otherwise. We develop a novel recurrent convolutional architecture suitable for large-scale visual learning which is end-to-end trainable, and demonstrate the value of these models on benchmark video recognition tasks, image description and retrieval problems, and video narration challenges. In contrast to current models which assume a fixed spatio-temporal receptive field or simple temporal averaging for sequential processing, recurrent convolutional models are “doubly deep” in that they can be compositional in spatial and temporal “layers”. Such models may have advantages when target concepts are complex and/or training data are limited. Learning long-term dependencies is possible when nonlinearities are incorporated into the network state updates. Long-term RNN models are appealing in that they directly can map variable-length inputs (e.g., video frames) to variable length outputs (e.g., natural language text) and can model complex temporal dynamics; yet they can be optimized with backpropagation. Our recurrent long-term models are directly connected to modern visual convnet models and can be jointly trained to simultaneously learn temporal dynamics and convolutional perceptual representations. Our results show such models have distinct advantages o…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cameras have been widely used in the HAR applications, however collecting video data presents many issues regarding privacy and computational requirements [10].”
From Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey (2020) · cites this paper as [10]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“Though RNNs have proven successful on tasks such as speech recognition [43] and text generation [38], it can be difficult to train them to learn long-term dynamics, likely due in part to the vanishing and exploding gradients problem [12] that can result from propagating the gradients down through the many layers of th…”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“The net is pre-trained on the 1.2M image ILSVRC-2012 [32] classification training subset of the ImageNet [7] dataset, giving the network a strong initialization to facilitate faster training and prevent over-fitting to the relatively small video datasets.”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“sual and language inputs up to time t. (The visual model ˚ V (x) used in this experiment is the base Caffe [15] reference model, very similar to the wellknown AlexNet [23], pre-trained on ILSVRC-2012 [31] as in Section4.) The third and fourth LSTMs transform the outputs of the LSTM below, and the fourth LSTM’s outputs…”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Research works printed but not linked: 3
- simonyan, 0, Very deep convolutional networks for large-scale image recognition, ICLR 2015
- kiros, 2014, Multimodal neurallanguage models, ICML
- barbu, 2012, Video in sentences out, UAI
Cited by
Items 301–374 of 374
Detect-and-Track: Efficient Pose Estimation in Videos
Girdhar, R., Gkioxari, G., Torresani, L., Paluri, M., & Tran, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00044 · Cited by 254
A Survey on Deepfake Video Detection
Yu, P., Xia, Z., Fei, J., & Lu, Y. (2021) · IET Biometrics · doi:10.1049/bme2.12031 · Cited by 253
Learning to Answer Questions from Image Using Convolutional Neural Network
Ma, L., Lu, Z., & Li, H. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10442 · Cited by 253
Machine Learning for Microcontroller-Class Hardware: A Review
Saha, S. S., Sandha, S. S., & Srivastava, M. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3210773 · Cited by 253
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_3 · Cited by 246
“Recently, there has been significant interest in Recurrent Neural Networks (RNNs), specifically those using Long Short Term Memory (LSTM) (e.g. [28,33,34]).”
BackgroundBEVT: BERT Pretraining of Video Transformers
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Jiang, Y.-G., Zhou, L., & Yuan, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01432 · Cited by 234
“In particular, temporal aggregation methods typically extract image features/scores frame-by-frame and then combine frame-level information to achieve video-level predictions through recurrent networks [15, 66] or average pooling [47, 57].”
MethodDeep Reinforcement Learning for Sequence-to-Sequence Models
Keneshloo, Y., Shi, T., Ramakrishnan, N., & Reddy, C. K. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929141 · Cited by 230
Connectionist Temporal Modeling for Weakly Supervised Action Labeling
Huang, D.-A., Fei-Fei, L., & Niebles, J. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_9 · Cited by 226
Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images
MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025814 · Cited by 200
“Research in this area has generally been focused on the necessary artificial intelligence advancements (particularly in computer vision and natural language generation) to be able to accomplish such a task with a reasonable level of accuracy [10,17,23,25,27,34].”
BackgroundEnd-to-End Learning of Driving Models with Surround-View Cameras and Route Planners
Hecker, S., Dai, D., & Van Gool, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_27 · Cited by 191
Weakly Supervised Action Learning with RNN Based Fine-to-Coarse Modeling
Richard, A., Kuehne, H., & Gall, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.140 · Cited by 195
“On the fine grained level, we use a discriminative representation of sub-actions, modeled by a recurrent neural network as e.g . used by [6, 35, 27, 33].”
MethodControllable Video Captioning With POS Sequence Guidance Based on Gated Fusion Network
Wang, B., Ma, L., Zhang, W., Jiang, W., Wang, J., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00273 · Cited by 199
“Obviously, the existing approahces with one single kind feature [11, 43, 45, 46] are hard to comprehensively exploit the semantic meaning of a video.”
BackgroundWu, Y., & Wang, G. (2018) · International Journal of Molecular Sciences · doi:10.3390/ijms19082358 · Cited by 193
“It innovates many fields, including pattern recognition, speech recognition [47,48], natural language processing [49,50], image and video recognitions [45,51,52], and life science [53,54].”
BackgroundAttention Correctness in Neural Image Captioning
Liu, C., Mao, J., Sha, F., & Yuille, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11197 · Cited by 210
Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition With Multimodal Training
Abavisani, M., Joze, H. R. V., & Patel, V. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00126 · Cited by 190
Learning to Generate Product Reviews from Attributes
Dong, L., Huang, S., Wei, F., Lapata, M., Zhou, M., & Xu, K. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1059 · Cited by 190
Chen, H., Dou, Q., Ni, D., Cheng, J.-Z., Qin, J., Li, S., & Heng, P.-A. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24553-9_62 · Cited by 180
“Recently, the recurrent neural network (RNN), especially the long short-term memory (LSTM) model, has achieved success in sequence learning tasks, such as speech recognition [7] and video recognition [8].”
BackgroundPark, S., Pan, F., Kang, S., & Yoo, C. D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54526-4_12 · Cited by 171
“Situations Human AlexNet[1] VGG-FaceNet[2] FlowImageNet[3] LRCN[3] DDD-FFA DDD-IAA Bareface 82.”
BackgroundMethodInfluentialMultimodal Gesture Recognition Based on the ResC3D Network
Miao, Q., Li, Y., Ouyang, W., Ma, Z., Xu, X., Shi, W., & Cao, X. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.360 · Cited by 168
Ultrasound Standard Plane Detection Using a Composite Neural Network Framework
Chen, H., Wu, L., Dou, Q., Qin, J., Li, S., Cheng, J.-Z., Ni, D., & Heng, P.-A. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2685080 · Cited by 164
“, speech recognition [43], video recognition [44], [45], and machine translation [46], with promising results.”
BackgroundVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Wang, X., Girshick, R., Gupta, A., & He, K. (2017) · arXiv · Cited by 11,689
HABNet: Machine Learning, Remote Sensing-Based Detection of Harmful Algal Blooms
Hill, P. R., Kumar, A., Temimi, M., & Bull, D. R. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3001445 · Cited by 143
“CNNs and LSTMs have often been combined to provide such characterization in applications such as video se-quenceclassification[41],[42].”
MethodLiDAR-Video Driving Dataset: Learning Driving Policies Effectively
Chen, Y., Wang, J., Li, J., Lu, C., Luo, Z., Xue, H., & Wang, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00615 · Cited by 131
“As is illustrated in Figure 8, inspired by plentiful previous works [5, 10, 16, 24], we decide to adopt two representative mainstream frameworks for policy prediction tasks, namely “DNN-only” and “DNN-LSTM”.”
MethodGan, C., Li, Y., Li, H., Sun, C., & Gong, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.201 · Cited by 116
“We have witnessed the resurgence of image captioning [42, 29, 18, 6, 40, 4, 47, 12, 33, 10] which is often addressed by jointly modeling visual and textual content with deep neural networks.”
MethodTowards automated semantic segmentation in prenatal volumetric ultrasound
Yang, X., Yu, L., Li, S., Wen, H., Luo, D., Bian, C., Qin, J., Ni, D., & Heng, P.-A. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2858779 · Cited by 115
Reasoning about Pragmatics with Neural Listeners and Speakers
Andreas, J., & Klein, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1125 · Cited by 115
“To implement this probability distribution, we first use a multilayer per-ceptron to compute a vector of scores s (one s i for each vocabulary item): and then normalize them to obtain probabilities:”
BackgroundMethodDeep learning-based spacecraft relative navigation methods: A survey
Song, J., Rondao, D., & Aouf, N. (2022) · Acta Astronautica · doi:10.1016/j.actaastro.2021.10.025 · Cited by 106
“The end-to-end DRCNN framework achieves an average Root Mean Square Error (RMSE) drift of 5.96 % and 6.12 deg per trajectory for position and attitude, respectively, on lengths of 100 m–800 m, showing a competitive performance relative to Monocular VISO2 [14].”
MethodInfluentialTowards Unsupervised Image Captioning With Shared Multimodal Embeddings
Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111
A Deep Structured Model with Radius–Margin Bound for 3D Human Activity Recognition
Lin, L., Wang, K., Zuo, W., Wang, M., Luo, J., & Zhang, L. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-015-0876-z · Cited by 104
Towards occupant activity driven smart buildings via WiFi-enabled IoT devices and deep learning
Zou, H., Zhou, Y., Yang, J., & Spanos, C. J. (2018) · Energy and Buildings · doi:10.1016/j.enbuild.2018.08.010 · Cited by 100
Mehta, N. K., Prasad, S. S., Saurav, S., Saini, R., & Singh, S. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03200-4 · Cited by 97
“Nevertheless, compared to LRCN and C3D, the proposed DenseAttNet model is computational efficient and more suitable for real-world applications.”
BackgroundInfluentialASFormer: Transformer for Action Segmentation
Yi, F., Wen, H., & Jiang, T. (2021) · Proceedings of the British Machine Vision Conference 2021 · doi:10.5244/c.35.49 · Cited by 170
“Other approaches model the temporal action sequence with Conditional Random Fields [30, 34, 39], Markov model [20, 38] and RNN [9, 29, 35, 42, 49] to classify framewise actions.”
MethodAn Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences
Guo, W., Tondi, B., & Barni, M. (2022) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2022.3190213 · Cited by 91
Uncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference
Subedar, M., Krishnan, R., Meyer, P. L., Tickoo, O., & Huang, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00640 · Cited by 85
Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition
Cao, C., Zhang, Y., Zhang, C., & Lu, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2756840 · Cited by 78
Detecting Deepfake Videos with Temporal Dropout 3DCNN
Zhang, D., Li, C., Lin, F., Zeng, D., & Ge, S. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/178 · Cited by 77
Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning
Dong, X., Long, C., Xu, W., & Xiao, C. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475439 · Cited by 74
FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions
Rotstein, N., Bensaïd, D., Brody, S., Ganz, R., & Kimmel, R. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00559 · Cited by 66
Generating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · arXiv · doi:10.48550/arxiv.1603.08507 · Cited by 64
“Our model is based on LRCN [8], which consists of a convolutional neural network, which extracts powerful high level visual features, and two stacked recurrent networks (specifically LSTMs), which learn how to generate a description conditioned on visual features.”
BackgroundMethodInfluentialSelf-citationLow latency RNN inference with cellular batching
Gao, P., Yu, L., Wu, Y., & Li, J. (2018) · Proceedings of the Thirteenth EuroSys Conference · doi:10.1145/3190508.3190541 · Cited by 66
Udriștoiu, A. L., Cazacu, I. M., Gruionu, L. G., Gruionu, G., Iacob, A. V., Burtea, D. E., Ungureanu, B. S., Costache, M. I., Constantin, A., Popescu, C. F., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251701 · Cited by 63
“other, while the basic assumption of LSTM is that there is an interaction between the input sequences [9].”
BackgroundMulti-Modality Multi-Task Recurrent Neural Network for Online Action Detection
Liu, J., Li, Y., Song, S., Xing, J., Lan, C., & Zeng, W. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2799968 · Cited by 62
Recurrent Neural Networks for Semantic Instance Segmentation
Salvador, A., Bellver, M., Campos, V., Baradad, M., Marques, F., Torres, J., & Giro-i-Nieto, X. (2017) · arXiv · doi:10.48550/arxiv.1712.00617 · Cited by 62
“video action classification [13, 43], image captioning [40, 21] or visual question answering [2]).”
BackgroundAkhter, I., Jalal, A., & Kim, K. (2021) · 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST) · doi:10.1109/ibcast51254.2021.9393293 · Cited by 58
“TABLE 2: COMPARISONS WITH THE STATE-OF-THE-ART METHODS Systems UCF101 Systems YouTube LRCN [40] 71.”
Multimodal Gesture Recognition Using Multi-stream Recurrent Neural Network
Nishida, N., & Nakayama, H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-29451-3_54 · Cited by 53
“(2014) [19] incorporated a ConvNet with an LSTM-RNN and applied it to action recognition.”
MethodExplainable Sentiment Analysis with Applications in Medicine
Zucco, C., Liang, H., Fatta, G. D., & Cannataro, M. (2018) · 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621359 · Cited by 59
“Their approach consists of a long term recurrent convolutional network [59] consisting of a convolutional neural network which extracts highly discriminative image features with the advantage of a compact bilinear classifier,”
BackgroundBilal, M., Maqsood, M., Yasmin, S., Hasan, N. U., & Rho, S. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-021-03957-4 · Cited by 51
Khan, S. A., & Dang-Nguyen, D.-T. (2024) · IEEE Access · doi:10.1109/access.2023.3348450 · Cited by 53
Sharma, H., Drukker, L., Chatelain, P., Droste, R., Papageorghiou, A. T., & Noble, J. A. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101973 · Cited by 48
Mitigating Gender Bias in Captioning Systems
Tang, R., Du, M., Li, Y., Liu, Z., Zou, N., & Hu, X. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449950 · Cited by 50
Seeing with Humans: Gaze-Assisted Neural Image Captioning
Sugano, Y., & Bulling, A. (2016) · arXiv · doi:10.48550/arxiv.1608.05203 · Cited by 47
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning
Biten, A. F., Gomez, L., & Karatzas, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00253 · Cited by 43
Time Series-Based GHG Emissions Prediction for Smart Homes
Riekstin, A. C., Langevin, A., Dandres, T., Gagnon, G., & Cheriet, M. (2020) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2018.2886164 · Cited by 41
“, speech recognition, language modeling, visual recognition, and description [42], [43], [44], [45].”
Lin, Z., Zhang, D., Shi, D., Xu, R., Tao, Q., Wu, L., He, M., & Ge, Z. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104281 · Cited by 39
DTR-HAR: deep temporal residual representation for human activity recognition
Basly, H., Ouarda, W., Sayadi, F. E., Ouni, B., & Alimi, A. M. (2022) · The Visual Computer · doi:10.1007/s00371-021-02064-y · Cited by 36
Video ReCap: Recursive Captioning of Hour-Long Videos
Islam, M. M., Ho, N., Yang, X., Nagarajan, T., Torresani, L., & Bertasius, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01723 · Cited by 34
“Subsequently, these methods were replaced by deep learning methods built using CNN-RNN encoder-decoder architectures [8, 17, 37, 38, 47, 55, 56, 64].”
MethodN-HAR: A Neuromorphic Event-Based Human Activity Recognition System using Memory Surfaces
Pradhan, B. R., Bethi, Y., Narayanan, S., Chakraborty, A., & Thakur, C. S. (2019) · 2019 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2019.8702581 · Cited by 31
“As an extension of the above model, a longterm recurrent convolutional network (LRCN) [15] model was used where the architecture consists of time-distributed 2D CNN models, followed by the use of time-pooling layers to capture the temporal information.”
MethodGlow in the Dark: Smartphone Inertial Odometry for Vehicle Tracking in GPS Blocked Environments
Gao, R., Xiao, X., Zhu, S., Xing, W., Li, C., Liu, L., Ma, L., & Chai, H. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3064342 · Cited by 31
A Hybrid Deep Learning System of CNN and LRCN to Detect Cyberbullying from SNS Comments
Bu, S.-J., & Cho, S.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-92639-1_47 · Cited by 30
Exposing DeepFakes Using Convolutional Neural Networks and Transfer Learning Approaches
Suratkar, S., Kazi, F., Sakhalkar, M., Abhyankar, N., & Kshirsagar, M. (2020) · 2020 IEEE 17th India Council International Conference (INDICON) · doi:10.1109/indicon49873.2020.9342252 · Cited by 25
Ryoo, M. S., Gopalakrishnan, K., Kahatapitiya, K., Xiao, T., Rao, K., Stone, A., Lu, Y., Ibarz, J., & Arnab, A. (2022) · arXiv · doi:10.48550/arxiv.2211.09119 · Cited by 21
A data-driven approach to assist offensive and defensive players in optimal decision making
Rahimian, P., & Toka, L. (2024) · International Journal of Sports Science & Coaching · doi:10.1177/17479541221149481 · Cited by 20
Hybrid Recurrent Deep Learning Model for DeepFake Video Detection
Jaiswal, G. (2021) · 2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) · doi:10.1109/upcon52273.2021.9667632 · Cited by 20
Automatic Medical Image Report Generation with Multi-view and Multi-modal Attention Mechanism
Yang, S., Niu, J., Wu, J., & Liu, X. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60248-2_48 · Cited by 17
Abbreviated View of Deepfake Videos Detection Techniques
Younus, M. A., & Hasan, T. M. (2020) · 2020 6th International Engineering Conference “Sustainable Technology and Development" (IEC) · doi:10.1109/iec49899.2020.9122916 · Cited by 17
Deep Learning for Video Classification: A Review
Rehman, A., & Belhaouari, S. B. (2021) · doi:10.36227/techrxiv.15172920 · Cited by 17
Kumar, P., & Chauhan, S. (2021) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-021-00063-5 · Cited by 13
RetractedThinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Miech, A., Alayrac, J.-B., Laptev, I., Sivic, J., & Zisserman, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00970 · Cited by 149
“Second, we propose to revisit the use of a captioning loss [14] to train retrieval models and discuss the benefits over standard alternatives that use classification or ranking loss [8, 37, 39, 46, 67, 68, 78].”
MethodVision to Language: Methods, Metrics and Datasets
Sharif, N., Nadeem, U., Shah, S. A. A., Bennamoun, M., & Liu, W. (2020) · Learning and Analytics in Intelligent Systems · doi:10.1007/978-3-030-49724-8_2 · Cited by 9
Application of C-LSTM Networks to Automatic Labeling of Vehicle Dynamic Response Data for Bridges
Shin, R., Okada, Y., & Yamamoto, K. (2022) · Sensors · doi:10.3390/s22093486 · Cited by 8
“Neural networks are used in various fields, most notably in speech recognition and natural language processing, where they have shown excellent performance (Donahue et al., 2015 [18]).”
MethodLi, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7
“Donahue et al. (2015) proposed a two-layer LSTM architecture.”
The potential of vision-language models for content moderation of children’s videos
Ahmed, S. H., Hu, S., & Sukthankar, G. (2023) · 2023 International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla58977.2023.00186 · Cited by 4
“VTN [31] 77.9 I3D [32] 72.1 ConvLSTM [33] 69.7 Vanilla (PL) 80.3”
InfluentialOrder Matters: Sequence to sequence for sets
Vinyals, O., Bengio, S., & Kudlur, M. (2015) · arXiv · doi:10.48550/arxiv.1511.06391
Metadata
- Authors
- Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K.
- Affiliations
- University of California, Berkeley (US)
- International Computer Science Institute (US)
- The University of Texas at Austin (US)
- University of Massachusetts Lowell (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 2625-2634
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1947481528
- Cited by
- 5,228
- References
- 49
- Citations by year
- 2026: 149 · 2025: 246 · 2024: 274 · 2023: 377 · 2022: 412 · 2021: 607 · 2020: 633 · 2019: 758 · 2018: 749 · 2017: 607 · 2016: 305 · 2015: 99
- FWCI
- 238.45
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- National Science Foundation
- Nvidia
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- recurrent convolutional networks, video recognition, image description generation, image retrieval, video narration, long-range dependencies, temporal dynamics, end-to-end learning, spatiotemporal receptive fields, visual representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.