The DOI matches its Crossref record; 569 of the 569 references it cites were found in a registry or an index.
Attention, please! A survey of neural attention models in deep learning
Abstract
In humans, Attention is a core property of all perceptual and cognitive operations. Given our limited ability to process competing sources, attention mechanisms select, modulate, and focus on the information most relevant to behavior. For decades, concepts and functions of attention have been studied in philosophy, psychology, neuroscience, and computing. For the last 6 years, this property has been widely explored in deep neural networks. Currently, the state-of-the-art in Deep Learning is represented by neural attention models in several application domains. This survey provides a comprehensive overview and analysis of developments in neural attention models. We systematically reviewed hundreds of architectures in the area, identifying and discussing those in which attention has shown a significant impact. We also developed and made public an automated methodology to facilitate the development of reviews in the area. By critically analyzing 650 works, we describe the primary uses of attention in convolutional, recurrent networks, and generative models, identifying common subgroups of uses and applications. Furthermore, we describe the impact of attention in different application domains and their impact on neural networks’ interpretability. Finally, we list possible trends and opportunities for further research, hoping that this review will provide a succinct overview of the main attentional models in the area and guide researchers in developing future approaches that will drive further improvements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–400 of 569
GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction
Liang, Y., Ke, S., Zhang, J., Yi, X., & Zheng, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/476 · Cited by 510
MORAN: A Multi-Object Rectified Attention Network for scene text recognition
Luo, C., Jin, L., & Sun, Z. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.01.020 · Cited by 510
Multi-attention Recurrent Network for Human Communication Comprehension
Zadeh, A., Liang, P. P., Poria, S., Vij, P., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12024 · Cited by 505
Attention-Aware Compositional Network for Person Re-identification
Xu, J., Zhao, R., Zhu, F., Wang, H., & Ouyang, W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00226 · Cited by 502
Where to Look: Focus Regions for Visual Question Answering
Shih, K. J., Singh, S., & Hoiem, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.499 · Cited by 493
Relation Classification via Multi-Level Attention CNNs
Wang, L., Cao, Z., de Melo, G., & Liu, Z. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1123 · Cited by 487
Baziotis, C., Pelekis, N., & Doulkeridis, C. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2126 · Cited by 485
End-to-End Comparative Attention Networks for Person Re-Identification
Liu, H., Feng, J., Qi, M., Jiang, J., & Yan, S. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2700762 · Cited by 482
Commonsense Knowledge Aware Conversation Generation with Graph Attention
Zhou, H., Young, T., Huang, M., Zhao, H., Xu, J., & Zhu, X. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/643 · Cited by 479
Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model
Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2851672 · Cited by 474
Jain, S., & Wallace, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10186 · Cited by 467
VideoLSTM convolves, attends and flows for action recognition
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., & Snoek, C. G. M. (2018) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.011 · Cited by 466
Dual Attention Matching Network for Context-Aware Feature Sequence Based Person Re-identification
Si, J., Zhang, H., Li, C.-G., Kuen, J., Kong, X., Kot, A. C., & Wang, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00562 · Cited by 466
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
Lee, C.-Y., & Osindero, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.245 · Cited by 458
OCNet: Object Context Network for Scene Parsing
Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00916 · Cited by 453
Video-Based Sign Language Recognition Without Temporal Segmentation
Huang, J., Zhou, W., Zhang, Q., Li, H., & Li, W. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11903 · Cited by 451
Dynamic Coattention Networks For Question Answering
Xiong, C., Zhong, V., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01604 · Cited by 450
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803025 · Cited by 449
Topic Aware Neural Response Generation
Xing, C., Wu, W., Wu, Y., Liu, J., Huang, Y., Zhou, M., & Ma, W.-Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10981 · Cited by 438
Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
Li, H., Wang, P., Shen, C., & Zhang, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018610 · Cited by 432
Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms
Wang, W., Pan, S. J., Dahlmeier, D., & Xiao, X. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10974 · Cited by 425
Emergent Tool Use From Multi-Agent Autocurricula
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019) · arXiv · doi:10.48550/arxiv.1909.07528 · Cited by 424
Grounding of Textual Phrases in Images by Reconstruction
Rohrbach, A., Rohrbach, M., Hu, R., Darrell, T., & Schiele, B. (2016) · Max Planck Digital Library · Cited by 422
DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation
Liu, J., Gao, C., Meng, D., & Hauptmann, A. G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00545 · Cited by 416
Adaptive Input Representations for Neural Language ModelingPreprint
Baevski, A., & Auli, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10853 · Cited by 414
Understanding Neural Networks through Representation Erasure
Li, J., Monroe, W., & Jurafsky, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08220 · Cited by 412
Mancs: A Multi-task Attentional Network with Curriculum Sampling for Person Re-Identification
Wang, C., Zhang, Q., Huang, C., Liu, W., & Wang, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_23 · Cited by 410
Neural Aggregation Network for Video Face Recognition
Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., & Hua, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.554 · Cited by 409
Learning Natural Language Inference with LSTM
Wang, S., & Jiang, J. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1170 · Cited by 408
Linguistically-Informed Self-Attention for Semantic Role Labeling
Strubell, E., Verga, P., Andor, D., Weiss, D., & McCallum, A. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1548 · Cited by 406
Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., & Yu, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.518 · Cited by 406
ActBERT: Learning Global-Local Video-Text Representations
Zhu, L., & Yang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00877 · Cited by 405
Attention-Based Multimodal Fusion for Video Description
Hori, C., Hori, T., Lee, T.-Y., Zhang, Z., Harsham, B., Hershey, J. R., Marks, T. K., & Sumi, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.450 · Cited by 401
Object-Part Attention Model for Fine-Grained Image Classification
Peng, Y., He, X., & Zhao, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2774041 · Cited by 393
Video Summarization With Attention-Based Encoder–Decoder Networks
Ji, Z., Xiong, K., Pang, Y., & Li, X. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2904996 · Cited by 392
Computational visual attention systems and their cognitive foundations: A survey
Frintrop, S., Rome, E., & Christensen, H. I. (2010) · ACM Transactions on Applied Perception · doi:10.1145/1658349.1658355 · Cited by 389
BERT for Joint Intent Classification and Slot Filling
Lew, M., Obuchowski, A., & Kutyła, M. (2019) · arXiv (Cornell University) · doi:10.4230/oasics.ldk.2021.30 · Cited by 389
Occluded Pedestrian Detection Through Guided Attention in CNNs
Zhang, S., Yang, J., & Schiele, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00731 · Cited by 385
Jetley, S., Lord, N. A., Lee, N., & Torr, P. H. S. (2018) · arXiv · doi:10.48550/arxiv.1804.02391 · Cited by 383
Temporal Graph Networks for Deep Learning on Dynamic Graphs
Rossi, E., Chamberlain, B., Frasca, F., Eynard, D., Monti, F., & Bronstein, M. (2020) · arXiv · doi:10.48550/arxiv.2006.10637 · Cited by 382
Kim, Y., Denton, C., Hoang, L., & Rush, A. M. (2017) · arXiv · doi:10.48550/arxiv.1702.00887 · Cited by 381
Relation-Aware Graph Attention Network for Visual Question Answering
Li, L., Gan, Z., Cheng, Y., & Liu, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01041 · Cited by 379
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
Wang, Y., Jiang, L., Yang, M. H., Li, L.-J., Long, M., & Fei-Fei, L. (2019) · International Conference on Learning Representations · Cited by 372
Human‐centered artificial intelligence and machine learning
Riedl, M. O. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.117 · Cited by 371
LSTM-based Deep Learning Models for Non-factoid Answer Selection
Tan, M., dos Santos, C., Xiang, B., & Zhou, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04108 · Cited by 370
Diversity Regularized Spatiotemporal Attention for Video-Based Person Re-identification
Li, S., Bak, S., Carr, P., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00046 · Cited by 370
An Integrated Model of Top-Down and Bottom-Up Attention for Optimizing Detection Speed
Navalpakkam, V., & Itti, L. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.54 · Cited by 369
Diversified Visual Attention Networks for Fine-Grained Object Classification
Zhao, B., Wu, X., Feng, J., Peng, Q., & Yan, S. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2648498 · Cited by 368
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Yu, A. W., Dohan, D., Luong, M.-T., Zhao, R., Chen, K., Norouzi, M., & Le, Q. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09541 · Cited by 368
Gated-Attention Readers for Text Comprehension
Dhingra, B., Liu, H., Yang, Z., Cohen, W., & Salakhutdinov, R. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1168 · Cited by 366
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
Chorowski, J., Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1602 · Cited by 362
Hao, Y., Zhang, Y., Liu, K., He, S., Liu, Z., Wu, H., & Zhao, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1021 · Cited by 359
Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions
Ji, G., Liu, K., He, S., & Zhao, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10953 · Cited by 357
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357
A Context-Dependent Attention System for a Social Robot
Breazeal, C., & Scassellati, B. M. (1999) · Cited by 355
ADCrowdNet: An Attention-Injective Deformable Convolutional Network for Crowd Understanding
Liu, N., Long, Y., Zou, C., Niu, Q., Pan, L., & Wu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00334 · Cited by 350
Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation
Xu, D., Wang, W., Tang, H., Liu, H., Sebe, N., & Ricci, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00412 · Cited by 348
Nguyen, D.-K., & Okatani, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00637 · Cited by 347
VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search
Frintrop, S. (2006) · Lecture Notes in Computer Science · doi:10.1007/11682110 · Cited by 345
Single Shot Text Detector with Regional Attention
He, P., Huang, W., He, T., Zhu, Q., Qiao, Y., & Li, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.331 · Cited by 344
Jointly Attentive Spatial-Temporal Pooling Networks for Video-Based Person Re-identification
Xu, S., Cheng, Y., Gu, K., Yang, Y., Chang, S., & Zhou, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.507 · Cited by 341
Neural Sentiment Classification with User and Product Attention
Chen, H., Sun, M., Tu, C., Lin, Y., & Liu, Z. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1171 · Cited by 339
Deep Learning for Fall Detection: Three-Dimensional CNN Combined With LSTM on Video Kinematic Data
Lu, N., Wu, Y., Feng, L., & Song, J. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2808281 · Cited by 339
Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks
Cho, K., Courville, A., & Bengio, Y. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477044 · Cited by 338
Adaptive Computation Time for Recurrent Neural Networks
Graves, A. (2016) · arXiv · doi:10.48550/arxiv.1603.08983 · Cited by 335 · cited 1×
“The seminal approach by Graves (2016) made minor modifications to an RNN, allowing the network to perform a variable number of state transitions and a variable number of outputs at each stage of the input.”
MethodSpatially Adaptive Computation Time for Residual Networks
Figurnov, M., Collins, M. D., Zhu, Y., Zhang, L., Huang, J., Vetrov, D., & Salakhutdinov, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.194 · Cited by 329
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Zhou, X., Li, L., Dong, D., Liu, Y., Chen, Y., Zhao, W. X., Yu, D., & Wu, H. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1103 · Cited by 328
Deep Joint Entity Disambiguation with Local Neural Attention
Ganea, O.-E., & Hofmann, T. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1277 · Cited by 327
Multi-label Image Recognition by Recurrently Discovering Attentional Regions
Wang, Z., Chen, T., Li, G., Xu, R., & Lin, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.58 · Cited by 326
Three-Stream Attention-Aware Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2891104 · Cited by 324
Zhou, Z., Huang, Y., Wang, W., Wang, L., & Tan, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.717 · Cited by 323
Hierarchical Transformers for Multi-Document Summarization
Liu, Y., & Lapata, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1500 · Cited by 322
Cross-modality interactive attention network for multispectral pedestrian detection
Zhang, L., Liu, Z., Zhang, S., Yang, X., Qiao, H., Huang, K., & Hussain, A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.015 · Cited by 320
A Decomposable Attention Model for Natural Language InferencePreprint
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01933 · Cited by 314
Attentional Correlation Filter Network for Adaptive Visual Tracking
Choi, J., Chang, H. J., Yun, S., Fischer, T., Demiris, Y., & Choi, J. Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.513 · Cited by 311
End-to-End Instance Segmentation with Recurrent Attention
Ren, M., & Zemel, R. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.39 · Cited by 309
Attentional modulation in human primary olfactory cortex
Zelano, C., Bensafi, M., Porter, J., Mainland, J., Johnson, B., Bremner, E., Telles, C., Khan, R., & Sobel, N. (2005) · Nature Neuroscience · doi:10.1038/nn1368 · Cited by 307
Deep Semantic Role Labeling With Self-Attention
Tan, Z., Wang, M., Xie, J., Chen, Y., & Shi, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11928 · Cited by 306
Learning to Transduce with Unbounded MemoryPreprint
Grefenstette, E., Hermann, K. M., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02516 · Cited by 305
Analyzing the Structure of Attention in a Transformer Language Model
Vig, J., & Belinkov, Y. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4808 · Cited by 301 · cited 2×
“On the other hand, some studies have found how attention in neural models captures various notions of syntax and co-reference (Vig and Belinkov 2019) (Clark et al.”
Backgrounddos Santos, C. N., Tan, M., Xiang, B., & Zhou, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03609 · Cited by 300
Action Recognition using Visual Attention
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 298
Pretrained Transformers Improve Out-of-Distribution Robustness
Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., & Song, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.244 · Cited by 298
Viewpoint-Aware Attentive Multi-view Inference for Vehicle Re-identification
Zhouy, Y., & Shao, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00679 · Cited by 297
Relational Deep Reinforcement LearningPreprint
Zambaldi, V., Raposo, D. C., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01830 · Cited by 297
Text Understanding with the Attention Sum Reader Network
Kadlec, R., Schmid, M., Bajgar, O., & Kleindienst, J. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1086 · Cited by 295
Hori, T., Watanabe, S., Zhang, Y., & Chan, W. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1296 · Cited by 294
Review Networks for Caption GenerationPreprint
Yang, Z., Yuan, Y., Wu, Y., Salakhutdinov, R., & Cohen, W. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07912 · Cited by 289
Spectral-Spatial Attention Networks for Hyperspectral Image Classification
Mei, X., Pan, E., Ma, Y., Dai, X., Huang, J., Fan, F., Du, Q., Zheng, H., & Ma, J. (2019) · Remote Sensing · doi:10.3390/rs11080963 · Cited by 285
Chan, W., Jaitly, N., Le, Q. V., & Vinyals, O. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.01211 · Cited by 284
Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction
Verga, P., Strubell, E., & McCallum, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1080 · Cited by 284
Graph Classification using Structural Attention
Lee, J. B., Rossi, R., & Kong, X. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219980 · Cited by 280
RIC-Unet: An Improved Neural Network Based on Unet for Nuclei Segmentation in Histology Images
Zeng, Z., Xie, W., Zhang, Y., & Lu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2896920 · Cited by 280
Adaptive Attention Span in Transformers
Sukhbaatar, S., Grave, E., Bojanowski, P., & Joulin, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1032 · Cited by 279
Tachibana, H., Uenoyama, K., & Aihara, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461829 · Cited by 274
SLAM: A connectionist model for attention in visual selection tasks
Phaf, R. H., Van der Heijden, A. H. C., & Hudson, P. T. W. (1990) · Cognitive Psychology · doi:10.1016/0010-0285(90)90006-p · Cited by 272
Describing Video With Attention-Based Bidirectional LSTM
Bin, Y., Yang, Y., Shen, F., Xie, N., Shen, H. T., & Li, X. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2831447 · Cited by 270
MONet: Unsupervised Scene Decomposition and Representation
Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., & Lerchner, A. (2019) · arXiv · doi:10.48550/arxiv.1901.11390 · Cited by 270 · cited 2×
“1 3 MONet (Burgess et al.”
BackgroundBeyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action Recognition
Wang, X., Gao, L., Song, J., & Shen, H. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2611485 · Cited by 265
Tree-to-Sequence Attentional Neural Machine Translation
Eriguchi, A., Hashimoto, K., & Tsuruoka, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1078 · Cited by 265
Video Object Segmentation with Episodic Graph Memory Networks
Lu, X., Wang, W., Danelljan, M., Zhou, T., Shen, J., & Van Gool, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_39 · Cited by 263
Hyperbolic Attention NetworksPreprint
Gülçehre, Ç., Denil, M., Malinowski, M., Razavi, A. S., Pascanu, R., Hermann, K. M., Battaglia, P., Bapst, V., Raposo, D. C., Santoro, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09786 · Cited by 262
Leveraging Knowledge Bases in LSTMs for Improving Machine Reading
Yang, B., & Mitchell, T. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1132 · Cited by 261
Re-Identification With Consistent Attentive Siamese Networks
Zheng, M., Karanam, S., Wu, Z., & Radke, R. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00588 · Cited by 261
Improved Training of End-to-end Attention Models for Speech Recognition
Zeyer, A., Irie, K., Schlüter, R., & Ney, H. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1616 · Cited by 257
Johnston, A., & Carneiro, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00481 · Cited by 255
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
Chen, K., Wang, J., Chen, L.-C., Gao, H., Xu, W., & Nevatia, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05960 · Cited by 254
Kaiser, L., Gomez, A. N., Shazeer, N., Vaswani, A., Parmar, N., Jones, L., & Uszkoreit, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05137 · Cited by 253
Neural Programmer: Inducing Latent Programs with Gradient DescentPreprint
Neelakantan, A., Le, Q. V., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04834 · Cited by 251
Graph2Seq: Graph to Sequence Learning with Attention-based Neural NetworksPreprint
Xu, K., Wu, L., Wang, Z., Feng, Y., Witbrock, M., & Sheinin, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00823 · Cited by 251
Direction-Aware Spatial Context Features for Shadow Detection
Hu, X., Zhu, L., Fu, C.-W., Qin, J., & Heng, P.-A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00778 · Cited by 249
An Introductory Survey on Attention Mechanisms in NLP Problems
Hu, D. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-29513-4_31 · Cited by 247
Deep Networks with Internal Selective Attention through Feedback ConnectionsPreprint
Stollenga, M., Masci, J., Gomez, F. J., & Schmidhuber, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.3068 · Cited by 246
Neumann, M., & Vu, N. T. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-917 · Cited by 243
Attention Interpretability Across NLP TasksPreprint
Vashishth, S., Upadhyay, S., Tomar, G. S., & Faruqui, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11218 · Cited by 243
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention
Lüscher, C., Beck, E., Irie, K., Kitza, M., Michel, W., Zeyer, A., Schlüter, R., & Ney, H. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1780 · Cited by 241
Wang, W., Xu, Y., Shen, J., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00449 · Cited by 241
Self-Attentive Speaker Embeddings for Text-Independent Speaker Verification
Zhu, Y., Ko, T., Snyder, D., Mak, B., & Povey, D. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1158 · Cited by 241
Unsupervised Attention-guided Image to Image Translation
Mejjati, Y. A., Richardt, C., Tompkin, J., Cosker, D. P., & Kim, K. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02311 · Cited by 237
Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks
Zhang, N., Deng, S., Sun, Z., Wang, G., Chen, X., Zhang, W., & Chen, H. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1306 · Cited by 236
Iterative Visual Reasoning Beyond Convolutions
Chen, X., Li, L.-J., Fei-Fei, L., & Gupta, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00756 · Cited by 236
Attention-Aware Face Hallucination via Deep Reinforcement Learning
Cao, Q., Lin, L., Shi, Y., Liang, X., & Li, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.180 · Cited by 235
Deep Attention-Based Spatially Recursive Networks for Fine-Grained Visual Recognition
Wu, L., Wang, Y., Li, X., & Gao, J. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2813971 · Cited by 234
Attention-Based Ensemble for Deep Metric Learning
Kim, W., Goyal, B., Chawla, K., Lee, J., & Kwon, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_45 · Cited by 234
An End-to-End TextSpotter with Explicit Alignment and Attention
He, T., Tian, Z., Huang, W., Shen, C., Qiao, Y., & Sun, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00527 · Cited by 234
Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification
Long, X., Gan, C., de Melo, G., Wu, J., Liu, X., & Wen, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00817 · Cited by 233
Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
Besold, T. R., Garcez, A. D., Bader, S., Bowman, H., Domingos, P., Hitzler, P., Kühnberger, K., Lamb, L. C., Lowd, D., Lima, P. M. V., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03902 · Cited by 232
Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks
Das, R., Neelakantan, A., Belanger, D., & McCallum, A. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1013 · Cited by 231
Attention Modeling for Targeted Sentiment
Liu, J., & Zhang, Y. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2091 · Cited by 231
Recurrent Attentional Networks for Saliency Detection
Kuen, J., Wang, Z., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.399 · Cited by 231
Attention Models in Graphs: A Survey
Lee, J. B., Rossi, R. A., Kim, S., Ahmed, N. K., & Koh, E. (2019) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3363574 · Cited by 231
Multi-level Attention Networks for Visual Question Answering
Yu, D., Fu, J., Mei, T., & Rui, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.446 · Cited by 230
Memory Fusion Network for Multi-view Sequential LearningPreprint
Zadeh, A., Liang, P. P., Mazumder, N., Poria, S., Cambria, E., & Morency, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00927 · Cited by 227
Guan, Q., Huang, Y., Zhong, Z., Zheng, Z., Zheng, L., & Yang, Y. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09927 · Cited by 226
Deep Feature Pyramid Reconfiguration for Object Detection
Kong, T., Sun, F., Huang, W., & Liu, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_11 · Cited by 224
Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning
Wang, J., Jiang, W., Ma, L., Liu, W., & Xu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00751 · Cited by 223
Grounding of Textual Phrases in Images by Reconstruction
Rohrbach, A., Rohrbach, M., Hu, R., Darrell, T., & Schiele, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_49 · Cited by 221
Tang, H., Xu, D., Sebe, N., Wang, Y., Corso, J. J., & Yan, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00252 · Cited by 221
Chen, D., Li, H., Xiao, T., Yi, S., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00128 · Cited by 219
Watch Your Step: Learning Node Embeddings via Graph AttentionPreprint
Abu-El-Haija, S., Perozzi, B., Al‐Rfou, R., & Alemi, A. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09599 · Cited by 219
Learning Image and User Features for Recommendation in Social Networks
Geng, X., Zhang, H., Bian, J., & Chua, T.-S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.486 · Cited by 218
Pyramid Attention Network for Semantic Segmentation
Li, H., Xiong, P., An, J., & Wang, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10180 · Cited by 217
Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
Zhou, J., Cao, Y., Wang, X., Li, P., & Xu, W. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00105 · Cited by 216
Hyperbolic Graph Attention Network
Zhang, Y., Wang, X., Shi, C., Jiang, X., & Ye, Y. F. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2021.3081431 · Cited by 216
Relational recurrent neural networksPreprint
Santoro, A., Faulkner, R., Raposo, D. C., Rae, J. W., Chrzanowski, M., Weber, T., Wierstra, D., Vinyals, O., Pascanu, R., & Lillicrap, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01822 · Cited by 214
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213
Multimodal Residual Learning for Visual QA
Kim, J.-H., Lee, S., Kwak, D., Heo, M.-O., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01455 · Cited by 209
Xie, S., Hu, H., & Wu, Y. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.03.019 · Cited by 209
Areas of Attention for Image Captioning
Pedersoli, M., Lucas, T., Schmid, C., & Verbeek, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.140 · Cited by 208
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 208
Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
Liu, Y., Sun, C., Lin, L., & Wang, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09090 · Cited by 207
Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
Li, X., & Loy, C. C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_6 · Cited by 207
Attentional Pooling for Action Recognition
Girdhar, R., & Ramanan, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01467 · Cited by 206
Son, L. H., Kumar, A., Sangwan, S. R., Arora, A., Nayyar, A., & Abdel-Basset, M. (2019) · IEEE Access · doi:10.1109/access.2019.2899260 · Cited by 206
Towards Universal Object Detection by Domain Attention
Wang, X., Cai, Z., Gao, D., & Vasconcelos, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00746 · Cited by 206
Recurrent Topic-Transition GAN for Visual Paragraph Generation
Liang, X., Hu, Z., Zhang, H., Gan, C., & Xing, E. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.364 · Cited by 205
Semantic Sentence Matching with Densely-Connected Recurrent and Co-Attentive Information
Kim, S., Kang, I., & Kwak, N. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016586 · Cited by 203
Minimum Word Error Rate Training for Attention-Based Sequence-to-Sequence Models
Prabhavalkar, R., Sainath, T. N., Wu, Y., Nguyen, P., Chen, Z., Chiu, C., & Kannan, A. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461809 · Cited by 201
Hierarchical Boundary-Aware Neural Encoder for Video Captioning
Baraldi, L., Grana, C., & Cucchiara, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.339 · Cited by 199
Online and Linear-Time Attention by Enforcing Monotonic Alignments
Raffel, C., Luong, M.-T., Liu, P. J., Weiss, R. J., & Eck, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.00784 · Cited by 199
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos
Du, W., Wang, Y., & Qiao, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.402 · Cited by 199
Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks
Xiao, S., Feng, J., Xing, J., Lai, H., Yan, S., & Kassim, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_4 · Cited by 198
Deep Reinforcement Learning With Visual Attention for Vehicle Classification
Zhao, D., Chen, Y., & Lv, L. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2614675 · Cited by 198
Crowd Counting using Deep Recurrent Spatial-Aware Network
Liu, L., Wang, H., Li, G., Ouyang, W., & Lin, L. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/118 · Cited by 198
Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation
Tang, H., Xu, D., Sebe, N., & Yan, Y. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851881 · Cited by 196
Inner Attention based Recurrent Neural Networks for Answer Selection
Wang, B., Liu, K., & Zhao, J. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1122 · Cited by 194
High dimensions and heavy tails for natural evolution strategies
Schaul, T., Glasmachers, T., & Schmidhuber, J. (2011) · Annual Conference on Genetic and Evolutionary Computation · doi:10.1145/2001576.2001692 · Cited by 193
Two Stream LSTM: A Deep Fusion Framework for Human Action Recognition
Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.27 · Cited by 192
Veldhuizen, M. G., Bender, G., Constable, R. T., & Small, D. M. (2007) · Chemical Senses · doi:10.1093/chemse/bjm025 · Cited by 190
Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation
Dai, N., Liang, J., Qiu, X., & Huang, X. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1601 · Cited by 190
Improving Referring Expression Grounding With Cross-Modal Attention-Guided Erasing
Liu, X., Wang, Z., Shao, J., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00205 · Cited by 188
Attentional Encoder Network for Targeted Sentiment Classification
Song, Y., Wang, J., Jiang, T., Liu, Z., & Rao, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09314 · Cited by 185
Attention-GAN for Object Transfiguration in Wild Images
Chen, X., Xu, C., Yang, X., & Tao, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_11 · Cited by 185
Hierarchical Recurrent Attention Network for Response Generation
Xing, C., Wu, Y., Wu, W., Huang, Y., & Zhou, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11965 · Cited by 184
VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection
Yuan, Y., Xiong, Z., & Wang, Q. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2896952 · Cited by 183
Attention-Aware Deep Reinforcement Learning for Video Face Recognition
Rao, Y., Lu, J., & Zhou, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.424 · Cited by 182
DA-GAN: Instance-Level Image Translation by Deep Attention Generative Adversarial Networks
Ma, S., Fu, J., Chen, C. W., & Mei, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00593 · Cited by 182
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models
Shao, Y., Gouws, S., Britz, D., Goldie, A., Strope, B., & Kurzweil, R. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1235 · Cited by 179
Neural Map: Structured Memory for Deep Reinforcement Learning
Parisotto, E., & Salakhutdinov, R. (2017) · arXiv · doi:10.48550/arxiv.1702.08360 · Cited by 178
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Fang, K., Toshev, A., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00063 · Cited by 178
Learning to Count Objects in Natural Images for Visual Question Answering
Zhang, Y., Hare, J. S., & Prügel‐Bennett, A. (2018) · ePrints Soton (University of Southampton) · doi:10.48550/arxiv.1802.05766 · Cited by 177
aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model
Yang, L., Ai, Q., Guo, J., & Croft, W. B. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983818 · Cited by 176
Fang, B., Li, Y., Zhang, H., & Chan, J. C.-W. (2019) · Remote Sensing · doi:10.3390/rs11020159 · Cited by 173
Wang, W., Yan, M., & Wu, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1158 · Cited by 172
Attention-over-Attention Neural Networks for Reading Comprehension
Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., & Hu, G. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1055 · Cited by 171
Sockeye: A Toolkit for Neural Machine Translation
Hieber, F., Domhan, T., Denkowski, M., Vilar, D., Sokolov, A., Clifton, A., & Post, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05690 · Cited by 171
Attention-Based Extraction of Structured Information from Street View Imagery
Wojna, Z., Gorban, A. N., Lee, D.-S., Murphy, K., Yu, Q., Li, Y., & Ibarz, J. (2017) · 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.143 · Cited by 170
Language Modeling with Deep Transformers
Irie, K., Zeyer, A., Schlüter, R., & Ney, H. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2225 · Cited by 170
Origins of the human pointing gesture: a training study
Matthews, D., Behne, T., Lieven, E., & Tomasello, M. (2012) · Developmental Science · doi:10.1111/j.1467-7687.2012.01181.x · Cited by 168
Attentional Landmarks and Active Gaze Control for Visual SLAM
Frintrop, S., & Jensfelt, P. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004977 · Cited by 168
FusionNet: Fusing via Fully-Aware Attention with Application to Machine ComprehensionPreprint
Huang, H.-Y., Zhu, C., Shen, Y., & Chen, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07341 · Cited by 167
Sequence-to-Sequence Acoustic Modeling for Voice Conversion
Zhang, J.-X., Ling, Z.-H., Liu, L.-J., Jiang, Y., & Dai, L.-R. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2892235 · Cited by 167
RecSys Challenge 2015 and the YOOCHOOSE Dataset
Ben-Shimon, D., Tsikinovsky, A., Friedmann, M., Shapira, B., Rokach, L., & Hoerle, J. (2015) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2792838.2798723 · Cited by 166
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
Shen, T., Zhou, T., Long, G., Jiang, J. B., Wang, S., & Zhang, C. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/604 · Cited by 166
Modal Control Of An Attentive Vision System
Clark, J. J., & Ferrier, N. J. (2005) · [1988 Proceedings] Second International Conference on Computer Vision · doi:10.1109/ccv.1988.590032 · Cited by 165
Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning
Song, J., Gao, L., Guo, Z., Liu, W., Zhang, D., & Shen, H. T. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/381 · Cited by 165
A CNN–RNN architecture for multi-label weather recognition
Zhao, B., Li, X., Lu, X., & Wang, Z. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.048 · Cited by 165
Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embedding
Wang, P., Han, J., Li, C., & Pan, R. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017152 · Cited by 165
Attention-Based Models for Text-Dependent Speaker Verification
Rezaur rahman Chowdhury, F. A., Wang, Q., Moreno, I. L., & Wan, L. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461587 · Cited by 164
Survey on the attention based RNN model and its applications in computer visionPreprint
Wang, F., & Tax, D. M. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06823 · Cited by 163
Research works printed but not linked: 57
- Ahmadi AHK, Hassani K, Moradi P, Lee L, Morris Q (2020) Memory-based graph networks. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020, OpenReview.net. https://openreview.net/forum?id=r1laNeBYPB
- Allamanis M, Peng H, Sutton C (2016) A convolutional attention network for extreme summarization of source code. In: International conference on machine learning, pp 2091–2100
- Arjovsky M (2020) Out of distribution generalization in machine learning. PhD thesis, New York University
- Baluja S, Pomerleau DA (1997) Expectation-based selective attention for visual monitoring and control of a robot vehicle. Robot Auton Syst 22(3–4):329–344 · DOI
- Bastings J, Titov I, Aziz W, Marcheggiani D, Sima’an K (2017) Graph convolutional encoders for syntax-aware neural machine translation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, association for computational linguistics, Copenhagen, Denmark, pp 1957–1967. https://doi.org/10.18653/v1/D17-1209 · DOI · Paper page
- Chen J (2016) Multi-modal learning: study on a large-scale micro-video data collection. In: Proceedings of the 24th ACM international conference on Multimedia, pp 1454–1458 · doi:10.1145/2964284.2971477
- Chen Y, Kalantidis Y, Li J, Yan S, Feng J (2018e) A$$^2$$-nets: Double attention networks. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31. Curran Associates, Inc., pp 352–361. http://papers.nips.cc/paper/7318-a2-nets-double-attention-networks.pdf
- Cheng Z, Bai F, Xu Y, Zheng G, Pu S, Zhou S (2017b) Focusing attention: towards accurate text recognition in natural images. In: Proceedings of the IEEE ICCV, pp 5076–5084 · DOI · Paper page
- Colombini EL, da Silva Simoes A, Ribeiro C (2014) An attentional model for intelligent robotics agents. PhD thesis, Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil
- Daniluk M, Rocktäschel T, Welbl J, Riedel S (2017) Frustratingly short attention spans in neural language modeling. In: 5th international conference on learning representations, ICLR 2017, Toulon, France, April 24–26, 2017, conference track proceedings, OpenReview.net. https://openreview.net/forum?id=ByIAPUcee
- Dehghani M, Gouws S, Vinyals O, Uszkoreit J, Kaiser L (2019) Universal transformers. In: 7th International conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=HyzdRiR9Y7
- Doughty H, Mayol-Cuevas W, Damen D (2019) The pros and cons: rank-aware temporal attention for skill determination in long videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7862–7871 · DOI
- Frintrop S, Rome E, Christensen HI (2010b) Computational visual attention systems and their cognitive foundations: a survey. ACM Trans Appl Percept (TAP) 7(1):1–39 · DOI · Paper page
- Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019b) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10705–10714 · DOI · Paper page
- Girdhar R, Carreira J, Doersch C, Zisserman A (2019) Video action transformer network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 244–253 · DOI
- Goyal A, Lamb A, Hoffmann J, Sodhani S, Levine S, Bengio Y, Schölkopf B (2019) Recurrent independent mechanisms. arXiv:190910893
- Gregor K, Danihelka I, Mnih A, Blundell C, Wierstra D (2014) Deep autoregressive networks. In: International conference on machine learning, PMLR, pp 1242–1250
- Han Z, Wang X, Vong C, Liu Y, Zwicker M, Chen CLP (2019) 3dviewgraph: learning global features for 3d shapes from A graph of unordered views with attention. In: Kraus S (ed) Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI 2019, Macao, China, August 10–16, 2019, ijcai.org, pp 758–765. https://doi.org/10.24963/ijcai.2019/107 · doi:10.24963/ijcai.2019/107
- Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2015) Session-based recommendations with recurrent neural networks. arXiv:151106939
- Hole KJ, Ahmad S (2021) A thousand brains: toward biologically constrained ai. SN Appl Sci 3(8):1–14
- Huang CZA, Vaswani A, Uszkoreit J, Simon I, Hawthorne C, Shazeer N, Dai AM, Hoffman MD, Dinculescu M, Eck D (2018a) Music transformer: generating music with long-term structure. In: International conference on learning representations
- Jaderberg M, Simonyan K, Zisserman A, kavukcuoglu k (2015) Spatial transformer networks. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28. Curran Associates, Inc., pp 2017–2025. http://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf
- Joulin A, Mikolov T (2015) Inferring algorithmic patterns with stack-augmented recurrent nets. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7–12, 2015, Montreal, pp 190–198
- Kim JH, Jun J, Zhang BT (2018a) Bilinear attention networks. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31. Curran Associates, Inc., pp 1564–1574. http://papers.nips.cc/paper/7429-bilinear-attention-networks.pdf
- Kiros R, Salakhutdinov R, Zemel R (2014) Multimodal neural language models. In: International conference on machine learning, PMLR, pp 595–603
- Kumar A, Irsoy O, Ondruska P, Iyyer M, Bradbury J, Gulrajani I, Zhong V, Paulus R, Socher R (2016) Ask me anything: Dynamic memory networks for natural language processing. In: International conference on machine learning, PMLR, pp 1378–1387
- Li J, Yang J, Hertzmann A, Zhang J, Xu T (2019d) Layoutgan: Generating graphic layouts with wireframe discriminators. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=HJxB5sRcFQ
- Liu J, Wang G, Hu P, Duan LY, Kot AC (2017d) Global context-aware attention lstm networks for 3d action recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR) · DOI · Paper page
- Liu S, Johns E, Davison AJ (2019c) End-to-end multi-task learning with attention. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1871–1880 · DOI · Paper page
- Mittal S, Lamb A, Goyal A, Voleti V, Shanahan M, Lajoie G, Mozer M, Bengio Y (2020) Learning to combine top-down and bottom-up signals in recurrent neural networks with attention over modules. In: International conference on machine learning, PMLR, pp 6972–6986
- (ed) Interspeech 2017, 18th annual conference of the international speech communication association, Stockholm, August 20-24, 2017, ISCA, pp 1263-1267. http://www.isca-speech.org/archive/Interspeech_2017/abstracts/0917.html
- Norouzian A, Mazoure B, Connolly D, Willett D (2019) Exploring attention mechanism for acoustic-based classification of speech utterances into system-directed and non-system-directed. In: ICASSP 2019–2019 IEEE international conference on acoustics. Speech and signal processing (ICASSP). IEEE, pp 7310–7314 · doi:10.1109/icassp.2019.8683565
- Oh J, Chockalingam V, Lee H, et al. (2016) Control of memory, active perception, and action in minecraft. In: International conference on machine learning, PMLR, pp 2790–2799
- Ouerhani N (2003) Visual attention: from bio-inspired modeling to real-time implementation. PhD thesis, Université de Neuchâtel
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
- Rekabdar B, Mousas C, Gupta B (2019) Generative adversarial network with policy gradient for text summarization. In: 2019 IEEE 13th international conference on semantic computing (ICSC). IEEE, pp 204–207 · DOI
- Ren M, Liao R, Fetaya E, Zemel R (2019) Incremental few-shot learning with attention attractor networks. In: Advances in neural information processing systems, pp 5275–5285
- Rezende DJ, Mohamed S, Danihelka I, Gregor K, Wierstra D (2016) One-shot generalization in deep generative models. arXiv:160305106
- Rotenstein A, Andreopoulos A, Fazl E, Jacob D, Robinson M, Shubina K, Zhu Y, Tsotsos J (2007) Towards the dream of intelligent, visually-guided wheelchairs. In: Proceedings of the 2nd international conference on technology and aging
- Scheier C, Egner S (1997) Visual attention in a mobile robot. In: ISIE’97 Proceeding of the IEEE international symposium on industrial electronics. vol 1. IEEE, pp SS48–SS52 · doi:10.1109/isie.1997.651734
- Seo PH, Lin Z, Cohen S, Shen X, Han B (2016) Hierarchical attention networks. arXiv:abs/160602393
- Sordoni A, Bachman P, Trischler A, Bengio Y (2016) Iterative alternating neural attention for machine reading. arXiv:160602245
- Tian T, Fang ZF (2019) Attention-based autoencoder topic model for short texts. Procedia Comput Sci 151:1134–1139 · doi:10.1016/j.procs.2019.04.161
- Tsai YH, Srivastava N, Goh H, Salakhutdinov R (2020) Capsules with inverted dot-product attention routing. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020, OpenReview.net. https://openreview.net/forum?id=HJe6uANtwH
- Van Den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. In: Neural information processing systems conference (NIPS 2013), neural information processing systems foundation (NIPS), vol 26
- Van der Velde F, de Kamps M et al (2004) Clam: closed-loop attention model for visual search. Neurocomputing 58:607–612 · doi:10.1016/j.neucom.2004.01.102
- Vinyals O, Fortunato M, Jaitly N (2015a) Pointer networks. In: Advances in neural information processing systems, pp 2692–2700
- Vinyals O, Kaiser Ł, Koo T, Petrov S, Sutskever I, Hinton G (2015b) Grammar as a foreign language. In: Advances in neural information processing systems, pp 2773–2781
- Walther D, Edgington DR, Koch C (2004) Detection and tracking of objects in underwater video. In: Proceedings of the IEEE CVPR, vol 1. IEEE, pp I–I · doi:10.1109/cvpr.2004.1315079
- Weston J, Chopra S, Bordes A (2014) Memory networks. arXiv:14103916
- Wu W, Chen Y, Xu J, Zhang Y (2018c) Attention-based convolutional neural networks for chinese relation extraction. In: Chinese computational linguistics and natural language processing based on naturally annotated big data. Springer, pp 147–158 · doi:10.1007/978-3-030-01716-3_13
- Yang F, Yang Z, Cohen WW (2017a) Differentiable learning of logical rules for knowledge base reasoning. In: Advances in neural information processing systems, pp 2319–2328
- Yu Z, Yu J, Xiang C, Fan J, Tao D (2018e) Beyond bilinear: generalized multimodal factorized high-order pooling for visual question answering. IEEE Trans Neural Netw Learn Syst 29(12):5947–5959 · DOI · Paper page
- Zhang B, Xiong D, Su J (2017a) Battrae: didimensional attention-based recursive autoencoders for learning bilingual phrase embeddings. In: Proceedings of the AAAI conference on artificial intelligence · doi:10.1609/aaai.v31i1.10969
- Zhang B, Xiong D, Su J, Zhang M (2018a) Learning better discourse representation for implicit discourse relation recognition via attention networks. Neurocomputing 275:1241–1249. https://doi.org/10.1016/j.neucom.2017.09.074 · DOI
- Zhang Y, Li K, Li K, Zhong B, Fu Y (2019j) Residual non-local attention networks for image restoration. In: 7th International conference on learning representations, ICLR 2019, New Orleans, May 6–9, 2019, OpenReview.net. https://openreview.net/forum?id=HkeGhoA5FX
- Zhang Y, Zhou C, Chang F, Kot AC (2019o) Multi-resolution attention convolutional neural network for crowd counting. Neurocomputing 329:144–152 · doi:10.1016/j.neucom.2018.10.058
Cited by
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
Feng, Y., Chen, J., Xie, J., Zhang, T., Lv, H., & Pan, T. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107646 · Cited by 262
Lv, H., Chen, J., Pan, T., Zhang, T., Feng, Y., & Liu, S. (2022) · Measurement · doi:10.1016/j.measurement.2022.111594 · Cited by 261
Metadata
- Authors
- de Santana Correia, A., & Colombini, E. L.
- Venue
- Artificial Intelligence Review
- Published
- 2022-03-03
- Volume
- 55
- Issue
- 8
- Pages
- 6037-6124
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W3147852756
- Cited by
- 298
- References
- 569
- FWCI
- 20.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.