The DOI matches its Crossref record; 569 of the 569 references it cites were found in a registry or an index.
Attention, please! A survey of neural attention models in deep learning
Abstract
In humans, Attention is a core property of all perceptual and cognitive operations. Given our limited ability to process competing sources, attention mechanisms select, modulate, and focus on the information most relevant to behavior. For decades, concepts and functions of attention have been studied in philosophy, psychology, neuroscience, and computing. For the last 6 years, this property has been widely explored in deep neural networks. Currently, the state-of-the-art in Deep Learning is represented by neural attention models in several application domains. This survey provides a comprehensive overview and analysis of developments in neural attention models. We systematically reviewed hundreds of architectures in the area, identifying and discussing those in which attention has shown a significant impact. We also developed and made public an automated methodology to facilitate the development of reviews in the area. By critically analyzing 650 works, we describe the primary uses of attention in convolutional, recurrent networks, and generative models, identifying common subgroups of uses and applications. Furthermore, we describe the impact of attention in different application domains and their impact on neural networks’ interpretability. Finally, we list possible trends and opportunities for further research, hoping that this review will provide a succinct overview of the main attentional models in the area and guide researchers in developing future approaches that will drive further improvements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 401–569 of 569
Song, M., Park, H., & Shin, K.-S. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2018.12.005 · Cited by 163
Attention Based Fully Convolutional Network for Speech Emotion Recognition
Zhang, Y., Du, J., Wang, Z., Zhang, J., & Tu, Y. (2018) · 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipa.2018.8659587 · Cited by 161
Commonsense for Generative Multi-Hop Question Answering Tasks
Bauer, L., Wang, Y., & Bansal, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1454 · Cited by 161
Jiang, H., Shi, T., Bai, Z., & Huang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2899608 · Cited by 161
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence ModelingPreprint
Shen, T., Zhou, T., Long, G., Jiang, J. B., & Zhang, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00857 · Cited by 160
LSTA: Long Short-Term Attention for Egocentric Action Recognition
Sudhakaran, S., Escalera, S., & Lanz, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01019 · Cited by 160
Crowd Counting Using Scale-Aware Attention Networks
Hossain, M., Hosseinzadeh, M., Chanda, O., & Wang, Y. (2019) · 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2019.00141 · Cited by 159
Attend and Interact: Higher-Order Object Interactions for Video Understanding
Ma, C., Kadav, A., Melvin, I., Kira, Z., AlRegib, G., & Graf, H. P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00710 · Cited by 159
Hypergraph Attention Networks for Multimodal Learning
Kim, E., Kang, W. Y., On, K.-W., Heo, Y., & Zhang, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01459 · Cited by 156
Weighted Transformer Network for Machine TranslationPreprint
Ahmed, K., Keskar, N. S., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02132 · Cited by 153
CAR-Net: Clairvoyant Attentive Recurrent Network
Sadeghian, A., Legros, F., Voisin, M., Vesel, R., Alahi, A., & Savarese, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_10 · Cited by 153
Description Generation for Remote Sensing Images Using Attribute Attention Mechanism
Zhang, X., Wang, X., Tang, X., Zhou, H., & Li, C. (2019) · Remote Sensing · doi:10.3390/rs11060612 · Cited by 153
Bhunia, A. K., Konwer, A., Bhunia, A. K., Bhowmick, A., Roy, P. P., & Pal, U. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.034 · Cited by 150
Context-Aware Self-Attention Networks
Yang, B., Li, J., Wong, D. F., Chao, L. S., Wang, X., & Tu, Z. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301387 · Cited by 150
Relational Network for Skeleton-Based Action Recognition
Wu, Z., Li, L., Zhang, Z., Huang, Y., & Wang, L. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00147 · Cited by 150
Attention for Fine-Grained CategorizationPreprint
Sermanet, P., Frome, A., & Real, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.7054 · Cited by 149
Attribute-Aware Attention Model for Fine-grained Representation Learning
Han, K., Guo, J., Zhang, C., & Zhu, M. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240550 · Cited by 149
Deep Compositional Question Answering with Neural Module NetworksPreprint
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2015) · arXiv (Cornell University) · Cited by 147
Recurrent Mixture Density Network for Spatiotemporal Visual AttentionPreprint
Bazzani, L., Larochelle, H., & Torresani, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08199 · Cited by 147
Lu, J., Kannan, A., Yang, J., Parikh, D., & Batra, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01554 · Cited by 147
Learning to detect chest radiographs containing pulmonary lesions using visual attention networks
Pesce, E., Withey, S. J., Ypsilantis, P.-P., Bakewell, R. J., Goh, V., & Montana, G. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.12.007 · Cited by 145
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition
Salazar, J., Kirchhoff, K., & Huang, Z. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682539 · Cited by 144
Lexicon Integrated CNN Models with Attention for Sentiment Analysis
Shin, B., Lee, T. J., & Choi, J. D. (2017) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/w17-5220 · Cited by 144
Unified Spatio-Temporal Attention Networks for Action Recognition in Videos
Li, D., Yao, T., Duan, L., Mei, T., & Rui, Y. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2862341 · Cited by 143
Salah, A. A., Alpaydın, E., & Akarun, L. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990146 · Cited by 140
Scene Segmentation with DAG-Recurrent Neural Networks
Shuai, B., Zuo, Z. C., Wang, B., & Wang, G. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2712691 · Cited by 139
Please Stop Explaining Black Box Models for High Stakes Decisions.Preprint
Rudin, C. D. (2018) · arXiv (Cornell University) · Cited by 138
DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional NetworksPreprint
Li, X., Xiong, H., Wang, H., Rao, Y., Liu, L., Chen, Z., & Huan, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09229 · Cited by 136
Guo, Q., Qiu, X., Liu, P., Shao, Y., Xue, X., & Zhang, Z. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1133 · Cited by 135
SDNet: Contextualized Attention-based Deep Network for Conversational Question AnsweringPreprint
Zhu, C., Zeng, M., & Huang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03593 · Cited by 132
What Do Recurrent Neural Network Grammars Learn About Syntax?
Kuncoro, A., Ballesteros, M., Kong, L., Dyer, C., Neubig, G., & Smith, N. A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1117 · Cited by 131
Kang, G., Zheng, L., Yan, Y., & Yang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_25 · Cited by 127
Stacked Semantic-Guided Attention Model for Fine-Grained Zero-Shot LearningPreprint
yu, Y., Ji, Z., Fu, Y., Guo, J., Pang, Y., & Zhang, Z. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08113 · Cited by 124
Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog
Gan, Z., Cheng, Y., Kholy, A. E., Li, L., Liu, J., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1648 · Cited by 124
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour PredictionPreprint
Xu, D., Ouyang, W., Alameda-Pineda, X., Ricci, E., Wang, X., & Sebe, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.00524 · Cited by 123
Asynchronous Bidirectional Decoding for Neural Machine Translation
Zhang, X., Su, J., Qin, Y., Liu, Y., Ji, R., & Wang, H. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11984 · Cited by 122
Suganuma, M., Liu, X., & Okatani, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00925 · Cited by 121
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
Zhou, S., Dong, L., Xu, S., & Xu, B. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1107 · Cited by 119
Adding Attentiveness to the Neurons in Recurrent Neural Networks
Zhang, P., Xue, J., Lan, C., Zeng, W., Gao, Z., & Zheng, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_9 · Cited by 115
Adding Interpretable Attention to Neural Translation Models Improves Word AlignmentPreprint
Zenkel, T., Wuebker, J., & DeNero, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11359 · Cited by 115
Latent Alignment and Variational AttentionPreprint
Deng, Y., Kim, Y., Chiu, J., Guo, D., & Rush, A. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03756 · Cited by 114
Enhancing Attention-Based LSTM With Position Context for Aspect-Level Sentiment Classification
Zeng, J., Ma, X., & Zhou, K. (2019) · IEEE Access · doi:10.1109/access.2019.2893806 · Cited by 114
Differentiable Dynamic Programming for Structured Prediction and AttentionPreprint
Mensch, A., & Blondel, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03676 · Cited by 112
Neural Tree Indexers for Text Understanding
Munkhdalai, T., & Yu, H. Q. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1002 · Cited by 109
Modeling Recurrence for Transformer
Hao, J., Wang, X., Yang, B., Wang, L., Zhang, J., & Tu, Z. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1122 · Cited by 109
Bi-Directional Spatial-Semantic Attention Networks for Image-Text Matching
Huang, F., Zhang, X., Zhao, Z., & Li, Z. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2882225 · Cited by 107
Sparse Attentive Backtracking: Temporal CreditAssignment Through RemindingPreprint
Ke, N. R., Goyal, A., Bilaniuk, O., Binas, J., Mozer, M. C., Pal, C., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.5555/3327757.3327863 · Cited by 105
Focal Visual-Text Attention for Visual Question Answering
Liang, J., Jiang, L., Cao, L., Li, L.-J., & Hauptmann, A. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00642 · Cited by 103
Understanding Subtitles by Character-Level Sequence-to-Sequence Learning
Zhang, H., Li, J., Ji, Y., & Yue, H. (2016) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2016.2601521 · Cited by 101
Attention Routing Between Capsules
Choi, J., Seo, H., Im, S., & Kang, M. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00247 · Cited by 101
The physiology and psychology of selective attention to touch
Johansen‐Berg, H., & Lloyd, D. M. (2000) · Frontiers in bioscience · doi:10.2741/a558 · Cited by 99
Attention in convolutional LSTM for gesture recognition
Zhang, L., Zhu, G., Mei, L., Shen, P., Shah, S. A. A., & Bennamoun, M. (2018) · neural information processing systems · Cited by 98
SCAN: Self-and-Collaborative Attention Network for Video Person Re-Identification
Zhang, R., Li, J., Sun, H., Ge, Y., Luo, P., Wang, X., & Lin, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2911488 · Cited by 98
Audio Set Classification with Attention Model: A Probabilistic Perspective
Kong, Q., Xu, Y., Wang, W., & Plumbley, M. D. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461392 · Cited by 98
Deep gated attention networks for large-scale street-level scene segmentation
Zhang, P., Liu, W., Wang, H., Lei, Y., & Lu, H. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.12.021 · Cited by 97
Consensus Attention-based Neural Networks for Chinese Reading ComprehensionPreprint
Cui, Y., Liu, T., Chen, Z., Wang, S., & Hu, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02250 · Cited by 94
Feature Fusion Encoder Decoder Network for Automatic Liver Lesion Segmentation
Chen, X., Zhang, R., & Yan, P. (2019) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2019.8759555 · Cited by 94
Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation
Sperber, M., Neubig, G., Niehues, J., & Waibel, A. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00270 · Cited by 91
Zhang, J., Du, J., & Dai, L.-R. (2017) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.152 · Cited by 91
Deep Saliency Hashing for Fine-Grained Retrieval
Jin, S., Yao, H., Sun, X., Zhou, S., Zhang, L., & Hua, X. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2971105 · Cited by 91
Temporal Attention-Gated Model for Robust Sequence Classification
Pei, W., Baltrušaitis, T., Tax, D. M. J., & Morency, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.94 · Cited by 90
Walther, D. B. (2006) · Europe PMC (PubMed Central) · doi:10.7907/p5ny-vc91 · Cited by 89
Attention-Based Temporal Weighted Convolutional Neural Network for Action Recognition
Zang, J., Wang, L., Liu, Z., Zhang, Q., Hua, G., & Zheng, N. (2018) · IFIP advances in information and communication technology · doi:10.1007/978-3-319-92007-8_9 · Cited by 89
Hamker, F. H. (2005) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.09.005 · Cited by 88
Convolutional Attention Networks for Scene Text Recognition
Xie, H., Fang, S., Zha, Z.-J., Yang, Y., Li, Y., & Zhang, Y. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3231737 · Cited by 88
SAC-Net: Spatial Attenuation Context for Salient Object Detection
Hu, X., Fu, C., Zhu, L., Wang, T., & Heng, P. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2995220 · Cited by 87
Input Combination Strategies for Multi-Source Transformer Decoder
Libovický, J., Helcl, J., & Mareček, D. (2018) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w18-6326 · Cited by 85
A Single Attention-Based Combination of CNN and RNN for Relation Classification
Guo, X., Zhang, H., Yang, H., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2891770 · Cited by 84
Modality Shifting Attention Network for Multi-Modal Video Question Answering
Kim, J., Ma, M., Pham, T. X., Kim, K., & Yoo, C. D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01012 · Cited by 84
Cross-Modal Attentional Context Learning for RGB-D Object Detection
Li, G., Gan, Y., Wu, H., Xiao, N., & Lin, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2878956 · Cited by 83
Dong, L., Wang, F., & Xu, B. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682954 · Cited by 83
Yasuda, Y., Wang, X., Takaki, S., & Yamagishi, J. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682353 · Cited by 80
Zhang, M., Wang, X., Fang, F., Li, H., & Yamagishi, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1357 · Cited by 80
Few-shot Autoregressive Density Estimation: Towards Learning to Learn DistributionsPreprint
Reed, S., Chen, Y., Paine, T., van den Oord, A., Eslami, S. M. A., Rezende, D. J., Vinyals, O., & de Freitas, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10304 · Cited by 79
Pointing Novel Objects in Image Captioning
Li, Y., Yao, T., Pan, Y., Chao, H., & Mei, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01278 · Cited by 79
Video-Based Person Re-identification via 3D Convolutional Networks and Non-local Attention
Liao, X., He, L., Yang, Z., & Zhang, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20876-9_39 · Cited by 78
Fully Convolutional Sequence Recognition Network for Water Meter Number Reading
Yang, F., Jin, L., Lai, S., Gao, X., & Li, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2891767 · Cited by 78
Quaternion Convolutional Neural Network for Color Image Classification and Forensics
Yin, Q., Wang, J., Luo, X., Zhai, J., Jha, S. K., & Shi, Y.-Q. (2019) · IEEE Access · doi:10.1109/access.2019.2897000 · Cited by 77
Attention-Based Multiview Re-Observation Fusion Network for Skeletal Action Recognition
Fan, Z., Zhao, X., Lin, T., & Su, H. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2859620 · Cited by 76
Coarse-grain Fine-grain Coattention Network for Multi-evidence Question AnsweringPreprint
Zhong, V., Xiong, C., Keskar, N. S., & Socher, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00603 · Cited by 76
Deep Joint Entity Disambiguation with Local Neural AttentionPreprint
Ganea, O.-E., & Hofmann, T. (2017) · arXiv (Cornell University) · doi:10.3929/ethz-b-000225539 · Cited by 76
Focal Visual-Text Attention for Memex Question Answering
Liang, J., Jiang, L., Cao, L., Kalantidis, Y., Li, L.-J., & Hauptmann, A. G. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2890628 · Cited by 76
Interpretable Visual Question Answering by Visual Grounding From Attention Supervision Mining
Zhang, Y., Niebles, J. C., & Soto, Á. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00043 · Cited by 71
Image captioning with triple-attention and stack parallel LSTM
Zhu, X., Li, L., Liu, J., Li, Z., Peng, H., & Niu, X. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.08.069 · Cited by 71
DeepHINT: understanding HIV-1 integration via deep learning with attention
Hu, H., Xiao, A., Zhang, S., Li, Y., Shi, X., Jiang, T., Zhang, L., Zhang, L., & Zeng, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty842 · Cited by 71
Multi-Stream Network with Temporal Attention for Environmental Sound Classification
Li, X., Chebiyyam, V., & Kirchhoff, K. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3019 · Cited by 71
Learning Embedding Adaptation for Few-Shot LearningPreprint
Ye, H.-J., Hu, H., Zhan, D., & Sha, F. (2018) · arXiv (Cornell University) · Cited by 70
Convolution-Based Neural Attention With Applications to Sentiment Classification
Du, J., Gui, L., He, Y., Xu, R., & Wang, X. (2019) · IEEE Access · doi:10.1109/access.2019.2900335 · Cited by 70
Zhang, X., Shi, H., Li, C., Zheng, K., Zhu, X., & Duan, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019227 · Cited by 70
Huang, K.-Y., Wu, C., & Su, M.-H. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.12.016 · Cited by 69
Improving Relation Extraction by Pre-trained Language RepresentationsPreprint
Alt, C., Hübner, M., & Hennig, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03088 · Cited by 68
DA-CapsNet: dual attention mechanism capsule network
Huang, W., & Zhou, F. (2020) · Scientific Reports · doi:10.1038/s41598-020-68453-w · Cited by 65
Option Comparison Network for Multiple-choice Reading ComprehensionPreprint
Ran, Q., Li, P., Hu, W., & Zhou, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03033 · Cited by 64
Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts
Schick, T., & Schütze, H. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1048 · Cited by 60
Modeling feature-based attention as an active top-down inference process
Hamker, F. H. (2006) · Biosystems · doi:10.1016/j.biosystems.2006.03.010 · Cited by 59
Choi, M., Park, C., Yang, S., Kim, Y., Choo, J., & Hong, S.-S. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300460 · Cited by 59
Variational Memory Addressing in Generative ModelsPreprint
Bornschein, J., Mnih, A., Zoran, D., & Rezende, D. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07116 · Cited by 58
DRAU: Dual Recurrent Attention Units for Visual Question Answering
Osman, A., & Samek, W. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.05.001 · Cited by 57
Pixel-Wise Attentional Gating for Scene Parsing
Kong, S., & Fowlkes, C. C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00114 · Cited by 56
Connection Sensitive Attention U-NET for Accurate Retinal Vessel SegmentationPreprint
Li, R., Li, M., Li, J., & Zhou, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.05558 · Cited by 55
Shuang, K., Ren, X., Yang, Q., Li, R., & Loo, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.11.084 · Cited by 54
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence ModelsPreprint
Shao, L. Y., Gouws, S., Britz, D., Goldie, A., Strope, B. P., & Kurzweil, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03185 · Cited by 54
Lattice-Based Transformer Encoder for Neural Machine Translation
Xiao, F., Li, J., Zhao, H., Wang, R., & Chen, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1298 · Cited by 51
Efficient Large-Scale Neural Domain Classification with Personalized Attention
Kim, Y., Kim, D., Kumar, A., & Sarikaya, R. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1206 · Cited by 51
Densely Connected Attention Propagation for Reading ComprehensionPreprint
Tay, Y., Tuan, L. A., Hui, S. C., & Su, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04210 · Cited by 50
VD-SAN: Visual-Densely Semantic Attention Network for Image Caption Generation
He, X., Yang, Y., Shi, B., & Bai, X. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.02.106 · Cited by 49
Attention based hierarchical LSTM network for context-aware microblog sentiment classification
Feng, S., Wang, Y., Liu, L., Wang, D., & Yu, G. (2018) · World Wide Web · doi:10.1007/s11280-018-0529-6 · Cited by 48
Character-based handwritten text transcription with attention networks
Poulos, J., & Valle, R. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05813-1 · Cited by 46
Convolutional Self-Attention NetworksPreprint
Yang, B., Wang, L., Wong, D. F., Chao, L. S., & Tu, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03107 · Cited by 45
Residual attention-based LSTM for video captioning
Li, X., Zhou, Z., Chen, L., & Gao, L. (2018) · World Wide Web · doi:10.1007/s11280-018-0531-z · Cited by 44
Learning Better Features for Face Detection with Feature Fusion and Segmentation SupervisionPreprint
Tian, W., Wang, Z., Shen, H., Deng, W., Meng, Y., Chen, B., Zhang, X., Zhao, Y., & Huang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08557 · Cited by 44
Deep Fusion: An Attention Guided Factorized Bilinear Pooling for Audio-video Emotion Recognition
Zhang, Y., Wang, Z.-R., & Du, J. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851942 · Cited by 44
Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
Ferret, J., Marinier, R., Geist, M., & Pietquin, O. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/368 · Cited by 44
Dynamic Attention-based Visual Odometry
Kuo, X.-Y., Liu, C., Lin, K.-C., & Lee, C. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00026 · Cited by 44
Commonsense for Generative Multi-Hop Question Answering TasksPreprint
Bauer, L., Wang, Y., & Bansal, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.06309 · Cited by 42
A Survey on Statistical Relational Learning
Khosravi, H., & Bina, B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13059-5_25 · Cited by 41
Automatic Question Tagging with Deep Neural Networks
Sun, B., Zhu, Y., Xiao, Y., Xiao, R., & Wei, Y. (2018) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2018.2808187 · Cited by 41
Ouyang, D., Zhang, Y., & Shao, J. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.05.009 · Cited by 39
Neural Multimodal Belief Tracker with Adaptive Attention for Dialogue Systems
Zhang, Z., Liao, L., Huang, M., Zhu, X., & Chua, T. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313598 · Cited by 39
Pay Attention! - Robustifying a Deep Visuomotor Policy Through Task-Focused Visual Attention
Abolghasemi, P., Mazaheri, A., Shah, M. A., & Bölöni, L. L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00438 · Cited by 39
Learning to Generate with MemoryPreprint
Li, C., Zhu, J., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07416 · Cited by 38
Attention-Aware Generative Adversarial Networks (ATA-GANs)
Kastaniotis, D., Ντίνου, Ι., Tsourounis, D., Economou, G., & Fotopoulos, S. (2018) · Multidimensional Signal Processing Workshop · doi:10.1109/ivmspw.2018.8448850 · Cited by 38
Digital Image Steganalysis Based on Visual Attention and Deep Reinforcement Learning
Hu, D., Zhou, S., Shen, Q., Zheng, S., Zhao, Z., & Fan, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2900076 · Cited by 37
Hierarchical attention-based multimodal fusion for video captioning
Wu, C., Wei, Y., Chu, X., Sun, W., Su, F., & Wang, L. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.029 · Cited by 33
Self-Attention Capsule Networks for Object ClassificationPreprint
Hoogi, A., Wilcox, B., Gupta, Y., & Rubin, D. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12483 · Cited by 33
Transformer guided geometry model for flow-based unsupervised visual odometry
Li, X., Hou, Y., Wang, P., Gao, Z., Xu, M., & Li, W. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05545-8 · Cited by 33
Doubly Attentive Transformer Machine TranslationPreprint
Arslan, H. S., Fishel, M., & Anbarjafari, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11605 · Cited by 32
Probabilistic Logic Graph Attention Networks for Reasoning
Vardhan, L. V. H., Guo, J., & Kok, S. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366424.3391265 · Cited by 32
Olivastri, S., Singh, G., & Fabio Cuzzolin, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00185 · Cited by 31
R-Trans: RNN Transformer Network for Chinese Machine Reading Comprehension
Liu, S., Zhang, S., Zhang, X., & Wang, H. (2019) · IEEE Access · doi:10.1109/access.2019.2901547 · Cited by 30
Zang, X., Pokle, A., Vázquez, M., Chen, K., Niebles, J. C., Soto, Á., & Savarese, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1286 · Cited by 30
Adaptive Feature Abstraction for Translating Video to Text
Pu, Y., Min, M. R., Gan, Z., & Carin, L. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12245 · Cited by 30
Robust and High Performance Face DetectorPreprint
Zhang, Y., Xu, X., & Liu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.02350 · Cited by 29
Deep Visual Odometry With Adaptive Memory
Xue, F., Wang, X., Wang, J., & Zha, H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3014100 · Cited by 29
MGANet: A Robust Model for Quality Enhancement of Compressed VideoPreprint
Meng, X., Deng, X., Zhu, S., Liu, S., Wang, C., Chen, C., & Zeng, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09150 · Cited by 28
Memory-augmented Attention Modelling for VideosPreprint
Fakoor, R., Mohamed, A., Mitchell, M., Kang, S. B., & Kohli, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02261 · Cited by 27
What Do Recurrent Neural Network Grammars Learn About Syntax?Preprint
Kuncoro, A., Ballesteros, M., Kong, L., Dyer, C., Neubig, G., & Smith, N. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05774 · Cited by 27
Attention-based Encoder-Decoder Networks for Spelling and Grammatical Error CorrectionPreprint
Ahmadi, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00660 · Cited by 26
Fast Adaptation in Generative Models with Generative Matching NetworksPreprint
Bartunov, S., & Vetrov, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02192 · Cited by 24
End-to-end Anchored Speech Recognition
Wang, Y., Fan, X., Chen, I., Liu, Y., Chen, T., & Hoffmeister, B. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682866 · Cited by 24
Tian, J., Li, C., Shi, Z., & Xu, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_78 · Cited by 23
Stream Attention-based Multi-array End-to-end Speech Recognition
Wang, X., Li, R., Mallidi, S. H., Hori, T., Watanabe, S., & Heřmanský, H. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682650 · Cited by 22
Fantastic Answers and Where to Find Them: Immersive Question-Directed Visual Attention
Jiang, M. X., Chen, S., Yang, J., & Zhao, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00305 · Cited by 21
Action Recognition Using Visual Attention with Reinforcement Learning
Li, H., Chen, J., Hu, R., Yu, M., Chen, H., & Xu, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05716-9_30 · Cited by 19
LSTM Networks for Online Cross-Network Recommendations
Perera, D., & Zimmermann, R. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/532 · Cited by 18
Yang, Z., Raymond, O. I., Sun, W., & Long, J. (2019) · IEEE Access · doi:10.1109/access.2019.2891894 · Cited by 18
Video Object Segmentation with Episodic Graph Memory NetworksPreprint
Lu, X., Wang, W., Danelljan, M., Zhou, T., Shen, J., & Gool, L. V. (2020) · Lirias · doi:10.3929/ethz-b-000460874 · Cited by 18
Gao, F., Yu, J., Shen, H., Wang, Y., & Yang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.09369 · Cited by 18
Semantic Adversarial Network with Multi-Scale Pyramid Attention for Video Classification
Xie, D., Deng, C. D., Wang, H., Li, C., & Tao, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019030 · Cited by 17
Bielski, A., & Trzcinski, T. P. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00309 · Cited by 16
Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling
Abdulnabi, A. H., Shuai, B., WINKLER, S. S., & Wang, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.665 · Cited by 15
A Multimodal LSTM for Predicting Listener Empathic Responses Over Time
Zhi‐Xuan, T., Goel, A., Nguyen, T.-S., & Ong, D. C. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756577 · Cited by 15
(Self-Attentive) Autoencoder-based Universal Language Representation for Machine TranslationPreprint
Escolano, C., Costa‐jussà, M. R., & Fonollosa, J. A. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06351 · Cited by 14
Memory Graph Networks for Explainable Memory-grounded Question Answering
Moon, S., Shah, P., Kumar, A., & Subba, R. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1068 · Cited by 14
Distant Supervision for Relation Extraction with Hierarchical Attention and Entity Descriptions
She, H., Wu, B., Wang, B., et al. (2018) · doi:10.1109/ijcnn.2018.8489631 · Cited by 13
Differentiable Adaptive Computation Time for Visual Reasoning
Eyzaguirre, C., & Soto, Á. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01283 · Cited by 13
Park, D., Kim, J., & Chun, S. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10157 · Cited by 12
Recurrent Temporal Sparse Autoencoder for attention-based action recognition
Xin, M., Zhang, H., Sun, M., & Yuan, D. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727234 · Cited by 10
Phrase-Based AttentionsPreprint
Nguyen, P. X., & Joty, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03444 · Cited by 10
Neural Phrase-to-Phrase Machine TranslationPreprint
Feng, J., Kong, L., Huang, P.-S., Wang, C., Huang, D. W., Mao, J., Qiao, K., & Zhou, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.02172 · Cited by 10
Information Aggregation for Multi-Head Attention with Routing-by-AgreementPreprint
Li, J., Yang, B., Dou, Z.-Y., Wang, X., Lyu, M. R.-T., & Tu, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03100 · Cited by 9
Exploring Self-Attention for Visual OdometryPreprint
Damirchi, H., Khorrambakht, R., & Taghirad, H. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08634 · Cited by 8
Dual Recurrent Attention Units for Visual Question AnsweringPreprint
Osman, A., & Samek, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00209 · Cited by 6
Inference, Learning and Attention Mechanisms that Exploit and Preserve Sparsity in CNNs
Häckel, T., Usvyatsov, M., Galliani, S., Wegner, J. D., & Konrad Schindler, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-12939-2_41 · Cited by 6
Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer
Chiang, T.-R., Huang, C.-W., Su, S., & Chen, Y.-N. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101073 · Cited by 6
A Painless Attention Mechanism for Convolutional Neural Networks
Rodríguez, P., Cucurull, G., Gonzàlez, J., & Gonfaus, J. M. (2018) · Cited by 5
An End-to-End Baseline for Video Captioning.Preprint
Olivastri, S., Singh, G., & Fabio Cuzzolin, A. (2019) · arXiv (Cornell University) · Cited by 3
Capacity Visual Attention Networks
Edel, M., & Lausch, J. (2018) · EPiC series in computing · doi:10.29007/lcmk · Cited by 2
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019) · arXiv · doi:10.48550/arxiv.1901.02860
Research works printed but not linked: 57
- Ahmadi AHK, Hassani K, Moradi P, Lee L, Morris Q (2020) Memory-based graph networks. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020, OpenReview.net. https://openreview.net/forum?id=r1laNeBYPB
- Allamanis M, Peng H, Sutton C (2016) A convolutional attention network for extreme summarization of source code. In: International conference on machine learning, pp 2091–2100
- Arjovsky M (2020) Out of distribution generalization in machine learning. PhD thesis, New York University
- Baluja S, Pomerleau DA (1997) Expectation-based selective attention for visual monitoring and control of a robot vehicle. Robot Auton Syst 22(3–4):329–344 · DOI
- Bastings J, Titov I, Aziz W, Marcheggiani D, Sima’an K (2017) Graph convolutional encoders for syntax-aware neural machine translation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, association for computational linguistics, Copenhagen, Denmark, pp 1957–1967. https://doi.org/10.18653/v1/D17-1209 · DOI · Paper page
- Chen J (2016) Multi-modal learning: study on a large-scale micro-video data collection. In: Proceedings of the 24th ACM international conference on Multimedia, pp 1454–1458 · doi:10.1145/2964284.2971477
- Chen Y, Kalantidis Y, Li J, Yan S, Feng J (2018e) A$$^2$$-nets: Double attention networks. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31. Curran Associates, Inc., pp 352–361. http://papers.nips.cc/paper/7318-a2-nets-double-attention-networks.pdf
- Cheng Z, Bai F, Xu Y, Zheng G, Pu S, Zhou S (2017b) Focusing attention: towards accurate text recognition in natural images. In: Proceedings of the IEEE ICCV, pp 5076–5084 · DOI · Paper page
- Colombini EL, da Silva Simoes A, Ribeiro C (2014) An attentional model for intelligent robotics agents. PhD thesis, Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil
- Daniluk M, Rocktäschel T, Welbl J, Riedel S (2017) Frustratingly short attention spans in neural language modeling. In: 5th international conference on learning representations, ICLR 2017, Toulon, France, April 24–26, 2017, conference track proceedings, OpenReview.net. https://openreview.net/forum?id=ByIAPUcee
- Dehghani M, Gouws S, Vinyals O, Uszkoreit J, Kaiser L (2019) Universal transformers. In: 7th International conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=HyzdRiR9Y7
- Doughty H, Mayol-Cuevas W, Damen D (2019) The pros and cons: rank-aware temporal attention for skill determination in long videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7862–7871 · DOI
- Frintrop S, Rome E, Christensen HI (2010b) Computational visual attention systems and their cognitive foundations: a survey. ACM Trans Appl Percept (TAP) 7(1):1–39 · DOI · Paper page
- Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019b) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10705–10714 · DOI · Paper page
- Girdhar R, Carreira J, Doersch C, Zisserman A (2019) Video action transformer network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 244–253 · DOI
- Goyal A, Lamb A, Hoffmann J, Sodhani S, Levine S, Bengio Y, Schölkopf B (2019) Recurrent independent mechanisms. arXiv:190910893
- Gregor K, Danihelka I, Mnih A, Blundell C, Wierstra D (2014) Deep autoregressive networks. In: International conference on machine learning, PMLR, pp 1242–1250
- Han Z, Wang X, Vong C, Liu Y, Zwicker M, Chen CLP (2019) 3dviewgraph: learning global features for 3d shapes from A graph of unordered views with attention. In: Kraus S (ed) Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI 2019, Macao, China, August 10–16, 2019, ijcai.org, pp 758–765. https://doi.org/10.24963/ijcai.2019/107 · doi:10.24963/ijcai.2019/107
- Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2015) Session-based recommendations with recurrent neural networks. arXiv:151106939
- Hole KJ, Ahmad S (2021) A thousand brains: toward biologically constrained ai. SN Appl Sci 3(8):1–14
- Huang CZA, Vaswani A, Uszkoreit J, Simon I, Hawthorne C, Shazeer N, Dai AM, Hoffman MD, Dinculescu M, Eck D (2018a) Music transformer: generating music with long-term structure. In: International conference on learning representations
- Jaderberg M, Simonyan K, Zisserman A, kavukcuoglu k (2015) Spatial transformer networks. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28. Curran Associates, Inc., pp 2017–2025. http://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf
- Joulin A, Mikolov T (2015) Inferring algorithmic patterns with stack-augmented recurrent nets. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7–12, 2015, Montreal, pp 190–198
- Kim JH, Jun J, Zhang BT (2018a) Bilinear attention networks. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31. Curran Associates, Inc., pp 1564–1574. http://papers.nips.cc/paper/7429-bilinear-attention-networks.pdf
- Kiros R, Salakhutdinov R, Zemel R (2014) Multimodal neural language models. In: International conference on machine learning, PMLR, pp 595–603
- Kumar A, Irsoy O, Ondruska P, Iyyer M, Bradbury J, Gulrajani I, Zhong V, Paulus R, Socher R (2016) Ask me anything: Dynamic memory networks for natural language processing. In: International conference on machine learning, PMLR, pp 1378–1387
- Li J, Yang J, Hertzmann A, Zhang J, Xu T (2019d) Layoutgan: Generating graphic layouts with wireframe discriminators. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=HJxB5sRcFQ
- Liu J, Wang G, Hu P, Duan LY, Kot AC (2017d) Global context-aware attention lstm networks for 3d action recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR) · DOI · Paper page
- Liu S, Johns E, Davison AJ (2019c) End-to-end multi-task learning with attention. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1871–1880 · DOI · Paper page
- Mittal S, Lamb A, Goyal A, Voleti V, Shanahan M, Lajoie G, Mozer M, Bengio Y (2020) Learning to combine top-down and bottom-up signals in recurrent neural networks with attention over modules. In: International conference on machine learning, PMLR, pp 6972–6986
- (ed) Interspeech 2017, 18th annual conference of the international speech communication association, Stockholm, August 20-24, 2017, ISCA, pp 1263-1267. http://www.isca-speech.org/archive/Interspeech_2017/abstracts/0917.html
- Norouzian A, Mazoure B, Connolly D, Willett D (2019) Exploring attention mechanism for acoustic-based classification of speech utterances into system-directed and non-system-directed. In: ICASSP 2019–2019 IEEE international conference on acoustics. Speech and signal processing (ICASSP). IEEE, pp 7310–7314 · doi:10.1109/icassp.2019.8683565
- Oh J, Chockalingam V, Lee H, et al. (2016) Control of memory, active perception, and action in minecraft. In: International conference on machine learning, PMLR, pp 2790–2799
- Ouerhani N (2003) Visual attention: from bio-inspired modeling to real-time implementation. PhD thesis, Université de Neuchâtel
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
- Rekabdar B, Mousas C, Gupta B (2019) Generative adversarial network with policy gradient for text summarization. In: 2019 IEEE 13th international conference on semantic computing (ICSC). IEEE, pp 204–207 · DOI
- Ren M, Liao R, Fetaya E, Zemel R (2019) Incremental few-shot learning with attention attractor networks. In: Advances in neural information processing systems, pp 5275–5285
- Rezende DJ, Mohamed S, Danihelka I, Gregor K, Wierstra D (2016) One-shot generalization in deep generative models. arXiv:160305106
- Rotenstein A, Andreopoulos A, Fazl E, Jacob D, Robinson M, Shubina K, Zhu Y, Tsotsos J (2007) Towards the dream of intelligent, visually-guided wheelchairs. In: Proceedings of the 2nd international conference on technology and aging
- Scheier C, Egner S (1997) Visual attention in a mobile robot. In: ISIE’97 Proceeding of the IEEE international symposium on industrial electronics. vol 1. IEEE, pp SS48–SS52 · doi:10.1109/isie.1997.651734
- Seo PH, Lin Z, Cohen S, Shen X, Han B (2016) Hierarchical attention networks. arXiv:abs/160602393
- Sordoni A, Bachman P, Trischler A, Bengio Y (2016) Iterative alternating neural attention for machine reading. arXiv:160602245
- Tian T, Fang ZF (2019) Attention-based autoencoder topic model for short texts. Procedia Comput Sci 151:1134–1139 · doi:10.1016/j.procs.2019.04.161
- Tsai YH, Srivastava N, Goh H, Salakhutdinov R (2020) Capsules with inverted dot-product attention routing. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020, OpenReview.net. https://openreview.net/forum?id=HJe6uANtwH
- Van Den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. In: Neural information processing systems conference (NIPS 2013), neural information processing systems foundation (NIPS), vol 26
- Van der Velde F, de Kamps M et al (2004) Clam: closed-loop attention model for visual search. Neurocomputing 58:607–612 · doi:10.1016/j.neucom.2004.01.102
- Vinyals O, Fortunato M, Jaitly N (2015a) Pointer networks. In: Advances in neural information processing systems, pp 2692–2700
- Vinyals O, Kaiser Ł, Koo T, Petrov S, Sutskever I, Hinton G (2015b) Grammar as a foreign language. In: Advances in neural information processing systems, pp 2773–2781
- Walther D, Edgington DR, Koch C (2004) Detection and tracking of objects in underwater video. In: Proceedings of the IEEE CVPR, vol 1. IEEE, pp I–I · doi:10.1109/cvpr.2004.1315079
- Weston J, Chopra S, Bordes A (2014) Memory networks. arXiv:14103916
- Wu W, Chen Y, Xu J, Zhang Y (2018c) Attention-based convolutional neural networks for chinese relation extraction. In: Chinese computational linguistics and natural language processing based on naturally annotated big data. Springer, pp 147–158 · doi:10.1007/978-3-030-01716-3_13
- Yang F, Yang Z, Cohen WW (2017a) Differentiable learning of logical rules for knowledge base reasoning. In: Advances in neural information processing systems, pp 2319–2328
- Yu Z, Yu J, Xiang C, Fan J, Tao D (2018e) Beyond bilinear: generalized multimodal factorized high-order pooling for visual question answering. IEEE Trans Neural Netw Learn Syst 29(12):5947–5959 · DOI · Paper page
- Zhang B, Xiong D, Su J (2017a) Battrae: didimensional attention-based recursive autoencoders for learning bilingual phrase embeddings. In: Proceedings of the AAAI conference on artificial intelligence · doi:10.1609/aaai.v31i1.10969
- Zhang B, Xiong D, Su J, Zhang M (2018a) Learning better discourse representation for implicit discourse relation recognition via attention networks. Neurocomputing 275:1241–1249. https://doi.org/10.1016/j.neucom.2017.09.074 · DOI
- Zhang Y, Li K, Li K, Zhong B, Fu Y (2019j) Residual non-local attention networks for image restoration. In: 7th International conference on learning representations, ICLR 2019, New Orleans, May 6–9, 2019, OpenReview.net. https://openreview.net/forum?id=HkeGhoA5FX
- Zhang Y, Zhou C, Chang F, Kot AC (2019o) Multi-resolution attention convolutional neural network for crowd counting. Neurocomputing 329:144–152 · doi:10.1016/j.neucom.2018.10.058
Cited by
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
Feng, Y., Chen, J., Xie, J., Zhang, T., Lv, H., & Pan, T. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107646 · Cited by 262
Lv, H., Chen, J., Pan, T., Zhang, T., Feng, Y., & Liu, S. (2022) · Measurement · doi:10.1016/j.measurement.2022.111594 · Cited by 261
Metadata
- Authors
- de Santana Correia, A., & Colombini, E. L.
- Venue
- Artificial Intelligence Review
- Published
- 2022-03-03
- Volume
- 55
- Issue
- 8
- Pages
- 6037-6124
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W3147852756
- Cited by
- 298
- References
- 569
- FWCI
- 20.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.