Natural Language Processing (almost) from Scratch

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

A fast learning algorithm for deep belief nets

Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336

Multitask Learning

Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636

Curriculum learning

Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 2×

Greedy layer-wise training of deep networks

Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841

Untitled work

(1992) · doi:10.5555/176313.176316 · Cited by 2,892

Phoneme recognition using time-delay neural networks

Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643

Transparency and accountability in AI decision support: Explaining and visualizing convolutional neural networks for text information

Kim, B., Park, J., & Suh, J. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113302 · Cited by 182

Coarse-to-Fine Question Answering for Long Documents

Choi, E., Hewlett, D., Uszkoreit, J., Polosukhin, I., Lacoste, A., & Berant, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1020 · Cited by 177

End-to-end acoustic modeling using convolutional neural networks for HMM-based automatic speech recognition

Palaz, D., Magimai-Doss, M., & Collobert, R. (2019) · Speech Communication · doi:10.1016/j.specom.2019.01.004 · Cited by 168

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049

Demographic Factors Improve Classification Performance

Hovy, D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1073 · Cited by 148

A systematic review of digital transformation technologies in museum exhibition

Li, J., Zheng, X., Watanabe, I., & Ochiai, Y. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108407 · Cited by 136

Knowledge-primed neural networks enable biologically interpretable deep learning on single-cell sequencing data

Fortelny, N., & Bock, C. (2020) · Genome Biology · doi:10.1186/s13059-020-02100-5 · Cited by 135

Recent advances in biomedical literature mining

Zhao, S., Su, C., Lu, Z., & Wang, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa057 · Cited by 134

SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks

Faraone, J., Fraser, N., Blott, M., & Leong, P. H. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00452 · Cited by 134

Cyberthreat Detection from Twitter using Deep Neural Networks

Dionisio, N., Alves, F., Ferreira, P. M., & Bessani, A. (2019) · 2019 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852475 · Cited by 126

Deep Learning for Detecting Robotic Grasps

Lenz, I., Lee, H., & Saxena, A. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.012 · Cited by 126

Applications of AI in classical software engineering

Barenkamp, M., Rebstadt, J., & Thomas, O. (2020) · AI Perspectives · doi:10.1186/s42467-020-00005-4 · Cited by 123

Audio deepfakes: A survey

Khanjani, Z., Watson, G., & Janeja, V. P. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1001063 · Cited by 121

Knowledge Graph Representation with Jointly Structural and Textual Encoding

Xu, J., Qiu, X., Chen, K., & Huang, X. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/183 · Cited by 121

EARL: Joint Entity and Relation Linking for Question Answering over Knowledge Graphs

Dubey, M., Banerjee, D., Chaudhuri, D., & Lehmann, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00671-6_7 · Cited by 111

SpanNER: Named Entity Re-/Recognition as Span Prediction

Fu, J., Huang, X., & Liu, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.558 · Cited by 112

A Comparison of Transfer Learning Performance Versus Health Experts in Disease Diagnosis from Medical Imaging

Malik, H., Farooq, M. S., Khelifi, A., Abid, A., Nasir Qureshi, J., & Hussain, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004766 · Cited by 104

Identifying Where to Focus in Reading Comprehension for Neural Question Generation

Du, X., & Cardie, C. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1219 · Cited by 100

Technical and clinical overview of deep learning in radiology

Ueda, D., Shimazaki, A., & Miki, Y. (2019) · Japanese Journal of Radiology · doi:10.1007/s11604-018-0795-3 · Cited by 100

Evaluating the carbon footprint of NLP methods: a survey and analysis of existing tools

Bannour, N., Ghannay, S., Névéol, A., & Ligozat, A.-L. (2021) · Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing · doi:10.18653/v1/2021.sustainlp-1.2 · Cited by 98

Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models

McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1910416117 · Cited by 91

Open Problems in Cooperative AI

Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93

Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance

Liao, C.-H., & Wu, J.-Y. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104599 · Cited by 89

Analyzing Multimodal Sentiment Via Acoustic- and Visual-LSTM With Channel-Aware Temporal Convolution Network

Mai, S., Xing, S., & Hu, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3068598 · Cited by 89

Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution

Ruder, S., Ghaffari, P., & Breslin, J. G. (2016) · arXiv · doi:10.48550/arxiv.1609.06686 · Cited by 86

Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training

Xu, P., Madotto, A., Wu, C.-S., Park, J. H., & Fung, P. (2018) · Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w18-6243 · Cited by 83

Importance of Search and Evaluation Strategies in Neural Dialogue Modeling

Kulikov, I., Miller, A., Cho, K., & Weston, J. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8609 · Cited by 83

A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks

Sadr, H., Pedram, M. M., & Teshnehlab, M. (2019) · Neural Processing Letters · doi:10.1007/s11063-019-10049-1 · Cited by 82

A general path-based representation for predicting program properties

Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2018) · Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3192366.3192412 · Cited by 79

Detecting predatory conversations in social media by deep convolutional neural networks

Ebrahimi, M., Suen, C. Y., & Ormandjieva, O. (2016) · Digital Investigation · doi:10.1016/j.diin.2016.07.001 · Cited by 78

Real or Fake? Learning to Discriminate Machine from Human Generated Text

Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., & Szlam, A. (2019) · arXiv · doi:10.48550/arxiv.1906.03351 · Cited by 77

Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting

Yang, X., Bian, J., Fang, R., Bjarnadottir, R. I., Hogan, W. R., & Wu, Y. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz144 · Cited by 74

Improving Deep Learning for HAR with Shallow LSTMs

Bock, M., Hölzemann, A., Moeller, M., & Van Laerhoven, K. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480419 · Cited by 74

Cyberbullying detection in social media text based on character-level convolutional neural network with shortcuts

Lu, N., Wu, G., Zhang, Z., Zheng, Y., Ren, Y., & Choo, K. R. (2020) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.5627 · Cited by 76

Topic Modeling and Characterization of Hate Speech against Immigrants on Twitter around the Emergence of a Far-Right Party in Spain

Calderón, C. A., de la Vega, G., & Herrero, D. B. (2020) · Social Sciences · doi:10.3390/socsci9110188 · Cited by 73

Improving reference prioritisation with PICO recognition.

Brockmeier, A. J., Ju, M., Przybyła, P., & Ananiadou, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0992-8 · Cited by 62

Multi-view Models for Political Ideology Detection of News Articles

Kulkarni, V., Ye, J., Skiena, S., & Wang, W. Y. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1388 · Cited by 71

Artificial intelligence applications in fake review detection: Bibliometric analysis and future avenues for research

Ben Jabeur, S., Ballouk, H., Ben Arfi, W., & Sahut, J.-M. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2022.113631 · Cited by 70

Evaluation of a Machine-Learning Algorithm for Treatment Planning in Prostate Low-Dose-Rate Brachytherapy

Nicolae, A., Morton, G., Chung, H., Loblaw, A., Jain, S., Mitchell, D., Lu, L., Helou, J., Al-Hanaqta, M., Heath, E., et al. (2017) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2016.11.036 · Cited by 69

Finding the Intersection of Neuroplasticity, Stroke Recovery, and Learning: Scope and Contributions to Stroke Rehabilitation

Carey, L., Walsh, A., Adikari, A., Goodin, P., Alahakoon, D., De Silva, D., Ong, K.-L., Nilsson, M., & Boyd, L. (2019) · Neural Plasticity · doi:10.1155/2019/5232374 · Cited by 68

The Impact of Personalisation on Human-Robot Interaction in Learning Scenarios

Churamani, N., Anton, P., Brügger, M., Fließwasser, E., Hummel, T., Mayer, J., Mustafa, W., Ng, H. G., Nguyen, T. L. C., Nguyen, Q., et al. (2017) · Proceedings of the 5th International Conference on Human Agent Interaction · doi:10.1145/3125739.3125756 · Cited by 67

Extracting Symptoms and their Status from Clinical Conversations

Du, N., Chen, K., Kannan, A., Tran, L., Chen, Y., & Shafran, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1087 · Cited by 66

Peer review analyze: A novel benchmark resource for computational analysis of peer reviews

Ghosal, T., Kumar, S., Bharti, P. K., & Ekbal, A. (2022) · PLOS ONE · doi:10.1371/journal.pone.0259238 · Cited by 66

MADEx: A System for Detecting Medications, Adverse Drug Events, and Their Relations from Clinical Notes

Yang, X., Bian, J., Gong, Y., Hogan, W. R., & Wu, Y. (2019) · Drug Safety · doi:10.1007/s40264-018-0761-0 · Cited by 66

Combining FastText and Glove Word Embedding for Offensive and Hate speech Text Detection

Badri, N., Kboubi, F., & Chaibi, A. H. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.09.132 · Cited by 65

How to mitigate the incident? An effective troubleshooting guide recommendation technique for online service systems

Jiang, J., Lu, W., Chen, J., Lin, Q., Zhao, P., Kang, Y., Zhang, H., Xiong, Y., Gao, F., Xu, Z., et al. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417054 · Cited by 56

MDLFace: Memorability augmented deep learning for video face recognition

Goswami, G., Bhardwaj, R., Singh, R., & Vatsa, M. (2014) · IEEE International Joint Conference on Biometrics · doi:10.1109/btas.2014.6996299 · Cited by 53

Explainable Sentiment Analysis with Applications in Medicine

Zucco, C., Liang, H., Fatta, G. D., & Cannataro, M. (2018) · 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621359 · Cited by 59

Bit Efficient Quantization for Deep Neural Networks

Nayak, P., Zhang, D., & Chai, S. (2019) · 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) · doi:10.1109/emc2-nips53020.2019.00020 · Cited by 51

Off the Beaten Path: Let's Replace Term-Based Retrieval with k-NN Search

Boytsov, L., Novak, D., Malkov, Y., & Nyberg, E. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983815 · Cited by 50

Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness

Lyu, L., He, X., & Li, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.213 · Cited by 51

Scaling up data curation using deep learning: An application to literature triage in genomic variation resources

Lee, K., Famiglietti, M. L., McMahon, A., Wei, C.-H., MacArthur, J. A. L., Poux, S., Breuza, L., Bridge, A., Cunningham, F., Xenarios, I., et al. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006390 · Cited by 47

EMR-based medical knowledge representation and inference via Markov random fields and distributed representation learning

Zhao, C., Jiang, J., Guan, Y., Guo, X., & He, B. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.03.005 · Cited by 41

The Challenges and Opportunities of Artificial Intelligence for Trustworthy Robots and Autonomous Systems

He, H., Gray, J., Cangelosi, A., Meng, Q., McGinnity, T. M., & Mehnen, J. (2020) · 2020 3rd International Conference on Intelligent Robotic and Control Engineering (IRCE) · doi:10.1109/irce50905.2020.9199244 · Cited by 43

Advancing PICO element detection in biomedical text via deep neural networks.

Jin, D., & Szolovits, P. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa256 · Cited by 41

A Semantic Role-based Approach to Open-Domain Automatic Question Generation

Flor, M., & Riordan, B. (2018) · Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0530 · Cited by 39

A General Framework for Information Extraction using Dynamic Span Graphs

Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., & Hajishirzi, H. (2019) · arXiv · doi:10.48550/arxiv.1904.03296 · Cited by 34

GatorTron: A Large Language Model for Clinical Natural Language Processing

Yang, X., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Flores, M. G., Zhang, Y., Magoc, T., et al. (2022) · doi:10.1101/2022.02.27.22271257 · Cited by 33

On the Decision Boundary of Deep Neural Networks

Li, Y., Ding, L., & Gao, X. (2018) · arXiv · doi:10.48550/arxiv.1808.05385 · Cited by 31

Asking questions like educational experts: automatically generating question-answer pairs on real-world examination data

Qu, F., Jia, X., & Wu, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.202 · Cited by 30

A Multi-task Approach to Learning Multilingual Representations

Singla, K., Can, D., & Narayanan, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2035 · Cited by 22

Insider Threats Detection Using CNN-LSTM Model

Saaudi, A., Al-Ibadi, Z., Tong, Y., & Farkas, C. (2018) · 2018 International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci46756.2018.00025 · Cited by 22

SemGloVe: Semantic Co-Occurrences for GloVe From BERT

Gan, L., Teng, Z., Zhang, Y., Zhu, L., Wu, F., & Yang, Y. (2022) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2022.3197316 · Cited by 22

Classifying temporal relations in clinical data: A hybrid, knowledge-rich approach

D’Souza, J., & Ng, V. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.003 · Cited by 18

It’s All in the Name: A Character-Based Approach to Infer Religion

Chaturvedi, R., & Chaturvedi, S. (2024) · Political Analysis · doi:10.1017/pan.2023.6 · Cited by 18

A Neural Candidate-Selector Architecture for Automatic Structured Clinical Text Annotation.

Singh, G., Marshall, I. J., Thomas, J., Shawe-Taylor, J., & Wallace, B. C. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132989 · Cited by 11

Joint extraction of entity and relation based on fine-tuning BERT for long biomedical literatures

Gao, T., Zhai, X., Yang, C., Lv, L., & Wang, H. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae194 · Cited by 2

A Scoping Review of Adopted Information Extraction Methods for RCTs.

Aletaha, A., Nemati-Anaraki, L., Keshtkar, A., Sedghi, S., Keramatfar, A., & Korolyova, A. (2023) · Medical Journal of The Islamic Republic of Iran · doi:10.47176/mjiri.37.95 · Cited by 2

Adversarial Unsupervised Representation Learning for Activity Time-Series

Aggarwal, K., Joty, S., Fernandez-Luque, L., & Srivastava, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301834

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks

Reimers, N., & Gurevych, I. (2017) · arXiv · doi:10.48550/arxiv.1707.06799

Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

Lee, C., Cho, K., & Kang, W. (2019) · arXiv · doi:10.48550/arxiv.1909.11299