A unified architecture for natural language processing: deep neural networks with multitask learning
Abstract
We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in state-of-the-art-performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Neural networks have made advancements in computer vision, speech recognition, and natural language processing.
“these networks have made advancements in diverse fields such as computer vision [21], speech recognition [7], and natural language processing [6].”
From DeepWalk: online learning of social representations (2014) · cites this paper as [6]
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451 · see citing works
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,698 · see citing works
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643 · see citing works
The Proposition Bank: An Annotated Corpus of Semantic Roles
Palmer, M. S., Gildea, D., & Kingsbury, P. (2005) · Computational Linguistics · doi:10.1162/0891201053630264 · Cited by 2,317 · see citing works
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,319 · see citing works
Bridle, J. S. (1990) · Springer eBooks · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,278 · see citing works
Caruana, R. (1998) · Cited by 1,110 · see citing works
Effective Self-Training for Parsing
Mcclosky, D., Charniak, E., & Johnson, M. (2006) · North American Chapter of the Association for Computational Linguistics · doi:10.3115/1220835.1220855 · Cited by 702 · see citing works
Shallow Semantic Parsing using Support Vector Machines.
Pradhan, S., Ward, W., Hacıoğlu, K., et al. (2004) · Cited by 368 · see citing works
Sutton, C. A., McCallum, A., & Rohanimanesh, K. (2007) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.1145/1015330.1015422 · Cited by 288 · see citing works
The necessity of parsing for predicate argument recognition
Gildea, D., & Palmer, M. S. (2001) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/1073083.1073124 · Cited by 218 · see citing works
A novel use of statistical parsing to extract information from text
Miller, S. L., Fox, H. J., Ramshaw, L., et al. (2000) · Cited by 201 · see citing works
Semi-Supervised Learning (Adaptive Computation and Machine Learning)
Chapelle, O., Schölkopf, B., & Zien, A. (2006) · The MIT Press eBooks · Cited by 185 · see citing works
Connectionist language modeling for large vocabulary continuous speech recognition
Schwenk, H., & Gauvain, J. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5743830 · Cited by 117 · see citing works
Transductive learning for statistical machine translation
Ueffing, N., Haffari, G., & Sarkar, A. (2007) · NPARC · Cited by 117 · see citing works
Fast Semantic Extraction Using a Novel Neural Network Architecture
Collobert, R., & Weston, J. (2007) · Cited by 63 · see citing works
Self-citationJoint parsing and semantic role labeling
Sutton, C. A., & McCallum, A. (2005) · doi:10.3115/1706543.1706587 · Cited by 54 · see citing works
Composition of conditional random fields for transfer learning
Sutton, C. A., & McCallum, A. (2005) · doi:10.3115/1220575.1220669 · Cited by 54 · see citing works
Using Corpus Statistics on Entities to Improve Semi-supervised Relation Extraction from the Web
Rosenfeld, B. L., & Feldman, R. (2007) · Cited by 51 · see citing works
A discriminative language model with pseudo-negative samples
Okanohara, D., & Tsujii, J. (2007) · Cited by 49 · see citing works
Robust Parsing of the Proposition Bank
Musillo, G., & Merlo, P. (2006) · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · see citing works
Self-citationDeep learning in medical image analysis
Shen, D., Wu, G., & Suk, H.-I. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071516-044442 · Cited by 4,968 · see citing works
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545 · see citing works
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239 · see citing works
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · see citing works
Traffic Flow Prediction With Big Data: A Deep Learning Approach
Lv, Y., Duan, Y., Kang, W., Li, Z., & Wang, F.-Y. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2345663 · Cited by 3,000 · see citing works
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921 · see citing works
Deep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · see citing works
Natural language processing: state of the art, current trends and challenges
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13428-4 · Cited by 1,823 · see citing works
Reading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
Self-citationERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482 · see citing works
Quantum-chemical insights from deep tensor neural networks
Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R., & Tkatchenko, A. (2017) · Nature Communications · doi:10.1038/ncomms13890 · Cited by 1,445 · see citing works
How to Fine-Tune BERT for Text Classification?
Sun, C., Qiu, X., Xu, Y., & Huang, X. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32381-3_16 · Cited by 1,534 · see citing works
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387 · see citing works
Norouzzadeh, M. S., Nguyen, A., Kosmala, M., Swanson, A., Palmer, M. S., Packer, C., & Clune, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1719367115 · Cited by 1,230 · see citing works
Majorana zero modes and topological quantum computation
Sarma, S. D., Freedman, M., & Nayak, C. (2015) · npj Quantum Information · doi:10.1038/npjqi.2015.1 · Cited by 1,180 · see citing works
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
Contrastive Representation Learning: A Framework and Review
Le-Khac, P. H., Healy, G., & Smeaton, A. F. (2020) · IEEE Access · doi:10.1109/access.2020.3031549 · Cited by 862 · see citing works
A survey on deep learning based bearing fault diagnosis
Hoang, D.-T., & Kang, H.-J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.06.078 · Cited by 866 · see citing works
Neural Network Methods for Natural Language Processing
Goldberg, Y. (2017) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02165-7 · Cited by 821 · see citing works
Fadlullah, Z. M., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2707140 · Cited by 810 · see citing works
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works
Bordes, A., Glorot, X., Weston, J., & Bengio, Y. (2014) · Machine Learning · doi:10.1007/s10994-013-5363-6 · Cited by 675 · see citing works
Self-citationKnowledge Enhanced Contextual Word Representations
Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., & Smith, N. A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1005 · Cited by 661 · see citing works
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., & Gurevych, I. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.39 · Cited by 636 · see citing works
“…trained simultaneously with the aim of learning a shared representation that will enable the model to generalize better on each task (Caruana, 1997; Collobert and Weston, 2008; Nam et al., 2014; Liu et al., 2016, 2017; Zhang and Yang, 2017; Ruder, 2017; Ruder et al., 2019; Sanh et al., 2019;…”
BackgroundImproving Vector Space Word Representations Using Multilingual Correlation
Faruqui, M., & Dyer, C. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1049 · Cited by 620 · see citing works
Cai, Y., Guan, K., Peng, J., Wang, S., Seifert, C., Wardlow, B., & Li, Z. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.02.045 · Cited by 540 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
Minerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446 · see citing works
Multi-layer Representation Learning for Medical Concepts
Choi, E., Bahadori, M. T., Searles, E., Coffey, C., Thompson, M., Bost, J., Tejedor-Sojo, J., & Sun, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939823 · Cited by 469 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
“…a form of unsupervised learning in which context within the text is used to predict missing words, have been shown to materialize representations of word meaning that can generalize across problems (Collobert & Weston, 2008; Dai & Le, 2015; Peters et al., 2018; Devlin et al., 2018).”
BackgroundBig Data: Deep Learning for financial sentiment analysis
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0111-6 · Cited by 390 · see citing works
Opinion Mining with Deep Recurrent Neural Networks
Irsoy, O., & Cardie, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1080 · Cited by 384 · see citing works
“…incorporate information from preceding as well as following tokens (Schuster and Paliwal, 1997) while recent advances in word embedding induction (Collobert and Weston, 2008; Mnih and Hinton, 2007; Mikolov et al., 2013; Turian et al., 2010) have enabled more effective training of RNNs by…”
BackgroundInfluentialDistributed deep learning networks among institutions for medical imaging
Chang, K., Balachandar, N., Lam, C., Yi, D., Brown, J., Beers, A., Rosen, B., Rubin, D. L., & Kalpathy-Cramer, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy017 · Cited by 340 · see citing works
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · see citing works
Bilingual Word Representations with Monolingual Quality in Mind
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing · doi:10.3115/v1/w15-1521 · Cited by 315 · see citing works
Machine learning-based imaging system for surface defect inspection
Park, J.-K., Kwon, B.-K., Park, J.-H., & Kang, D.-J. (2016) · International Journal of Precision Engineering and Manufacturing-Green Technology · doi:10.1007/s40684-016-0039-x · Cited by 311 · see citing works
Learning deep physiological models of affect
Martinez, H. P., Bengio, Y., & Yannakakis, G. N. (2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2247823 · Cited by 287 · see citing works
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300 · see citing works
Combining structured and unstructured data for predictive models: a deep learning approach
Zhang, D., Yin, C., Zeng, J., Yuan, X., & Zhang, P. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01297-6 · Cited by 269 · see citing works
A patch-based convolutional neural network for remote sensing image classification
Sharma, A., Liu, X., Yang, X., & Shi, D. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.07.017 · Cited by 260 · see citing works
Rationale-Augmented Convolutional Neural Networks for Text Classification
Zhang, Y., Marshall, I., & Wallace, B. C. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1076 · Cited by 249 · see citing works
Demystifying Multitask Deep Neural Networks for Quantitative Structure–Activity Relationships
Xu, Y., Ma, J., Liaw, A., Sheridan, R. P., & Svetnik, V. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00087 · Cited by 238 · see citing works
A neural network multi-task learning approach to biomedical named entity recognition
Crichton, G., Pyysalo, S., Chiu, B., & Korhonen, A. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1776-8 · Cited by 229 · see citing works
Papadimitroulas, P., Brocki, L., Christopher Chung, N., Marchadour, W., Vermet, F., Gaubert, L., Eleftheriadis, V., Plachouris, D., Visvikis, D., Kagadis, G. C., et al. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.03.009 · Cited by 226 · see citing works
A survey of word embeddings based on deep learning
Wang, S., Zhou, W., & Jiang, C. (2020) · Computing · doi:10.1007/s00607-019-00768-7 · Cited by 193 · see citing works
Naseem, U., Razzak, I., Khan, S. K., & Prasad, M. (2021) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3434237 · Cited by 194 · see citing works
CoLAKE: Contextualized Language and Knowledge Embedding
Sun, T., Shao, Y., Qiu, X., Guo, Q., Hu, Y., Huang, X., & Zhang, Z. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.327 · Cited by 185 · see citing works
Context-aware convolutional neural network for grading of colorectal cancer histology images
Shaban, M., Awan, R., Fraz, M. M., Azam, A., Tsang, Y.-W., Snead, D., & Rajpoot, N. M. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2971006 · Cited by 179 · see citing works
. Hyper-class augmented and regularized deep learning for fine-grained image classification
Xie, S., Yang, T., Wang, X., & Lin, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298880 · Cited by 169 · see citing works
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · see citing works
Limitations of transformers on clinical text classification
Gao, S., Alawad, M., Young, M. T., Gounley, J., Schaefferkoetter, N., Yoon, H. J., Wu, X.-C., Durbin, E. B., Doherty, J., Stroup, A., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062322 · Cited by 157 · see citing works
Moral foundations twitter corpus: a collection of 35k tweets annotated for moral sentiment
Hoover, J., Portillo-Wightman, G., Yeh, L., Havaldar, S., Davani, A. M., Lin, Y., Kennedy, B., Atari, M., Kamel, Z., Mendlen, M., et al. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619876629 · Cited by 177 · see citing works
Self-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155 · see citing works
Using natural language processing to analyse text data in behavioural science
Feuerriegel, S., Maarouf, A., Bär, D., Geissler, D., Schweisthal, J., Pröllochs, N., Robertson, C. E., Rathje, S., Hartmann, J., Mohammad, S. M., et al. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-024-00392-z · Cited by 140 · see citing works
A Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136 · see citing works
“To highlight a few notable trends, we see audience is under-represented compared to the other task dimensions, suggesting future work may want to more explicitly consider who is the audience of a piece of writing. overall, as well as relative to other technology dimensions, suggesting that there…”
Altszyler, E., Ribeiro, S., Sigman, M., & Fernández Slezak, D. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.09.004 · Cited by 128 · see citing works
Learning for Personalized Medicine: A Comprehensive Review From a Deep Learning Perspective
Zhang, S., Bamakan, S. M. H., Qu, Q., & Li, S. (2019) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2864254 · Cited by 125 · see citing works
Automated Landslides Detection for Mountain Cities Using Multi-Temporal Remote Sensing Imagery
Chen, Z., Zhang, Y., Ouyang, C., Zhang, F., & Ma, J. (2018) · Sensors · doi:10.3390/s18030821 · Cited by 111 · see citing works
Knowledge-based approaches in software documentation: A systematic literature review
Ding, W., Liang, P., Tang, A., & van Vliet, H. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.01.008 · Cited by 107 · see citing works
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays
Shi, Y., Nguyen, L., Oh, S., Liu, X., Koushan, F., Jameson, J. R., & Kuzum, D. (2018) · Nature Communications · doi:10.1038/s41467-018-07682-0 · Cited by 106 · see citing works
From Word Embeddings to Pre-Trained Language Models: A State-of-the-Art Walkthrough
Mars, M. (2022) · Applied Sciences · doi:10.3390/app12178805 · Cited by 91 · see citing works
Word Representations in Vector Space and their Applications for Arabic
Zahran, M. A., Magooda, A., Mahgoub, A. Y., Raafat, H., Rashwan, M., & Atyia, A. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-18111-0_32 · Cited by 90 · see citing works
McInnes, G., Sharo, A. G., Koleske, M. L., Brown, J. E. H., Norstad, M., Adhikari, A. N., Wang, S., Brenner, S. E., Halpern, J., Koenig, B. A., et al. (2021) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2021.03.003 · Cited by 86 · see citing works
“47 67 Machine learning is a type of artificial intelligence in which algorithms are taught, or ‘‘trained,’’ to make predictions based on existing data.”
BackgroundWiebe, N., Kapoor, A., & Svore, K. M. (2016) · Quantum Information and Computation · doi:10.26421/qic16.7-8-1 · Cited by 85 · see citing works
DAO to HANOI via DeSci: AI Paradigm Shifts from AlphaGo to ChatGPT
Miao, Q., Zheng, W., Lv, Y., Huang, M., Ding, W., & Wang, F.-Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123561 · Cited by 78 · see citing works
Lu, N., Wu, G., Zhang, Z., Zheng, Y., Ren, Y., & Choo, K. R. (2020) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.5627 · Cited by 76 · see citing works
Georgiev, P., Bhattacharya, S., Lane, N. D., & Mascolo, C. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3131895 · Cited by 67 · see citing works
Multi-task Learning for Multilingual Neural Machine Translation
Wang, Y., Zhai, C., & Hassan, H. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.75 · Cited by 59 · see citing works
Recognizing Human Activities from Raw Accelerometer Data Using Deep Neural Networks
Zhang, L., Wu, X., & Luo, D. (2015) · 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2015.48 · Cited by 59 · see citing works
Arabic Cyberbullying Detection: Using Deep Learning
Haidar, B., Chamoun, M., & Serhrouchni, A. (2018) · 2018 7th International Conference on Computer and Communication Engineering (ICCCE) · doi:10.1109/iccce.2018.8539303 · Cited by 58 · see citing works
A systematic deep learning model selection for p300-based brain-computer interfaces
Abibullaev, B., & Zollanvari, A. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3051136 · Cited by 53 · see citing works
Automated Emerging Cyber Threat Identification and Profiling Based on Natural Language Processing
Marinho, R., & Holanda, R. (2023) · IEEE Access · doi:10.1109/access.2023.3260020 · Cited by 50 · see citing works
Exploring Numeracy in Word Embeddings
Naik, A., Ravichander, A., Rose, C., & Hovy, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1329 · Cited by 49 · see citing works
Teijema, J. J., Hofstee, L., Brouwer, M., de Bruin, J., Ferdinands, G., de Boer, J., Vizan, P., van den Brand, S., Bockting, C., van de Schoot, R., et al. (2023) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2023.1178181 · Cited by 36 · see citing works
Real-Time Activity Recognition on Smartphones Using Deep Neural Networks
Zhang, L., Wu, X., & Luo, D. (2015) · 2015 IEEE 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom) · doi:10.1109/uic-atc-scalcom-cbdcom-iop.2015.224 · Cited by 35 · see citing works
Deep learning for online harassment detection in tweets
Marwa, T., Salima, O., & Souham, M. (2018) · 2018 3rd International Conference on Pattern Analysis and Intelligent Systems (PAIS) · doi:10.1109/pais.2018.8598530 · Cited by 35 · see citing works
A General Framework for Information Extraction using Dynamic Span Graphs
Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., & Hajishirzi, H. (2019) · arXiv · doi:10.48550/arxiv.1904.03296 · Cited by 34 · see citing works
“Unlike previous BIO-based entity recognition systems (Collobert and Weston, 2008; Lample et al., 2016; Ma and Hovy, 2016) that assign a text span to at most one entity, our framework enumerates and represents all possible spans to recognize arbitrarily overlapping entities.”
BackgroundA Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y., & Cheng, X. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591631 · Cited by 32 · see citing works
A Hybrid Deep Learning System of CNN and LRCN to Detect Cyberbullying from SNS Comments
Bu, S.-J., & Cho, S.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-92639-1_47 · Cited by 30 · see citing works
Kong, W., Khadanga, S., Li, C., Gupta, S. K., Zhang, M., Xu, W., & Bendersky, M. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539137 · Cited by 27 · see citing works
Kazi, S., Khoja, S., & Daud, A. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100756 · Cited by 9 · see citing works
Wiebe, N., Kapoor, A., & Svore, K. M. (2014) · arXiv · doi:10.48550/arxiv.1412.3489 · Cited by 5 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
- Authors
- Collobert, R., & Weston, J.
- Venue
- Proceedings of the 25th international conference on Machine learning - ICML '08
- Publisher
- ACM Press
- Published
- 2008
- Pages
- 160-167
- OpenAlex
- W2117130368
- Cited by
- 5,058
- References
- 22
- Citations by year
- 2026: 40 · 2025: 117 · 2024: 184 · 2023: 251 · 2022: 307 · 2021: 490 · 2020: 609 · 2019: 694 · 2018: 686 · 2017: 485 · 2016: 385 · 2015: 354 · 2014: 191 · 2013: 91 · 2012: 61
- FWCI
- 27.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- unified architecture, convolutional neural networks, multi-task learning, parameter sharing, semi-supervised learning, part-of-speech tagging, chunking, named entity recognition, semantic role labeling, word similarity, language models, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review