Deep multi-task learning with low level tasks supervised at lower layers
Open accessAbstract
In all previous work on deep multi-task learning we are aware of, all task supervisions are on the same (outermost) layer. We present a multi-task learning architecture with deep bi-directional RNNs, where different tasks supervision can happen at different layers. We present experiments in syntactic chunking and CCG supertagging, coupled with the additional task of POS-tagging. We show that it is consistently better to have POS supervision at the innermost rather than the outermost layer. We argue that this is because “lowlevel” tasks are better kept at the lower layers, enabling the higher-level tasks to make use of the shared representation of the lower-level tasks. Finally, we also show how this architecture can be used for domain adaptation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For POS tagging, accuracies using the first biLM layer are higher than the top layer.
“However, unlike WSD, accuracies using the first biLM layer are higher than the top layer, consistent with results from deep biL-STMs in multi-task training (Søgaard and Goldberg, 2016;Hashimoto et al., 2017) and MT (Belinkov et al., 2017).”
From Deep Contextualized Word Representations (2018) · cites this paper as Søgaard and Goldberg, 2016
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,920 · see citing works
A Primer on Neural Network Models for Natural Language Processing
Goldberg, Y. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4992 · Cited by 1,259 · see citing works
Self-citationIntroduction to the CoNLL-2000 shared task
Sang, E. T. K., & Buchholz, S. (2000) · Conference on Computational Natural Language Learning · doi:10.3115/1117601.1117631 · Cited by 704 · see citing works
The CoNLL 2007 Shared Task on Dependency Parsing
Nivre, J., Hall, J., Kübler, S., McDonald, R., Nilsson, J., Riedel, S., & Yüret, D. (2007) · Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) · Cited by 667 · see citing works
Multi-task Sequence to Sequence LearningPreprint
Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, Ł. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06114 · Cited by 606 · see citing works
Opinion Mining with Deep Recurrent Neural Networks
Irsoy, O., & Cardie, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1080 · Cited by 384 · cited 3× · see citing works
“We experiment with a multi-task learning (MTL) architecture based on deep bi-directional recurrent neural networks (bi-RNNs) (Schuster and Paliwal, 1997; Irsoy and Cardie, 2014).”
MethodRotating your face using multi-task deep neural network
Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., & Kim, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298667 · Cited by 279 · see citing works
Improving multiview face detection with multi-task deep convolutional neural networks
Zhang, C., & Zhang, Z. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6835990 · Cited by 210 · see citing works
Voting Between Multiple Data Representations for Text Chunking
Shen, H., & Sarkar, A. (2005) · Lecture notes in computer science · doi:10.1007/11424918_40 · Cited by 57 · see citing works
CCG Supertagging with a Recurrent Neural Network
Xu, W., Auli, M., & Clark, S. C. (2015) · doi:10.3115/v1/p15-2041 · Cited by 53 · see citing works
Natural Language Understanding with Distributed RepresentationPreprint
Cho, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07916 · Cited by 40 · see citing works
Zhu, X., Gibson, B. R., & Rogers, T. T. (2009) · Cited by 19 · see citing works
Flagged first, then by citations
1 of the 16 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,195 · see citing works
A Primer on Neural Network Models for Natural Language Processing
Goldberg, Y. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4992 · Cited by 1,259 · see citing works
Self-citationContextual String Embeddings for Sequence Labeling
Akbik, A., Blythe, D. A. J., & Vollgraf, R. (2018) · International Conference on Computational Linguistics · Cited by 1,086 · see citing works
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987 · see citing works
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask NetworksPreprint
Chen, Z., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02257 · Cited by 838 · see citing works
Semi-supervised sequence tagging with bidirectional language models
Peters, M., Ammar, W., Bhagavatula, C., & Power, R. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1161 · Cited by 655 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018) · arXiv · doi:10.48550/arxiv.1804.07461 · Cited by 562 · see citing works
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Hashimoto, K., xiong, C., Tsuruoka, Y., & Socher, R. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1206 · Cited by 535 · see citing works
Reimers, N., & Gurevych, I. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1035 · Cited by 463 · see citing works
Multi-Task Learning with Deep Neural Networks: A Survey
Crawshaw, M. (2020) · arXiv · doi:10.48550/arxiv.2009.09796 · Cited by 428 · see citing works
Linguistically-Informed Self-Attention for Semantic Role Labeling
Strubell, E., Verga, P. W., Andor, D., Weiss, D., & McCallum, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1548 · Cited by 414 · see citing works
Dynamic Task Prioritization for Multitask Learning
Guo, M., Haque, A., Huang, D.-A., Yeung, S. Y., & Fei-Fei, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_17 · Cited by 377 · see citing works
Semi-Supervised Sequence Modeling with Cross-View Training
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. V. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1217 · Cited by 366 · see citing works
DyNet: The Dynamic Neural Network ToolkitPreprint
Neubig, G., Dyer, C., Goldberg, Y., Matthews, A., Ammar, W., Anastasopoulos, A., Ballesteros, M., Chiang, D., Clothiaux, D., Cohn, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03980 · Cited by 351 · see citing works
Self-citationEmpower Sequence Labeling with Task-Aware Neural Language Model
Liu, L., Shang, J., Ren, X., Xu, F. F., Gui, H., Peng, J., & Han, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12006 · Cited by 349 · see citing works
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling TasksPreprint
Reimers, N., & Gurevych, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06799 · Cited by 321 · see citing works
UniT: Multimodal Multitask Learning with a Unified Transformer
Hu, R., & Singh, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00147 · Cited by 303 · see citing works
Cross-type biomedical named entity recognition with deep multi-task learning
Wang, X., Zhang, Y., Ren, X., Zhang, Y., Žitnik, M., Shang, J., Langlotz, C. P., & Han, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty869 · Cited by 276 · see citing works
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773 · Cited by 272 · see citing works
Towards Automated Reentrancy Detection for Smart Contracts Based on Sequential Models
Qian, P., Liu, Z., He, Q., Zimmermann, R., & Wang, X. (2020) · IEEE Access · doi:10.1109/access.2020.2969429 · Cited by 234 · see citing works
A neural network multi-task learning approach to biomedical named entity recognition
Crichton, G., Pyysalo, S., Chiu, B., & Korhonen, A. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1776-8 · Cited by 229 · see citing works
Identifying beneficial task relations for multi-task learning in deep neural networks
Bingel, J., & Søgaard, A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2026 · Cited by 220 · see citing works
Self-citationZhao, W., Wang, L., Shen, K., Jia, R., & Liu, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1014 · Cited by 220 · see citing works
BAM! Born-Again Multi-Task Networks for Natural Language Understanding
Clark, K., Luong, M.-T., Khandelwal, U., Manning, C. D., & Le, Q. V. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1595 · Cited by 219 · see citing works
A Hierarchical Multi-Task Approach for Learning Embeddings from Semantic Tasks
Sanh, V., Wolf, T., & Ruder, S. (2019) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v33i01.33016949 · Cited by 216 · see citing works
Tan, B., Song, Y., Zhong, E., & Yang, Q. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783295 · Cited by 198 · see citing works
Neural Sequence Learning Models for Word Sense Disambiguation
Raganato, A., Bovi, C. D., & Roberto Navigli, A. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1120 · Cited by 196 · see citing works
Neural End-to-End Learning for Computational Argumentation Mining
Eger, S., Daxenberger, J., & Gurevych, I. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1002 · Cited by 183 · see citing works
Multitask Learning for Mental Health Conditions with Limited Social Media Data
Benton, A., Mitchell, M., & Hovy, D. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1015 · Cited by 175 · see citing works
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift
Ruder, S., & Plank, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1096 · Cited by 168 · see citing works
Multi-Task Learning in Natural Language Processing: An Overview
Chen, S., Zhang, Y., & Yang, Q. (2024) · ACM Computing Surveys · doi:10.1145/3663363 · Cited by 167 · see citing works
A Multi-task Approach for Named Entity Recognition in Social Media Data
Aguilar, G., Maharjan, S., López-Monroy, A. P., & Solorio, T. (2017) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w17-4419 · Cited by 164 · see citing works
Robust Multilingual Part-of-Speech Tagging via Adversarial Training
Yasunaga, M., Kasai, J., & Radev, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1089 · Cited by 161 · see citing works
Learning Sparse Sharing Architectures for Multiple Tasks
Sun, T., Shao, Y., Li, X., Liu, P., Yan, H. R., Qiu, X., & Huang, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6424 · Cited by 149 · see citing works
Training Complex Models with Multi-Task Weak Supervision
Ratner, A., Hancock, B. J., Dunnmon, J., Sala, F., Pandey, S., & Ré, C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014763 · Cited by 147 · see citing works
Deep Multitask Learning for Semantic Dependency Parsing
Peng, H., Thomson, S., & Smith, N. A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1186 · Cited by 145 · see citing works
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Tu, L., Lalwani, G., Gella, S., & He, H. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00335 · Cited by 125 · see citing works
All-in-One: Emotion, Sentiment and Intensity Prediction Using a Multi-Task Ensemble Framework
Akhtar, M. S., Ghosal, D., Ekbal, A., Bhattacharyya, P., & Kurohashi, S. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2926724 · Cited by 122 · see citing works
Enhanced Multi-Task Learning and Knowledge Graph-Based Recommender System
Gao, M., Li, J.-Y., Chen, C., Li, Y., Zhang, J., & Zhan, Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3251897 · Cited by 122 · see citing works
Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition
Toshniwal, S., Tang, H., Lu, L., & Livescu, K. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1118 · Cited by 118 · see citing works
An overview of biomimetic robots with animal behaviors
Gao, Z., Shi, Q., Fukuda, T., Li, C., & Huang, Q. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.12.071 · Cited by 116 · see citing works
Zhao, S., Liu, T., Zhao, S., & Wang, F. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301817 · Cited by 115 · see citing works
Multi-Task Learning for Mental Health using Social Media TextPreprint
Benton, A., Mitchell, M., & Hovy, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03538 · Cited by 114 · see citing works
Sluice networks: Learning what to share between loosely related tasks.Preprint
Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A. (2017) · arXiv (Cornell University) · Cited by 112 · see citing works
Self-citationA Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling
Lin, Y., Yang, S., Stoyanov, V., & Ji, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1074 · Cited by 109 · see citing works
Multi-task Domain Adaptation for Sequence Tagging
Peng, N., & Dredze, M. H. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2612 · Cited by 109 · see citing works
HTML: Hierarchical Transformer-based Multi-task Learning for Volatility Prediction
Yang, L., Ng, T. L. J., Smyth, B., & Dong, R. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380128 · Cited by 109 · see citing works
An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models
Chronopoulou, A., Baziotis, C., & Potamianos, A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1213 · Cited by 105 · see citing works
Hierarchically-Refined Label Attention Network for Sequence Labeling
Cui, L., & Zhang, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1422 · Cited by 105 · see citing works
Global Inference for Aspect and Opinion Terms Co-Extraction Based on Multi-Task Neural Networks
Yu, J., Jiang, J., & Xia, R. (2018) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2018.2875170 · Cited by 104 · see citing works
GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling
Liu, Y., Meng, F., Zhang, J., Xu, J., Chen, Y., & Zhou, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1233 · Cited by 100 · see citing works
When is multitask learning effective? Semantic sequence prediction under varying data conditionsPreprint
Alonso, H. M., & Plank, B. (2017) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 99 · see citing works
Sequence Classification with Human Attention
Barrett, M., Bingel, J., Hollenstein, N., Rei, M., & Søgaard, A. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1030 · Cited by 98 · see citing works
Self-citationSyntactic Scaffolds for Semantic Structures
Swayamdipta, S., Thomson, S., Lee, K., Zettlemoyer, L., Dyer, C., & Smith, N. A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1412 · Cited by 94 · see citing works
Transfer Learning in Natural Language Processing
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773.020 · Cited by 94 · see citing works
What is the Essence of a Claim? Cross-Domain Claim Identification
Daxenberger, J., Eger, S., Habernal, I., Stab, C., & Gurevych, I. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1218 · Cited by 89 · see citing works
When is multitask learning effective? Semantic sequence prediction under varying data conditions
Alonso, H. M., & Plank, B. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1005 · Cited by 87 · see citing works
Multi-Task Learning of Keyphrase Boundary Classification
Augenstein, I., & Søgaard, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2054 · Cited by 84 · see citing works
Self-citationZhang, Z., Yu, W., Yu, M., Guo, Z., & Jiang, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.66 · Cited by 84 · see citing works
A Survey on Green Deep LearningPreprint
Xu, J., Zhou, W., Fu, Z., Zhou, H., & Li, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05193 · Cited by 84 · see citing works
Empirical Analysis of Multi-Task Learning for Reducing Identity Bias in Toxic Comment Detection
Vaidya, A., Mai, F., & Ning, Y. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7334 · Cited by 83 · see citing works
Robust Lexical Features for Improved Neural Network Named-Entity RecognitionPreprint
Ghaddar, A., & Langlais, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03489 · Cited by 81 · see citing works
BridgeTower: Building Bridges between Encoders in Vision-Language Representation Learning
Xiao, X., Wu, C., Rosenman, S., Lal, V., Che, W., & Duan, N. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26263 · Cited by 75 · see citing works
Seq2Edits: Sequence Transduction Using Span-level Edit Operations
Stahlberg, F., & Kumar, S. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.418 · Cited by 74 · see citing works
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP
van der Goot, R., Üstün, A., Ramponi, A., Sharaf, I., & Plank, B. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-demos.22 · Cited by 73 · see citing works
Multi-Task Learning for Argumentation Mining in Low-Resource Settings
Schulz, C., Eger, S., Daxenberger, J., Kahse, T., & Gurevych, I. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2006 · Cited by 72 · see citing works
Multitask Parsing Across Semantic Representations
Hershcovich, D., Abend, O., & Rappoport, A. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1035 · Cited by 71 · see citing works
Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling
Zhang, Q., Liao, X., Liu, Q., Xu, J., & Zheng, B. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498479 · Cited by 70 · see citing works
Learning Joint Semantic Parsers from Disjoint Data
Peng, H., Thomson, S., Swayamdipta, S., & Smith, N. A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1135 · Cited by 69 · see citing works
Multi-modal transportation recommendation with unified route representation learning
Liu, H., Han, J., Fu, Y., Zhou, J., Lu, X., & Xiong, H. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3430915.3430924 · Cited by 65 · see citing works
Multi-Task Learning for Sequence Tagging: An Empirical StudyPreprint
Changpinyo, S., Hu, H., & Sha, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04151 · Cited by 65 · see citing works
Neural Paraphrase Identification of Questions with Noisy Pretraining
Tomar, G. S., Duque, T. S. P. C., Täckström, O., Uszkoreit, J., & Das, D. (2017) · Workshop on Subword and Character Level Models in NLP (SCLeM) · doi:10.18653/v1/w17-4121 · Cited by 64 · see citing works
Learning what to share between loosely related tasksPreprint
Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A. (2017) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 64 · see citing works
Self-citationMultiple Relational Attention Network for Multi-task Learning
Zhao, J., Du, B., Sun, L., Zhuang, F., Lv, W., & Xiong, H. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330861 · Cited by 62 · see citing works
Li, C., Song, D., & Tao, D. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330983 · Cited by 58 · see citing works
Multi-task Character-Level Attentional Networks for Medical Concept Normalization
Niu, J., Yang, Y., Zhang, S., Sun, Z., & Zhang, W. (2018) · Neural Processing Letters · doi:10.1007/s11063-018-9873-x · Cited by 52 · see citing works
Transformer-Based Argument Mining for Healthcare Applications
Mayer, T., Cabrio, E., & Villata, S. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200334 · Cited by 52 · see citing works
Hierarchical Multitask Learning With CTC
Sanabria, R., & Metze, F. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639530 · Cited by 50 · see citing works
A multi-task deep learning model for the classification of Age-related Macular Degeneration
Chen, Q., Peng, Y., Keenan, T. D. L., Dharssi, S., Agrón, E., Wong, W. T., Chew, E. Y., & Lu, Z. (2018) · PubMed Central · doi:10.48550/arxiv.1812.00422 · Cited by 48 · see citing works
Hierarchical Multi-task Learning for Organization Evaluation of Argumentative Student Essays
Song, W., Song, Z., Liu, L., & Fu, R. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/536 · Cited by 47 · see citing works
Multi-Task Learning for Abstractive and Extractive Summarization
Chen, Y., Ma, Y., Mao, X., & Li, Q. (2019) · Data Science and Engineering · doi:10.1007/s41019-019-0087-7 · Cited by 45 · see citing works
End-to-end aspect-based sentiment analysis with hierarchical multi-task learning
Wang, X., Xu, G., Zhang, Z., Jin, L., & Sun, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.100 · Cited by 45 · see citing works
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural NetworksPreprint
Mao, H. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00800 · Cited by 45 · see citing works
Multi-task Learning in Deep Neural Networks at EVALITA 2018
Cimino, A., Mattei, L. D., & Dell’Orletta⋄, F. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.4527 · Cited by 44 · see citing works
Improving sentiment analysis with multi-task learning of negation
Barnes, J., Velldal, E., & Øvrelid, L. (2020) · Natural Language Engineering · doi:10.1017/s1351324920000510 · Cited by 44 · see citing works
Snorkel MeTaL: Weak Supervision for Multi-Task Learning
Ratner, A., Hancock, B., Dunnmon, J., Goldman, R., & Ré, C. (2018) · Proceedings of the Second Workshop on Data Management for End-To-End Machine Learning · doi:10.1145/3209889.3209898 · Cited by 43 · see citing works
Liang, W., Zhang, K., Cao, P., Liu, X., Yang, J., & Zai͏̈ane, O. R. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104935 · Cited by 42 · see citing works
MuMu: Cooperative Multitask Learning-Based Guided Multimodal Fusion
Islam, M. M., & Iqbal, T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19988 · Cited by 42 · see citing works
Semi-Supervised Sequence Modeling with Cross-View TrainingPreprint
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08370 · Cited by 42 · see citing works
Generating Product Descriptions from User Reviews
Novgorodov, S., Guy, I., Elad, G., & Radinsky, K. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313532 · Cited by 41 · see citing works
Hierarchical shared transfer learning for biomedical named entity recognition
Chai, Z., Han, J., Shi, S., Zhan, S., Zhuo, L., & Yang, Y. (2022) · BMC Bioinformatics · doi:10.1186/s12859-021-04551-4 · Cited by 39 · see citing works
Narayanan, S., Mannam, K., Achan, P., Ramesh, M. V., Rangan, P. V., & Rajan, S. P. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103960 · Cited by 38 · see citing works
Hierarchical Multitask Learning for CTC-based Speech RecognitionPreprint
Krishna, K., Toshniwal, S., & Livescu, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06234 · Cited by 38 · see citing works
A Multi-task Ensemble Framework for Emotion, Sentiment and Intensity PredictionPreprint
Akhtar, M. S., Ghosal, D., Ekbal, A., Bhattacharyya, P., & Kurohashi, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01216 · Cited by 37 · see citing works
Enhancing evidence-based medicine with natural language argumentative analysis of clinical trials.
Mayer, T., Marro, S., Cabrio, E., & Villata, S. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102098 · Cited by 36 · see citing works
Word Emotion Induction for Multiple Languages as a Deep Multi-Task Learning Problem
Buechel, S., & Hahn, U. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1173 · Cited by 36 · see citing works
Argumentative Link Prediction using Residual Networks and Multi-Objective Learning
Galassi, A., Lippi, M., & Torroni, P. (2018) · Workshop on Argument Mining (ArgMining) · doi:10.18653/v1/w18-5201 · Cited by 36 · see citing works
Multitask Learning for Arabic Offensive Language and Hate-Speech Detection
Farha, I. A., & Magdy, W. (2020) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 36 · see citing works
- Authors
- Søgaard, A., & Goldberg, Y.
- Affiliations
- University of Copenhagen (DK)
- Venue
- Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Published
- 2016
- Pages
- 231-235
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2516255829
- Cited by
- 445
- References
- 16
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 12 · 2023: 32 · 2022: 35 · 2021: 42 · 2020: 87 · 2019: 95 · 2018: 86 · 2017: 37 · 2016: 2 · 2015: 1
- FWCI
- 39.51
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- multi-task deep learning, bidirectional RNN, part-of-speech tagging, shared representations, domain adaptation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review