An Overview of Multi-Task Learning in Deep Neural Networks
Open accessThe DOI matches its OpenAlex record; 39 of the 39 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261 · cited 1×
“The latter is also known as group lasso and was first proposed by [Yuan and Lin, 2006].”
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
Regularized multi--task learning
Evgeniou, T., & Pontil, M. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014067 · Cited by 1,433
Cross-Stitch Networks for Multi-task Learning
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.433 · Cited by 1,397
Facial Landmark Detection by Deep Multi-task Learning
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_7 · Cited by 1,365
Argyriou, A., Evgeniou, T., & Pontil, M. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0010 · Cited by 1,333
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,311
Deng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Caruana, R. (1998) · Cited by 1,110
Learning Multiple Tasks with Kernel Methods
Evgeniou, T., Micchelli, C. A., & Pontil, M. (2005) · doi:10.5555/1046920.1088693 · Cited by 799
A Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730
A model of inductive bias learning
Baxter, J. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.731 · Cited by 582 · cited 2×
“[Baxter, 2000] argues only theoretically that related tasks share a common optimal hypothesis class, i.e. have the same inductive bias.”
BackgroundA Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Hashimoto, K., xiong, C., Tsuruoka, Y., & Socher, R. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1206 · Cited by 535
A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling
Baxter, J. (1997) · Machine Learning · doi:10.1023/a:1007327622663 · Cited by 462
Deep multi-task learning with low level tasks supervised at lower layers
Søgaard, A., & Goldberg, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2038 · Cited by 445 · cited 1×
“In contrast, in natural language processing (NLP), recent work focused on finding better task hierarchies for multi-task learning: [Søgaard and Goldberg, 2016] show that low-level tasks, i.e. NLP tasks typically used for preprocessing such as part-of-speech tagging and named entity recognition,…”
BackgroundExploiting Task Relatedness for Multiple Task Learning
Ben-David, S., & Schuller, R. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-45167-9_41 · Cited by 425
Low Resource Dependency Parsing: Cross-lingual Parameter Sharing in a Neural Network Parser
Duong, L., Cohn, T., Bird, S., & Cook, P. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.3115/v1/p15-2139 · Cited by 402
Clustered Multi-Task Learning: A Convex Formulation
Jacob, L., Bach, F., & Vert, J. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0809.2085 · Cited by 400
Deep Voice: Real-time Neural Text-to-Speech
Arık, S. Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., Li, X., Miller, J. J., Ng, A. Y., Raiman, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.07825 · Cited by 381
Learning Gaussian processes from multiple tasks
Yu, K., Tresp, V., & Schwaighofer, A. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102479 · Cited by 380
Massively Multitask Networks for Drug Discovery
Ramsundar, B., Kearnes, S. M., Riley, P., Webster, D. R., Konerding, D. E., & Pande, V. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02072 · Cited by 378
Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity
Kim, S., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476414 · Cited by 374
A Dirty Model for Multi-task Learning
Jalali, A., Sanghavi, S., Ruan, C., & Ravikumar, P. (2010) · Cited by 368
Learning with Whom to Share in Multi-task Feature Learning
Kang, Z., Grauman, K. L., & Sha, F. (2011) · Cited by 320
Girshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 276
Semi-supervised Multitask Learning for Sequence Labeling
Rei, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1194 · Cited by 257
Learning from hints in neural networks
Abu-Mostafa, Y. S. (1990) · Journal of Complexity · doi:10.1016/0885-064x(90)90006-y · Cited by 241
Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification
Yu, J., & Jiang, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1023 · Cited by 212
Discovering Structure in Multiple Learning Tasks: The TC Algorithm.
Thrun, S., & O’Sullivan, J. A. (1996) · Cited by 208
Sluice networks: Learning what to share between loosely related tasks.Preprint
Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A. (2017) · arXiv (Cornell University) · Cited by 155
Self-citationWhen is multitask learning effective? Semantic sequence prediction under varying data conditions
Alonso, H. M., & Plank, B. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1005 · Cited by 144
Deep Multi-task Representation Learning: A Tensor Factorisation Approach
Yang, Y., & Hospedales, T. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06391 · Cited by 96
Empirical Bayes for Learning to Learn
Heskes, T. (2000) · Radboud Repository (Radboud University) · doi:10.5555/645529.658133 · Cited by 94
Learning Multiple Tasks with Deep Relationship NetworksPreprint
Long, M., Wang, J., & Yu, P. S. (2015) · arXiv (Cornell University) · Cited by 91
Bayesian Multitask Learning with Latent HierarchiesPreprint
Daumé, H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.2032 · Cited by 83
Distributed Multi-Task Relationship Learning
Liu, S., Pan, S. J., & Ho, Q. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098136 · Cited by 61
Open-Domain Name Error Detection using a Multi-Task RNN
Cheng, H., Fang, H., & Ostendorf, M. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1085 · Cited by 51
Learning Multiple Tasks using Shared Hypotheses
Crammer, K., & Mansour, Y. (2012) · Cited by 49
Promoting Poor Features to Supervisors: Some Inputs Work Better as Outputs
Caruana, R., de Sa, V. R., & de Sa, V. R. (1997) · Cited by 36
Cited by
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The future of digital health with federated learning
Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00323-1 · Cited by 4,015
ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data
Diakogiannis, F. I., Waldner, F., Caccetta, P., & Wu, C. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.01.013 · Cited by 2,153
Crop yield prediction using machine learning: A systematic literature review
van Klompenburg, T., Kassahun, A., & Catal, C. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105709 · Cited by 1,858
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
“Techniques in transfer learning [134], meta-learning [52], and multi-task learning [144] may be useful to combine the knowledge of various kinds of parties.”
Background
Metadata
- Authors
- Ruder, S.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-06-15
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1706.05098
- OpenAlex
- W2624871570
- Cited by
- 2,265
- References
- 39
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.