CoChat
Sign Up
Cited by 1,128Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsAdvanced Vision and Imaging

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Taskonomy: Disentangling Task Transfer Learning

Open access
Zamir, Amir R., Sax, Alexander, Shen, William, Guibas, Leonidas, Malik, Jitendra, Savarese, Silvio·2018·2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
DOI 10.1109/cvpr.2018.00391
Open this paper in CoChat — free accountOpen PDF

Abstract

Do visual tasks have relationships, or are they unrelated? For instance, could having surface normals simplify estimating the depth of an image? Intuition answers these questions positively, implying existence of a certain structure among visual tasks. Knowing this structure has notable values; it provides a principled way for identifying relationships across tasks, for instance, in order to reuse supervision among tasks with redundancies or solve many tasks in one system without piling up the complexity. We propose a fully computational approach for modeling the transfer learning structure of the space of visual tasks. This is done via finding transfer learning dependencies across tasks in a dictionary of twenty-six 2D, 2.5D, 3D, and semantic tasks. The product is a computational taxonomic map among tasks for transfer learning, and we exploit it to reduce the demand for labeled data. For example, we show that the total number of labeled datapoints needed for solving a set of 10 tasks can be reduced by roughly 2/3 (compared to training independently) while keeping the performance nearly the same. We provide a set of tools for computing and visualizing this taxonomical structure at http://taskonomy.vision.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr.2018.00391

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 15 references

Research works printed but not linked: 86

  1. gurobi, 2016, Optimization Gurobi Optimizer Reference Manual
  2. graves, 2014, Neural turing machines, CoRR abs/1410 5401
  3. 10.1090/qam/1939008 · DOI
  4. ge, 2013, Provable algorithmsfor machine learning problems
  5. 10.1016/S1364-6613(99)01294-2 · DOI · Paper page
  6. fodor, 2002, A survey of dimension reduction techniques, Technical Report
  7. gopnik, 1999, The scientist in the crib Minds brains and how children learn
  8. 10.1037/0033-295X.111.1.3 · DOI
  9. gopnik, 2004, A theory of causal learning in children Causal maps and bayes nets, 111, 3
  10. 10.1109/ICCV.2011.6126344 · DOI
  11. finn, 2016, Generalizing skills with semi-supervised reinforcement learning, CoRR abs/1612 00429
  12. 10.1109/ICRA.2016.7487173 · DOI
  13. finn, 2017, One-shot visual imitation learning via meta-learning, CoRR abs/1709 04905
  14. erhan, 2010, Why does unsupervised pre-training help deep learning?, Journal of Machine Learning Research, 625
  15. 10.1109/ICCV.2003.1238476 · DOI
  16. faktor, 2012, clustering by composition-unsupervised discovery of image categories, European Conference on Computer Vision, 474
  17. 10.1109/ICCV.2013.368 · DOI
  18. 10.1109/TPAMI.2006.79 · DOI · Paper page
  19. zhou, 2014, Learning deep features for scene recognition using places database, Advances in neural information processing systems, 487
  20. finn, 2016, Guided cost learning: Deep inverse optimal control via policy optimization, CoRR abs/1603 00448
  21. 10.1007/978-3-319-46487-9_40 · DOI
  22. finn, 2017, Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks[J]
  23. 10.1109/CVPR.2011.5995702 · DOI
  24. 10.1109/3DV.2016.32 · DOI
  25. 10.1016/j.physrep.2017.07.007 · DOI
  26. 10.1016/j.patrec.2016.01.019 · DOI
  27. luo, 0, Label efficient learning of transferable representations across domains and tasks
  28. 10.1016/j.patcog.2015.04.019 · DOI
  29. lin, 2014, Microsoft coco: Common objects in context, European Conference on Computer Vision, 740
  30. li, 2016, Fully convolutional instance-aware semantic segmentation
  31. henry, 2008, The Theory and Applications of Homomorphic Cryptography
  32. arora, 2014, Provable bounds for learning some deep representations, International Conference on Machine Learning, 584
  33. armeni, 2017, Joint 2d-3d-semantic data for indoor scene understanding
  34. 10.1109/TPAMI.2013.50 · DOI · Paper page
  35. 10.1109/ICCV.2011.6126504 · DOI
  36. bienenstock, 1997, Compositionality, mdl priors, and object recognition, Advances in neural information processing systems, 838
  37. krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems, 1097
  38. 10.1007/3-540-28349-8_2 · DOI
  39. bilen, 2016, Integrated Integrated with recurrent multi-task neural networks, Advances in neural information processing systems, 235
  40. 10.1109/CVPR.2012.6247924 · DOI
  41. kokkinos, 2016, Ubernet Training auniversal’ convolutional neural network for low- mid- and high-level vision using diverse datasets and limited memory
  42. 10.1109/CVPR.2014.116 · DOI
  43. hinton, 2015, Distilling the knowledge in a neural network
  44. hu, 2015, Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery, Remote Sensing, 7, 14680, 10.3390/rs71114680 · DOI
  45. hoshen, 2015, Visual learning of arithmetic operations, CoRR abs/1506 02264
  46. 10.1109/ICCV.2017.243 · DOI
  47. pratt, 1993, Discriminability-based transfer between neural networks, Advances in neural information processing systems, 204
  48. 10.1037/11494-000 · DOI · Paper page
  49. pentina, 2017, Multi-task learning with labeled and unlabeled tasks, Stat, 1050, 1
  50. saaty, 0, The analytic hierarchy process - what it is and how it is used, Mathematical Modeling, 9, 161, 10.1016/0270-0255(87)90473-8 · DOI · Paper page
  51. roweis, 2000, Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 2323, 10.1126/science.290.5500.2323 · DOI · Paper page
  52. salakhutdinov, 2012, One-shot learning with a hierarchical nonparametric bayesian model, Proc ICML Workshop Unsupervised Transfer Learn, 195
  53. schulman, 2015, Trust region policy optimization, CoRR abs/1502 05477
  54. mikolov, 2013, Exploiting similarities among languages for machine translation, CoRR abs/1309 4168
  55. mihalkova, 2007, Mapping and revising markov logic networks for transfer learning, AAAI, 7, 608
  56. 10.1109/CVPR.2016.433 · DOI
  57. niculescu-mizil, 2007, Inductive transfer for bayesian network structure learning, Artificial Intelligence and Statistics, 339
  58. noroozi, 2016, Unsupervised learning of visual representations by solving jigsaw puzzles, European Conference on Computer Vision, 69
  59. noroozi, 2017, Representation learning by learning to count
  60. norouzi, 2013, Zero-shot learning by convex combination of semantic embeddings
  61. ovsjanikov, 2012, Functional maps: a flexible representation of maps between shapes, ACM Transactions on Graphics (TOG), 31, 30, 10.1145/2185520.2185526 · DOI
  62. andrychowicz, 2016, Learning to learn by gradient descent by gradient descent, Advances in neural information processing systems, 3981
  63. 10.1109/CVPR.2016.278 · DOI · Paper page
  64. 10.1109/ICCV.2015.13 · DOI
  65. winograd, 1991, Thinking machines Can there be? Are we, 200
  66. wang, 2017, Transitive invariance for self-supervised visual representation learning
  67. thrun, 2012, Learning to Learn
  68. tessler, 2017, A deep hierarchical approach to lifelong learning in minecraft, AAAI, 1553
  69. zamir, 2018, Gibson Env: Real-world Gibson Env: Real-world for embodied agents, 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  70. zhang, 2016, Understanding deep learning requires rethinking generalization, CoRR abs/1611 03530
  71. 10.1109/ICDMW.2007.37 · DOI
  72. zamir, 2016, Generic 3d representation via pose estimation and matching, European Conference on Computer Vision, 535
  73. bingel, 2017, Identifying beneficial task relations for multi-task learning in deep neural networks
  74. boiman, 2007, Similarity by composition, Advances in neural information processing systems, 177
  75. 10.2200/S00737ED1V01Y201610AIM033 · DOI · Paper page
  76. cplex, 2009, V12. 1: Users manual for cplex, International Business Machines Corporation, 46, 157
  77. 10.1109/ICCV.2015.167 · DOI · Paper page
  78. doersch, 2017, Multi-task self-supervised visual learning
  79. silver, 2013, Lifelong machine learning systems: Beyond learning algorithms, Proc AAAI Spring Symp Series
  80. 10.1007/s10994-008-5087-1 · DOI
  81. donahue, 2016, Adversarial feature learning
  82. srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, 15, 1929
  83. duan, 2016, RL2 Fast reinforcement learning via slow reinforcement learning
  84. socher, 2013, Zero-shot learning through cross-modal transfer, Advances in neural information processing systems, 935
  85. szegedy, 2013, Intriguing properties of neural networks, CoRR abs/1312 6199
  86. 10.1017/S0140525X01000061 · DOI

Cited by

Sorted by citations

See all 1,128

Metadata

Authors
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S.
Affiliations
  • University of California, Berkeley (US)
  • Stanford University (US)
ORCID
Venue
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Publisher
IEEE
Published
2018-06
Pages
3712-3722
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr.2018.00391
OpenAlex
W2798512429
Cited by
1,128
References
15
Citations by year
2026: 36 · 2025: 94 · 2024: 136 · 2023: 164 · 2022: 137 · 2021: 211 · 2020: 165 · 2019: 158 · 2018: 27
FWCI
57.35
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • International Joint Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (N00014-13-1-0341, 1521608)
  • Nvidia
  • Amazon Web Services
  • Tencent
  • Multidisciplinary University Research Initiative (N00014-13-1-0341)
  • Office of Naval Research (N00014-13-1-0341, N000141410671, N00014)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
visual task, task dependencies, latent space, surface normal estimation, depth estimation, data efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review