The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Open accessAbstract
Numerous deep learning applications benefit from multitask learning with multiple regression and classification objectives. In this paper we make the observation that the performance of such systems is strongly dependent on the relative weighting between each task's loss. Tuning these weights by hand is a difficult and expensive process, making multi-task learning prohibitive in practice. We propose a principled approach to multi-task deep learning which weighs multiple loss functions by considering the homoscedastic uncertainty of each task. This allows us to simultaneously learn various quantities with different units or scales in both classification and regression settings. We demonstrate our model learning per-pixel depth regression, semantic and instance segmentation from a monocular input image. Perhaps surprisingly, we show our model can learn multi-task weightings and outperform separate models trained individually on each task.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
(1967) · Cited by 23,719
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Self-citationZhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Stereo Processing by Semiglobal Matching and Mutual Information
Hirschmuller, H. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1166 · Cited by 4,195
Ankerst, M. (1999) · doi:10.1145/304181.304187 · Cited by 3,982
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
Self-citationEigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
(2011) · Cited by 2,752
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Self-citationConditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1×
“Firstly, we learn to classify objects at a pixel level, also known as semantic segmentation [32, 3, 42, 8, 45].”
MethodAccurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information
Hirschmuller, H. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.56 · Cited by 1,698
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., BG, V. K., Carneiro, G., & Reid, I. (2016) · arXiv · doi:10.48550/arxiv.1603.04992 · Cited by 1,632
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Cross-Stitch Networks for Multi-task Learning
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.433 · Cited by 1,397
OPTICS: ordering points to identify the clustering structure
Ankerst, M., Breunig, M. M., Kriegel, H.-P., & Sander, J. (1999) · Proceedings of the 1999 ACM SIGMOD international conference on Management of data · doi:10.1145/304182.304187 · Cited by 1,335
Instance-aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.04412 · Cited by 1,273
Instance-Aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.343 · Cited by 1,244
Caruana, R. (1998) · Cited by 1,110
Robust Object Detection with Interleaved Categorization and Segmentation
Leibe, B., Leonardis, A., & Schiele, B. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0095-3 · Cited by 933
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
A Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
Teichmann, M., Weber, M., Zollner, M., Cipolla, R., & Urtasun, R. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500504 · Cited by 681
Self-citationLearning to Segment Object Candidates
Pinheiro, P. O., Collobert, R., & Dollár, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06204 · Cited by 648
Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers
Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639081 · Cited by 595
A model of inductive bias learning
Baxter, J. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.731 · Cited by 582
Caruana, R. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_5 · Cited by 577
Deep Watershed Transform for Instance Segmentation
Bai, M., & Urtasun, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.305 · Cited by 539
(1995) · Cited by 520
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06870 · Cited by 406
Agrawal, P., Carreira, J., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.13 · Cited by 395
In-place Activated BatchNorm for Memory-Optimized Training of DNNs
Bulo, S. R., Porzi, L., & Kontschieder, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00591 · Cited by 320
Proposal-Free Network for Instance-Level Object Segmentation
Liang, X., Lin, L., Wei, Y., Shen, X., Yang, J., & Yan, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2775623 · Cited by 224
Pixel-Level Encoding and Depth Layering for Instance-Level Semantic Labeling
Uhrig, J., Cordts, M., Franke, U., & Brox, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45886-1_2 · Cited by 152
Liao, Y., Kodagoda, S., Wang, Y., Shi, L., & Liu, Y. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487381 · Cited by 109
Convolutional networks for real-time 6-DOF camera relocalization.Preprint
Kendall, A., Grimes, M. K., & Cipolla, R. (2015) · arXiv (Cornell University) · Cited by 50
Self-citation
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
An Overview of Multi-Task Learning in Deep Neural Networks
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,265
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking
Zhang, Y., Wang, C., Wang, X., Zeng, W., & Liu, W. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01513-4 · Cited by 1,861
Libra R-CNN: Towards Balanced Learning for Object Detection
Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00091 · Cited by 1,763
Metadata
- Authors
- Cipolla, R., Gal, Y., & Kendall, A.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 7482-7491
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963677766
- Cited by
- 3,151
- References
- 52
- FWCI
- 86.46
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.