Learning with Side Information through Modality Hallucination
Abstract
We present a modality hallucination architecture for training an RGB object detection model which incorporates depth side information at training time. Our convolutional hallucination network learns a new and complementary RGB image representation which is taught to mimic convolutional mid-level features from a depth network. At test time images are processed jointly through the RGB and hallucination networks to produce improved detection performance. Thus, our method transfers information commonly extracted from depth training data to a network which can extract that information from the RGB counterpart. We present results on the standard NYUDv2 dataset and report improvement on the RGB detection task.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In action recognition, prior missing modality studies follow transfer learning via knowledge distillation.
“In the context of action recognition (Tran et al. 2015; Wang et al. 2016; Carreira and Zisserman 2017; Feichtenhofer et al. 2019), there are handful of studies towards this end (Hoffman, Gupta, and Darrell 2016; Luo et al. 2018; Garcia, Morerio, and Murino 2018, 2019; Garcia et al. 2019; Stroud et al. 2020), all of which follow transfer learning scheme via knowledge distillation (Hinton et al. 201”
From Towards Good Practices for Missing Modality Robust Action Recognition (2023) · cites this paper as Hoffman, Gupta, and Darrell 2016
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Self-citationGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Self-citationCaffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · see citing works
Self-citationEigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862 · see citing works
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
Geodesic flow kernel for unsupervised domain adaptation
Gong, B., Shi, Y., Sha, F., & Grauman, K. L. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247911 · Cited by 2,254 · see citing works
Do Deep Nets Really Need to be Deep?Preprint
Ba, J. L., & Caruana, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6184 · Cited by 1,592 · see citing works
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528 · see citing works
Self-citationSimultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,203 · see citing works
Self-citationMultiscale Combinatorial Grouping
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.49 · Cited by 1,155 · see citing works
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005 · see citing works
Deep convolutional neural fields for depth estimation from a single image
Liu, F., Shen, C., & Lin, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299152 · Cited by 892 · see citing works
A new learning paradigm: Learning using privileged information
Vapnik, V., & Vashist, A. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.06.042 · Cited by 774 · see citing works
What you saw is not what you get: Domain adaptation using asymmetric kernel transforms
Kulis, B., Saenko, K., & Darrell, T. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995702 · Cited by 698 · see citing works
Self-citationCross Modal Distillation for Supervision Transfer
Gupta, S., Hoffman, J., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.309 · Cited by 530 · cited 4× · see citing works
“This choice is inspired by prior work which uses similar techniques for model distillation [17], task correlation transfer across domains [33], and supervision transfer from a well labeled modality to one with limited labels [16].”
BackgroundMethodInfluentialSelf-citationTabula rasa: Model transfer for object category detection
Aytar, Y., & Zisserman, A. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126504 · Cited by 339 · see citing works
Learning with Augmented Features for Heterogeneous Domain Adaptation
Duan, L., Xu, D., & Tsang, I. (2012) · arXiv · doi:10.48550/arxiv.1206.4660 · Cited by 320 · see citing works
A Category-Level 3D Object Dataset: Putting the Kinect to Work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2012) · Advances in computer vision and pattern recognition · doi:10.1007/978-1-4471-4640-7_8 · Cited by 254 · see citing works
Self-citationA category-level 3-D object dataset: Putting the Kinect to work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130382 · Cited by 250 · see citing works
Self-citationAligning 3D models to RGB-D images of cluttered scenes
Gupta, S., Arbeláez, P., Girshick, R., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299105 · Cited by 199 · see citing works
Self-citationLearning to Rank Using Privileged Information
Sharmanska, V., Quadrianto, N., & Lampert, C. H. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.107 · Cited by 196 · see citing works
Weakly Supervised Object Localization with Latent Category Learning
Wang, C., Ren, W., Huang, K., & Tan, T. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_28 · Cited by 187 · see citing works
Histogram of Oriented Normal Vectors for Object Recognition with a Depth Sensor
Tang, S., Wang, X., Lv, X., Han, T. X., Keller, J. F., He, Z., Skubic, M., & Lao, S. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37444-9_41 · Cited by 178 · see citing works
Efficient Learning of Domain-invariant Image RepresentationsPreprint
Hoffman, J., Rodner, E., Donahue, J., Darrell, T., & Saenko, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3224 · Cited by 167 · see citing works
Self-citationLarge-Margin Multi-Modal Deep Learning for RGB-D Object Recognition
Wang, A., Lu, J., Cai, J., Cham, T.-J., & Wang, G. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2476655 · Cited by 153 · see citing works
Recognizing RGB Images by Learning from RGB-D Data
Chen, L., Li, W., & Xu, D. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.184 · Cited by 62 · see citing works
Accurate Localization of 3D Objects from RGB-D Data Using Segmentation Hypotheses
Kim, B.-S., Xu, S., & Savarese, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.409 · Cited by 60 · see citing works
Deep CNN Ensemble with Data Augmentation for Object DetectionPreprint
Guo, J. F., & Gould, S. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.07224 · Cited by 44 · see citing works
Building Part-Based Object Detectors via 3D Geometry
Shrivastava, A., & Gupta, A. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.219 · Cited by 39 · see citing works
Leveraging RGB-D Data: Adaptive fusion and domain adaptation for object detection
Spinello, L., & Arras, K. O. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225137 · Cited by 33 · see citing works
Learning to Recognize Objects from Unseen Modalities
Christoudias, C. M., Urtasun, R., Salzmann, M., & Darrell, T. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_49 · Cited by 23 · see citing works
Self-citation
Flagged first, then by citations
2 of the 42 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931 · see citing works
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · see citing works
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315 · see citing works
End-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884 · see citing works
Self-citationContrastive Representation DistillationPreprint
Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10699 · Cited by 888 · see citing works
Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks
Audebert, N., Saux, B. L., & Lefèvre, S. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.11.011 · Cited by 678 · see citing works
Domain Adaptation for Visual Applications: A Comprehensive SurveyPreprint
Csurka, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05374 · Cited by 324 · see citing works
Skeleton Aware Multi-modal Sign Language Recognition
Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., & Fu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00380 · Cited by 309 · see citing works
MARS: Motion-Augmented RGB Stream for Action Recognition
Crasto, N., Weinzaepfel, P., Alahari, K., & Schmid, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00807 · Cited by 293 · see citing works
A2dele: Adaptive and Attentive Depth Distiller for Efficient RGB-D Salient Object Detection
Piao, Y., Rong, Z., Zhang, M., Ren, W., & Lu, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00908 · Cited by 262 · see citing works
Learning Cross-Modal Deep Representations for Robust Pedestrian Detection
Xu, D., Ouyang, W., Ricci, E., Wang, X., & Sebe, N. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.451 · Cited by 240 · see citing works
Modality Distillation with Multiple Stream Networks for Action Recognition
Garcia, N. C., Morerio, P., & Murino, V. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_7 · Cited by 201 · see citing works
Synthetic Data Generation for End-to-End Thermal Infrared Tracking
Zhang, L., González-García, A., van de Weijer, J., Danelljan, M., & Khan, F. S. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2879249 · Cited by 193 · see citing works
DADA: Depth-Aware Domain Adaptation in Semantic Segmentation
Vu, T.-H., Jain, H., Bucher, M., Cord, M., & Pérez, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00746 · Cited by 190 · see citing works
Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation
Zhou, H., Zhou, W., Zhou, Y., & Li, H. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3059098 · Cited by 182 · see citing works
Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation
Chen, L.-Z., Lin, Z., Wang, Z., Yang, Y., & Cheng, M.-M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3049332 · Cited by 177 · see citing works
You, H., Feng, Y., Ji, R., & Gao, Y. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240702 · Cited by 161 · see citing works
Label Efficient Learning of Transferable Representations across Domains and TasksPreprint
Luo, Z., Zou, Y., Hoffman, J., & Fei-Fei, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00123 · Cited by 172 · see citing works
Self-citationLezama, J., Qiu, Q., & Sapiro, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.720 · Cited by 148 · see citing works
Learning with Privileged Information for Efficient Image Super-Resolution
Lee, W.-K., Lee, J., Kim, D., & Ham, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_28 · Cited by 146 · see citing works
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations
Nekrasov, V., Dharmasiri, T., Spek, A., Drummond, T. W., Shen, C., & Reid, I. D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794220 · Cited by 143 · see citing works
Generative Multi-View Human Action Recognition
Wang, L., Ding, Z.-M., Tao, Z., Liu, Y., & Fu, Y. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.00631 · Cited by 117 · see citing works
Liu, T., Lam, K., Zhao, R., & Qiu, G. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3060162 · Cited by 117 · see citing works
Graph Distillation for Action Detection with Privileged Modalities
Luo, Z., Hsieh, J.-T., Jiang, L., Niebles, J. C., & Fei-Fei, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_11 · Cited by 111 · see citing works
Semantics-Aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
Liu, Y., Wang, K., Li, G., & Lin, L. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3086590 · Cited by 116 · see citing works
Geometry-Aware Distillation for Indoor Semantic Segmentation
Jiao, J., Wei, Y., Jie, Z., Shi, H., Lau, R. W. H., & Huang, T. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00298 · Cited by 105 · see citing works
MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding
Kong, Q., Wu, Z., Deng, Z., Klinkigt, M., Tong, B., & Murakami, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00875 · Cited by 96 · see citing works
Zero-Shot Deep Domain Adaptation
Peng, K., Wu, Z., & Ernst, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_47 · Cited by 90 · see citing works
VPN++: Rethinking Video-Pose Embeddings for Understanding Activities of Daily Living
Das, S., Dai, R., Yang, D., & Brémond, F. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3127885 · Cited by 92 · see citing works
Deep Learning Under Privileged Information Using Heteroscedastic Dropout
Lambert, J., Sener, O., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00926 · Cited by 93 · see citing works
“Modality Hallucination[18]: Distillation-based LUPI method designed for multi-modal CNNs.”
InfluentialDeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination
Saputra, M. R. U., de Gusmão, P. P. B., Lu, C. X., Almalıoğlu, Y., Rosa, S., Chen, C., Wahlström, J., Wang, W., Markham, A., & Trigoni, N. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969170 · Cited by 86 · see citing works
Shi, Z., & Kim, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.498 · Cited by 83 · see citing works
Learning with Privileged Information via Adversarial Discriminative Modality Distillation
Garcia, N. C., Morerio, P., & Murino, V. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929038 · Cited by 82 · see citing works
Robust Deep Multi-modal Learning Based on Gated Information Fusion Network
Kim, J., Koh, J., Kim, Y., Choi, J., Hwang, Y., & Choi, J. W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_6 · Cited by 76 · see citing works
Wei, S., Luo, C., & Luo, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01919 · Cited by 75 · see citing works
Learning Joint Visual Semantic Matching Embeddings for Language-Guided Retrieval
Chen, Y., & Bazzani, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_9 · Cited by 74 · see citing works
SPIGAN: Privileged Adversarial Learning from SimulationPreprint
Lee, K.-H., Ros, G., Li, J., & Gaidon, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03756 · Cited by 80 · see citing works
Attention-Based Autism Spectrum Disorder Screening With Privileged Modality
Chen, S., & Zhao, Q. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00127 · Cited by 72 · see citing works
Yu, J., Liu, J., Cheng, Y., Feng, R., & Zhang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547868 · Cited by 70 · see citing works
Kampffmeyer, M., Salberg, A.-B., & Jenssen, R. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2834961 · Cited by 66 · see citing works
A General Dynamic Knowledge Distillation Method for Visual Analytics
Tu, Z., Liu, X., & Xiao, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3212905 · Cited by 67 · see citing works
Domain Generalization in Vision: A SurveyPreprint
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2021) · arXiv (Cornell University) · Cited by 65 · see citing works
Towards Good Practices for Missing Modality Robust Action Recognition
Woo, S., Lee, S., Park, Y., Nugroho, M. A., & Kim, C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25378 · Cited by 60 · see citing works
Classification of femur fracture in pelvic X-ray images using meta-learned deep neural network
Lee, C., Jang, J., Lee, S., Kim, Y. S., Jo, H. J., & Kim, Y.-S. (2020) · Scientific Reports · doi:10.1038/s41598-020-70660-4 · Cited by 59 · see citing works
Teaching Compositionality to CNNs
Stone, A., Wang, H., Stark, M., Liu, Y., Phoenix, D. S., & Dileep George, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.85 · Cited by 57 · see citing works
Cross-Modal Distillation to Improve MRI-Based Brain Tumor Segmentation With Missing MRI Sequences
Rahimpour, M., Bertels, J., Radwan, A., Vandermeulen, H., Sunaert, S. G., Vandermeulen, D., Maes, F., Goffin, K. E., & Koole, M. J. B. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3137561 · Cited by 56 · see citing works
Effective Pan-Sharpening by Multiscale Invertible Neural Network and Heterogeneous Task Distilling
Zhou, M., Huang, J., Fu, X., Zhao, F., & Hong, D. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3199210 · Cited by 52 · see citing works
Adaptive Modality Distillation for Separable Multimodal Sentiment Analysis
Peng, W., Hong, X., & Zhao, G. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2021.3057757 · Cited by 52 · see citing works
Human Pose Estimation in Extremely Low-Light Conditions
Lee, S., Rim, J., Jeong, B., Kim, G., Woo, B., Lee, H., Cho, S., & Kwak, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00075 · Cited by 51 · see citing works
DOPE: Distillation of Part Experts for Whole-Body 3D Pose Estimation in the Wild
Weinzaepfel, P., Brégier, R., Combaluzier, H., Leroy, V., & Rogez, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_23 · Cited by 50 · see citing works
Multimodal vision-based human action recognition using deep learning: a review
Shafizadegan, F., Naghsh‐Nilchi, A. R., & Shabaninia, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10730-5 · Cited by 55 · see citing works
Prediction of Manipulation Actions
Fermuller, C. M., Wang, F., Yang, Y., Zampogiannis, K., Zhang, Y., Barranco, F., & Pfeiffer, M. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-0992-z · Cited by 48 · see citing works
Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision
Kim, N., Choi, Y., Hwang, S., & Kweon, I. S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12297 · Cited by 46 · see citing works
Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection
Mordan, T., Thome, N., Hénaff, G., & Cord, M. (2018) · HAL · Cited by 45 · see citing works
Huang, Y., Jahanshahi, M. R., Shen, F., & Mondal, T. G. (2023) · Journal of Transportation Engineering Part B Pavements · doi:10.1061/jpeodx.pveng-1194 · Cited by 44 · see citing works
Visual Recognition in RGB Images and Videos by Learning from RGB-D Data
Li, W., Chen, L., Xu, D., & Gool, L. V. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2734890 · Cited by 44 · see citing works
Learning Scene Geometry for Visual Localization in Challenging Conditions
Piasco, N., Désiré Sidibé, A., Gouet-Brunet, V., & Demonceaux, C. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794221 · Cited by 44 · see citing works
Dai, R., Das, S., & Brémond, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01281 · Cited by 45 · see citing works
When Pansharpening Meets Graph Convolution Network and Knowledge Distillation
Yan, K., Zhou, M., Liu, L., Xie, C., & Hong, D. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3168192 · Cited by 42 · see citing works
Hard pixel mining for depth privileged semantic segmentation
Gu, Z., Niu, L., Zhao, H., & Zhang, L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3035231 · Cited by 41 · see citing works
A Survey of Imbalanced Learning on Graphs: Problems, Techniques, and Future Directions
Liu, Z., Yuan, L., Chen, N., Wang, Q., Hooi, B., & He, B. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3549299 · Cited by 52 · see citing works
Learning Hand Articulations by Hallucinating Heat Distribution
Choi, C., Kim, S., & Karthik Ramani, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.337 · Cited by 40 · see citing works
Alehdaghi, M., Josi, A., Cruz, R. M. O., Shamsolmoali, P., & Granger, É. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3541969 · Cited by 39 · see citing works
Cross-View Policy Learning for Street Navigation
Li, A., Hu, H., Mirowski, P., & Farajtabar, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00819 · Cited by 37 · see citing works
LGD: Label-Guided Self-Distillation for Object Detection
Zhang, P., Kang, Z., Yang, T., Zhang, X., Zheng, N., & Sun, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20240 · Cited by 38 · see citing works
BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentation
Truong, T.-D., Duong, C. N., Le, N., Phung, S. L., Rainwater, C. E., & Luu, K. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00843 · Cited by 37 · see citing works
(DE)$^2$CO: Deep Depth Colorization
Carlucci, F. M., Russo, P., & Barbara Caputo, B. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2812225 · Cited by 41 · see citing works
Robust learning with imperfect privileged information
Li, X., Du, B., Xu, C., Zhang, Y., Zhang, L., & Tao, D. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103246 · Cited by 36 · see citing works
VALHALLA: Visual Hallucination for Machine Translation
Li, Y., Panda, R., Kim, Y., Chen, C., Feris, R., Cox, D. D., & Vasconcelos, N. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00515 · Cited by 60 · see citing works
Generative-Discriminative Complementary Learning
Xu, Y., Gong, M., Chen, J., Liu, T., Zhang, K., & Batmanghelich, K. N. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6126 · Cited by 38 · see citing works
Progressive Cross-modal Knowledge Distillation for Human Action Recognition
Ni, J., Ngu, A. H. H., & Yan, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548238 · Cited by 35 · see citing works
Enhancing Place Recognition Using Joint Intensity - Depth Analysis and Synthetic Data
Sizikova, E., Singh, V. K., Georgescu, B., Halber, M., Ma, K., & Chen, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_74 · Cited by 33 · see citing works
Cross-Modality Distillation: A Case for Conditional Generative Adversarial Networks
Roheda, S., Riggan, B. S., Krim, H., & Dai, L. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462082 · Cited by 33 · see citing works
Visible-Infrared Person Re-Identification Using Privileged Intermediate Information
Alehdaghi, M., Josi, A., Cruz, R. M. O., & Granger, É. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25072-9_48 · Cited by 33 · see citing works
Distilling Privileged Knowledge for Anomalous Event Detection From Weakly Labeled Videos
Liu, T., Lam, K., & Kong, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3263966 · Cited by 32 · see citing works
Distillation Multiple Choice Learning for Multimodal Action Recognition
Garcia, N. C., Bargal, S. A., Ablavsky, V., Morerio, P., Murino, V., & Sclaroff, S. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00280 · Cited by 32 · see citing works
Li, X., Lei, L., Sun, Y., & Kuang, G. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3048332 · Cited by 32 · see citing works
Sign Language Recognition via Skeleton-Aware Multi-Model EnsemblePreprint
Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., & Fu, Y. (2021) · arXiv.org · doi:10.48550/arxiv.2110.06161 · Cited by 60 · see citing works
Huang, H., Tseng, H.-Y., Lee, H.-Y., & Huang, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58610-2_35 · Cited by 31 · see citing works
He, J., Li, C., Ye, J., Qiao, Y., & Gu, L. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102491 · Cited by 29 · see citing works
CNN-Based RGB-D Salient Object Detection: Learn, Select, and Fuse
Chen, H., Li, Y., Deng, Y., & Lin, G. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01452-0 · Cited by 29 · see citing works
StructVPR: Distill Structural Knowledge with Weighting Samples for Visual Place Recognition
Shen, Y., Zhou, S., Fu, J., Wang, R., Chen, S., & Zheng, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01079 · Cited by 28 · see citing works
Multimodal Across Domains Gaze Target Detection
Tonini, F., Beyan, C., & Ricci, E. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556624 · Cited by 28 · see citing works
PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape RecognitionPreprint
You, H., Feng, Y., Ji, R., & Gao, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07659 · Cited by 28 · see citing works
Deep Learning-Based RGB-D Fusion for Multimodal Condition Assessment of Civil Infrastructure
Mondal, T. G., Jahanshahi, M. R., & Wu, Z. Y. (2023) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-5197 · Cited by 29 · see citing works
Audio-Visual Model Distillation Using Acoustic Images
Pérez, A. F., Sanguineti, V., Morerio, P., & Murino, V. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093307 · Cited by 27 · see citing works
Huang, J.-T., Lu, C.-L., Chang, P.-K., Huang, C.-I., Hsu, C.-C., Ewe, Z. L., Huang, P.-J., & Wang, H. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3062011 · Cited by 27 · see citing works
Cooperative Learning with Visual AttributesPreprint
Batra, T., & Parikh, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05512 · Cited by 28 · see citing works
Progressive modality cooperation for multi-modality domain adaptation
Zhang, W., Xu, D., Zhang, J., & Ouyang, W. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3052083 · Cited by 25 · see citing works
Bischke, B., Helber, P., Koenig, F., Borth, D., & Dengel, A. R. (2018) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi.2018.8516271 · Cited by 26 · see citing works
Pande, S., Banerjee, A., Kumar, S., Banerjee, B., & Chaudhuri, S. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00558 · Cited by 26 · see citing works
3D Hand Pose Estimation from RGB Using Privileged Learning with Depth Data
Yuan, S., Stenger, B., & Kim, T. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00348 · Cited by 26 · see citing works
Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution
Shi, W., Ye, M., & Du, B. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547873 · Cited by 24 · see citing works
Li, X., Lei, L., Zhang, C., & Kuang, G. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3176936 · Cited by 23 · see citing works
Robust Multimodal Learning via Representation Decoupling
Wei, S., Luo, Y., Wang, Y., & Luo, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72946-1_3 · Cited by 22 · see citing works
Piasco, N., Désiré Sidibé, A., Gouet-Brunet, V., & Demonceaux, C. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01363-6 · Cited by 22 · see citing works
Improving Unimodal Object Recognition with Multimodal Contrastive Learning
Meyer, J., Eitel, A., Brox, T., & Burgard, W. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341029 · Cited by 22 · see citing works
Learning to see colours: Biologically relevant virtual staining for adipocyte cell images
Wieslander, H., Gupta, A., Bergman, E., Hallström, E., & Harrison, P. J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0258546 · Cited by 22 · see citing works
Liu, X., Jin, F., Wang, S., Rui, J., Zuo, X., Yang, X., & Cheng, C. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3388604 · Cited by 21 · see citing works
Auxiliary Task Reweighting for Minimum-data LearningPreprint
Shi, B., Hoffman, J., Saenko, K., Darrell, T., & Xu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08244 · Cited by 21 · see citing works
Self-citation
- Authors
- Hoffman, J., Gupta, S., & Darrell, T.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 826-834
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2463402750
- Cited by
- 207
- References
- 42
- Citations by year
- 2026: 3 · 2025: 12 · 2024: 15 · 2023: 21 · 2022: 16 · 2021: 39 · 2020: 27 · 2019: 31 · 2018: 25 · 2017: 17 · 2016: 1
- FWCI
- 8.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- side information, mid-level features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
