CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow
Open accessAbstract
Despite impressive performance for high-level downstream tasks, self-supervised pre-training methods have not yet fully delivered on dense geometric vision tasks such as stereo matching or optical flow. The application of self-supervised concepts, such as instance discrimination or masked image modeling, to geometric tasks is an active area of research. In this work, we build on the recent cross-view completion framework, a variation of masked image modeling that leverages a second view from the same scene which makes it well suited for binocular downstream tasks. The applicability of this concept has so far been limited in at least two ways: (a) by the difficulty of collecting real-world image pairs – in practice only synthetic data have been used – and (b) by the lack of generalization of vanilla transformers to dense downstream tasks for which relative position is more meaningful than absolute position. We explore three avenues of improvement. First, we introduce a method to collect suitable real-world image pairs at large scale. Second, we experiment with relative positional embeddings and show that they enable vision transformers to perform substantially better. Third, we scale up vision transformer based cross-completion architectures, which is made possible by the use of large amounts of data. With these improvements, we show for the first time that state-of-the-art results on stereo matching and optical flow can be reached without using any classical task-specific techniques like correlation volume, iterative estimation, image warping or multi-scale reasoning, thus paving the way towards universal vision models.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The robustness of SenseAct-NN methods to image noise speaks to the robustness of visual features from cross-view consistent representations from the CroCo-v2 encoder used for the SenseAct-NN Skill Chain baseline and CLIP used for the monolithic policy.
“This speaks to the robustness of the visual features from cross-view consistent representations from the CroCo-v2 encoder [34] used for the SenseAct-NN Skill Chain baseline and CLIP used for the monolithic policy.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [34]
Cites
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · see citing works
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 3× · see citing works
“The success of instance discrimination [12, 13, 16, 28, 30] has drawn a lot of attention to self-supervised learning in computer vision [37].”
BackgroundMethodFlownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · see citing works
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00781 · Cited by 3,657 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,361 · see citing works
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
(2020) · doi:10.1007/978-3-030-58536-5_24 · Cited by 2,824 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,741 · see citing works
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679 · see citing works
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,634 · see citing works
Digging Into Self-Supervised Monocular Depth Estimation
Godard, C., Aodha, O. M., Firman, M., & Brostow, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00393 · Cited by 2,561 · see citing works
Self-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2074 · Cited by 2,505 · see citing works
Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., & Brox, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.438 · Cited by 2,488 · see citing works
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1× · see citing works
“image color [97], patch ordering [57] or image orientation [25]) and trying to recover it.”
BackgroundVision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · cited 2× · see citing works
“We finetune our pre-trained model, referred to as CroCo v2, with this improved cross-view completion scheme on stereo matching and optical flow using a Dense Prediction Transformer (DPT) [59] head.”
MethodObject scene flow for autonomous vehicles
Menze, M., & Geiger, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298925 · Cited by 2,270 · see citing works
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · see citing works
RoFormer: Enhanced transformer with rotary position embedding
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., & Liu, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2023.127063 · Cited by 1,957 · see citing works
Pyramid Stereo Matching Network
Chang, J.-R., & Chen, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00567 · Cited by 1,882 · see citing works
A Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842 · see citing works
End-to-End Learning of Geometry and Context for Deep Stereo Regression
Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.17 · Cited by 1,531 · see citing works
High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth
Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_3 · Cited by 1,377 · see citing works
SimMIM: a Simple Framework for Masked Image Modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., & Hu, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00943 · Cited by 1,348 · see citing works
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q. V., Vaswani, A., & Shlens, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,320 · see citing works
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and LanguagePreprint
Baevski, A., Hsu, W.-N., Xu, Q., Babu, A., Gu, J., & Auli, M. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2202.03555 · Cited by 1,303 · see citing works
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · see citing works
MegaDepth: Learning Single-View Depth Prediction from Internet Photos
Li, Z., & Snavely, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00218 · Cited by 1,164 · see citing works
A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos
Schops, T., Schonberger, J. L., Galliani, S., Sattler, T., Schindler, K., Pollefeys, M., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.272 · Cited by 915 · see citing works
Sun, D., Roth, S., & Black, M. J. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0644-x · Cited by 698 · see citing works
Dense Contrastive Learning for Self-Supervised Visual Pre-Training
Wang, X., Zhang, R., Shen, C., Kong, T., & Li, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00304 · Cited by 660 · see citing works
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587 · see citing works
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching
Lipson, L., Teed, Z., & Deng, J. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00032 · Cited by 570 · see citing works
TartanAir: A Dataset to Push the Limits of Visual SLAM
Wang, W., Zhu, D., Wang, X., Hu, Y., Qiu, Y., Wang, C., Hu, Y., Kapoor, A., & Scherer, S. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341801 · Cited by 456 · see citing works
GMFlow: Learning Optical Flow via Global Matching
Xu, H., Zhang, J., Cai, J., Rezatofighi, H., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00795 · Cited by 437 · see citing works
StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction
Khamis, S., Fanello, S., Rhemann, C., Kowdle, A., Valentin, J., & Izadi, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_35 · Cited by 418 · see citing works
Real-World Robot Learning with Masked Visual Pre-trainingPreprint
Radosavovic, I., Xiao, T.-T., James, S., Abbeel, P., Malik, J., & Darrell, T. (2022) · Conference on Robot Learning · doi:10.48550/arxiv.2210.03109 · Cited by 389 · see citing works
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Yu, J. J., Harley, A. W., & Derpanis, K. G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_1 · Cited by 377 · see citing works
FlowFormer: A Transformer Architecture for Optical Flow
Huang, Z., Shi, X., Zhang, C., Wang, Q., Cheung, K. C., Qin, H., Dai, J., & Li, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_40 · Cited by 373 · see citing works
Xie, Z., Lin, Y., Zhang, Z., Cao, Y., Lin, S., & Hu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01641 · Cited by 370 · see citing works
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation
Li, J., Wang, P., Xiong, P., Cai, T., Yan, Z., Yang, L., Liu, J., Fan, H., & Liu, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01578 · Cited by 367 · see citing works
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation
Laga, H., Jospin, L. V., Boussaïd, F., & Bennamoun, M. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3032602 · Cited by 340 · see citing works
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
Tankovich, V., Häne, C., Zhang, Y., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01413 · Cited by 311 · see citing works
Attention Concatenation Volume for Accurate and Efficient Stereo Matching
Xu, G., Cheng, J., Guo, P., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01264 · Cited by 291 · see citing works
DeepMatching: Hierarchical Deformable Dense Matching
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0908-3 · Cited by 274 · see citing works
Self-citationMultiMAE: Multi-modal Multi-task Masked Autoencoders
Bachmann, R., Mizrahi, D., Atanov, A., & Zamir, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_20 · Cited by 268 · see citing works
Unifying Flow, Stereo and Depth Estimation
Xu, H., Zhang, J., Cai, J., Rezatofighi, H., Yu, F., Tao, D., & Geiger, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298645 · Cited by 255 · see citing works
Hierarchical Discrete Distribution Decomposition for Match Density Estimation
Yin, Z., Darrell, T., & Yu, F. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00620 · Cited by 253 · see citing works
Masked Siamese Networks for Label-Efficient Learning
Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F., Vincent, P., Joulin, A., Rabbat, M., & Ballas, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19821-2_26 · Cited by 234 · see citing works
Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2894353 · Cited by 200 · see citing works
Optical flow and scene flow estimation: A survey
Zhai, M., Xiang, X., Lv, N., & Kong, X. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107861 · Cited by 198 · see citing works
What to Hide from Your Students: Attention-Guided Masked Image Modeling
Kakogeorgiou, I., Gidaris, S., Psomas, B., Avrithis, Y., Bursuc, A., Karantzalos, K., & Komodakis, N. (2022) · European Conference on Computer Vision · doi:10.1007/978-3-031-20056-4_18 · Cited by 183 · see citing works
Unsupervised Learning of Stereo Matching
Zhou, C., Zhang, H., Shen, X., & Jia, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.174 · Cited by 163 · see citing works
CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow
Sui, X., Li, S., Geng, X., Wu, Y., Xu, X., Liu, Y., Goh, R. S. M., & Zhu, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01708 · Cited by 122 · see citing works
Multi-Level Context Ultra-Aggregation for Stereo Matching
Nie, G.-Y., Cheng, M., Liu, Y., Liang, Z., Fan, D.-P., Liu, Y., & Wang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00340 · Cited by 117 · see citing works
Separable Flow: Learning Motion Cost Volumes for Optical Flow Estimation
Zhang, F., Woodford, O. J., Prisacariu, V. A., & Torr, P. H. S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01063 · Cited by 110 · see citing works
PCW-Net: Pyramid Combination and Warping Cost Volume for Stereo Matching
Shen, Z., Dai, Y., Song, X., Rao, Z., Zhou, D., & Zhang, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_17 · Cited by 109 · see citing works
Global Matching with Overlapping Attention for Optical Flow Estimation
Zhao, S., Zhao, L., Zhang, Z., Zhou, E., & Metaxas, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01707 · Cited by 101 · see citing works
AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
Song, X., Yang, G., Zhu, X., Zhou, H., Wang, Z., & Shi, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01019 · Cited by 91 · see citing works
Learning Optical Flow from a Few Matches
Jiang, S., Lu, Y., Li, H., & Hartley, R. I. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01632 · Cited by 89 · see citing works
Left-Right Comparative Recurrent Model for Stereo Matching
Jie, Z., Wang, P., Ling, Y., Zhao, B., Wei, Y., Feng, J., & Liu, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00404 · Cited by 85 · see citing works
SiT: Self-supervised vIsion Transformer
Atito, S., Awais, M., & Kittler, J. (2021) · arXiv · doi:10.48550/arxiv.2104.03602 · Cited by 75 · see citing works
Generic 3D Representation via Pose Estimation and Matching
Zamir, A., Wekel, T., Agrawal, P., Wei, C., Malik, J., & Savarese, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_33 · Cited by 74 · see citing works
Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo
Mehl, L., Schmalfuß, J., Jahedi, A., Nalivayko, Y., & Bruhn, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00482 · Cited by 71 · see citing works
Large-scale Localization Datasets in Crowded Indoor Spaces
Lee, D., Ryu, S., Yeon, S., Lee, Y., Kim, D.-H., Han, C., Cabon, Y., Weinzaepfel, P., Guérin, N., Csurka, G., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00324 · Cited by 56 · see citing works
Self-citationDIP: Deep Inverse Patchmatch for High-Resolution Optical Flow
Zheng, Z., Nie, N., Ling, Z., Xiong, P., Liu, J., Wang, H., & Li, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00872 · Cited by 51 · see citing works
Open Challenges in Deep Stereo: the Booster Dataset
Ramirez, P. Z., Tosi, F., Poggi, M., Salti, S., Mattoccia, S., & Stefano, L. D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02049 · Cited by 49 · see citing works
Deep Equilibrium Optical Flow Estimation
Bai, S., Geng, Z., Savani, Y., & Kolter, J. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00070 · Cited by 49 · see citing works
High Resolution Multi-Scale RAFT (Robust Vision Challenge 2022)Preprint
Jahedi, A., Luz, M., Mehl, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.16900 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation
Khanna, M., Ramrakhya, R., Chhablani, G., Yenamandra, S., Gervet, T., Chang, M., Kira, Z., Chaplot, D. S., Batra, D., & Mottaghi, R. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01549 · Cited by 23 · see citing works
“For example, one-hot encodings or detections for object goals, SuperGLUE-based keypoint matching [18, 33] or cross-view consistent encoders [34] for image goals, or linear projection for 3D coordinates [6].”
BackgroundMethodInfluential
- Authors
- Weinzaepfel, P., Lucas, T., Leroy, V., Cabon, Y., Arora, V., Brégier, R., Csurka, G., Antsfeld, L., Chidlovskii, B., & Revaud, J.
- Affiliations
- Naver (South Korea) (KR)
- ORCID
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 17923-17934
- OpenAlex
- W4390872625
- Cited by
- 110
- References
- 78
- Citations by year
- 2026: 26 · 2025: 51 · 2024: 33
- FWCI
- 11.75
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- stereo matching, optical flow, self-supervised pre-training, masked image modeling, vision transformer, binocular vision
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
