The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
Open accessAbstract
This paper presents HITNet, a novel neural network architecture for real-time stereo matching. Contrary to many recent neural network approaches that operate on a full cost volume and rely on 3D convolutions, our approach does not explicitly build a volume and instead relies on a fast multi-resolution initialization step, differentiable 2D geometric propagation and warping mechanisms to infer disparity hypotheses. To achieve a high level of accuracy, our network not only geometrically reasons about disparities but also infers slanted plane hypotheses allowing to more accurately perform geometric warping and upsampling operations. Our architecture is inherently multi-resolution allowing the propagation of information across different levels. Multiple experiments prove the effectiveness of the proposed approach at a fraction of the computation required by state-of-the-art methods. At the time of writing, HITNet ranks 1st-3rdon all the metrics published on the ETH3D website for two view stereo, ranks 1ston most of the metrics amongst all the end-to-end learning approaches on Middlebury-v3, ranks 1ston the popular KITTI 2012 and 2015 benchmarks among the published methods faster than 100 ms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Stereo Processing by Semiglobal Matching and Mutual Information
Hirschmuller, H. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1166 · Cited by 4,195
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531330 · Cited by 2,880
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., & Brox, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.438 · Cited by 2,488
Pyramid Stereo Matching Network
Chang, J.-R., & Chen, Y.-S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00567 · Cited by 1,807
End-to-End Learning of Geometry and Context for Deep Stereo Regression
Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.17 · Cited by 1,470
Marr, D., & Poggio, T. (1976) · Science · doi:10.1126/science.968482 · Cited by 1,377
Efficient Belief Propagation for Early Vision
Felzenszwalb, P. F., & Huttenlocher, D. P. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7899-4 · Cited by 1,345
Optical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.291 · Cited by 1,344
High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth
Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11752-2_3 · Cited by 1,305
Mayer, N., Ilg, E., Cremers, D., Dosovitskiy, A., & Brox, T. (2016) · Cited by 1,214
Learning to compare image patches via convolutional neural networks
Zagoruyko, S., & Komodakis, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299064 · Cited by 1,140
Computing visual correspondence with occlusions using graph cuts
Kolmogorov, V., & Zabih, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937668 · Cited by 1,119
A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos
Schops, T., Schonberger, J. L., Galliani, S., Sattler, T., Schindler, K., Pollefeys, M., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.272 · Cited by 915
Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure
Klaus, A., Sormann, M., & Karner, K. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.1033 · Cited by 837
GA-Net: Guided Aggregation Net for End-To-End Stereo Matching
Zhang, F., Prisacariu, V., Yang, R., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00027 · Cited by 811
Computing the stereo matching cost with a convolutional neural network
Zbontar, J., & LeCun, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298767 · Cited by 796
Efficient Deep Learning for Stereo Matching
Luo, W., Schwing, A. G., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.614 · Cited by 741
PatchMatch Stereo - Stereo Matching with Slanted Support Windows
Bleyer, M., Rhemann, C., & Rother, C. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.14 · Cited by 736
A General and Adaptive Robust Loss Function
Barron, J. T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00446 · Cited by 689
Fast Cost-Volume Filtering for Visual Correspondence and Beyond
Hosni, A., Rhemann, C., Bleyer, M., Rother, C., & Gelautz, M. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.156 · Cited by 673
Cascade Residual Learning: A Two-Stage Convolutional Neural Network for Stereo Matching
Pang, J., Sun, W., Ren, J. S., Yang, C., & Yan, Q. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.108 · Cited by 491
Learning for Disparity Estimation Through Feature Constancy
Liang, Z., Feng, Y., Guo, Y., Liu, H., Chen, W., Qiao, L., Zhou, L., & Zhang, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00297 · Cited by 354
JOINT 3D ESTIMATION OF VEHICLES AND SCENE FLOW
Menze, M., Heipke, C., & Geiger, A. (2015) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprsannals-ii-3-w5-427-2015 · Cited by 341
DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch
Duggal, S., Wang, S., Ma, W.-C., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00448 · Cited by 317
Literature Survey on Stereo Vision Disparity Map Algorithms
Hamzah, R. A., & Ibrahim, H. (2015) · Journal of Sensors · doi:10.1155/2016/8742920 · Cited by 279
SGM-Nets: Semi-Global Matching with Neural Networks
Seki, A., & Pollefeys, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.703 · Cited by 269
Hierarchical Deep Stereo Matching on High-Resolution Images
Yang, G., Manela, J., Happold, M., & Ramanan, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00566 · Cited by 261
Locally Adaptive Support-Weight Approach for Visual Correspondence Search
Yoon, K.-J., & Kweon, I.-S. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.218 · Cited by 230
Hierarchical Neural Architecture Search for Deep Stereo Matching
Cheng, X., Zhong, Y., Harandi, M. T., Dai, Y., Chang, X., Drummond, T. W., Li, H., & Ge, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13501 · Cited by 222
End-to-End Learning of Geometry and Context for Deep Stereo Regression
Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04309 · Cited by 205
EdgeStereo: An Effective Multi-task Learning Network for Stereo Matching and Edge Detection
Song, X., Zhao, X., Fang, L., Hu, H., & Yu, Y. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01287-w · Cited by 199
Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning
Shaked, A., & Wolf, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.730 · Cited by 185
Lu, J.-B., Yang, H., Min, D., & Do, M. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.242 · Cited by 175
Continuous 3D Label Stereo Matching Using Local Expansion Moves
Taniai, T., Matsushita, Y., Sato, Y., & Naemura, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2766072 · Cited by 170
Detect, Replace, Refine: Deep Structured Prediction for Pixel Wise Labeling
Gidaris, S., & Komodakis, N. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.760 · Cited by 147
StereoDRNet: Dilated Residual StereoNet
Chabra, R., Straub, J., Sweeney, C., Newcombe, R. A., & Fuchs, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01206 · Cited by 130
AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
Song, X., Yang, G., Zhu, X., Zhou, H., Wang, Z., & Shi, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01019 · Cited by 106
HyperDepth: Learning Depth from Structured Light without Matching
Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Escolano, S. O., Kim, D., & Izadi, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.587 · Cited by 103
Self-citationLearning to be a depth camera for close-range human capture and interaction
Fanello, S., Keskin, C., Izadi, S., Kohli, P., Kim, D., Sweeney, D., Criminisi, A., Shotton, J., Kang, S. B., & Paek, T. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601223 · Cited by 83
Self-citationSIMPLE BUT EFFECTIVE TREE STRUCTURES FOR DYNAMIC PROGRAMMING-BASED STEREO MATCHING
Bleyer, M., & Gelautz, M. (2008) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0001072904150422 · Cited by 80
SPM-BP: Sped-Up PatchMatch Belief Propagation for Continuous MRFs
Li, Y., Min, D., Brown, M. S., Do, M. N., & Lu, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.456 · Cited by 79
UltraStereo: Efficient Learning-Based Matching for Active Stereo Systems
Fanello, S., Valentin, J., Rhemann, C., Kowdle, A., Tankovich, V., Davidson, P., & Izadi, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.692 · Cited by 68
Self-citationThe need 4 speed in real-time dense visual tracking
Kowdle, A., Rhemann, C., Fanello, S., Tagliasacchi, A., Taylor, J., Davidson, P., Dou, M., Guo, K., Keskin, C., Khamis, S., et al. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275062 · Cited by 42
Self-citationFast Deep Stereo with 2D Convolutional Processing of Cost Signatures
Yee, K., & Chakrabarti, A. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093273 · Cited by 38
Real-Time Stereo Matching Network with High Accuracy
Lee, H., & Shin, Y.-H. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803514 · Cited by 32
Low Compute and Fully Parallel Computer Vision with HashMatch
Fanello, S., Valentin, J., Kowdle, A., Rhemann, C., Tankovich, V., Ciliberto, C., Davidson, P., & Izadi, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.418 · Cited by 24
Self-citationSuperpixel alpha-expansion and normal adjustment for stereo matching
Ji, P., Li, J., Li, H., & Liu, X. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103238 · Cited by 15
CrossPatch-Based Rolling Label Expansion for Dense Stereo Matching
Xu, H., Chen, X., Liang, H., et al. (2020) · IEEE Access · doi:10.1109/access.2020.2985106 · Cited by 14
Cited by
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching
Lipson, L., Teed, Z., & Deng, J. (2021) · 2021 International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00032 · Cited by 536
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
Shen, Z., Dai, Y., & Rao, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01369 · Cited by 351
Iterative Geometry Encoding Volume for Stereo Matching
Xu, G., Wang, X., Ding, X., & Yang, X. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02099 · Cited by 349
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation
Li, J., Wang, P., Xiong, P., Cai, T., Yan, Z., Yang, L., Liu, J., Fan, H., & Liu, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01578 · Cited by 345
Attention Concatenation Volume for Accurate and Efficient Stereo Matching
Xu, G., Cheng, J., Guo, P., & Yang, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01264 · Cited by 291
Metadata
- Authors
- Tankovich, V., Hane, C., Zhang, Y., Kowdle, A., Fanello, S., & Bouaziz, S.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 14357-14367
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3045038724
- Cited by
- 309
- References
- 56
- FWCI
- 19.32
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.