The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
DeepFlow: Large Displacement Optical Flow with Deep Matching
Open accessAbstract
Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox and Malik, our approach, termed Deep Flow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable asset for integration into an energy minimization framework for optical flow estimation. Deep Flow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Computer vision: algorithms and applications
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5140 · Cited by 3,911
Lucas-Kanade 20 Years On: A Unifying Framework
Baker, S., & Matthews, I. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011205.11775.fd · Cited by 3,155
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Self-citationHigh Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 2×
“The matching algorithm builds upon a multi-stage architecture with about 6 layers (depending on the image size), inter-leaving convolutions and max-pooling, a construction akin to deep convolutional nets [15].”
A Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Self-citationSIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields
Black, M. J., & Anandan, P. (1996) · Computer Vision and Image Understanding · doi:10.1006/cviu.1996.0006 · Cited by 1,570
Gorelick, L., Blank, M., Shechtman, E., Irani, M., & Basri, R. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70711 · Cited by 1,430
Secrets of optical flow estimation and their principles
Sun, D., Roth, S., & Black, M. J. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539939 · Cited by 1,389
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Preattentive texture discrimination with early vision mechanisms
Malik, J., & Perona, P. (1990) · Journal of the Optical Society of America A · doi:10.1364/josaa.7.000923 · Cited by 960
The Generalized PatchMatch Correspondence Algorithm
Barnes, C., Shechtman, E., Goldman, D. B., & Finkelstein, A. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15558-1_3 · Cited by 530
Motion Detail Preserving Optical Flow Estimation
Xu, L., Jia, J., & Matsushita, Y. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.236 · Cited by 493
A fast local descriptor for dense matching
Tola, E., Lepetit, V., & Fua, P. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587673 · Cited by 480
Laptev, I., & Perez, P. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409105 · Cited by 416
Highly Accurate Optic Flow Computation with Theoretically Justified Warping
Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-005-3960-y · Cited by 409
Anisotropic Huber-L1 Optical Flow
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., & Bischof, H. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.108 · Cited by 354
Deformation Models for Image Recognition
Keysers, D., Deselaers, T., Gollan, C., & Ney, H. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1153 · Cited by 255
Deformable Spatial Pyramid Matching for Fast Dense Correspondences
Kim, J., Liu, C., Sha, F., & Grauman, K. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.299 · Cited by 243
Trajectons: Action recognition through the motion analysis of tracked features
Matikainen, P., Hebert, M., & Sukthankar, R. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457659 · Cited by 224
Zimmer, H., Bruhn, A., & Weickert, J. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0422-6 · Cited by 215
Towards ultimate motion estimation: combining highest accuracy with real-time performance
Bruhn, A., & Weickert, J. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.240 · Cited by 138
Large displacement optical flow computation withoutwarping
Steinbrücker, F., Pock, T., & Cremers, D. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459364 · Cited by 137
Soranus, Q. V. (1993) · Oxford University Press eBooks · doi:10.1093/oseo/instance.00076674 · Cited by 118
An Evaluation of Data Costs for Optical Flow
Vogel, C., Roth, S., & Konrad Schindler, B. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40602-7_37 · Cited by 89
Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Leordeanu, M., Zanfir, A., & Sminchisescu, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.216 · Cited by 80
A Feature-based Approach for Dense Segmentation and Estimation of Large Disparity Motion
Wills, J., Agarwal, S., & Belongie, S. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-6660-3 · Cited by 40
Adaptive Integration of Feature Matches into Variational Optical Flow Methods
Stoll, M., Volz, S., & Bruhn, A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37431-9_1 · Cited by 27
A hierarchical non-parametric method for capturing non-rigid deformations
Ecker, A., & Ullman, S. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2006.10.006 · Cited by 6
Research works printed but not linked: 2
- revaud, 2013, Deep Matching and Its Application to Large Displacement Optical Flow
- 10.1109/ICPR.1998.711195 · doi:10.1109/icpr.1998.711195
Cited by
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
(2020) · doi:10.1007/978-3-030-58536-5_24 · Cited by 2,824
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
Metadata
- Authors
- Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C.
- Venue
- 2013 IEEE International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 1385-1392
- OpenAlex
- W2113221323
- Cited by
- 997
- References
- 33
- FWCI
- 35.84
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.