The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
DeepFlow: Large Displacement Optical Flow with Deep Matching
Open accessAbstract
Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox and Malik, our approach, termed Deep Flow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable asset for integration into an energy minimization framework for optical flow estimation. Deep Flow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Computer vision: algorithms and applications
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5140 · Cited by 3,911
Lucas-Kanade 20 Years On: A Unifying Framework
Baker, S., & Matthews, I. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011205.11775.fd · Cited by 3,155
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Self-citationHigh Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 2×
“The matching algorithm builds upon a multi-stage architecture with about 6 layers (depending on the image size), inter-leaving convolutions and max-pooling, a construction akin to deep convolutional nets [15].”
A Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Self-citationSIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
Research works printed but not linked: 2
- revaud, 2013, Deep Matching and Its Application to Large Displacement Optical Flow
- 10.1109/ICPR.1998.711195 · DOI
Cited by
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
(2020) · doi:10.1007/978-3-030-58536-5_24 · Cited by 2,824
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2897538 · Cited by 2,174
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876
Optical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.291 · Cited by 1,344
Neural Style Transfer: A Review
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., & Song, M. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2921336 · Cited by 856
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Jiang, H., Sun, D., Jampani, V., Yang, M.-H., Learned-Miller, E., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00938 · Cited by 829
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
Hui, T.-W., Tang, X., & Loy, C. C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00936 · Cited by 817
Video Frame Interpolation via Adaptive Separable Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.37 · Cited by 725
Flow-Guided Feature Aggregation for Video Object Detection
Zhu, X., Wang, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.52 · Cited by 729
EpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298720 · Cited by 697
Self-citationDeep learning in video multi-object tracking: A survey
Ciaparrone, G., Luque Sánchez, F., Tabik, S., Troiano, L., Tagliaferri, R., & Herrera, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.023 · Cited by 665
Quicksilver: Fast predictive image registration – A deep learning approach
Yang, X., Kwitt, R., Styner, M., & Niethammer, M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.008 · Cited by 630
“This is unsurprising as, similar to existing optical flow approaches [12, 13], directly predicting”
BackgroundMethodResultInfluentialDeep Feature Flow for Video Recognition
Zhu, X., Xiong, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.441 · Cited by 653
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
An Unsupervised Learning Model for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Dalca, A. V., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00964 · Cited by 599
GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence
Bian, J., Lin, W.-Y., Matsushita, Y., Yeung, S.-K., Nguyen, T.-D., & Cheng, M.-M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.302 · Cited by 594
Multiple People Tracking by Lifted Multicut and Person Re-identification
Tang, S., Andriluka, M., Andres, B., & Schiele, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.394 · Cited by 543
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · arXiv · doi:10.48550/arxiv.1807.05520 · Cited by 533
Learning a Neural Solver for Multiple Object Tracking
Braso, G., & Leal-Taixe, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00628 · Cited by 504
Flowing ConvNets for Human Pose Estimation in Videos
Pfister, T., Charles, J., & Zisserman, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.222 · Cited by 500
Fusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
Video Frame Interpolation via Adaptive Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.244 · Cited by 499
Convolutional Neural Network Architecture for Geometric Matching
Rocco, I., Arandjelovic, R., & Sivic, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.12 · Cited by 496
Visual attribute transfer through deep image analogy
Liao, J., Yao, Y., Yuan, L., Hua, G., & Kang, S. B. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073683 · Cited by 487
DF-Net: Unsupervised Joint Learning of Depth and Flow Using Cross-Task Consistency
Zou, Y., Luo, Z., & Huang, J.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_3 · Cited by 469
SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
Cheng, J., Tsai, Y.-H., Wang, S., & Yang, M.-H. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.81 · Cited by 446
Optical flow modeling and computation: A survey
Fortun, D., Bouthemy, P., & Kervrann, C. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.02.008 · Cited by 431
A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training
Cui, R., Liu, H., & Zhang, C. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2889563 · Cited by 421
GMFlow: Learning Optical Flow via Global Matching
Xu, H., Zhang, J., Cai, J., Rezatofighi, H., & Tao, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00795 · Cited by 418
Context-Aware Synthesis for Video Frame Interpolation
Niklaus, S., & Liu, F. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00183 · Cited by 411
Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model
Nguyen, T., Chen, S. W., Shivakumar, S. S., Taylor, C. J., & Kumar, V. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2809549 · Cited by 374
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Yu, J. J., Harley, A. W., & Derpanis, K. G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_1 · Cited by 368
Bao, W., Lai, W.-S., Zhang, X., Gao, Z., & Yang, M.-H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2941941 · Cited by 362
Dynamic FAUST: Registering Human Bodies in Motion
Bogo, F., Romero, J., Pons-Moll, G., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.591 · Cited by 358
Fast Optical Flow Using Dense Inverse Search
Kroeger, T., Timofte, R., Dai, D., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_29 · Cited by 351
Learning Dense Correspondence via 3D-Guided Cycle Consistency
Zhou, T., Krahenbuhl, P., Aubry, M., Huang, Q., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.20 · Cited by 350
No More Discrimination: Cross City Adaptation of Road Scene Segmenters
Chen, Y.-H., Chen, W.-Y., Chen, Y.-T., Tsai, B.-C., Wang, Y.-C. F., & Sun, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.220 · Cited by 331
SelFlow: Self-Supervised Learning of Optical Flow
Liu, P., Lyu, M., King, I., & Xu, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00470 · Cited by 323
A novel application of deep learning for single-lead ECG classification
Mathews, S. M., Kambhamettu, C., & Barner, K. E. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.05.013 · Cited by 322
Occlusion Aware Unsupervised Learning of Optical Flow
Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., & Xu, W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00513 · Cited by 320
Video Segmentation via Object Flow
Tsai, Y.-H., Yang, M.-H., & Black, M. J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.423 · Cited by 318
Deep learning in multi-object detection and tracking: state of the art
Pal, S. K., Pramanik, A., Maiti, J., & Mitra, P. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02293-7 · Cited by 298
Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation
Yamaguchi, K., McAllester, D., & Urtasun, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_49 · Cited by 297
Unsupervised deep learning for optical flow estimation
Ren, Z., Yan, J., Ni, B., Liu, B., Yang, X., & Zha, H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10723 · Cited by 296
Deep exemplar-based colorization
He, M., Chen, D., Liao, J., Sander, P. V., & Yuan, L. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201365 · Cited by 293
Coherent Online Video Style Transfer
Chen, D., Liao, J., Yuan, L., Yu, N., & Hua, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.126 · Cited by 281
FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation
Tassano, M., Delon, J., & Veit, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00143 · Cited by 276
AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation
Lee, H., Kim, T., Chung, T.-Y., Pak, D., Ban, Y., & Lee, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00536 · Cited by 270
Unsupervised Pre-Training of Image Features on Non-Curated Data
Caron, M., Bojanowski, P., Mairal, J., & Joulin, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00305 · Cited by 258
DeepMatching: Hierarchical Deformable Dense Matching
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0908-3 · Cited by 267
Self-citationSegmentation-Aware Convolutional Networks Using Local Attention Masks
Harley, A. W., Derpanis, K. G., & Kokkinos, I. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.539 · Cited by 142
MoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation
Jain, A., Tompson, J., LeCun, Y., & Bregler, C. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16808-1_21 · Cited by 142
Fast Predictive Image Registration
Yang, X., Kwitt, R., & Niethammer, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46976-8_6 · Cited by 127
Merkle, N., Luo, W., Auer, S., Müller, R., & Urtasun, R. (2017) · Remote Sensing · doi:10.3390/rs9060586 · Cited by 120
Dilated FCN for Multi-Agent 2D/3D Medical Image Registration
Miao, S., Piat, S., Fischer, P., Tuysuzoglu, A., Mewes, P., Mansi, T., & Liao, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11576 · Cited by 113
Training cnns for image registration from few samples with model-based data augmentation
Uzunova, H., Wilms, M., Handels, H., & Ehrhardt, J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_26 · Cited by 110
Optical Flow Estimation with Deep Learning, a Survey on Recent Advances
Savian, S., Elahi, M., & Tillo, T. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-32583-1_12 · Cited by 29
Recaptured screen image demoiréing in raw domain
Yue, H., Cheng, Y., Mao, Y., Cao, C., & Yang, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3198333 · Cited by 26
Siamese Encoding and Alignment by Multiscale Learning with Self-Supervision
Mitchell, E., Keselj, S., Popovych, S., Buniatyan, D., & Seung, H. S. (2019) · arXiv · doi:10.48550/arxiv.1904.02643
Metadata
- Authors
- Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C.
- Venue
- 2013 IEEE International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 1385-1392
- OpenAlex
- W2113221323
- Cited by
- 997
- References
- 33
- FWCI
- 35.84
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.