The DOI matches its Crossref record; 41 of the 44 references it cites were found in a registry or an index.
FlowNet: Learning Optical Flow with Convolutional Networks
Abstract
Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks CNNs succeeded at. In this paper we construct CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations. Since existing ground truth data sets are not sufficiently large to train a CNN, we generate a large synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“On the other hand, Scene Flow model is trained in highly dynamic synthetic environments, i.e. able to estimate large flow magnitude caused by moving objects.”
From DF-VO: What Should Be Learnt for Visual Odometry? (2021) · cites this paper as Dosovitskiy et al. 2015
In the citing paper
“LiteFlowNet shows good generalization ability. LiteFlowNet trained on a synthetic dataset (Scene Flow(Dosovitskiy et al. 2015)) can generalize well in realworld scenarios, though sometimes artifacts present in some regions.”
From DF-VO: What Should Be Learnt for Visual Odometry? (2021) · cites this paper as Dosovitskiy et al. 2015
In the citing paper
“Uzunova et al. (78) designed a network for the registration between 2D brain MRI and 2D cardiac MRI. Their ground truth is generated utilizing statistical appearance models (SAMs). They adapted FlowNet (79) architecture and obtained outperforming results.”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [79]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,634
Self-citationLearning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 1×
“The Middlebury dataset [2] contains only 8 image pairs for training, with ground truth flows generated using four different techniques.”
MethodA Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Self-citationAdaptive deconvolutional networks for mid and high level feature learning
Zeiler, M. D., Taylor, G. W., & Fergus, R. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126474 · Cited by 1,293
(2012) · neural information processing systems · Cited by 1,291
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.175 · Cited by 1,057
IM2GPS: estimating geographic information from a single image
Hays, J. H., & Efros, A. A. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587784 · Cited by 875
Computing the stereo matching cost with a convolutional neural network
Žbontar, J., & LeCun, Y. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298767 · Cited by 838
EpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298720 · Cited by 757
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_11 · Cited by 652
Learning to generate chairs with convolutional neural networks
Dosovitskiy, A., Springenberg, J. T., & Brox, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298761 · Cited by 640
Self-citationSeeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models
Aubry, M., Maturana, D., Efros, A. A., Russell, B. C., & Šivic, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.487 · Cited by 471
Sun, D., Roth, S., Lewis, J. P., & Black, M. J. (2008) · European Conference on Computer Vision · doi:10.1007/978-3-540-88690-7_7 · Cited by 303
Dense estimation and object-based segmentation of the optical flow with robust techniques
Mémin, É., & Pérez, P. (1998) · IEEE Transactions on Image Processing · doi:10.1109/83.668027 · Cited by 291
$$N^4$$-Fields: Neural Network Nearest Neighbor Fields for Image Transforms
Ganin, Y., & Lempitsky, V. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16808-1_36 · Cited by 262
Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFTPreprint
Fischer, P., Dosovitskiy, A., & Brox, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.5769 · Cited by 251
Self-citationStructure- and motion-adaptive regularization for high accuracy optic flow
Wedel, A., Cremers, D., Pock, T., & Bischof, H. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459375 · Cited by 178
Self-citationLearning parameterized models of image motion
Black, M. J., Yacoob, Y., Jepson, A. D., & Fleet, D. J. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1997.609381 · Cited by 126
Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow
Bao, L., Yang, Q., & Jin, H. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2359374 · Cited by 109
Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow
Bao, L., Yang, Q., & Jin, H. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.452 · Cited by 90
Efficient Closed-Form Solution to Generalized Boundary Detection
Leordeanu, M., Sukthankar, R., & Sminchisescu, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_37 · Cited by 85
Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames
Kennedy, R., & Taylor, C. J. (2015) · Energy Minimization Methods in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-14612-6_27 · Cited by 84
Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Leordeanu, M., Zanfir, A., & Sminchisescu, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.216 · Cited by 68
Unsupervised learning of depth and motionPreprint
Konda, K., & Memisevic, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.3429 · Cited by 37
Learning the Local Statistics of Optical Flow
Rosenbaum, D., Zoran, D., & Weiss, Y. (2013) · Cited by 18
Flagged first, then by citations
3 of the 44 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.179 · Cited by 3,177
Self-citation
Metadata
- Authors
- Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T.
- Affiliations
- University of Freiburg (DE)
- Technical University of Munich (DE)
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2758-2766
- OpenAlex
- W764651262
- Cited by
- 4,313
- References
- 44
- Citations by year
- 2026: 182 · 2025: 342 · 2024: 462 · 2023: 550 · 2022: 559 · 2021: 647 · 2020: 579 · 2019: 469 · 2018: 278 · 2017: 184 · 2016: 57 · 2015: 3
- FWCI
- 71.36
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Funders
- Deutsche Telekom Stiftung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Keywords
- optical flow, convolutional neural networks, supervised learning, synthetic training data, KITTI, video frame rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
