The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Abstract
The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a subnetwork specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1981) · Cited by 11,963
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“The strategy of training machine learning models on a series of gradually increasing tasks is known as curriculum learning [5].”
BackgroundUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
Self-citationHigh Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
Self-citationMayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., & Brox, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.438 · Cited by 2,488
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 1×
“The idea dates back at least to Elman [11], who showed that both the evolution of tasks and the network architectures can be beneficial in the language processing scenario.”
BackgroundChen, Y., & Pock, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2596743 · Cited by 1,344
Research works printed but not linked: 2
- tran, 2016, Deep end2end voxe12voxel prediction (the 3rd workshop on deep learning in computer vision), IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- savva, 2015, Semantically-Enriched 3D Models for Common-sense Knowledge (Workshop on Functionality, Physics, Intentionality and Causality), IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Cited by
Items 101–120 of 120
Dilated FCN for Multi-Agent 2D/3D Medical Image Registration
Miao, S., Piat, S., Fischer, P., Tuysuzoglu, A., Mewes, P., Mansi, T., & Liao, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11576 · Cited by 113
Recognizing Facial Expressions Using a Shallow Convolutional Neural Network
Miao, S., Xu, H., Han, Z., & Zhu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2921220 · Cited by 110
Visual SLAM for Automated Driving: Exploring the Applications of Deep Learning
Milz, S., Arbeiter, G., Witt, C., Abdallah, B., & Yogamani, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00062 · Cited by 107
Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Wang, C., Lucey, S., Perazzi, F., & Wang, O. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00046 · Cited by 103
Morales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
Online Model Distillation for Efficient Video Inference
Mullapudi, R. T., Chen, S., Zhang, K., Ramanan, D., & Fatahalian, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00367 · Cited by 96
Tracking cell lineages in 3D by incremental deep learning
Sugawara, K., Çevrim, Ç., & Averof, M. (2022) · eLife · doi:10.7554/elife.69380 · Cited by 85
SMSnet: Semantic motion segmentation using deep convolutional neural networks
Vertens, J., Valada, A., & Burgard, W. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202211 · Cited by 69
MemFlow: Optical Flow Estimation and Prediction with Memory
Dong, Q., & Fu, Y. (2024) · arXiv · doi:10.48550/arxiv.2404.04808 · Cited by 49
Approaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data
Kwon, H., Wang, B., Abowd, G. D., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3478096 · Cited by 48
Deep learning in ultrasound elastography imaging: A review
Li, H., Bhatt, M., Qu, Z., Zhang, S., Hartel, M. C., Khademhosseini, A., & Cloutier, G. (2022) · Medical Physics · doi:10.1002/mp.15856 · Cited by 47
Motion and Depth Augmented Semantic Segmentation for Autonomous Navigation
Rashed, H., El Sallab, A., Yogamani, S., & ElHelw, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00049 · Cited by 38
High Speed and High Dynamic Range Video with an Event Camera
Rebecq, H., Ranftl, R., Koltun, V., & Scaramuzza, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2963386 · Cited by 50
“DAVIS frames, whichwe obtainwith FlowNet2 [40].”
DF-VO: What Should Be Learnt for Visual Odometry?
Zhan, H., Weerasekera, C. S., Bian, J.-W., Garg, R., & Reid, I. (2021) · arXiv · doi:10.48550/arxiv.2103.00933 · Cited by 31
Optical Flow Estimation with Deep Learning, a Survey on Recent Advances
Savian, S., Elahi, M., & Tillo, T. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-32583-1_12 · Cited by 29
Video demoiréing with relation-based temporal consistency
Dai, P., Yu, X., Ma, L., Zhang, B., Li, J., Li, W., Shen, J., & Qi, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01710 · Cited by 26
Attention-Based Deep Metric Learning for Near-Duplicate Video Retrieval
Wang, K.-H., Cheng, C.-C., Chen, Y.-L., Song, Y., & Lai, S.-H. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412710 · Cited by 19
Ma, T., & Tian, W. (2021) · The Visual Computer · doi:10.1007/s00371-020-01843-3 · Cited by 16
Haq, E. U., Huarong, X., Xuhui, C., Wanqing, Z., Jianping, F., & Abid, F. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08167-y · Cited by 12
Video Object Detection Base on RGB and Optical Flow Analysis
Zhang, S., Wang, T., Wang, C., Wang, Y., Shan, G., & Snoussi, H. (2019) · 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI) · doi:10.1109/cchi.2019.8901921 · Cited by 7
Metadata
- Authors
- Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 1647-1655
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2560474170
- Cited by
- 3,152
- References
- 35
- FWCI
- 94.24
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.