The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
UnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss
Open accessAbstract
In the era of end-to-end deep learning, many advances in computer vision are driven by large amounts of labeled data. In the optical flow setting, however, obtaining dense per-pixel ground truth for real scenes is difficult and thus such data is rare. Therefore, recent end-to-end convolutional networks for optical flow rely on synthetic datasets for supervision, but the domain mismatch between training and test scenarios continues to be a challenge. Inspired by classical energy-based optical flow methods, we design an unsupervised loss based on occlusion-aware bidirectional flow estimation and the robust census transform to circumvent the need for ground truth flow. On the KITTI benchmarks, our unsupervised approach outperforms previous unsupervised deep networks by a large margin, and is even more accurate than similar supervised methods trained on synthetic datasets alone. By optionally fine-tuning on the KITTI training data, our method achieves competitive optical flow accuracy on the KITTI 2012 and 2015 benchmarks, thus in addition enabling generic pre-training of supervised networks for datasets with limited amounts of ground truth.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Self-citationVision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., & Lopez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.352 · Cited by 2,338
Object scene flow for autonomous vehicles
Menze, M., & Geiger, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298925 · Cited by 2,270
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 1×
“Similar observations hold, for example, on the Middlebury dataset (Baker et al. 2011).”
ResultSelf-citationA Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842
Non-parametric local transforms for computing visual correspondence
Zabih, R., & Woodfill, J. (1994) · Lecture Notes in Computer Science · doi:10.1007/bfb0028345 · Cited by 1,763
Optical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.291 · Cited by 1,344
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Mayer, N., Ilg, E., Cremers, D., Dosovitskiy, A., & Brox, T. (2016) · Cited by 1,214
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174
Self-citationLucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods
Bruhn, A., Weickert, J., & Schnörr, C. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000045324.43199.43 · Cited by 1,160
Sun, D., Roth, S., & Black, M. J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0644-x · Cited by 614
Self-citationDense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow
Sundaram, N., Brox, T., & Keutzer, K. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_32 · Cited by 422
Fleet, D., & Weiss, Y. (2005) · Handbook of Mathematical Models in Computer Vision · doi:10.1007/0-387-28831-7_15 · Cited by 372
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Yu, J. J., Harley, A. W., & Derpanis, K. G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_1 · Cited by 368
Unsupervised deep learning for optical flow estimation
Ren, Z., Yan, J., Ni, B., Liu, B., Yang, X., & Zha, H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10723 · Cited by 296
Robust dynamic motion estimation over time
Black, M. J., & Anandan, P. (2002) · Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1991.139705 · Cited by 241
Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection
Xiao, J., Cheng, H., Sawhney, H., Rao, C., & Isnardi, M. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_17 · Cited by 229
Efficient Computation of Optical Flow Using the Census Transform
Stein, F. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28649-3_10 · Cited by 174
Exploiting Semantic Information and Deep Matching for Optical Flow
Bai, M., Luo, W., Kundu, K., & Urtasun, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_10 · Cited by 125
Unsupervised convolutional neural networks for motion estimation
Ahmadi, A., & Patras, I. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532634 · Cited by 120
MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation
Hur, J., & Roth, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.42 · Cited by 106
Self-citationPatchBatch: A Batch Augmented Loss for Optical Flow
Gadot, D., & Wolf, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.459 · Cited by 102
An Evaluation of Data Costs for Optical Flow
Vogel, C., Roth, S., & Konrad Schindler, B. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40602-7_37 · Cited by 89
Self-citationWhy Is the Census Transform Good for Robust Optic Flow Computation?
Hafner, D., Demetz, O., & Weickert, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38267-3_18 · Cited by 82
An Unbiased Second-Order Prior for High-Accuracy Motion Estimation
Trobin, W., Pock, T., Cremers, D., & Bischof, H. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69321-5_40 · Cited by 54
Guided Optical Flow LearningPreprint
Zhu, Y., Lan, Z., Newsam, S. D., & Hauptmann, A. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.02295 · Cited by 36
As-Rigid-As-Possible Stereo under Second Order Smoothness Priors
Zhang, C., Li, Z., Cai, R., Chao, H., & Rui, Y. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_8 · Cited by 33
Optical Flow Requires Multiple Strategies (but Only One Network)
Schuster, T., Wolf, L., & Gadot, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.732 · Cited by 16
CNN based Patch Matching for Optical Flow with Thresholded Hinge Loss.Preprint
Bailer, C., Varanasi, K., & Stricker, D. (2016) · arXiv (Cornell University) · Cited by 10
Cited by
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00212 · Cited by 1,256
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Li, Z., Niklaus, S., Snavely, N., & Wang, O. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00643 · Cited by 674
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
Ranjan, A., Jampani, V., Balles, L., Kim, K., Sun, D., Wulff, J., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01252 · Cited by 632
Metadata
- Authors
- Meister, S., Hur, J., & Roth, S.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-27
- Volume
- 32
- Issue
- 1
- ISSN
- 2374-3468
- 2159-5399
- OpenAlex
- W2963891416
- Cited by
- 520
- References
- 35
- FWCI
- 32.15
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.