FusionFlow: Discrete-continuous optimization for optical flow estimation
Abstract
Accurate estimation of optical flow is a challenging task, which often requires addressing difficult energy optimization problems. To solve them, most top-performing methods rely on continuous optimization algorithms. The modeling accuracy of the energy in this case is often traded for its tractability. This is in contrast to the related problem of narrow-baseline stereo matching, where the top-performing methods employ powerful discrete optimization algorithms such as graph cuts and message-passing to optimize highly non-convex energies. In this paper, we demonstrate how similar non-convex energies can be formulated and optimized discretely in the context of optical flow estimation. Starting with a set of candidate solutions that are produced by fast continuous flow estimation algorithms, the proposed method iteratively fuses these candidate solutions by the computation of minimum cuts on graphs. The obtained continuous-valued fusion result is then further improved using local gradient descent. Experimentally, we demonstrate that the proposed energy is an accurate model and that the proposed discrete-continuous optimization scheme not only finds lower energy solutions than traditional discrete or continuous optimization techniques, but also leads to flow estimates that outperform the current state-of-the-art.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Only a few algorithms (e.g., DPOF, Fusion, and Dynamic MRF) perform well in this region.
“Only a few algorithms (e.g., DPOF-Lei and Yang 2009, Fusion-Lempitsky et al. 2008, and Dynamic MRF-Glocker et al. 2008) perform well in this region.”
From A Database and Evaluation Methodology for Optical Flow (2011) · cites this paper as Lempitsky et al. 2008
Cites
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988 · see citing works
Boros, E., & Hammer, P. L. (2002) · Discrete Applied Mathematics · doi:10.1016/s0166-218x(01)00341-9 · Cited by 827 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · see citing works
“Fusion Flow ( Lempitsky et al. 2008 )u ses as equence of binary graph-cut optimizations to refine the current flow estimate by selectively replacing portions with one of the candidate solutions.”
BackgroundMethodInfluentialSelf-citationSecrets of optical flow estimation and their principles
Sun, D., Roth, S., & Black, M. J. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539939 · Cited by 1,656 · see citing works
Self-citationLarge Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295 · see citing works
Sun, D., Roth, S., & Black, M. J. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0644-x · Cited by 698 · see citing works
“…al. 2010; Zimmer et al. 2009) to reduce the influence of lighting changes, incremental warping (Bergen et al. 1992), warping with bicubic interpolation (Lempitsky et al. 2008; Wedel et al. 2008b), temporal averaging of image derivatives (Horn 1986; Wedel et al. 2008b),…”
BackgroundSelf-citationGlobal stereo reconstruction under second order smoothness priors
Woodford, O. J., Torr, P. H. S., Reid, I. D., & Fitzgibbon, A. W. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587672 · Cited by 120 · see citing works
“This has allowed graph-cuts-based multi-label optimization to be extended to both consistently labelled problems with non-submodular terms [47], [51], [52], [53], and problems with inconsistent labels [5], [44], [ 54 ].”
BackgroundSfM with MRFs: Discrete-Continuous Optimization for Large-Scale Structure from Motion
Crandall, D., Owens, A., Snavely, N., & Huttenlocher, D. P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.218 · Cited by 157 · see citing works
“Combining discrete and continuous optimization techniques has been found to work well on other vision problems, such as optical flow [31].”
Power Watershed: A Unifying Graph-Based Optimization Framework
Couprie, C., Grady, L., Najman, L., & Talbot, H. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.200 · Cited by 337 · see citing works
“Viewed as energy minimization algorithms, graph cuts, random walker, and shortest paths have found many different applications in the computer vision field that go beyond image segmentation, such as stereo correspondence, optical flow, and image restoration (e.g., [82], [76], [ 53 ]).”
BackgroundSun, D., Roth, S., Lewis, J. P., & Black, M. J. (2008) · European Conference on Computer Vision · doi:10.1007/978-3-540-88690-7_7 · Cited by 303 · see citing works
Self-citationFusion Moves for Markov Random Field Optimization
Lempitsky, V., Rother, C., Roth, S., & Blake, A. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.143 · Cited by 199 · see citing works
“An important point we advocate here as well as in our previous work [15], [16], which distinguishes this from other preceding work, is that the proposals employed during the iterative application of fusion moves do not necessarily have to be primitive, constant proposals or permutations of the…”
BackgroundMethodResultInfluentialSelf-citation3D Scene Flow Estimation with a Piecewise Rigid Scene Model
Vogel, C., Schindler, K., & Roth, S. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0806-0 · Cited by 246 · see citing works
“(11), the pseudo-Boolean function (Lempitsky et al. 2008) representing the local energy has at most ( 2 N S − 1 ) 2 variables, which makes the optimization efficient.”
BackgroundSelf-citationVogel, C., Schindler, K., & Roth, S. (2013) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.174 · Cited by 246 · see citing works
“6) using fusion moves [11] and QPBO [15].”
MethodSelf-citationEfficient sparse-to-dense optical flow estimation using a learned basis and layers
Wulff, J., & Black, M. J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298607 · Cited by 186 · see citing works
“Several methods have been propose in the literature for related problems [27, 32].”
MethodOptical Flow with Semantic Segmentation and Localized Layers
Sevilla-Lara, L., Sun, D., Jampani, V., & Black, M. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.422 · Cited by 147 · see citing works
Motion detail preserving optical flow estimation
Xu, L., Jia, J.-Y., & Matsushita, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539820 · Cited by 193 · see citing works
“The fusion move step [ 14 ] is used to repeatedly fuse the candidates until each gets visited twice.”
BackgroundMethodInfluentialLarge Displacement Optical Flow from Nearest Neighbor Fields
Chen, Z., Jin, H., Lin, Z., Cohen, S. D., & Wu, Y. N. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.316 · Cited by 145 · see citing works
SimpleFlow: A Non‐iterative, Sublinear Optical Flow Algorithm
Tao, M. W., Bai, J., Kohli, P., & Paris, S. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03013.x · Cited by 139 · see citing works
Higher-order clique reduction in binary graph cut
Ishikawa, H. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206689 · Cited by 128 · see citing works
““fusion move”[18, 19] of the α-expansion algorithm[5].”
Jointly Optimized Regressors for Image Super‐resolution
Dai, D., Timofte, R., & Gool, L. V. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12544 · Cited by 162 · see citing works
Couprie, C., Grady, L., Najman, L., & Talbot, H. (2009) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2009.5459284 · Cited by 154 · see citing works
“Viewed as energy minimization algorithms, graph cuts, random walker and shortest paths have found many different applications in the computer vision field that go beyond image segmentation, such as stereo correspondence, optical flow and image restoration [35, 32, 25].”
BackgroundSurface stereo with soft segmentation
Bleyer, M., Rother, C., & Kohli, P. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539783 · Cited by 151 · see citing works
Self-citationMotion segmentation and activity representation in crowds
Barneva, R., Brimkov, V., & Kanev, K. (2009) · doi:10.1002/ima.v19:2 · Cited by 149
Layered segmentation and optical flow estimation over time
Sun, D., Sudderth, E. B., & Black, M. J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247873 · Cited by 90 · see citing works
“This is a binary segmentation problem similar to the FusionFlow [17] work.”
BackgroundMethodTransformation of General Binary MRF Minimization to the First-Order Case
Ishikawa, H. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.91 · Cited by 144 · see citing works
“Ç”
Robust Monocular Epipolar Flow Estimation
Yamaguchi, K., McAllester, D. A., & Urtasun, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.243 · Cited by 136 · see citing works
Discrete Optimization for Optical Flow
Menze, M., Heipke, C., & Geiger, A. (2015) · German Conference on Pattern Recognition · doi:10.1007/978-3-319-24947-6_2 · Cited by 130 · see citing works
“To avoid this difficulty, a number of approaches formulate optical flow as a segmentation problem: Based on a small set of dense flow field proposals, the most likely flow field at each pixel subject to regularization constraints is approximated via quadratic pseudo-boolean optimization (QPBO) or belief propagation (B…”
BackgroundExploiting Semantic Information and Deep Matching for Optical Flow
Bai, M., Luo, W., Kundu, K., & Urtasun, R. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46466-4_10 · Cited by 127 · see citing works
“Many approaches formulate flow as inference in a Markov random field (MRF) [14,15,16,17,18].”
BackgroundGlobally consistent registration of terrestrial laser scans via graph optimization
Theiler, P. W., Wegner, J. D., & Schindler, K. (2015) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2015.08.007 · Cited by 116 · see citing works
A 3D Morphable Model of Craniofacial Shape and Texture Variation
Dai, H., Pears, N. E., Smith, W. A. P., & Duncan, C. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.335 · Cited by 124 · see citing works
Optical Flow in Mostly Rigid Scenes
Wulff, J., Sevilla-Lara, L., & Black, M. J. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.731 · Cited by 121 · see citing works
A Fully-Connected Layered Model of Foreground and Background Flow
Sun, D., Wulff, J., Sudderth, E. B., Pfister, H., & Black, M. J. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.317 · Cited by 86 · see citing works
Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data
Janai, J., Güney, F., Wulff, J., Black, M. J., & Geiger, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.154 · Cited by 82 · see citing works
“To cope with large displacements, sparse feature correspondences [9,11,62,67] and discrete inference techniques [4,13,34,37,40,55,71] have been proposed.”
MethodUnwrap mosaics: a new representation for video editing
Rav-Acha, A., Kohli, P., Rother, C., & Fitzgibbon, A. (2008) · ACM Transactions on Graphics · doi:10.1145/1399504.1360616 · Cited by 108 · see citing works
“An intermediate step which increases the overall energy can be rejected, or re-run with different algorithmic parameters, but with the same energy parameters [Woodford et al. 2008; Lempitsky et al. 2008].”
BackgroundSelf-citationMirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation
Hur, J., & Roth, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.42 · Cited by 94 · see citing works
Self-citationEnergy based multiple model fitting for non-rigid structure from motion
Russell, C., Fayad, J. K., & Agapito, L. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995383 · Cited by 83 · see citing works
“As all of these approaches are based upon either -expansion, or its extension fusion moves [ 19 , 30], they make the implicit assumption that each point of interest belongs to exactly one model.”
BackgroundMethodInfluentialRobust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera
Zhang, G., Jia, J., Hua, W., & Bao, H. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.115 · Cited by 77 · see citing works
“There have been quite a number of methods to improve the robustness, accuracy, and efficiency [5], [9], [10], [22], [40], [42].”
Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames
Kennedy, R., & Taylor, C. J. (2015) · Energy Minimization Methods in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-14612-6_27 · Cited by 84 · see citing works
“Although fusion-based methods have been used previously [7], our approach is fundamentally different and does not require optimizing an NP-hard quadratic binary program.”
BackgroundMethodGüney, F., & Geiger, A. (2016) · Asian Conference on Computer Vision · doi:10.1007/978-3-319-54190-7_13 · Cited by 75 · see citing works
Layered RGBD scene flow estimation
Sun, D., Sudderth, E. B., & Pfister, H. (2015) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2015.7298653 · Cited by 73 · see citing works
Deformable image registration by combining uncertainty estimates from supervoxel belief propagation
Heinrich, M., Simpson, I. J. A., Papież, B., Brady, M., & Schnabel, J. (2016) · Medical Image Anal. · doi:10.1016/j.media.2015.09.005 · Cited by 72 · see citing works
Local Layering for Joint Motion Estimation and Occlusion Detection
Sun, D., Liu, C., & Pfister, H. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.144 · Cited by 69 · see citing works
Optical flow estimation on coarse-to-fine region-trees using discrete optimization
Lei, C., & Yang, Y.-H. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459253 · Cited by 67 · see citing works
Depth from Defocus in the Wild
Tang, H., Cohen, S. D., Price, B. L., Schiller, S., & Kutulakos, K. N. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.507 · Cited by 63 · see citing works
“This approach has been used before for depth cues other than defocus [5– 7, 15, 30, 33, 34].”
MethodSegmenting video into classes of algorithm-suitability
Aodha, O. M., Brostow, G., & Pollefeys, M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540099 · Cited by 62 · see citing works
“Optical Flow Of the existing approaches to computing optical flow, the iterative FusionFlow [ 13 ] is still very different technically, but the closest to our approach in terms of its philosophy.”
MethodBootstrap optical flow confidence and uncertainty measure
Kybic, J., & Nieuwenhuis, C. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2011.06.008 · Cited by 53 · see citing works
“This is different from approaches such as the FusionFlow [29] which apply different algorithms to the same input images and combine the results to obtain a better estimate of the flow.”
MethodPORTR: Pre-Operative and Post-Recurrence Brain Tumor Registration
Kwon, D., Niethammer, M., Akbari, H., Bilello, M., Davatzikos, C., & Pohl, K. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2293478 · Cited by 52
“2) Hybrid Optimization Method: We determine the minimum of (12) via a hybrid approach combining discrete with continuous optimization, a concept recently explored in optical flow estimations [38], [39].”
MethodConstrained Optical Flow Estimation as a Matching Problem
Mozerov, M. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2244221 · Cited by 49 · see citing works
Optical flow estimation using learned sparse model
Jia, K., Wang, X., & Tang, X.-O. (2011) · Vision · doi:10.1109/iccv.2011.6126522 · Cited by 48 · see citing works
Ayed, I. B., Chen, H.-M., Punithakumar, K., Ross, I., & Li, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540045 · Cited by 39 · see citing works
“Several studies have shown that graph cut optimization can be quite effective in various computer vision problems, for instance, image segmentation [16, 17, 18], object tracking [19], motion estimation [20], visual correspondence [21], and restoration [11].”
BackgroundAhmadi, M., Arabi, M., Hoag, D., & Engel, B. (2013) · doi:10.1002/2013wr013656 · Cited by 45
TriangleFlow: Optical Flow with Triangulation-Based Higher-Order Likelihoods
Glocker, B., Heibel, H., Navab, N., Kohli, P., & Rother, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15558-1_20 · Cited by 35 · see citing works
“Recently, methods have been proposed [3, 4] which successfully combine both worlds towards discrete-continuous optimization which is able to avoid local minima and obtain highly accurate (continuous) flow estimates at the same time.”
MethodSelf-citationSparsity model for robust optical flow estimation at motion discontinuities
Shen, X.-H., & Wu, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539944 · Cited by 44
Continuous Maximal Flows and Wulff Shapes: Application to MRFs
Zach, C., Niethammer, M., & Frahm, J.-M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206565 · Cited by 43 · see citing works
“Wellestablished approaches for MRF optimization in computer vision include belief propagation [26], graph cut methods [7, 15], message passing approaches [14, 16], and sequential fusion of labeling proposals [ 17 , 25].”
MethodGraph based construction of textured large field of view mosaics for bladder cancer diagnosis
Weibel, T., Daul, C., Wolf, D., Rösch, R., & Guillemin, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.05.023 · Cited by 42 · see citing works
Low-level vision by consensus in a spatial hierarchy of regions
Chakrabarti, A., Xiong, Y., Gortler, S., & Zickler, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1411.4894 · Cited by 40
MAP Estimation of Semi-Metric MRFs via Hierarchical Graph CutsPreprint
Kumar, M., & Koller, D. (2009) · Conference on Uncertainty in Artificial Intelligence · doi:10.48550/arxiv.1205.2633 · Cited by 39 · see citing works
“Move making algorithms are generally preferred in applications which involve a large number of random variables, e.g. on image-sizedMRFs in computer vision, due to their efficiency.”
BackgroundCho, J., Jung, Y., Kim, D.-S., Lee, S., & Jung, Y.-H. (2019) · Italian National Conference on Sensors · doi:10.3390/s19143217 · Cited by 37
Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow
Fortun, D., Bouthemy, P., & Kervrann, C. (2014) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.11.020 · Cited by 36 · see citing works
“To combine the subpixel accuracy of the continuous variational approach and the efficiency of discrete minimization, the authors of [40] built a discrete motion space from motion fields delivered by several global variational estimations with different parameter settings.”
BackgroundMethodInfluentialHigher-order gradient descent by fusion-move graph cut
Ishikawa, H. (2009) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2009.5459187 · Cited by 35 · see citing works
“In this section, we first formulate the higher-order graph cuts in more detail and then present the new technique for generating the proposal labelings.”
MethodAlphabet SOUP: A framework for approximate energy minimization
Gould, S. J., Amat, F., & Koller, D. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206650 · Cited by 28 · see citing works
“The novel fusion-move approach [ 18 ] makes moves by combining two proposed solutions x0 and x1. These fusion-moves usually”
MethodScene Flow Estimation: A SurveyPreprint
Yan, Z.-K., & Xiang, X.-Z. (2016) · arXiv.org · doi:10.48550/arxiv.1612.02590 · Cited by 29 · see citing works
“Using a superpixel segmentation for initialization [92], the energy is alternatively optimized using fusion moves [93] and quadratic pseudo-boolean optimization (QPBO) [94], and it achieves the stateof-the-art performance.”
MethodGlobal Stereo Reconstruction under Second-Order Smoothness Priors
Woodford, O. J., Torr, P. H. S., Reid, I. D., & Fitzgibbon, A. W. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.131 · Cited by 247 · see citing works
Discrete Graphical Models - An Optimization Perspective
Savchynskyy, B. (2019) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000084 · Cited by 28
“The fusion moves were proposed and discussed in [68, 52, 69].”
BackgroundOPTICAL FLOW FOR GLACIER MOTION ESTIMATION
Vogel, C., Bauder, A., & Schindler, K. (2012) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprsannals-i-3-359-2012 · Cited by 28 · see citing works
“Notable exceptions are (Lei and Yang, 2009, Lempitsky et al., 2008).”
BackgroundA new multiview spacetime-consistent depth recovery framework for free viewpoint video rendering
Lei, C., Chen, X.-D., & Yang, H. (2009) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2009.5459357 · Cited by 28
Dynamic Graph Cuts and Their Applications in Computer Vision
Kohli, P., & Torr, P. H. S. (2010) · Computer Vision: Detection, Recognition and Reconstruction · doi:10.1007/978-3-642-12848-6_3 · Cited by 28 · see citing works
“Similarly, successive sub-problems solved in move making algorithms algorithms such as swap, expansion [9], or fusion move [44, 74] are also similar.”
BackgroundLinear and quadratic reformulations of nonlinear optimization problems in binary variables
Rodríguez-Heck, E. (2018) · 4OR · doi:10.1007/s10288-018-0392-4 · Cited by 24 · see citing works
“Later, α-expansion moves were generalized to fusion moves [82, 83].”
BackgroundWeighted local intensity fusion method for variational optical flow estimation
Tu, Z., Poppe, R. W., & Veltkamp, R. C. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.09.002 · Cited by 24 · see citing works
Discrete-Continuous Optimization for Optical Flow Estimation
Roth, S., Lempitsky, V., & Rother, C. (2009) · Statistical and Geometrical Approaches to Visual Motion Analysis · doi:10.1007/978-3-642-03061-1_1 · Cited by 27 · see citing works
“Only a few authors have attempted to use discrete optimization for optical flow [7,10,11,12,13], however, and only very recently state-of-the-art performance has been achieved [ 14 ,15] 1 .”
BackgroundMethodResultInfluentialSelf-citationStochastic Uncertainty Models for the Luminance Consistency Assumption
Corpetti, T., & Mémin, É. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2162742 · Cited by 25 · see citing works
“Readers can refer to [4]–[7], [18], [22]–[24], [28], [34], [35], and [37] for a nonexhaustive panel or to [14] for a recent review on estimators dedicated to fluid flows.”
BackgroundRobust image alignment with multiple feature descriptors and matching-guided neighborhoods
Hsu, K.-J., Lin, Y.-Y., & Chuang, Y.-Y. (2015) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2015.7298802 · Cited by 22 · see citing works
“[21] proposed FusionFlow, which compiles the flow map by adaptively fusing multiple flow proposals obtained by different flow estimation methods with various parameter settings.”
MethodResultInfluentialDistribution Matching with the Bhattacharyya Similarity: A Bound Optimization Framework
Ayed, I. B., Punithakumar, K., & Li, S. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2382104 · Cited by 20 · see citing works
“It has been effective in various computer vision problems [39], for instance, segmentation [19], [20], [21], [40], tracking [41], [42], motion estimation [43], visual correspondence [44] and restoration [37].”
MethodObservation Model Based on Scale Interactions for Optical Flow Estimation
Zille, P., Corpetti, T., Shao, L., & Chen, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2328893 · Cited by 20
“Basic approaches penalize solution variation but some advanced terms have also been designed in order to deal with occlusions, discontinuities or relying on physical grounds (see [3], [7], [9], [15], [21], [25], [34], [41], [42], [44]).”
BackgroundHierarchical Scan-Line Dynamic Programming for Optical Flow Using Semi-Global Matching
Hermann, S., & Klette, R. (2012) · ACCV Workshops · doi:10.1007/978-3-642-37484-5_45 · Cited by 20 · see citing works
“[11], and Lei and Yang [12].”
An Eigenvalue Approach to Detect Flows and Events in Crowd Videos
Sharif, M. H. (2017) · J. Circuits Syst. Comput. · doi:10.1142/s0218126617501109 · Cited by 19 · see citing works
Progressive Feature Matching with Alternate Descriptor Selection and Correspondence Enrichment
Hu, Y.-T., & Lin, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.44 · Cited by 20 · see citing works
Combinatorial mesh optimization
Vidal, V., Wolf, C., & Dupont, F. (2011) · The Visual Computer · doi:10.1007/s00371-011-0649-9 · Cited by 19
Over-Parameterized Optical Flow Using a Stereoscopic Constraint
Rosman, G., Shem-Tov, S., Bitton, D., Nir, T., Adiv, G., Kimmel, R., Feuer, A., & Bruckstein, A. (2011) · Scale Space and Variational Methods in Computer Vision · doi:10.1007/978-3-642-24785-9_64 · Cited by 19 · see citing works
“Improving this convergence using discrete, graph-based, techniques, has been the focus of several attempts (see [23, 24, 25], for example), and is beyond the scope of this paper.”
BackgroundDecomposing and regularizing sparse/non-sparse components for motion field estimation
Chen, Z., Wang, J., & Wu, Y. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247874 · Cited by 18 · see citing works
“9 as in [16], the regularized TV penalty φ(s(2)) = √ s2 + 2 (also called “Charbonnier penalty”) as in [8, 7, 9], the non-convex Lorentzian function [5, 26, 22], and the Geman-McClure function [17], etc.”
BackgroundMethodGenerating Classic Mosaics with Graph Cuts
Liu, Y., Veksler, O., & Juan, O. (2010) · Computer graphics forum (Print) · doi:10.1111/j.1467-8659.2010.01752.x · Cited by 17
Combinatorial Regularization of Descriptor Matching for Optical Flow Estimation
Drayer, B., & Brox, T. (2015) · British Machine Vision Conference · doi:10.5244/c.29.42 · Cited by 16
Two Applications of Graph-Cuts to Image Processing
Cooke, T. (2008) · 2008 Digital Image Computing: Techniques and Applications · doi:10.1109/dicta.2008.32 · Cited by 15 · see citing works
“A more recent application of graph-cuts to the optical flow problem is FusionFlow [10], which is currently the best performing optical flow method on the Middlebury optical flow evaluation data set [12] (as measured by average end-point error).”
MethodSIFT flow for large-displacement object tracking.
Zhang, H.-L., Hu, S.-Q., & Zhang, X. (2014) · Applied Optics · doi:10.1364/ao.53.006194 · Cited by 14
“To overcome the problem, most nonconvex energy functions have been designed [22,23].”
BackgroundRigid Slice-To-Volume Medical Image Registration Through Markov Random Fields
Porchetto, R., Stramana, F., Paragios, N., & Ferrante, E. (2016) · MCV/BAMBI@MICCAI · doi:10.1007/978-3-319-61188-4_16 · Cited by 13 · see citing works
“This is coherent with the conclusions presented by [20] for the case of optical flow estimation.”
BackgroundMethodResultInfluentialMulti-step flow fusion: towards accurate and dense correspondences in long video shots
Crivelli, T., Conze, P.-H., Robert, P., Fradet, M., & Pérez, P. (2012) · British Machine Vision Conference · doi:10.5244/c.26.107 · Cited by 13 · see citing works
“We then apply the method recently presented in [15, 16] in the context of instantaneous optical flow estimation by flow fusion.”
MethodLow complexity optical flow using neighbor-guided semi-global matching
Xiang, J., Li, Z., Blaauw, D., Kim, H.-S., & Chakrabarti, C. (2016) · International Conference on Information Photonics · doi:10.1109/icip.2016.7533208 · Cited by 12 · see citing works
Improving sub-pixel correspondence through upsampling
Xu, L., Jia, J.-Y., & Kang, S.-B. (2012) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2011.11.003 · Cited by 12
A Continuous Labeling for Multiphase Graph Cut Image Partitioning
Salah, M. K. B., Mitiche, A., & Ayed, I. B. (2008) · International Symposium on Visual Computing · doi:10.1007/978-3-540-89639-5_26 · Cited by 11
“The general motivation of this recent trend is to take advantage of the ability of graph-cut algorithms to obtain nearly-global optima while avoiding the drawbacks of purely discrete methods, namely the huge number of labels as well as discretization artifacts [16].”
BackgroundReliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps
Liu, J., Li, C., Fan, X.-F., & Wang, Z. (2015) · Italian National Conference on Sensors · doi:10.3390/s150820894 · Cited by 9 · see citing works
“Because of the given lowest average number of unlabeled pixels, we used Quadratic Pseudo Boolean Optimization with Probing (QPBO-P) [38] and Quadratic Pseudo Boolean Optimization with Improving (QPBO-I) [39] as our fusion strategies.”
MethodHigher order maximum persistency and comparison theorems
Shekhovtsov, A. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.05.002 · Cited by 9 · see citing works
“This basic method is again extended to multilabel problems by solving crossover problems [42] and to general higher order 0-1 problems by reduction (quadratization) techniques expressing the function as a quadratic function with auxiliary variables [22, 15, 5].”
MethodLocal smoothness for global optical flow
Raket, L. L. (2012) · 2012 19th IEEE International Conference on Image Processing · doi:10.1109/icip.2012.6674231 · Cited by 9 · see citing works
“[5], where simple flow candidates were fused to produce superior flows by minimizing a highly non-convex energy.”
BackgroundReadjusting Unstable Regions to Improve the Quality of High Accuracy Optical Flow
Ha, S. V.-U., & Jeon, J. (2010) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2010.2041818 · Cited by 9
Pun, C.-M., & Huang, G.-H. (2016) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2016.10.017 · Cited by 8
“Lempitsky, Roth and Rother proposed a new energy minimization approach for optical flow estimation, named FusionFlow [45].”
BackgroundFast Optical Flow Estimation Without Parallel Architectures
Zhu, E., Li, Y.-W., & Shi, Y. (2017) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2016.2592322 · Cited by 7 · see citing works
“FusionFlow proposed in [17] estimates optical flow by fusing three kinds of optical flow proposals.”
BackgroundUnsupervised learning-based long-term superpixel tracking
Conze, P.-H., Tilquin, F., Lamard, M., Heitz, F., & Quellec, G. (2019) · Image and Vision Computing · doi:10.48550/arxiv.1902.09596 · Cited by 6
FusionNet and AugmentedFlowNet: Selective Proxy Ground Truth for Training on Unlabeled Images
Makansi, O., Ilg, E., & Brox, T. (2018) · arXiv.org · doi:10.48550/arxiv.1808.06389 · Cited by 6
“Lempitsky et al. [27] proposed a combinatorial optimization approach to combine flow fields from multiple methods based on a smoothness loss.”
MethodMulti-Exposure Motion Estimation Based on Deep Convolutional Networks
Xie, Z., Guo, Y.-C., Zhang, S.-H., Zhang, W.-J., & Ma, L.-Z. (2018) · Journal of Computational Science and Technology · doi:10.1007/s11390-018-1833-4 · Cited by 6 · see citing works
Motion estimation within the MPEG video compressed domain for JET plasma diagnostics
Craciunescu, T., Murari, A., Tiseanu, I., Lang, P., & Contributors, J. (2011) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/j.nima.2011.08.011 · Cited by 6 · see citing works
Large Displacement Optical Flow Estimation Based on Robust Interpolation of Sparse Correspondences
Shi, S., Zhang, D.-W., Zhang, C.-X., Chen, Z., Feng, C., & Fan, B. (2020) · IEEE Access · doi:10.1109/access.2020.3045764 · Cited by 6 · see citing works
“To improve both of the accuracy and robustness of optical flow estimation under large displacements and occlusions, Lempitsky et al. [51] proposed a discrete-continuous optimization for optical flow, which the proposed method fuses multiple optical flows to obtain a robust dense flow field.”
MethodLi, Z., Xiang, J., Gong, L., Blaauw, D., Chakrabarti, C., & Kim, H.-S. (2019) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2018.2854284 · Cited by 5 · see citing works
Deep Semantic Matching for Optical Flow
Bai, M., Luo, W., Kundu, K., & Urtasun, R. (2016) · arXiv.org · doi:10.48550/arxiv.1604.01827 · Cited by 5
“Discrete flow methods: A number of discrete approaches to optical flow estimation have been proposed in recent years [10,11,12,13,14].”
Background
- Authors
- Lempitsky, V., Roth, S., & Rother, C.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Technische Universität Darmstadt (DE)
- Venue
- 2008 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2008-06
- Pages
- 1-8
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2157394577
- Cited by
- 145
- References
- 0
- Citations by year
- 2022: 1 · 2021: 1 · 2020: 1 · 2019: 2 · 2018: 9 · 2017: 13 · 2016: 14 · 2015: 20 · 2014: 8 · 2013: 12 · 2012: 14
- FWCI
- 13.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Keywords
- optical flow, discrete-continuous optimization, minimum cut, graph cuts, energy modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
