Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 110 of the 112 references it cites were found in a registry or an index; 2 of the 68 sources its claims cite were checked for support.

A Database and Evaluation Methodology for Optical Flow

Open access
Baker, Simon, Scharstein, Daniel, Lewis, J. P., Roth, Stefan, Black, Michael J., Szeliski, Richard · 2011 · International Journal of Computer Vision

Abstract

The quantitative evaluation of optical flow algorithms by Barron et al. (1994) led to significant advances in performance. The challenges for optical flow algorithms today go beyond the datasets and evaluation methods proposed in that paper. Instead, they center on problems associated with complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. We propose a new set of benchmarks and evaluation methods for the next generation of optical flow algorithms. To that end, we contribute four types of data to test different aspects of optical flow algorithms: (1) sequences with nonrigid motion where the ground-truth flow is determined by tracking hidden fluorescent texture, (2) realistic synthetic sequences, (3) high frame-rate video used to study interpolation error, and (4) modified stereo sequences of static scenes. In addition to the average angular error used by Barron et al., we compute the absolute flow endpoint error, measures for frame interpolation error, improved statistics, and results at motion discontinuities and in textureless regions.

Overview

CoChat Summary

TL;DR

The paper introduces a new optical flow benchmark of four data types (hidden fluorescent texture, realistic synthetic, high-speed interpolation, and modified stereo) plus an extended evaluation methodology with endpoint error, interpolation error, robustness/accuracy statistics, and region masks. It reports that performance improved substantially over the preliminary 2007 study, but motion discontinuities and diverse datasets remain difficult, and top flow methods still lag top stereo methods on the modified Teddy stereo data.

Key findings

  • The benchmark contains 12 training and 12 test sequences; test ground truth is withheld except for Yosemite. As of December 2009 the online evaluation contained results for 24 published methods, all using only 2 input frames and 10 using color.
  • Best average endpoint error (Avg. EE) improved dramatically over 2D-CLG (Bruhn et al. 2005): Army 0.28→0.09, Mequon 0.67→0.18, Schefflera 1.12→0.24, Wooden 1.07→0.18, Grove 1.23→0.74, Urban 1.54→0.39, Yosemite 0.10→0.08, Teddy 1.38→0.50. Best Avg. EE was below 0.2 pixels on four datasets (Yosemite, Army, Mequon, Wooden).
  • Endpoint error (EE) and angular error (AE) rankings were highly correlated (r = 0.989), but the paper argues EE should be preferred: Complementary OF on Urban had Avg. AE 4.64° (rank 6) but Avg. EE 1.78 pixels (rank 20), because AE downweights large motions.
  • Motion discontinuity (Disc) regions were the hardest; textureless (Untext) regions were typically easiest. Dataset results were less correlated, with Yosemite poorly or negatively correlated with others. Flow and interpolation errors were only moderately correlated (Avg4 EE vs Avg4 IE r = 0.763). On Teddy, top stereo methods outperformed top flow methods: median lowest-five R1.0 was 9.9 for flow vs 6.5 for stereo in All regions (34% reduction), and 27.6 vs 10.0 in Disc regions (64% reduction).

Methods

  • Four data types were collected: (1) real nonrigid scenes with dense ground truth from hidden fluorescent texture, captured with a Canon EOS 20D at 3504×2336, downsampled every 40th visible-light frame by factor 6 to 584×388, with SSD tracking (15×15 windows), forward/backward crosschecking, and Lucas-Kanade subpixel refinement; (2) realistic synthetic scenes rendered with 3Delight at 640×480, with ground truth from a custom shader and dominant-cluster flow selection; (3) high-speed video for frame interpolation captured with a PointGrey Dragonfly Express at 60 fps, providing every other frame and retaining intermediate frames as ground truth; and (4) modified stereo data (Teddy) from structured light, cropped to make the disparity range roughly symmetric.
  • Evaluation metrics include average angular error (AE), absolute flow endpoint error (EE), interpolation error (IE), and gradient-normalized interpolation error (NE). Statistics include averages, robustness measures R_X, and percentile accuracy measures A_X, computed over All, Disc, and Untext region masks.
  • Analysis used the online evaluation at http://vision.middlebury.edu/flow/, comparing 24 published methods across metrics, statistics, masks, and datasets, with Pearson correlations between rankings and comparisons to state-of-the-art stereo methods on Teddy.

Limitations

  • The paper states that the hidden fluorescent texture scenes are laboratory scenes, not real-world scenes, and have no motion blur due to stop-motion capture; ground truth is unavailable where cross-checking failed, especially occluded regions, and the measured ground truth is not perfect, though significant errors are limited to a small fraction of pixels. Synthetic sequences were generated without motion blur. The modified stereo ground truth is only accurate to 0.25 pixels, and algorithms may exploit the knowledge that motions are horizontal; overfitting to rigid data should be visible across the 12 evaluation images. Interpolation ground truth is an intermediate image, not a flow field, so the flow definition is apparent motion.
  • The paper also states that no single method yet achieves strong performance across a wide variety of datatypes, that any dataset has a limited lifespan, and that future studies should include more challenging realistic synthetic data, longer sequences, and more difficult materials, illumination changes, atmospheric effects, and transparency.
  • Evident from the text: the online tables do not provide an overall ranking among methods; runtimes are not normalized for programming environment, CPU speed, cores, or hardware acceleration; algorithms are required to produce dense flow and cannot submit confidence maps or sparse/incomplete fields; all 24 published methods used only 2 frames despite 8-frame sequences being available; and the paper cannot definitively answer whether temporal smoothing is less suited to the more challenging sequences.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11263-010-0390-2

Cites

110 of 110 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

2 Valid · 16 Not valid · 58 Not checked · 185 Own results and interpretations

96 more claims

Create a free account to see all 261 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 114
  • It may state two separate claims as one. 76
  • The quote only partly states this claim. 6

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 112 references

Research works printed but not linked: 3

  1. Anandan, P., & Weiss, R. (1985). Introducing smoothness constraint in a matching approach for the computation of displacement fields. In Proceedings of the DARPA image understanding workshop (pp. 186–196).
  2. Bleyer, M., & Chambon, S. (2010). Does color really help in dense stereo matching? In Proceedings of the international symposium 3D data processing, visualization and transmission.
  3. Rannacher, J. (2009). Realtime 3D motion estimation on graphics hardware. Undergraduate thesis, Heidelberg University.

Cited by

See all 2,273

Metadata

Authors
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R.
Affiliations
  • Microsoft (United States) (US)
  • Middlebury College (US)
  • Technische Universität Darmstadt (DE)
  • Brown University (US)
Corresponding author
Scharstein, Daniel
Venue
International Journal of Computer Vision
Published
2010-11-30
Volume
92
Issue
1
Pages
1-31
ISSN
  • 0920-5691 (ISSN-L)
  • 1573-1405
OpenAlex
W2147253850
Cited by
2,205
References
112
Citations by year
2026: 49 · 2025: 81 · 2024: 82 · 2023: 117 · 2022: 129 · 2021: 152 · 2020: 174 · 2019: 163 · 2018: 144 · 2017: 156 · 2016: 187 · 2015: 180 · 2014: 154 · 2013: 129 · 2012: 118
FWCI
146.62
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • International Journal of Computer Vision (publishedVersion, cc-by-nc)
  • Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (publishedVersion)
  • Max Planck Institute for Plasma Physics (submittedVersion)
  • Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
Funders
  • National Science Foundation (0534858, 0535075, 0413169)
  • Intel Corporation
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
  • Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
Keywords
optical flow, benchmark dataset, evaluation methodology, non-rigid motion, motion discontinuities, textureless regions, sensor noise

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Database and Evaluation Methodology for Optical Flow. https://app.cochat.ai/papers/10.1007/s11263-010-0390-2
The paper
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011). A Database and Evaluation Methodology for Optical Flow. International Journal of Computer Vision. https://doi.org/10.1007/s11263-010-0390-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.