The DOI matches its Crossref record; 110 of the 112 references it cites were found in a registry or an index; 2 of the 68 sources its claims cite were checked for support.
A Database and Evaluation Methodology for Optical Flow
Open accessAbstract
The quantitative evaluation of optical flow algorithms by Barron et al. (1994) led to significant advances in performance. The challenges for optical flow algorithms today go beyond the datasets and evaluation methods proposed in that paper. Instead, they center on problems associated with complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. We propose a new set of benchmarks and evaluation methods for the next generation of optical flow algorithms. To that end, we contribute four types of data to test different aspects of optical flow algorithms: (1) sequences with nonrigid motion where the ground-truth flow is determined by tracking hidden fluorescent texture, (2) realistic synthetic sequences, (3) high frame-rate video used to study interpolation error, and (4) modified stereo sequences of static scenes. In addition to the average angular error used by Barron et al., we compute the absolute flow endpoint error, measures for frame interpolation error, improved statistics, and results at motion discontinuities and in textureless regions.
Overview
CoChat Summary
TL;DR
The paper introduces a new optical flow benchmark of four data types (hidden fluorescent texture, realistic synthetic, high-speed interpolation, and modified stereo) plus an extended evaluation methodology with endpoint error, interpolation error, robustness/accuracy statistics, and region masks. It reports that performance improved substantially over the preliminary 2007 study, but motion discontinuities and diverse datasets remain difficult, and top flow methods still lag top stereo methods on the modified Teddy stereo data.
Key findings
- The benchmark contains 12 training and 12 test sequences; test ground truth is withheld except for Yosemite. As of December 2009 the online evaluation contained results for 24 published methods, all using only 2 input frames and 10 using color.
- Best average endpoint error (Avg. EE) improved dramatically over 2D-CLG (Bruhn et al. 2005): Army 0.28→0.09, Mequon 0.67→0.18, Schefflera 1.12→0.24, Wooden 1.07→0.18, Grove 1.23→0.74, Urban 1.54→0.39, Yosemite 0.10→0.08, Teddy 1.38→0.50. Best Avg. EE was below 0.2 pixels on four datasets (Yosemite, Army, Mequon, Wooden).
- Endpoint error (EE) and angular error (AE) rankings were highly correlated (r = 0.989), but the paper argues EE should be preferred: Complementary OF on Urban had Avg. AE 4.64° (rank 6) but Avg. EE 1.78 pixels (rank 20), because AE downweights large motions.
- Motion discontinuity (Disc) regions were the hardest; textureless (Untext) regions were typically easiest. Dataset results were less correlated, with Yosemite poorly or negatively correlated with others. Flow and interpolation errors were only moderately correlated (Avg4 EE vs Avg4 IE r = 0.763). On Teddy, top stereo methods outperformed top flow methods: median lowest-five R1.0 was 9.9 for flow vs 6.5 for stereo in All regions (34% reduction), and 27.6 vs 10.0 in Disc regions (64% reduction).
Methods
- Four data types were collected: (1) real nonrigid scenes with dense ground truth from hidden fluorescent texture, captured with a Canon EOS 20D at 3504×2336, downsampled every 40th visible-light frame by factor 6 to 584×388, with SSD tracking (15×15 windows), forward/backward crosschecking, and Lucas-Kanade subpixel refinement; (2) realistic synthetic scenes rendered with 3Delight at 640×480, with ground truth from a custom shader and dominant-cluster flow selection; (3) high-speed video for frame interpolation captured with a PointGrey Dragonfly Express at 60 fps, providing every other frame and retaining intermediate frames as ground truth; and (4) modified stereo data (Teddy) from structured light, cropped to make the disparity range roughly symmetric.
- Evaluation metrics include average angular error (AE), absolute flow endpoint error (EE), interpolation error (IE), and gradient-normalized interpolation error (NE). Statistics include averages, robustness measures R_X, and percentile accuracy measures A_X, computed over All, Disc, and Untext region masks.
- Analysis used the online evaluation at http://vision.middlebury.edu/flow/, comparing 24 published methods across metrics, statistics, masks, and datasets, with Pearson correlations between rankings and comparisons to state-of-the-art stereo methods on Teddy.
Limitations
- The paper states that the hidden fluorescent texture scenes are laboratory scenes, not real-world scenes, and have no motion blur due to stop-motion capture; ground truth is unavailable where cross-checking failed, especially occluded regions, and the measured ground truth is not perfect, though significant errors are limited to a small fraction of pixels. Synthetic sequences were generated without motion blur. The modified stereo ground truth is only accurate to 0.25 pixels, and algorithms may exploit the knowledge that motions are horizontal; overfitting to rigid data should be visible across the 12 evaluation images. Interpolation ground truth is an intermediate image, not a flow field, so the flow definition is apparent motion.
- The paper also states that no single method yet achieves strong performance across a wide variety of datatypes, that any dataset has a limited lifespan, and that future studies should include more challenging realistic synthetic data, longer sequences, and more difficult materials, illumination changes, atmospheric effects, and transparency.
- Evident from the text: the online tables do not provide an overall ranking among methods; runtimes are not normalized for programming environment, CPU speed, cores, or hardware acceleration; algorithms are required to produce dense flow and cannot submit confidence maps or sparse/incomplete fields; all 24 published methods used only 2 frames despite 8-frame sequences being available; and the paper cannot definitively answer whether temporal smoothing is less suited to the more challenging sequences.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 16 Not valid · 58 Not checked · 185 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Rather than collecting a single type of data (with its inherent limitations) we instead collected four different types of data, each satisfying a different subset of desirable properties.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the combination of fluorescent paint, downsampling high-resolution images, and sequential tracking of small motions, we are able to obtain dense, subpixel accurate ground truth for a nonrigid scene.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To allow for measures that predict performance on such tasks, we collected a variety of data suitable for frame interpolation.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have developed a technique for capturing imagery of nonrigid scenes with ground-truth optical flow.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To go beyond previous synthetic ground truth (e.g., the Yosemite sequence), we generated two types of fairly complex synthetic outdoor scenes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
96 more claims
Create a free account to see all 261 claims and every source verdict.
- Probably not a substantial claim. 114
- It may state two separate claims as one. 76
- The quote only partly states this claim. 6
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Notable works include the Middlebury dataset (Baker et al, 2011) and the KITTI datasets (Geiger et al, 2012; Menze and Geiger, 2015), but due to the difficulty of computing accurate ground truth, these have been restricted to 8 and 2 · 200 pairs of frames, respectively (the latter are also limited by the special driving setting).”
From What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation? (2018) · cites this paper as Baker et al, 2011
Experimental — accuracy not yet measured
Citations
Cites
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Proceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
(1981) · Cited by 11,963
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394 · cited 2×
“A baseline approach is to use an L2 norm as in the Horn and Schunck algorithm (Horn and Schunck 1981):”
BackgroundMethodFast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740 · cited 10×
“Many areas of computer vision, such as stereo [63], face recognition [28, 55, 68], and object recognition [23, 24], have challenging datasets to track the progress made by leading algorithms and to stimulate new ideas.”
BackgroundMethodResultInfluentialSelf-citationProceedings European Conference on Computer Vision 2002
Heyden, A., Sparr, G., Nielsen, M., & Johansen, P. (2002) · Lund University Publications (Lund University) · Cited by 5,153
From few to many: illumination cone models for face recognition under variable lighting and pose
Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927464 · Cited by 4,912 · cited 1×
“Many areas of computer vision, such as stereo (Scharstein and Szeliski 2002), face recognition (Philips et al. 2005; Sim et al. 2003; Gross et al. 2008; Georghiades et al. 2001), and object recognition (Fei-Fei et al.”
Background
Research works printed but not linked: 3
- Anandan, P., & Weiss, R. (1985). Introducing smoothness constraint in a matching approach for the computation of displacement fields. In Proceedings of the DARPA image understanding workshop (pp. 186–196).
- Bleyer, M., & Chambon, S. (2010). Does color really help in dense stereo matching? In Proceedings of the international symposium 3D data processing, visualization and transmission.
- Rannacher, J. (2009). Realtime 3D motion estimation on graphics hardware. Undergraduate thesis, Heidelberg University.
Cited by
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
“Some notable datasets include the Middlebury datasets for stereo vision [20], multi-view stereo [36] and optical flow [21].”
BackgroundAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
“Perhaps not surprisingly, many algorithms that do well on established datasets such as Middlebury [41, 2] struggle on our benchmark.”
InfluentialThe Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
“The Middlebury dataset [2] contains only 8 image pairs for training, with ground truth flows generated using four different techniques.”
MethodA benchmark for the evaluation of RGB-D SLAM systems
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385773 · Cited by 3,976
“All sequences and movies are available on our website [1].”
Background
Metadata
- Authors
- Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R.
- Affiliations
- Microsoft (United States) (US)
- Middlebury College (US)
- Technische Universität Darmstadt (DE)
- Brown University (US)
- Corresponding author
- Scharstein, Daniel
- Venue
- International Journal of Computer Vision
- Published
- 2010-11-30
- Volume
- 92
- Issue
- 1
- Pages
- 1-31
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2147253850
- Semantic Scholar
- 804836b8ad86ef8042e3dcbd45442a52f031ee03
- Cited by
- 2,205
- References
- 112
- Citations by year
- 2026: 49 · 2025: 81 · 2024: 82 · 2023: 117 · 2022: 129 · 2021: 152 · 2020: 174 · 2019: 163 · 2018: 144 · 2017: 156 · 2016: 187 · 2015: 180 · 2014: 154 · 2013: 129 · 2012: 118
- FWCI
- 146.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by-nc)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (publishedVersion)
- Max Planck Institute for Plasma Physics (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- National Science Foundation (0534858, 0535075, 0413169)
- Intel Corporation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Keywords
- optical flow, benchmark dataset, evaluation methodology, non-rigid motion, motion discontinuities, textureless regions, sensor noise
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Database and Evaluation Methodology for Optical Flow. https://app.cochat.ai/papers/10.1007/s11263-010-0390-2
- The paper
- Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011). A Database and Evaluation Methodology for Optical Flow. International Journal of Computer Vision. https://doi.org/10.1007/s11263-010-0390-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.