Light field video stabilization
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We describe a method for producing a smooth, stabilized video from the shaky input of a hand-held light field video camera—specifically, a small camera array. Traditional stabilization techniques dampen shake with 2D warps, and thus have limited ability to stabilize a significantly shaky camera motion through a 3D scene. Other recent stabilization techniques synthesize novel views as they would have been seen along a virtual, smooth 3D camera path, but are limited to static scenes. We show that video camera arrays enable much more powerful video stabilization, since they allow changes in viewpoint for a single time instant. Furthermore, we point out that the straightforward approach to light field video stabilization requires computing structure-from-motion, which can be brittle for typical consumer-level video of general dynamic scenes. We present a more robust approach that avoids input camera path reconstruction. Instead, we employ a spacetime optimization that directly computes a sequence of relative poses between the virtual camera and the camera array, while minimizing acceleration of salient visual features in the virtual image plane. We validate our novel method by comparing it to state-of-the-art stabilization software, such as Apple iMovie and 2d3 SteadyMove Pro, on a number of challenging scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Light-field cameras offer several advantages over standard cameras, such as depth estimation, noise reduction [57], video stabilization [227], isolation of distractors [58], and specularity removal [119].”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [227]
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 2×
“Conditions for success include the presence of enough distinct image features, minimal motion blur and noise, and scenes composed mostly of rigid background with only small dynamic scene elements whose motion can be filtered as outliers.”
BackgroundMethodA Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004 · cited 1×
“To address this issue, we do not use edge pixels that have significantly different depth values compared with their matches in adjacent frames.”
BackgroundMultiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
(2021) · A First Course in Statistical Programming with R · doi:10.1017/9781108993456.011 · Cited by 13,724
Multiple View Geometry in Computer Vision
(2001) · Kybernetes · doi:10.1108/k.2001.30.9_10.1333.2 · Cited by 3,336
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 1×
“[8] for the optical flow computation, although other state-of-the-art optical flow methods reviewed in [6] may also be used.”
BackgroundLight Field Photography with a Hand-held Plenoptic Camera
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., & Hanrahan, P. (2005) · HAL · Cited by 1,749
High-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015766 · Cited by 1,385
Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods
Bruhn, A., Weickert, J., & Schnörr, C. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000045324.43199.43 · Cited by 1,160
Single lens stereo with a plenoptic camera
Adelson, E. H., & Wang, J. Y. A. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121783 · Cited by 1,101 · cited 2×
“We briefly describe our first approach as a baseline method, which motivates our second approach.”
Method
Research works printed but not linked: 7
- burt, 1994, Image stabilization by registration to a reference mosaic, DARPA Image Understanding Workshop
- georgiev, 0, Spatio-angular resolution tradeoffs in integral photography, Rendering Techniques 2006 17th Eurographics Workshop on Rendering
- 0, ViewPLUS Inc ProFUSION 25 5*5 camera array system
- 0, Fujifilm finepix rea13d system, 2
- 0, Apple iMovie'09
- 0, 2d3 SteadyMove Pro
- morimoto, 1996, Automatic digital image stabilization, ICPR
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“Light-field cameras offer several advantages over standard cameras, such as depth estimation, noise reduction [58], video stabilization [227], isolation of distractors [59], and specularity removal [119].”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Light Field Image Processing: An Overview
Wu, G., Masia, B., Jarabo, A., Zhang, Y., Wang, L., Dai, Q., Chai, T., & Liu, Y. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2747126 · Cited by 586
“[192] described a video stabilization approach using an integrated camera array, ProFUSION-25C [21].”
MethodBundled camera paths for video stabilization
Liu, S., Yuan, L., Tan, P., & Sun, J. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461995 · Cited by 352
“Feature track smoothing is also used in light-field camera video stabilization work [Smith et al. 2009].”
MethodAuto-directed video stabilization with robust L1 optimal camera paths
Grundmann, M., Kwatra, V., & Essa, I. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995525 · Cited by 351
“tionally, image-based rendering techniques [1] or light-field rendering (if the video was captured by a camera array [13]) can be used to recast the original video.”
Method
Metadata
- Authors
- Smith, B. M., Zhang, L., Jin, H., & Agarwala, A.
- Affiliations
- University of Wisconsin–Madison (US)
- Adobe Systems (United States) (US)
- Venue
- 2009 IEEE 12th International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2009-09
- Pages
- 341-348
- OpenAlex
- W2112673789
- Cited by
- 107
- References
- 34
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 6 · 2023: 8 · 2022: 6 · 2021: 4 · 2020: 6 · 2019: 11 · 2018: 4 · 2017: 9 · 2016: 11 · 2015: 5 · 2014: 10 · 2013: 6 · 2012: 3
- FWCI
- 3.54
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Funders
- Adobe Systems
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Stabilization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- light field video, video stabilization, camera array, relative pose estimation, structure from motion, dynamic scenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Light field video stabilization. https://app.cochat.ai/papers/10.1109/iccv.2009.5459270
- The paper
- Smith, B. M., Zhang, L., Jin, H., & Agarwala, A. (2009). Light field video stabilization. 2009 IEEE 12th International Conference on Computer Vision. https://doi.org/10.1109/iccv.2009.5459270
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.