The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Burst photography for high dynamic range and low-light imaging on mobile cameras
Abstract
Cell phone cameras have small apertures, which limits the number of photons they can gather, leading to noisy images in low light. They also have small sensor pixels, which limits the number of electrons each pixel can store, leading to limited dynamic range. We describe a computational photography pipeline that captures, aligns, and merges a burst of frames to reduce noise and increase dynamic range. Our system has several key features that help make it robust and efficient. First, we do not use bracketed exposures. Instead, we capture frames of constant exposure, which makes alignment more robust, and we set this exposure low enough to avoid blowing out highlights. The resulting merged image has clean shadows and high bit depth, allowing us to apply standard HDR tone mapping methods. Second, we begin from Bayer raw frames rather than the demosaicked RGB (or YUV) frames produced by hardware Image Signal Processors (ISPs) common on mobile platforms. This gives us more bits per pixel and allows us to circumvent the ISP's unwanted tone mapping and spatial denoising. Third, we use a novel FFT-based alignment algorithm and a hybrid 2D/3D Wiener filter to denoise and merge the frames in a burst. Our implementation is built atop Android's Camera2 API, which provides per-frame camera control and access to raw imagery, and is written in the Halide domain-specific language (DSL). It runs in 4 seconds on device (for a 12 Mpix image), requires no user intervention, and ships on several mass-produced cell phones.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1981) · Cited by 11,963
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
De-noising by soft-thresholding
Donoho, D. L. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382009 · Cited by 9,619
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Overview of the H.264/AVC video coding standard
Wiegand, T., Sullivan, G. J., Bjontegaard, G., & Luthra, A. (2003) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2003.815165 · Cited by 8,184
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394
The Design and Implementation of FFTW3
Frigo, M., & Johnson, S. G. (2005) · Proceedings of the IEEE · doi:10.1109/jproc.2004.840301 · Cited by 5,086 · cited 2×
“For the small DFTs in our pipeline, this implementation is five times faster than FFTW [Frigo and Johnson 2005] on an ARM-based mobile phone.”
MethodRecovering high dynamic range radiance maps from photographs
Debevec, P. E., & Malik, J. (2008) · ACM SIGGRAPH 2008 classes · doi:10.1145/1401132.1401174 · Cited by 2,399
Object scene flow for autonomous vehicles
Menze, M., & Geiger, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298925 · Cited by 2,270
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 1×
“In the computer vision literature, optical flow techniques are evaluated primarily by quality on established benchmarks [Baker et al. 2011; Menze and Geiger 2015].”
MethodRecovering high dynamic range radiance maps from photographs
Debevec, P. E., & Malik, J. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258884 · Cited by 1,775
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting
Reinhard, E., Ward, G., Pattanaik, S., Debevec, P. E., Heidrich, W., & Myszkowski, K. (2010) · Cited by 1,359
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Digital photography with flash and no-flash image pairs
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015777 · Cited by 1,056
High performance imaging using large camera arrays
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., & Levoy, M. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073259 · Cited by 1,042
Self-citationRadiometric CCD camera calibration and noise estimation
Healey, G. E., & Kondepudy, R. (1994) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.276126 · Cited by 739
Demosaicking: color filter array interpolation
Gunturk, B. K., Glotzbach, J., Altunbasak, Y., Schafer, R. W., & Mersereau, R. M. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407714 · Cited by 582
Mertens, T., Kautz, J., & Van Reeth, F. (2007) · 15th Pacific Conference on Computer Graphics and Applications (PG'07) · doi:10.1109/pg.2007.17 · Cited by 500
Image restoration by sparse 3D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008) · SPIE Proceedings · doi:10.1117/12.766355 · Cited by 461
Video Denoising By Sparse 3D Transform-Domain Collaborative Filtering
Dabov, K., Foi, A., & Egiazarian, K. (2007) · doi:10.5281/zenodo.40233 · Cited by 373
Reinhard, E., Ward, G., Pattanaik, S. N., & Debevec, P. E. (2005) · Cited by 372
Optimal Inversion of the Generalized Anscombe Transformation for Poisson-Gaussian Noise
Makitalo, M., & Foi, A. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2202675 · Cited by 363
A fast direct Fourier-based algorithm for subpixel registration of images
Stone, H. S., Orchard, M. T., Chang, E., & Martucci, S. A. (2001) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/36.957286 · Cited by 358
Multiframe demosaicing and super-resolution of color images
Farsiu, S., Elad, M., & Milanfar, P. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.860336 · Cited by 319
FlexISP: a flexible camera image processing framework
Heide, F., Steinberger, M., Tsai, Y.-T., Rouf, M., Pająk, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661260 · Cited by 298
Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation
Yamaguchi, K., McAllester, D., & Urtasun, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_49 · Cited by 297
High performance imaging using large camera arrays
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., & Levoy, M. (2005) · ACM SIGGRAPH 2005 Papers · doi:10.1145/1186822.1073259 · Cited by 258
Self-citationDecoupling algorithms from schedules for easy optimization of image processing pipelines
Ragan-Kelley, J., Adams, A., Paris, S., Levoy, M., Amarasinghe, S., & Durand, F. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185528 · Cited by 251
Self-citationVideo enhancement using per-pixel virtual exposures
Bennett, E. P., & McMillan, L. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073272 · Cited by 241
Fast Local Laplacian Filters: Theory and Applications
Aubry, M., Paris, S., Hasinoff, S. W., Kautz, J., & Durand, F. (2014) · ACM Transactions on Graphics · doi:10.1145/2629645 · Cited by 197
Self-citationA New In-Camera Imaging Model for Color Computer Vision and Its Application
Kim, S. J., Lin, H. T., Lu, Z., Süsstrunk, S., Lin, S., & Brown, M. S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.58 · Cited by 167
SimpleFlow: A Non‐iterative, Sublinear Optical Flow Algorithm
Tao, M. W., Bai, J., Kohli, P., & Paris, S. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03013.x · Cited by 163
Polynomial Expansion for Orientation and Motion Estimation
Farnebäck, G. (2002) · Cited by 144
Adams, A., Talvala, E.-V., Park, S. H., et al. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778766 · Cited by 131
Self-citationLiu, Z., Yuan, L., Tang, X., Uyttendaele, M., & Sun, J. Z. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661277 · Cited by 120
Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal
Joshi, N. S., & Cohen, M. F. (2010) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2010.5585096 · Cited by 97
Hand-Held Video Deblurring Via Efficient Fourier Aggregation
Delbracio, M., & Sapiro, G. (2015) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2015.2501245 · Cited by 79
Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences
Farsiu, S., Elad, M., & Milanfar, P. (2006) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/asp/2006/61859 · Cited by 67
Stack-Based Algorithms for HDR Capture and Reconstruction
Gallo, O., & Sen, P. (2016) · Elsevier eBooks · doi:10.1016/b978-0-08-100412-8.00003-6 · Cited by 15
Cited by
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
On the use of deep learning for computational imaging
Barbastathis, G., Ozcan, A., & Situ, G. (2019) · Optica · doi:10.1364/optica.6.000921 · Cited by 869
Deep bilateral learning for real-time image enhancement
Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073592 · Cited by 800
Self-citationStereo magnification: learning view synthesis using multiplane images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201323 · Cited by 776
Deep high dynamic range imaging of dynamic scenes
Kalantari, N. K., & Ramamoorthi, R. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073609 · Cited by 596
Metadata
- Authors
- Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M.
- Venue
- ACM Transactions on Graphics
- Published
- 2016-11-11
- Volume
- 35
- Issue
- 6
- Pages
- 1-12
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2552290192
- Cited by
- 483
- References
- 40
- FWCI
- 17.34
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.