Color image processing pipeline
The DOI matches its Crossref record; 45 of the 50 references it cites were found in a registry or an index.
Abstract
Digital still color cameras (DSCs) have gained significant popularity in recent years, with projected sales in the order of 44 million units by the year 2005. Such an explosive demand calls for an understanding of the processing involved and the implementation issues, bearing in mind the otherwise difficult problems these cameras solve. This article presents an overview of the image processing pipeline, first from a signal processing perspective and later from an implementation perspective, along with the tradeoffs involved.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Color Science, Concepts and Methods. Quantitative Data and Formulas
Wright, W. D. (1967) · Physics Bulletin · doi:10.1088/0031-9112/18/10/010 · Cited by 3,540
Recovering high dynamic range radiance maps from photographs
Debevec, P. E., & Malik, J. (2008) · ACM SIGGRAPH 2008 classes · doi:10.1145/1401132.1401174 · Cited by 2,399
Recovering high dynamic range radiance maps from photographs
Debevec, P. E., & Malik, J. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258884 · Cited by 1,775 · cited 1×
“To address this problem, researchers have used the approach of capturing multiple images of the same scene at varying exposure levels and combining them to obtain a fused image that represents the highlight (bright) and shadow (dark) regions of an image in reasonable detail [11].”
MethodDigital Color Imaging Handbook
(2017) · CRC Press eBooks · doi:10.1201/9781420041484 · Cited by 644
Color plane interpolation using alternating projections
Gunturk, B. K., Altunbasak, Y., & Mersereau, R. M. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.801121 · Cited by 621
Demosaicking: color filter array interpolation
Gunturk, B. K., Glotzbach, J., Altunbasak, Y., Schafer, R. W., & Mersereau, R. M. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407714 · Cited by 582
High dynamic range imaging: spatially varying pixel exposures
Nayar, S. K., & Mitsunaga, T. (2002) · Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662) · doi:10.1109/cvpr.2000.855857 · Cited by 542
Demosaicing: image reconstruction from color CCD samples
Kimmel, R. (1999) · IEEE Transactions on Image Processing · doi:10.1109/83.784434 · Cited by 472
Barnard, K., Cardei, V., & Funt, B. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.802531 · Cited by 440
Sharma, G., & Trussell, H. J. (1997) · IEEE Transactions on Image Processing · doi:10.1109/83.597268 · Cited by 440
Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882270 · Cited by 395
Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882270 · Cited by 387
Selecting the optimal focus measure for autofocusing and depth-from-focus
Subbarao, M., & Tyan, J.-K. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.709612 · Cited by 323
Demosaicking methods for Bayer color arrays
Ramanath, R., Snyder, W. E., Bilbro, G. L., & Sander, W. A. (2002) · Journal of Electronic Imaging · doi:10.1117/1.1484495 · Cited by 289
Self-citationKeelan, B. W. (2002) · CRC Press eBooks · doi:10.1201/9780203910825 · Cited by 254
Color processing in digital cameras
Adams, J., Parulski, K., & Spaulding, K. (1998) · IEEE Micro · doi:10.1109/40.743681 · Cited by 233 · cited 1×
“Further resources on DSC issues include an illustrative overview of some of the processing steps involved in a DSC by Adams et al. [4].”
MethodSüsstrunk, S., Buckley, R., & Swen, S. (1999) · Color and Imaging Conference · doi:10.2352/cic.1999.7.1.art00024 · Cited by 191 · cited 5×
“(a) (b) (c) IEEE SIGNAL PROCESSING MAGAZINE [36] JANUARY 2005 the resulting energy as a measure of focus.”
MethodInfluentialGamut Constrained Illuminant Estimation
Finlayson, G. D., Hordley, S. D., & Tastl, I. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-4100-z · Cited by 166 · cited 1×
“, [25]) are slowly entering the industry.”
Color image processing [basics and special issue overview]
Trussell, H. J., Saber, E. S., & Vrhel, M. J. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407711 · Cited by 132
Eyeing the Camera: Into the Next Century
Lyon, R. F., & Hubel, P. M. (2002) · Color and Imaging Conference · doi:10.2352/cic.2002.10.1.art00064 · Cited by 110 · cited 1×
“In an exciting new development, the invention of entirely new sensor arrays [20] that more or less emulate the capture”
BackgroundRamanath, R., & Snyder, W. E. (2003) · Journal of Electronic Imaging · doi:10.1117/1.1606459 · Cited by 107
Self-citationTrussell, H. J., & Hartwig, R. E. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.999681 · Cited by 102
Constrained least-squares regression in color spaces
Finlayson, G. D., & Drew, M. S. (1997) · Journal of Electronic Imaging · doi:10.1117/12.278080 · Cited by 92 · cited 9×
“(a) (b) (c) IEEE SIGNAL PROCESSING MAGAZINE [36] JANUARY 2005 the resulting energy as a measure of focus.”
BackgroundMethodInfluentialSelf-citationImplementation of a passive automatic focusing algorithm for digital still camera
Lee, J., Kim, K.-S., Nam, B., Lee, J.-C., Kwon, Y.-M., & Kim, H.-G. (1995) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.468047 · Cited by 82
Color image processing for digital cameras
Parulski, K., & Spaulding, K. E. (2017) · CRC Press eBooks · doi:10.1201/9781420041484-12 · Cited by 54
Video camera system using fuzzy logic
Haruki, T., & Kikuchi, K. (1992) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.156746 · Cited by 53 · cited 1×
“The image is divided into blocks [9], [10] as shown in Figure 2(a).”
MethodWhite-point preserving color correction
Finlayson, G. D., & Drew, M. S. (1997) · Color and Imaging Conference · doi:10.2352/cic.1997.5.1.art00051 · Cited by 53
Self-citationColor dependent optical prefilter for the suppression of aliasing artifacts
Greivenkamp, J. E. (1990) · Applied Optics · doi:10.1364/ao.29.000676 · Cited by 51 · cited 1×
“As a means of reducing these artifacts, most cameras use an antialiasing filter using one of two approaches: using a birefringent material that uses polarization to spread a beam of light over multiple pixels [19] or using phase delay techniques [4] that shift neighboring rays out of phase, consequently attenuating hi…”
MethodMatrix Calculations for Digital Photography
Hubel, P. M., Holm, J. M., Finlayson, G. D., & Drew, M. S. (1997) · Color and Imaging Conference · doi:10.2352/cic.1997.5.1.art00021 · Cited by 44 · cited 1×
“Not simply science, but human tastes dictate the final steps in the pipeline: a “preferred reproduction” is required [50], rather than an accurate one.”
BackgroundSelf-citationColor filter array interpolation using color correlations and directional derivatives
Kehtarnavaz, N., Oh, H., & Yoo, Y. F. (2003) · Journal of Electronic Imaging · doi:10.1117/1.1607966 · Cited by 43
Self-citationDSP solutions run the gamut for color systems
Trussell, H. J. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.207697 · Cited by 36
Programmable DSP platform for digital still cameras
Illgner, K., Gruber, H.-G., Gelabert, P., Liang, J., Yoo, Y. F., Rabadi, W. A., & Talluri, R. (1999) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1999.758381 · Cited by 36
Self-citationWard, G. (2001) · Color and Imaging Conference · doi:10.2352/cic.2001.9.1.art00003 · Cited by 29
Digital Techniques for Reducing Television Noise
Rossi, J. P. (1978) · SMPTE Journal · doi:10.5594/j17407 · Cited by 26 · cited 1×
“Coring is used to remove detail information that has no significant contribution to image detail and behaves much like noise [44].”
MethodDevelopment and Real-Time Implementation of Auto White Balancing Scoring Algorithm
Kehtarnavaz, N., Oh, H.-J., & Yoo, Y. F. (2002) · Real-Time Imaging · doi:10.1006/rtim.2001.0287 · Cited by 25
Self-citationMorimura, A., Uomori, K., Kitamura, Y., Fujioka, A., Harada, J., Iwamura, S., & Hirota, M. (1990) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.61549 · Cited by 24 · cited 1×
“The image is divided into blocks [9], [10] as shown in Figure 2(a).”
MethodGamut constrained illuminant estimation
Finlayson, Hordley, & Tastl (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238429 · Cited by 23
Constrained least-squares regression in color spaces
Finlayson, G. D. (1997) · Journal of Electronic Imaging · doi:10.1117/1.482558 · Cited by 22
Reference Input/Output Medium Metric RGB Color Encodings (RIMM/ROMM RGB)
Spaulding, K. E., Woolfe, G. J., & Giorgianni, E. J. (2000) · PICS · Cited by 21
Color image processing for digital cameras
Spaulding, K. E., & Parulski, K. (2002) · Electrical engineering and applied signal processing series/The electrical engineering and applied signal processing series · doi:10.1201/9781420041484.ch12 · Cited by 19
System optimization in digital color imaging
Bala, R., & Gaurav Sharma, A. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407715 · Cited by 12 · cited 1×
“Each of these blocks may be fine-tuned to achieve better systemic performance [7], e.g., introducing a small amount of blur using the lens system increases the correlation between neighboring pixels, which in turn may be used in the demosaicking step.”
BackgroundIssues Relating to the Transformation of Sensor Data into Standard Color Spaces
Holm, J. M. (1997) · Color and Imaging Conference · doi:10.2352/cic.1997.5.1.art00057 · Cited by 11 · cited 3×
“The interested reader is referred to the work by Holm [43] for a study of considerations in color space standards design.”
MethodHigh dynamic range imaging & image-based lighting
Ward, G., Reinhard, E., & Debevec, P. E. (2008) · doi:10.1145/1401132.1401170 · Cited by 10
Low-Complexity and Low-Power Color Space Conversion for Digital Video
Holm, K., & Gustafsson, O. (2006) · doi:10.1109/norchp.2006.329205 · Cited by 8
Parulski, K. (2000) · Cited by 8
Research works printed but not linked: 15
- 2003, Multimedia Systems and Equipment— Colour Measurement and Management— Part 2-2 Colour Management— Extended RGB Colour Space— scRGB
- 1998, Graphic Technology and Photography— Colour Characterisation of Digital Still Cameras (DSCs)— Part 1 Stimuli Metrology and Test Procedures
- mccamy, 1976, a color-rendition chart, J Appl Photog Eng, 2, 95
- 2002, Photography— Electronic Still Picture Imaging— Reference Input Medium Metric RGB Color Encoding (RIMM-RGB)
- 1999, Photography— Electronic Still-Picture Cameras— Methods for Measuring Opto-Electronic Conversion Functions (OECFs)
- wyszecki, 1982, Color Science Concepts Methods Quantiitative Data and Formulae
- 2002, Photography-Psychophysical experimental methods to estimate image quality
- 1999, Multimedia Systems and Equipment— Colour Measurement and Management— Part 2-1 Colour Management— Default RGB Colour Space— sRGB
- pregara, 2003, Autofocus sensor
- stauffer, 1980, Range determination system
- bayer, 1976, Color imaging array
- 2000, JPEG 2000 Image Coding System
- tiff-ep, 0, Image file format overview
- 1998, Digital Still Camera Image File Format Standard (Exchangeable Image File Format for Digital Still Camera Exif)
- 1998, Parameter Values for the HDTV Standards for Production and International Programme Exchange
Cited by
CycleISP: Real Image Restoration via Improved Data Synthesis
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00277 · Cited by 393
“Digital cameras apply a series of operations on RAW sensor data in order to generate the monitor-ready sRGB images [46].”
BackgroundMethodEntropy Minimization for Shadow Removal
Finlayson, G. D., Drew, M. S., & Lu, C. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0243-z · Cited by 349
“Here, the image processing software applied is typically aimed at a “preferred” (i.e., pleasing) rendition, rather than photometric accuracy, and the number of processing steps in the camera software can be substantial (Ramanath et al. 2005).”
MethodSelf-citationVector filtering for color imaging
Lukac, R., Smolka, B., Martin, K., Plataniotis, K. N., & Venetsanopoulos, A. N. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407717 · Cited by 334
FlexISP: a flexible camera image processing framework
Heide, F., Steinberger, M., Tsai, Y.-T., Rouf, M., Pająk, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661260 · Cited by 298
“The complexity of this process is traditionally tackled by splitting the image processing into several independent pipeline stages [Ramanath et al. 2005].”
MethodCVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms
Nuutinen, M., Virtanen, T., Vaahteranoksa, M., Vuori, T., Oittinen, P., & Hakkinen, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2562513 · Cited by 227
“Then, for example, white balance, sharpness, noise reduction and colors are processed [22]–[24] before the output is encoded.”
Background
Metadata
- Authors
- Ramanath, R., Snyder, W. E., Yoo, Y., & Drew, M. S.
- Affiliations
- North Carolina State University (US)
- Texas Instruments (United States) (US)
- ORCID
- Yoo, Y.
- Venue
- IEEE Signal Processing Magazine
- Published
- 2005-01
- Volume
- 22
- Issue
- 1
- Pages
- 34-43
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2122410868
- Cited by
- 369
- References
- 50
- Citations by year
- 2026: 17 · 2025: 19 · 2024: 16 · 2023: 23 · 2022: 18 · 2021: 22 · 2020: 13 · 2019: 21 · 2018: 21 · 2017: 22 · 2016: 16 · 2015: 22 · 2014: 21 · 2013: 15 · 2012: 14
- FWCI
- 12.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Funders
- North Carolina State University
- Samsung
- Army Research Office
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › CCD and CMOS Imaging Sensors
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Keywords
- color image processing, digital still cameras, image processing pipeline, camera image processing, signal processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.