Evaluation of Noise Characteristics
in Image Pipeline

Annabel Huo, Hyung Suk Kim, Sung Hee Park
March 21, 2009
Psych 221 : Applied Vision and Image Systems

  |   Introduction   |   Methods   |   Results   |   Conclusions   |   References   |   Appendix   |  

Conclusions
    We have done various experiments to evaluate and characterize the performance of image pipelines to do demosaic and denoise. To get a better insight in each operation, we analyzed each stage separately and compared with several intermediate results. From these observations we were able to explain what is really happening in the middle of image pipelines.
    Demosaic algorithms are designed to interpolate pixel values that are not measured directly from the sensor. However, many methods do not make assumptions about input noise, which make them vulnerable to low quality input images. In addition, demosaic operation changes input noise characteristics and this fact can affect rest of the following image processing stages.
    Many state-of-art denoising methods work very well when input noise level is low. But when noise level gets higher or noise characteristics are different from their assumptions, the performance drops significantly.
    Usually, demosaicing is done before denoising. However, demosaic process suffer from bad input noise while correlated noise passed from demosaic operation degrades the output of denoising algorithms. Thus, it is reasonable to consider doing denoising before applying demosaic method. Demosaic can be done better at low noise level without color channel correlation. Also, denoising will not have to deal with weird noise patterns, making better quality of output images.
    Joint demosaic and denoise algorithm is proposed to handle this problem at the same time. For the best cases, it outperforms the case when we do operations separately. However, it requires more detailed characteristics about input noise and takes much longer time to get results. More study is needed to improve the performance of joint algorithms.
    There are several more things we can think about for the later works. Applying denoising algorithms to RAW images are not quite straight forward depending on the method used in the algorithm. Decomposing raw image and then denoising may not be a good choice and it can introduce another parameter to tweak. Choosing the optimal parameters for denoise algorithm is cumbersome because the performance highly depends on the correct choice of parameters. In addition, the optimal parameters vary with input image and input noise level, making it impossible to get the best one for every case. It will be desirable to come up with a way to automatically find good values for denoising. Or introducing an explicit parameter that clearly represents the trade-off between blurring and reducing noise will be useful. If we can have a fine control on denoising algorithms, it will be possible to perform fair comparisons between various image pipelines in more quantitative way.


References Top
Demosaic Algorithms
  • Gunturk, B.K., Glotzbach, J., Altunbasak, Y., Schafer, R.W., Mersereau, R.M., Demosaicking: color filter array interpolation, Signal Processing Magazine, IEEE , vol.22, no.1, pp. 44-54, Jan. 2005

  • Hirakawa, K., Parks, T.W., Adaptive homogeneity-directed demosaicing algorithm, Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on , vol.3, no., pp. III-669-72 vol.2, 14-17 Sept. 2003

  • Gunturk, B.K., Altunbasak, Y., Mersereau, R.M., Color plane interpolation using alternating projections, Image Processing, IEEE Transactions on , vol.11, no.9, pp. 997-1013, Sep 2002

  • Dubois, E., Frequency-domain methods for demosaicking of Bayer-sampled color images, Signal Processing Letters, IEEE , vol.12, no.12, pp. 847-850, Dec. 2005

Denoise Algorithms
  • Dabov, K., Foi, A., Katkovnik, V. , and Egiazarian, K., Image denoising with block-matching and 3D filtering, in Electronic Imaging'06, Proc. SPIE 6064, no. 6064A-30, San Jose, California USA, 2006.

  • Chen, J., Paris, S., and Durand, F. Real-time edge-aware image processing with the bilateral grid. In ACM SIGGRAPH 2007 Papers (San Diego, California, August 05 - 09, 2007). SIGGRAPH '07. ACM, New York, NY, 103. DOI= http://doi.acm.org/10.1145/1275808.1276506

  • Portilla J., Strela V., Wainwright M., Simoncelli, E. P., Image Denoising using Scale Mixtures of Gaussians in the Wavelet Domain. IEEE Transactions on Image Processing. vol 12, no. 11, pp. 1338-1351, November 2003.

  • Martinec E., Noise, Dynamic Range and Bit Depth in Digital SLRs, http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html

Joint Demosaic-Denoise Algorithms
  • Hirakawa, K., Parks, T.W., Joint demosaicing and denoising, Image Processing, IEEE Transactions on , vol.15, no.8, pp.2146-2157, Aug. 2006

Image Evaluation and Simulation
  • Ramanath, R., Snyder, W.E., Yoo, Y., Drew, M.S., Color image processing pipeline, Signal Processing Magazine, IEEE , vol.22, no.1, pp. 34-43, Jan. 2005

  • Zhang, X., Farrell, J., Wandell, B. A., Applications of a spatial extension to CIELAB, in: SPIE, 1997

  • Farrell, J. E., Image Quality Evaluation, In Color Imaging: Vision and Technology, eds. L.W. Macdonald and M. R. Luo, Wiley Press, 1999, pp. 285-313

  • Farrell, J. E., Okincha, M., Parmar, M., Sensor calibration and simulation, In Proceedings of the SPIE Electronic Imaging '2008 Conference, Vol. 6817, 2008

  • Farrell, J. E., Xiao, F., Catrysse, P., Wandell, B., A simulation tool for evaluating digital camera image quality. In Proc. SPIE Int. Soc. Opt. Eng., Vol. 5294, 124, 2004.

Appendix Top
Source Codes and Input Images
   
[Presentation Slides]    

[Source Codes]    



|Folders|
[Algorithms]
=Demosaic=
/MNdemosaic - Adaptive Homogeneity
/demosaick - POCS
/frequency - Adaptive Frequency Method

=Denoise=
/BM3D - BM3D
/bilateral - Bilateral Filter
/BLS_GSM - BLS-GSM

=Joint=
/JointDemosaic

=Image Quality=
/scielab - sCIELAB. use
/errorPlots - error plot functions.

=Data=
/images - contains the image files in .mat format.
/data - contains the data files with MSE, sCIELAB values in .CSV and .XLS format.

|Files|
[Executable Files]
iset_*.m - files for acquiring RAW and TRUE images from ISET.
ipl_*.m - files for testing image pipelines.
noise_test_*.m - files for acquiring MSE and sCIELAB values and create .CSV files.

[Functions]
applyDemosaic.m - applies demosaic to RAW image.
applyDenoise.m - applies denoise to RGB image.
applyDenoiseSingle.m - applies denoise to single channel image.

decomposeRaw.m - decomposes RAW image into each color channel.
decomposeRaw2.m - decomposes RAW image into each color channel for denoising. G channel is interpolated.
mosaicRGB.m - creates RAW image from RGB image.

displayRGB.m - displays RGB image
displayRAW.m - displays RAW image in proper color.
displayError.m - method for obtaining error plots.
evaluateQuality - method for getting MSE and sCIELAB values.

loadImage.m - a helper function for loading an image from the /image folder.



Work Distribution
  • Sung Hee Park
    • - Image Acquisition
    - Image Pipeline Simulation
    - Demosaic Algorithms
    - Denoise Algorithms
    - Image Plots
    - Webpage Setup

  • Hyung Suk Kim
    • - Image Pipeline Simulation
    - Data Acquisition/Analysis
    - Denoise Algorithms
    - Error Plots

  • Annabel Huo
    • - Joint Demosaic Denoise
    - Demosaic Algorithms
    - Image Pipeline Simulation