Weighted Average
In this method, the algorithm takes the weighted average of the differently exposed images. The algorithm weighs the media exposure more heavily since the most objects are likely to be within the dynamic range of the camera. The weights are the same for all pixels and depend on the number of exposures.
Contrast Blending
This method is similar to the weighted average in that each pixel in the resulting image is an average of the pixels from all the exposures, but the weight for each pixel is different. This algorithm works for sets of multiple images even with just two images. With two exposures, there is a long (L) exposure and a short (S) exposure. We use the grayscale value of the long exposure as the weight of the short exposure because the bright pixels in the long exposure may be blown out or actually a bright object. In either case, we would want to use the pixel value in the short exposure. Dark pixels in the long exposure (since exposed longer) most likely represent a dark object in the scene. For each pixel, the resulting pixel is a weighted average of the short and long exposure pixel where the grayscale value of the long exposure pixel is the weight for the short pixel. The pixel is scaled such that energy of the pixels is not increased. This easily extends to multiple images. First, blend the two images with the longest exposure as described above, and blend the result with the next longest exposure. This is repeated until all the images are used.
Divide, Choose & Blend
This algorithm is based on the idea that one exposure will capture a part of the scene best. The algorithm blends the different exposures by placing the most weight on the best exposure for the pixel in question.
The algorithm works as follows. First, divide the scene into square blocks. For each of these blocks, the block with the most details as determined by entropy is chosen. Blocks of the same best exposure tend to appear in clusters because of object coherence, but the boundaries between blocks of different exposures will be visible. A weighted average of the exposures is needed to smoothly blend the edges of the blocks. A pixel value will be strongly weighted by the pixel value from the best exposure chosen for the block in which the pixel is located and will be influenced by the pixel value at the same location in the best exposures of nearby blocks. The pixels are blended in LAB coordinates to ensure proper blending of colors. Optimization of the size of blocks and the widths of the Gaussians used in the blending process is needed to produce an image with the most entropy.
For a more detailed explanation, check out the paper by Arthur Goshtasby that is listed in the reference section. An important thing to note is that the optimization described in the paper does not always lead to the most visually pleasing image. In addition, the main assumption of the chosen optimization method is usually true, but not always. Therefore, the size of the blocks and the widths of the Gaussians may need to be determined manually. If this is the case, a good rule of thumb is to keep the width of the Gaussians larger than the width of the square block.
Radiance Mapping
For this method, RGB can be treated separately and then combined at the end. First, the multiple exposures are used to recover the camera response function. It is better to use as many exposures as possible when recovering the response function. Given a response function, a radiance map is created for the scene. We use a weighted average so that the saturated pixels are weighted less. Afterwards, the radiance map is stretched to fit the 0 to 255 range. The radiance map is in terms of the natural log of the irradiance value (which is proportional to the radiance value). Lastly, the radiance values are tone mapped logarithmically to fit the 0 to 255 range.
Results
Original Images
 |
 |
 |
 |
 |
| Exposure Time: 0.1s | Exposure Time: 0.025s | Exposure Time: 0.00625s | Exposure Time: 0.002s | Exposure Time: 0.0005s |
 |
 |
 |
 |
 |
| Exposure Time: 0.03333s | Exposure Time: 0.01666s | Exposure Time: 0.008s | Exposure Time: 0.004s | Exposure Time: 0.002s |
 |
 |
| Exposure Time: 0.25s | Exposure Time: 0.03333s |
 |
 |
 |
 |
 |
 |
 |
| Exposure Time: 30s | Exposure Time: 8s | Exposure Time: 2s | Exposure Time: 0.5s | Exposure Time: 0.125s | Exposure Time: 0.03333 | Exposure Time: 0.008s |
Histogram Flattener
This method is the only method that works with one image, so we didn't really have anything to compare it to. The transformation is monotonic but nonlinear. This method showed more details in some of the regions but had to sacrifice the contrast in other regions.
In the example below, the sunset picture is taken from the original set above. The histogram is displayed below it where the intensities are mostly in the high and low ends of the histogram, leaving the mid-intensities pretty much unused. The histogram is remapped into something more flat. The resulting image (in the upper right) shows more detail in the grass and sand.
|
Figure 5: Original Sunset Image |
 Figure 6: Histogram Flattener Result |
 Figure 7: Histogram of Sunset Image |
 Figure 8: Histogram of Result |
The algorithm was tried on all the test images above and the results are shown below.
This method is computationally easy and yields decent recents. The algorithm can fail when the dynamic range of the scene is too great such that the entire image is either near the minimum or maximum intensity values except for a small region in the image. As seen in the Memorial Church images, the dynamic range of the scene required 7 exposures, many of which are completely dark except for the stained glass windows. The algorithm fails because the median exposure is too dark and draws a lot of weight, making the resulting image dark.
 Figure 9: Result of Weighted Average on Sunset Images |
 Figure 10: Result of Weighted Average on Golf Images |
 Figure 11: Result of Weighted Average on Beach Images |
 Figure 12: Result of Weighted Average on Memorial Church Images |
This algorithm suffers greatly in the case of misaligned images. Below is the result of the weighted average algorithm with one image from the Memorial Church set slightly rotated from the others.
Figure 13: Result of Weighted Average on Memorial Church Images with 1 image slightly misregistered
Contrast Blending
The method works well and is computationally easy. This method also suffers from the same problem when the dynamic range is large since this also averages over all the exposures although this method does not suffer from this problem as badly as the weighted average method.
Without blurring the alpha mask, the resulting image of the Contrast Blending algorithm will have artifacts at the edge, so we need to apply a blurring function to the mask. But as a result of the blurring function, there are small halos around objects as can be seen in the tree on the right-hand side of Figure 15 (Note, Figure 15 only blends 2 images instead of the entire Sunset set) . The size of the halo grows with the size of the Gaussian width.
 |
 |
In general, this method does a pretty good job at blending the multiple exposures. The ground in the golf image (Figure 18) is still somewhat dark and lacking contrast. The problem of the large dynamic range with several dark images is shown especially in the Memorial Church image (Figure 21). In order to get nice contrast and details in the stained glass windows, the algorithm needed to use the really short exposure photo, which had the entire scene black except the windows. Thus, the interior of the church is really dark as well. This method generally produces nice skies with suns visible in the sky because the algorithms blends in the shortest exposure last, which makes it capable of incorporating the sun in the sky.
 Figure 18: Result of Contrast Blending on Sunset Images |
 Figure 18: Result of Contrast Blending on Golf Images |
 Figure 19: Result of Contrast Blending on Beach Images |
 Figure 20: Result of Contrast Blending on Memorial Church Images |
Divide, Choose & Blend
The resulting images have great color contrast and arguably are the best pictures in our project, but the optimization code takes a long time to run (2 hours or more). This method provides the best walkway for the golf pictures (Figure 22). The interior of the Memorial Church image (Figure 24) looks the best in this method, but the stained glass windows are still blown out. This results from the blending of nearby best exposure blocks. Since the windows are the only portions of the image from the short exposure and everything else is best captured by longer exposures, the longer exposures have a lot of weight due to the large number of them in the rest of the scene. Fine details can become blurry due to blending. The stained-glass windows in the Memorial Church photo are the best example of this.
 Figure 21: Result of Divide, Choose & Blend Method on Sunset Images |
 Figure 22: Result of Divid, Choose & Blend Method on Golf Images |
 Figure 23: Result of Divide, Choose & Blend Method on Beach Images |
 Figure 24: Result of Divide, Choose & Blend Method on Memorial Church Images |
Radiance Mapping
The fine details are preserved, which can be seen in the stain-glass windows in the Memorial Church photo. Abnormal colors may appear in the final images due to bad sampling of pixels for the recovery of the camera response function. All the resulting images look faded as a result of stretching the radiance map from 0 to 255.
 Figure 25: Result of Radiance Mapping on Sunset Images |
 Figure 26: Result of Radiance Mapping on Golf Images |
 Figure 27: Result of Radiance Mapping on Beach Images |
 Figure 28: Result of Radiance Mapping on Memorial Church Images |
Conclusions
We found that the Block method (Divide, Choose & Blend) worked the best since it chose the best exposure for each block. This method was the most robust since it did not incorporate an image if it didn't contribute to the details in any part of the image.
The contrast blending method was good as well. We believe Radiance Mapping should have worked better given the sample photos we've seen in papers. In our results, our pictures have abnormal colors and overall looks faded. We believe that we're not accurately recovering the response function.
Future Work
We would like to improve the algorithms we've implemented to be more robust to the set of input images. For example, the exposure blending algorithms can throw out exposures if they are close to the extremes. Although the Block method worked the best, it was way too slow to run. We would like to look into optimizing the code. We believe we should be able to yield good results with the Radiance Mapping method, so we would also want to look into this method in more detail.
We would also like to look at combining several algorithms to yield better results or to make the process more robust.
References
P.E. Debevec and J. Malik, "Recovering High Dynamic Range Radiance Maps from Photographs," Proceedings of SIGGRAPH 1997, ACM Press / ACM SIGGRAPH, 369-378, 1997.
http://www.debevec.org/Research/HDR/debevec-siggraph97.pdf
A. Goshtasby, "High Dynamic Range Reduction Via Maximization of Image Information"
http://www.cs.wright.edu/~agoshtas/goshtasby_hdr.pdf
Reinhard, Erik, G. Ward, S. Pattanaik, P. Debevec. High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting. Morgan Kaufman Publishers: San Francisco, 2005.
Appendix 1
Histogram Flattener
hist_flat
Weighted Average
weighted_average
Contrast Blending
contrast_blend
Divide, Choose & Blend
blend7
choose
choose7
Gaussian
GaussianTable
ieNotDefined
imageLAB2RGB
imageRGB2LAB
lab2xyz
monitor.mat
optimize7
vcXYZ2lab
XYZ.mat
xyz2lab
Radiance Mapping
gsolve
map
radmap7
radmapbeach
radmapchurch
radmapmountain
weight
Appendix 2
Work Done
Tina: Histogram Flattener, Contrast Blending, presentation slides, writeup, compiliation of slides and writeup
Sufeng: Weighted Average, presentation slides, writeup
Michael: Divide, Choose & Blend, Radiance Map, presentation slides, writeup