An Adaptive Richardson-Lucy Algorithm for Single Image Deblurring

Journal of Applied Science and Engineering, Vol. 16, No. 3, pp. 269-276 (2013)
DOI: 10.6180/jase.2013.16.3.06
An Adaptive Richardson-Lucy Algorithm for Single
Image Deblurring Using Local Extrema Filtering
Jiunn-Lin Wu*, Chia-Feng Chang and Chun-Shih Chen
Department of Computer Science and Engineering, National Chung Hsing University,
Taichung, Taiwan 402, R.O.C.
Abstract
Motion Blur is one of the common artifacts in digital photographing. With the population of
handheld camera and smart phone, image deblurring becomes an important problem. RichardsonLucy algorithm is well-known deconvolution algorithm. But the ringing artifacts usually appear while
the estimated point spread function is not accurate. In this paper, we proposed an improved
Richardson-Lucy deconvolution algorithm. Before deconvolution step, we separate the blurred image
into smooth part and edge part which is called the edge map. The blurred image and edge map are then
used for image deblurring. By using the proposed edge map, the ringing artifacts in the deblurred
image are significantly reduced while preserving the sharp edge information.
Key Words: Motion Blur, Deconvolution, Richardson-Lucy, Ringing Artifacts, Edge Map
1. Introduction
Motion Blur is common artifact which produces disappointing blurred image under dim light in digital photography. It is caused by camera shaking or the target
object moving during exposure time. Recently, handheld
camera and smart phone are getting more and more popular; restoring a blurred image is becoming an important
issue. Figure 1 shows the example of image deblurring.
In general, motion blur can be separated into two
types. The first one is camera shake caused the image
blurred; the second one is the target object moving. A
blurred image can be viewed as an unblurred image convolution with point spread function (PSF) which is the
movement of camera or the movement of target object. If
the point spread function is shift-invariant, the image
deblurring problem can be transformed into image deconvolution. According to the point spread function is
known or not, the deconvolution can be classified into
non-blind deconvolution [1-4] and blind deconvolution
[5-8]. Blind deconvolution is an ill-pose problem, it uses
only blurred image to restore image. Therefore image*Corresponding author. E-mail: [email protected]
pair deblurring method [9,10] is proposed to solve motion blur problem, but the additional image is hard to acquire. Nonblind deconvolution uses known point spread
function and blurred image to restore deblurred image.
Wiener filter [11] and Richardson-Lucy deconvolution
[2] are well-known nonblind deconvolution algorithms
which are good at image deblurring. But if the point
spread function is not accurate, the deblurring result will
appear undesired ringing artifacts at the smooth region
around edges. To solve this problem, Wang et al. [3] proposed using different weight values for smooth region
and edge region during the deblurring process to suppress ringing. But it still failed around some edges. Zhao
et al. [12] suggested that using only two weight values
for describing smoothness information of image is not
enough.
Figure 1. Example of image deblurring.
270
Jiunn-Lin Wu et al.
In this paper, we proposed an adaptive image deblurring algorithm using local extrema filtering. To reduce ringing artifacts from deblurred image, the extrema-based multiscale decomposition filter is used to
distinguish the edge region and smooth region, and then
edge map is obtained. According to Zhao et al. suggestion, it is better that existing more than two weight values
in edge map instead of only two weight values in edge
map. Thus we use blurred image and edge map which
contains more than two weight values are both used to
deconvolution procedure. Finally the deblurred image is
obtained with less ringing.
The rest of the paper are organized as follows. Related works which is mentioned in Introduction are described in detail. Then according to related works our
proposed method is presented. And then experiments are
designed for verifying our deblurring algorithm. Finally,
we conclude this paper.
2. Related Works
In general, a blurred image can be modeled as
(1)
where I is blurred image; L is latent unblurred image; k
is point spread function; n is noise and Ä is convolution
operator. According to point spread function is known
or not, image deconvolution can be separated into nonblind deconvolution and blind deconvolution.
Richardson-Lucy algorithm [2] is a well-known iterative deconvolution method for image deblurring. A
more clearly deblurred image is generated for each time
of iteration. After several iterations, a sharp and unblurred result image can be obtained. According to Bayes’
theorem, (1) can be transform into
spread function is not accurate enough, the ringing artifacts will appear in the result image. Figure 2 shows
that if the point spread function is not precise enough,
then the ringing artifacts will be generated in the result
image. Therefore, to reduce ringing artifacts is an important issue for image deblurring. Several methods
have been proposed to solve this problem.
Shan et al. [13] suggested that the smooth region in
the blurred image is also smooth in deblurred image.
Therefore, Shan included a constraint about the smooth
region called local prior during the restoration process.
The formulation of local prior is defined as
(4)
where i is pixel position in image, L is latent unblurred
image, I is blurred image s is the standard deviation and
W is smooth region in image. Figure 3 shows the corresponding smooth region of original image.
For this reason Wang et al. proposed improved Richardson-Lucy algorithm using local prior. It includes an
adaptive weight map called edge map during deblurring
process. Before deconvolution, the blurred image is used
to extract the edge map which indicates where the smooth
or edge region it. And then the edge map and blurred
Figure 2. Example of the ringing artifacts. (a) Original image. (b) Blurred image. (c) The result of Richardson-Lucy algorithm.
(2)
Then the formulation of Richardson-Lucy algorithm
can be defined as
(3)
where * is correlation coefficient operator, Ä is convolution operator, t is the tth of iteration, Lt is current iteration deblurred image, Lt+1 is the next iteration deblurred
image, k is PSF and I is blurred image. If the point
Figure 3. Example of edge map. (a) Blurred Image. (b) The
corresponding smooth region of blurred image.
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring
image are used as inputs for each time of iteration. When
the ringing is appear during deconvolution process, the
edge map will reduce the effect of ringing by assigning
the smooth region where ringing appeared large weight
to update it. Then the ringing artifacts can be efficiently
reduced from result.
From Figure 4, we can notice that Wang’s method
reduces most ringing artifacts, but the ringing artifacts
around the region where between two edges is not reduced successfully. In next section, we will discuss this
problem, and then proposed our deblurring algorithm.
3. Proposed Method
In this section, we will propose an adaptive Richardson-Lucy deconvolution algorithm using local extrema
filtering. Observe on the result which is deblurred by
Wang et al, the region between two edges is not fully
described by Wang’s edge map. Consequently, the ringing artifacts still exist in deconvolution result. To get an
appropriate edge map is an important issue for obtaining
a good deconvolution result. As a result, there are two
problems for extracting edge map, first is well description edge map about the smooth and edge region; the
other is only two weight values in edge map is not enough.
Extrema-based multiscale decomposition (EMD) [14]
is good at separating detail and mean layer. Compare
with existing edge-preserving image decomposition algorithms, EMD can extract the detail from input image
Figure 4. Comparison of deblurring results. (a) The result
using Richardson-Lucy. (b) The result using Wang’s
method. (c) The corresponding magnitude of (a).
(d) The corresponding magnitude of (b).
271
while preserving edge mean and contrast properly. Its
idea is getting the local extrema such that maxima or
minima, and then getting the mean layer by averaging
maximum layer and minima layer, the mean layer is also
called smooth layer. Then we extract the detail layer by
subtracting smooth layer from input image.
As mentioned before the edge map must have good
description at smooth region, thus we improved EMD
algorithm for purpose. First EMD is used for separating
the detail layer and mean layer, and then the contour is
extracted from mean layer by calculate the gradient of
input image, called smooth map. For further edge map
information, the gradient of detail layer is also used for
describing the detail part called detail map. Finally,
Combining the detail map into smooth map, then the
result map is the proposed edge map. The flowchart of
our improved EMD method is shown in Figure 6.
In Figure 7(a), we can observe that the smooth map
contains most contours, so we can extract the contours
which are strong edges (Figure 7(c)) from smooth map.
And the detail map (Figure 7(b)) contains most texture
information, thus we extract further detail information
(Figure 7(d)) from detail map. But the detail map also
contains undesired noise information, so a simple threshold method is used for filtering out the noise. Compare
proposed edge map with the edge map extracted by
Wang’s method in Figure 8; the region between two
strong edges is clear defined in our proposed edge map.
Wang’s edge map only separate the blurred image into
smooth region and edge region, it is too strict to define
Figure 5. Overview of extrema-based multiscale decomposition algorithm [12].
272
Jiunn-Lin Wu et al.
Figure 6. Flowchart of our improved EMD method.
the edge map using only two weight values. For example, some regions contain ringing artifacts or noise may
be classified as an edge region. As a result, the ringing
artifacts will still exist in result image. On the other hand,
our edge map defines the smooth region and edge region
more flexible, so the deblurring result can be reduced
more ringing artifacts.
After obtained a reasonable edge map, blurred image
and edge map are both used as inputs for deconvolution.
During the deconvolution process, the edge map is used
as the local prior of Richardson-Lucy algorithm, and then
the cost function Epl (L) = -log(pl (L)) can be defined as
Figure 7. Intermediate results of our improved EMD. (a)
Smooth map. (b) Detail map. (c) The result of gradient of smooth map and then use threshold method. (d) The gradient of detail map.
Figure 8. Comparison between our method and others. (a)
Proposed edge map. (b) Wang’s edge map.
(5)
where ÑEpl (Lt) is defined as
where L is latent unblurred image, I is blurred image,
M is edge map which describes the smoothness information of the image, ¶x is the partial derivative operator
in x direction and ¶y is partial derivative operator in y
direction.
Combine (5) with Richardson-Lucy algorithm, it can
be defined as
(6)
(7)
where * is correlation coefficient operator, ¶x* is conjugate matrix of ¶x, ¶y* is conjugate matrix of ¶y, t is the
number of iteration, Lt is current deblurred image, Lt+1
is the deblurred image of next iteration, k is PSF, I is
blurred image, M is edge map and l is the predefined
parameter. The comparison of the proposed method
with Wang’s approach is shown in Figure 9; we observe
that the ringing artifacts within red rectangle are sig-
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring
nificantly reduced in our result. In the other word, our
deblurred image has better visual result than Wang’s
method with similar PSNR. It is obvious that the ringing artifacts can be efficiently reduced by using the proposed edge map in the deconvolution step. The flowchart of our proposed method is shown in Figure 10.
4. Experiments and Discussions
The aim of the proposed method is to reduce the
ringing artifacts in the deblurred images. Therefore, in
this section, we will use several well-known natural images which failed in Richardson-Lucy algorithm, and the
synthesis images to demonstrate the effectiveness of the
proposed method.
For all test images, at first the traditional Richardson-Lucy algorithm is used as a failed example with
ringing artifacts. Then several deconvolution methods
with edge map are used to recover blurred image and reduce ringing artifacts. Then we will compare our result
with the traditional Richardson-Lucy result and other
273
methods in detail. For the first experiment, we use the
blurred image “old man” which its size is 800 ´ 532 and
PSF which its size is 27 ´ 27 to verify our method. There
are many wrinkles on the old man face and hands, so we
can observe these two parts to verify our method. Figure
11 shows the undesired result of Richardson-Lucy algorithm. And then we compare our result and edge map
with Wang’s method in Figure 12. We can observe that
compare with Wang’s method, our edge map provides
more edge information. Therefore we can reduce more
ringing artifact while preserving the edge.
Figure 13 is a magnitude view of our result and
others. Compare with traditional Richardson-Lucy algorithm, our proposed method and Wang’s method can
efficient reduce the ringing artifacts, but our method has
better image vision than Wang’s. Next experiment we
Figure 10. The flowchart of our proposed method.
Figure 9. Comparison of deconvolution result (a) Original
image (512 ´ 512). (b) PSF (21 ´ 21). (c) Wang’
method with 40 times iteration (PSNR is 27.16). (d)
Our proposed method with 40 times iteration
(PSNR is 26.72).
Figure 11. The result of traditional Richardson-Lucy deconvolution algorithm using “Old man”. (a) Blurred
image and PSF. (b) Deblurred result.
274
Jiunn-Lin Wu et al.
use blurred image “flower” which its size is 701 ´ 494
and bigger PSF which its size is 35 ´ 35 to verify our
method. The “flower” has more texture then “old man”.
Figure 14 is the result of Richardson-Lucy algorithm.
We can observe that Figure 15(a) has less ringing artifacts, but some region is failed to recover sharp edge. In
contrast, our result in Figure 15(c) has sharper result.
From Figure 16, we can see that our method has the best
image vision of the three. Next, we want use an image
which contains a lot of narrow detailed to verify the robustness of proposed method. So we use he blurred image “doll” which its size is 800 ´ 558 and bigger PSF
which its size is 41 ´ 41 to verify our method. Figure 17
is the result of Richardson-Lucy algorithm. From Figure
18, the detail part is well preserved, and the ringing artifacts are reduced successfully.
Figure 14. The result of traditional Richardson-Lucy deconvolution algorithm using “Flowers”. (a) Blurred
image and PSF. (b) Deblurred result.
Figure 12. Comparison results with other methods using “Old
man” (a) Wang’s result. (b) Wang’s edge map. (c)
Our proposed method. (d) Our corresponding edge
map.
Figure 15. Comparison results with other methods using
“Flowers” (a) Wang’s result. (b) Wang’s edge map.
(c) Our proposed method. (d) Our corresponding
edge map.
Figure 13. Close-ups which is extracted from Figures 11(b)
and 12(a), (c). (a) Richardson-Lucy algorithm. (b)
Wang’s method. (c) Our proposed method.
Figure 16. Close-ups which is extracted from Figures 14(b)
and 15(a), (c). (a) Richardson-Lucy algorithm. (b)
Wang’s method. (c) Our proposed method.
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring
The close-ups show proposed method reduces more
ringing artifacts where near the contour of the doll’s face
than other deconvolution methods.
The processing time of deblurring using all natural
images is show in Table 1.
Finally, we use the synthesis images shown in Figure
20 to verify our proposed method. The synthesis images
include, “Lena”, “Airplane” and “Fruit”. The image size
Figure 17. The result of traditional Richardson-Lucy deconvolution algorithm using “Doll”. (a) Blurred image
and PSF. (b) Deblurred result.
275
of burred “Lena”, burred “Airplane” and burred “Fruit”
are 490 ´ 490, 486 ´ 486 and 490 ´ 458 respectively. The
size of corresponding PSF are 23 ´ 23, 19 ´ 19 and 23 ´
23 respectively. The result comparisons of the proposed
method with Wang’s algorithm are shown in Figure 20. It
is obvious that the proposed method has better visual result than Wang’s approach. Table 2 shows that the proposed method has similar PSNR with Wang’s method.
The processing times for deblurring examples is shown in
Table 2. We implemented our method with Visual C++.
Net language. The testing environment was as PC running
Window 7 64 bit version with AMD Phenom II X4 945
3.4 GHz. 4 GB Ram. The processing time of the proposed
method is too long to be waiting for. The FFT (Fast Fourier transform) which is mentioned in many existence
deconvolution methods can be used to solve this problem.
Table 1. The comparison of processing time between the
proposed method and other approaches
Processing time (second)
Image
Proposed
method
Wang’s
method
Richardson-Lucy
Old man
Flower
Doll
610.5927
556.0106
816.6494
178.457
200.890
342.269
139.32
170.90
0302.936
Figure 18. Comparison results with other methods using
“Doll” (a) Wang’s result. (b) Wang’s edge map. (c)
Our proposed method. (d) Our corresponding edge
map.
Figure 19. Close-ups which is extracted from Figures 17(b)
and 18(a), (c). (a) Richardson-Lucy algorithm. (b)
Wang’s method. (c) Our proposed method.
Figure 20. Deblurring result of synthesis image. First row:
blurred image and its PSF. Second row: the result of
proposed method. Third row: the result of Wang’s
method. (a) “Lena”. (b) “Airplane”. (c) “Fruit”.
276
Jiunn-Lin Wu et al.
Table 2. The comparison between the proposed method
and Wang’s method
Processing time
(second)
PSNR (dB)
Image
Lena
Airplane
Fruit
Proposed
method
Wang’s
method
Proposed
method
Wang’s
method
28.9105
27.7854
30.7137
29.2214
27.7699
30.8389
320.7099
298.3994
298.7241
77.227
61.108
71.791
To reduce time complexity of the proposed algorithm will be the main task for future work. Our method
shares common limitation with others deconvolution
method. Huge PSF, bright spot, severe noise, and nonuniform blur will cause the deblurring failed. To solve
these problems in the proposed method will be interesting future work.
5. Conclusions
In this paper, we proposed an adaptive RichardsonLucy deconvolution algorithm using local extrema filtering. First, the local extrema filtering called EMD is used
to extract the edge map of the image which contains well
definition on smooth regions, and then the edge map and
blurred image are both used as inputs for deblurring algorithm. With the effort of edge map, the ringing artifact
can be reduced for each time of iteration. Finally, a shaper
deblurred image without ringing can be obtained. The
experiments show that our proposed method can reduce
the ringing artifacts effectively while preserving the edge
of the image.
Acknowledgements
This research is partially supported by National Science Council Grant NSC 101-2221-E-005-088.
References
[1] Levin, A., Fergus, R., Durand, F. and Freeman, W. T.,
“Image and Depth from a Conventional Camera with a
Coded Aperture,” ACM Trans. Graph., Vol. 26, No. 3,
p. 70 (2007). doi: 10.1145/1276377.1276464
[2] Richardson, W. H., “Bayesian-Based Iterative Method
of Image Restoration,” JOSA, Vol. 62, No. 1, pp.
55-59 (1972). doi: 10.1364/JOSA.62.000055
[3] Wang, Y., Feng, H., Xu, Z., Li, Q. and Dai, C., “An Improved Richardson-Lucy Algorithm Based on Local
Prior,” Optics & Laser Technology, Vol. 42, No. 5, pp.
845-849 (2010). doi: 10.1016/j.optlastec.2010.01.001
[4] Yuan, L., Sun, J., Quan, L. and Shum, H., “Progressive
Inter-Scale and Intra-Scale Non-Blind Image Deconvolution,” ACM Trans. Graph., Vol. 27, No. 3 (2008).
doi: 10.1145/1360612.1360673
[5] Levin, A., Weiss, Y., Durand, F. and Freeman, W. T.,
“Understanding and Evaluating Blind Deconvolution
Algorithms,” Proc. CVPR, pp. 1964-1971 (2009). doi:
10.1109/CVPRW.2009.5206815
[6] Kundur, D. and Hatzinakos, D., “Blind Image Deconvolution,” IEEE Signal Processing Magazine, Vol.
13, No. 3, pp. 43-64 (1996). doi: 10.1109/79.489268
[7] Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T.
and Freeman, W. T., “Removing Camera Shake from a
Single Photograph,” ACM Trans. Graph., Vol. 25, No.
3, pp. 787-794 (2006). doi: 10.1145/1141911.1141956
[8] Xu, L. and Jia, J., “Two-Phase Kernel Estimation for
Robust Motion Deblurring,” Lecture Notes in Computer Science, Vol. 6311, Computer Vision - ECCV,
pp. 157-170 (2010). doi: 10.1007/978-3-642-155499_12
[9] Yuan, L., Sun, J., Quan, L. and Shum, H., “Image
Deblurring with Blurred/Noisy Image Pairs,” ACM
Trans. Graph., Vol. 26, No. 3 (2007). doi: 10.1145/
1276377.1276379
[10] Zhuo, S., Guo, D. and Sim, T., “Robust Flash Deblurring,” in Proc. CVPR, pp. 2440-2447 (2010). doi:
10.1109/CVPR.2010.5539941
[11] Gonzalez, R. C. and Woods, R. E., Digital Image
Processing, 2nd ed., Prentice Hall (2002).
[12] Zhao, J. F., Feng, H. J., Xu, Z. H. and Li, Q., “An Improved Image Restoration Approach Using Adaptive
Local Constraint,” Optik, Vol. 123, pp. 982-985 (2012).
doi: 10.1016/j.ijleo.2011.07.014
[13] Shan, Q., Jia, J. and Agarwala, A., “High-Quality Motion Deblurring from a Single Image,” ACM Trans.
Graph., Vol. 27, No. 3 (2008). doi: 10.1145/1360612.
1360672
[14] Subr, K., Soler, C. and Durand, F., “Edge-Preserving
Multiscale Image Decomposition Based on Local Extrema,” in SIGGRAPH Asia, Singapore (2008).
Manuscript Received: Feb. 27, 2013
Accepted: Jun. 28, 2013