Digital Image Processing

Chapter 5:
Image Restoration

1
Concept of Image Restoration

Image restoration is to restore a degraded image back to

the original image while image enhancement is to
manipulate the image so that it is suitable for a specific
application.

Degradation model:
g(x, y) = f (x, y)  h( x, y) +  ( x, y)

where h(x,y) is a system that causes image distortion and

(x,y) is noise, * is the convolution. (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
2
White Noise
White noise : Fourier spectrum of noise is constant

Fourier transform of unit impulse function :


F ( (t)) =  (t)e− jt dt =1
−

Unit impulse contains Component at Every Frequency

3
 .
White Noise(white random process)

A continuous time random process w(t) is a white noise

process if and only if its mean function and autocorrelation
function satisfy the following:

→ Stochastically independent in t1 and t2

White Gaussian noise (identically distributed) : pij=p=N(0,)

Salt and pepper noise

4
Noise Models
Noise cannot be predicted but can be approximately described in
statistical way using the probability density function (PDF)

5
Noise Models (cont.)

6
PDF: Statistical Way to Describe Noise

PDF tells how much

each z value occurs.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition. 7
Image Degradation with Additive Noise

g(x, y) = f (x, y) +  ( x, y)

Degraded images

Original image

Histogram

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
8
Image Degradation with Additive Noise (cont.)

g(x, y) = f (x, y) +  ( x, y)

Degraded images

Original image

Histogram

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
9
Periodic Noise

• Periodic noise :
by electrical or
electromechanical
interference during
image acquisition

• Can be reduced
by frequency
domain filtering

Periodic noise looks

like dots In the
frequency domain

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
10
Estimation of Noise

We cannot use the image

histogram to estimate
noise PDF.

 =  zi p(zi )
zi S

 2 =  (z i − )2 p(z i)
zi S

It is better to use the

histogram of one area
of an image that has
constant intensity to
estimate noise PDF.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
11
Periodic Noise Reduction by Freq. Domain Filtering
Degraded image DFT
Periodic noise
can be reduced by
setting frequency
components
corresponding to
noise to zero.

Band reject filter Restored image

(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
12
Band Reject Filters
Use to eliminate frequency components in some bands

Periodic noise from the

previous slide that is
Filtered out.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
13
Notch Reject Filters
A notch reject filter is used to eliminate some frequency components.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
14
Notch Reject Filter:

15
Example: Image Degraded by Periodic Noise
Degraded image

DFT
(no shift)

DFT of noise Noise Restored image

(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
16
Mean Filters
Degradation model:

g(x, y) = f (x, y)  h( x, y) +  ( x, y)

To remove this part

Arithmetic mean filter or moving average filter (from Chapter 3)

fˆ( x, y)=
1
 g(s, t)
mn ( s,t )S xy
Geometric mean filter

• mn = size of moving window

• Achieves smoothing, lose less
image detail

17
Geometric Mean Filter: Example

Image
Original
corrupted
image
byAWGN

By 3x3 By 3x3
arithmetic geometric
mean filter mean filter
:less blurring

AWGN: Additive White Gaussian Noise (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
18
Harmonic and Contraharmonic Filters
Harmonic mean filter

Works well for salt noise

but fails for pepper noise

Contraharmonic mean filter mn = size of moving window

Q = order of the filter
 g(s, t) Q+1
Positive Q is suitable for
fˆ (x, y) =
( s,t )Sxy eliminating pepper noise.

 g(s, t) Q
eliminating salt noise.
( s,t )Sxy

For Q = 0, the filter reduces to an arithmetic mean filter.

For Q = -1, the filter reduces to a harmonic mean filter.

19
Contraharmonic Filters: Example

Image Image
corrupted corrupted
by pepper by salt
noise with Noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q = 1.5 With Q= -1.5

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
20
Contraharmonic Filters: Incorrect Use Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q= -1.5 With Q=1.5

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
21
Order-Statistic Filters: Revisit
Original image
subimage

Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.
Moving
window

Order-statistics filters are nonlinear spatial filters

whose response is based on ordering (ranking) the
pixels contained in the image area encompassed by
the filter,
and then replacing the value of the center pixel with
the value determined by the ranking result.
: median filter Output image

22
Order-Statistics Filters
Median filter

fˆ(x, y) = mediang(s, t)

( s,t )Sxy

Max filter

fˆ( x, y) = max g(s, t) Reduce “dark” noise

( s,t )Sxy
(pepper noise)
Min filter

fˆ( x, y) = min g(s,t) Reduce “bright” noise

(salt noise)
( s,t )S xy

Midpoint filter : order statistics + averaging, best for randomly distributed noise

23
Median Filter : How it works
A median filter is good for removing impulse, isolated noise

24
Median Filter : Example
1 2
Image
corrupted
by salt-and-
pepper
noise with
pa=pb= 0.1

3 4

Images obtained using a 3x3 median filter

25
Max and Min Filters: Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
max filter min filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
26
Alpha-trimmed Mean Filter

Formula:

where gr(s,t) represent the remaining mn-d pixels after

removing the d/2 highest and d/2 lowest values of g(s,t).
If d=0, reduces to arithmetic mean filter

This filter is useful in situations involving multiple types

of noise such as a combination of salt-and-pepper and
Gaussian noise.

27
Alpha-trimmed Mean Filter: Example
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2 Image 2
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
28
Alpha-trimmed Mean Filter: Example (cont.)
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2
obtained
Image 2
using a 5x5
obtained
alpha-
using a 5x5
trimmed
median filter
mean filter
with d = 5

29
Alpha-trimmed Mean Filter: Example (cont.)

Image Image
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

Image
Image obtained
obtained using a 5x5
using a 5x5 alpha-
median filter trimmed
mean filter
with d = 5

30
Adaptive Filter
General concept:
-Filter behavior changes based on statistical characteristics of local areas
inside mxn moving window
- More complex but superior performance compared with “fixed”
filters

Statistical characteristics:

31
Adaptive, Local Noise Reduction Filter
Purpose: want to preserve edges

32
Adaptive Noise Reduction Filter: Example
Image Image
corrupted obtained
by additive using a 7x7
Gaussian arithmetic
noise with mean filter
zero mean
and  2 =1000

Image
Image obtained
obtained using a 7x7
using a 7x7 adaptive
geometric noise
mean filter reduction
filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
33
Adaptive Median Filter
Purpose: want to remove impulse noise while preserving edges
1) Impulse noise reduction
2) Other noises reduction : smoothing
3) Edge preserving
Level A: A1= zmedian – zmin
A2= zmedian – zmax
If A1 > 0 and A2 < 0, goto level B
Algorithm: Else increase window size
If window size <= Smax repeat level A
Else return zxy
Level B: B1= zxy – zmin
B2= zxy – zmax
If B1 > 0 and B2 < 0, return zxy
Else return zmedian
zmin = minimum gray level value in Sxy
zmax = maximum gray level value in Sxy
Where, zmedian = median of gray levels in Sxy
zxy = gray level value at pixel (x,y)
Smax = maximum allowed size of Sxy
34
Adaptive Median Filter: How it works
Level A: A1= zmedian – zmin
Determine
A2= zmedian – zmax
whether zmedian
is an impulse or not
If A1 > 0 and A2 < 0, goto level B
Else → Window is not big enough
increase window size
If window size <= Smax repeat level A
Else return zxy

Level B: → zmedian is not an impulse

B1= zxy – zmin Determine whether zxy
B2= zxy – zmax is an impulse or not
If B1 > 0 and B2 < 0, → zxy is not an impulse
return zxy → to preserve original details
Else
return zmedian → to remove impulse

35
Adaptive Median Filter: Example

Image corrupted Image obtained Image obtained

by salt-and-pepper using a 7x7 using an adaptive
noise with median filter median filter with
pa=pb= 0.25 Smax = 7

More small details are preserved

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
36
Estimation of Degradation Model
Degradation model:
g(x, y) = f (x, y)  h( x, y) +  ( x, y)
or
G(u, v) = F(u, v)H (u, v) + N (u, v)

Purpose: to estimate h(x,y) or H(u,v)

Why? If we know exact h(x,y), regardless of noise, we can do
deconvolution to get f(x,y) back from g(x,y).

Methods:
1. Estimation by Image Observation

2. Estimation by Experiment

3. Estimation by Modeling (Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
37
Estimation by Image Observation
Original image (unknown) Degraded image

f(x,y) f(x,y)*h(x,y) g(x,y)

Observation

DFT Subimage
Estimated Transfer Gs (u,v) gs (x, y)
function
Restoration
Gs (u,v)
H (u, v)  H s (u, v) = process by
F̂s (u,v) estimation
DFT Reconstructed
This case is used when we Fˆs(u, v)
know only g(x,y) and cannot Subimage
repeat the experiment! fˆs( x, y)
38
Estimation by Experiment
Used when we have the same equipment set up and can repeat the
experiment.
Response image from
Input impulse image the system

39
Estimation by Modeling
Used when we know physical mechanism underlying the image
formation process that can be expressed mathematically.

40
Estimation by Modeling: Motion Blurring
Assume that camera velocity is (x0 (t), y0 (t))
The blurred image is obtained by
T

g(x, y) =  f (x − x0 (t), y − y0 (t))dt

0
where T = exposure time.

41
Estimation by Modeling: Motion Blurring (cont.)

42
Motion Blurring Example
For constant motion
T
H (u, v) = sin( (ua + vb))e− j (ua+vb)
(ua + vb)

Original image Motion blurred image

a = b = 0.1, T = 1
(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
43
Inverse Filter
From degradation model:

G(u, v) = F(u, v)H (u, v) + N (u, v)

after we obtain H(u,v), we can estimate F(u,v) by the inverse filter:

G(u, v) N (u, v)
Fˆ(u,v) = = F (u, v) +
H (u, v) H (u, v)

Noise is enhanced
when H(u,v) is small. To avoid the side effect of enhancing
noise, we can apply this formulation
to freq. component (u,v) within a
radius D0 from the center of H(u,v).

In practical, the inverse filter is not popularly used.

(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
44
Inverse Filter: Example

45
Wiener Filter: Minimum Mean Square Error Filter

46
Approximation of Wiener Filter
Wiener Filter Formula:

47
Wiener Filter: Example

48
Example: Wiener Filter and Motion Blurring

49
Constrained Least Squares Filter

Assumptions on Wiener filtering:

• degradation function H is known
• power spectra of undegraded image and noise is known
• a constant estimate of the ratio of power spectra
• minimize a statistical criterion, is optimal in an average sense

Constrained Least Squares Filtering

• needs only the mean and variance of the noise?
• optimal result for each image

50
Constrained Least Squares Filter
Degradation model: Written in a matrix form
g( x, y) = f (x, y)  h( x, y) +  ( x, y) g = Hf + η
Restoration problem is reduced to simple matrix manipulations?
1) g = (MxN), H = (MNxMN), matrices are too big to handle
2) H is highly sensitive to noise

Noise sensitivity problem → measure of smoothness

Objective: to find the minimum of a criterion function

Subject to the constraint

ˆ w 2 = wT w
2
g − Hf = η
2 where

51
Constrained Least Squares Filter

The frequency domain solution to the problem:

a constrained least square filter

52
Constrained Least Squares Filter: Example
Constrained least square filter

 is adaptively adjusted to achieve the best result.

Results from the previous slide obtained

from the constrained least square filter

53
Constrained Least Squares Filter: Example (cont.)

54
Constrained Least Squares Filter:Adjusting 
( ) =rT r = r
2
r = g − Hfˆ Monotonically increasing:

r = η a
2 2
We want to adjust gamma so that 1
where a = accuracy factor
1. Specify an initial value of
2
2. Compute r

3. Stop if 1 is satisfied
Otherwise return step 2 after increasing  if r  η −a
2 2

or decreasing  if r  η +a
2 2

Use the new value of  to recompute

55
Constrained Least Squares Filter:Adjusting  (cont.)

56
Constrained Least Squares Filter: Example
Original image Use correct noise
parameters Correct parameters:
Initial  = 10-5
Correction factor = 10-6
a = 0.25
 2 =10-5

Blurred image Use wrong noise Wrong noise parameter

Due to Turbulence parameters 2 = 10-2

(Images from Rafael C.

Gonzalez and Richard E.
Wood, Digital Image
Processing, 2nd Edition.

Results obtained from constrained least square filters

57
Geometric Mean filter
This filter represents a family of filters combined into a
single expression

58
Geometric Transformation
Geometric transformations modify the spatial relationships between pixels
in an image.

These transformations are often called rubber-sheet transformations:

Printing an image on a rubber sheet and then stretch this sheet according
to some predefined set of rules.

A geometric transformation consists of 2 basic operations:

1. spatial transformation :
rearrangement of pixels on the image plane
2. gray level interpolation :
Assign gray level values to pixels in the spatially
transformed image.

59
Geometric Transformation : Algorithm

60
Spatial Transformation
To map between pixel coordinate (x,y) of f and pixel coordinate
(x’,y’) of g

For a bilinear transformation mapping between a pair of

x' = r( x, y) = c1x + c2 y + c3 xy + c4

y' = s( x, y) = c5 x + c6 y + c7 xy + c8

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
61
Gray Level Interpolation: Nearest Neighbor

Example interpolation methods that can be used:

1. Nearest neighbor selection
2. Bilinear interpolation
3. Bicubic interpolation

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
62
Geometric Distortion and Restoration Example

63
Geometric Distortion and Restoration Example (cont.)

64
Example: Geometric Restoration
Original image Geometrically distorted
image
Use the same
Spatial Trans.
as in the previous
example

Restored image

Difference between (Images from Rafael C.

Gonzalez and Richard E.
2 above images Wood, Digital Image
Processing, 2nd Edition.
65