Digital Image Processing

Unit - 3
Image Restoration

 Image restoration is to restore a degraded image back to the original image

 Restoration improves image in some predefined sense.
 It is an objective process.
 Restoration attempts to reconstruct an image that has been degraded by using a priori
knowledge of the degradation phenomenon.
 Restoration techniques are based on mathematical or probabilistic models of image
processing.
 Image Restoration refers to a class of methods that aim to remove or reduce the
degradations that have occurred while the digital image was being obtained.

 Image restoration in digital image processing refers to the process of recovering an

image that has been degraded by various factors such as noise, blur, or other
distortions.
 The goal is to reconstruct the original image with minimal loss of information and
improved visual quality.
 This process involves several techniques and algorithms designed to reverse the
effects of degradation and enhance the image's clarity and accuracy.

Key aspects of image restoration include:

1. Noise Reduction: Removing random variations in pixel intensity that can occur during
image capture or transmission. Common types of noise include Gaussian noise, salt-and-
pepper noise, and speckle noise. Techniques like filtering (e.g., median filter, Wiener filter)
and advanced methods like wavelet denoising are used.

2. Deblurring: Correcting the blurriness caused by motion, out-of-focus capture, or

atmospheric conditions. Methods like deconvolution are applied to reverse the blurring
process.

3. Inpainting: Filling in missing or corrupted parts of an image. This technique is used in

situations where parts of the image are lost or corrupted, such as in old photographs or
images with scratches.

4. Correction of Geometric Distortions: Addressing distortions caused by lens imperfections

or perspective issues. Techniques involve geometric transformations to align the image
correctly.

5. Super-Resolution: Enhancing the resolution of an image by reconstructing high-resolution

details from a low-resolution input. This involves algorithms that can infer and add fine
details.
6. Artifact Removal: Eliminating artifacts that result from compression, such as blockiness in
JPEG images.

Image restoration is distinct from image enhancement, which primarily focuses on improving
the visual appearance of an image without necessarily aiming to recover the original image's
exact content. Restoration aims to retrieve the original scene as accurately as possible,
leveraging mathematical models and knowledge about the degradation process.

Techniques and Algorithms in Image Restoration

1. Filtering Techniques:
* Linear Filters: Such as Gaussian filters to reduce Gaussian noise.
* Non-Linear Filters: Such as median filters to reduce salt-and-pepper noise.

2. Inverse Filtering: Used to deblur images by reversing the blurring process, assuming the
blur function is known.

3. Wiener Filtering: A statistical approach that reduces noise and blur based on known signal
and noise characteristics.

4. Regularization Methods:
*Tikhonov Regularization: Balances the fidelity of the solution to the observed data and
smoothness of the solution.
*Total Variation Regularization: Preserves edges while reducing noise.

5. Iterative Methods:
*Expectation Maximization (EM): An iterative approach for maximum likelihood estimation
in the presence of missing or incomplete data.
*Richardson-Lucy Deconvolution: An iterative method particularly used for image
deblurring.

6. Machine Learning and Deep Learning:

*Convolutional Neural Networks (CNNs): Used for tasks like denoising, deblurring, and
super-resolution.
*Generative Adversarial Networks (GANs): Applied for high-quality image restoration,
leveraging a generator and discriminator to produce realistic results.

Image restoration plays a crucial role in various fields, including medical imaging, astronomy,
photography, and surveillance, where high-quality and accurate images are essential.
Degradation model :-

In image restoration, the degradation model is a mathematical representation of how an

original image 𝑓(𝑥,𝑦)f(x,y) gets corrupted to produce a degraded image 𝑔(𝑥,𝑦)g(x,y).
Understanding this model is crucial because it allows for the formulation of appropriate
restoration techniques to recover the original image.
The degradation process can be described by a combination of factors such as blurring, noise,
and geometric distortions. The general form of the degradation model is:

g(x,y)=h(x,y)∗f(x,y)+η(x,y)

A model of the image degradation/restoration process

g(x,y)= f(x,y)*h(x,y)+ (x,y) – Spatial domain

G(u,v)= F(u,v) H(u,v)+ N(u,v) – Frequency domain
Where,
f(x,y) - input image
f^(x,y) - estimated original image
g(x,y) - degraded image
h(x,y) - degradation function
(x,y) - additive noise term
 Image degradation is the loss of image quality for a variety of reasons. When there is
image deterioration, the quality of the image is greatly diminished and becomes hazy.
All-natural images when displayed have gone through some sort of degradation:
1. During display mode
2. Acquisition mode, or
3. Processing mode
4. Sensor noise
5. Blur due to camera mis focus
6. Relative object-camera motion
7. Random atmospheric turbulence
8. Others

Components of the Degradation Model

1. Blurring (PSF h(x,y) ):
*Motion Blur: Caused by the relative movement between the camera and the object during
exposure.
*Out-of-Focus Blur: Due to the lens being out of focus.
*Atmospheric Blur: Caused by atmospheric conditions such as turbulence.

2. Noise (n(x,y) )
*Gaussian Noise: Characterized by a normal distribution of noise values, common in
electronic imaging systems.
*Salt-and-Pepper Noise: Manifests as random black and white pixels, often caused by faulty
pixel elements in sensors.
*Poisson Noise: Arises from the discrete nature of light and is common in photon-limited
imaging scenarios like microscopy.

Formulation of the Degradation Model

The degradation process can be visualized in steps:
1. Original Image : f(x,y)
2. Blurring : The original image f(x,y) is convolved with the PSF h(x,y) :
fb(x,y) = h(x,y) * f(x,y)
3. Noise Addition: Noise n(x,y) is added to the blurred image:
g(x,y) = f_b(x,y) + n(x,y) = h(x,y) * f(x,y) + n(x,y)

degradation models
1. Linear Degradation Model
2. Gaussian Blur Model
3. Motion Blur Model
4. Out-of-Focus Blur Model
5. Geometric Transformations
6. Noise Models
*Gaussian Noise
*Salt-and-Pepper Noise
*Poisson Noise
7. Combined Degradation Model

Properties
Image restoration involves several key properties and concepts that are crucial for effectively
recovering a degraded image. These properties guide the development and selection of
appropriate restoration techniques to achieve the best possible results. Here are the main
properties and concepts in image restoration:
1. Linearity
refers to the property of a system where the output is a linear combination of the inputs. In
the context of image restoration, if the degradation model is linear (e.g., convolution with a
PSF and additive noise), linear restoration techniques like inverse filtering can be applied.
Linear systems simplify mathematical analysis and solution derivation.
2. Shift Invariance
implies that the system's properties do not change with shifts in the input image. For a shift-
invariant system, the degradation process is consistent across the entire image. This property
allows the use of Fourier transforms to efficiently solve convolution problems.
3. Stability
ensures that small changes in the input result in small changes in the output. In restoration,
an algorithm must be stable to avoid amplifying noise and other artifacts during the
restoration process. Stability is essential for reliable and robust image restoration.
4. Inversibility
refers to the ability to reverse the degradation process. If a degradation process can be
accurately modeled, it may be possible to design an inverse filter or algorithm that recovers
the original image. However, perfect inversibility is challenging, especially in the presence of
noise.
5. Noise Sensitivity
indicates how susceptible a restoration algorithm is to noise amplification. Techniques like
inverse filtering are highly sensitive to noise, often requiring regularization or other noise
suppression methods to achieve usable results.
6. Regularization
involves adding additional information or constraints to stabilize the restoration process and
reduce noise sensitivity. Common regularization techniques include Tikhonov regularization,
which adds a smoothness constraint, and Total Variation (TV) regularization, which preserves
edges while smoothing.
7. Prior Knowledge
incorporates information about the original image or the degradation process into the
restoration algorithm. This can include assumptions about the image's smoothness, sparsity,
or specific structural features. Bayesian approaches and machine learning models often
leverage prior knowledge to improve restoration quality.
8. Edge Preservation
is crucial in maintaining the sharpness and clarity of important image features during
restoration. Techniques like Total Variation regularization and certain non-linear filters are
designed to preserve edges while reducing noise and blur.
9. Computational Efficiency
is important for practical applications, especially when dealing with large images or real-time
processing requirements. Efficient algorithms leverage fast computational techniques, such as
the Fast Fourier Transform (FFT) for convolution operations or GPU acceleration for deep
learning models.
10. Robustness
refers to the ability of a restoration method to perform well under varying conditions and
with different types of degradation. A robust method should handle diverse scenarios without
significant performance degradation.
Examples of Restoration Techniques Incorporating These Properties:
1. Wiener Filtering
 Balances noise reduction and deblurring by incorporating statistical properties of the
image and noise.
 Regularized to handle noise sensitivity and stability issues.
2. Total Variation Regularization
 Preserves edges while smoothing other parts of the image.
 Uses prior knowledge about the image structure to guide the restoration.
3. Iterative Deconvolution (e.g., Richardson-Lucy)
 Iteratively refines the estimate of the original image.
 Designed to converge to a stable solution while minimizing artifacts.
4. Deep Learning Models (e.g., CNNs, GANs)
 Leverage large datasets and prior knowledge to learn complex mappings from degraded
to restored images.
 Often incorporate regularization and other constraints to enhance robustness and
prevent overfitting.
Understanding and leveraging these properties allows for the development of effective image
restoration techniques that can handle a wide range of degradation scenarios and produce
high-quality restored images.
Noise Models:-

Noise tells unwanted information in digital images. Noise produces undesirable effects such as
artifacts, unrealistic edges, unseen lines, corners, blurred objects and disturbs background
scenes. To reduce these undesirable effects, prior learning of noise models is essential for
further processing. Digital noise may arise from various kinds of sources such as Charge
Coupled Device (CCD) and Complementary Metal Oxide Semiconductor (CMOS) sensors. In
some sense, points spreading function (PSF) and modulation transfer function (MTF) have
been used for timely, complete and quantitative analysis of noise models

There are different types of image noise. They can typically be divided into 3 types.

Fig.3 classification of noise

1. Gaussian Noise:
Gaussian Noise is a statistical noise having a probability density function equal to normal
distribution, also known as Gaussian Distribution. Random Gaussian function is added to
Image function to generate this noise. It is also called as electronic noise because it arises
in amplifiers or detectors. Source: thermal vibration of atoms and discrete nature of
radiation of warm objects.

Fig.4 Plot of Probability Distribution Function

The side image is a bell shaped probability distribution function which have mean 0 and
standard deviation(sigma) 1.
1.1 Implementation of Gaussian Noise with OpenCV-Python:

1.2 Effect of Standard Deviation(sigma) on Gaussian noise:

Fig.5 Effect of Sigma on Gaussian Noise

The magnitude of Gaussian Noise depends on the Standard Deviation(sigma). Noise

Magnitude is directly proportional to the sigma value.

2. Impulse Noise:
Impulse Function: In the discrete world impulse function on a vale of 1 at a single location
and In continuous world impulse function is an idealised function having unit area.

Fig.6 Impulse function in discrete world and continuous world

2.1 Types of Impulse Noise:
There are three types of impulse noises. Salt Noise, Pepper Noise, Salt and Pepper Noise.

Salt Noise: Salt noise is added to an image by addition of random bright (with 255 pixel
value) all over the image.

Pepper Noise: Salt noise is added to an image by addition of random dark (with 0 pixel
value) all over the image.

Salt and Pepper Noise: Salt and Pepper noise is added to an image by addition of both
random bright (with 255 pixel value) and random dark (with 0 pixel value) all over the
image.This model is also known as data drop noise because statistically it drop the original
data values [5]. Source: Malfunctioning of camera’s sensor cell.

2.2 Implementation of Salt and Pepper Noise with OpenCV-Python:

3. Poisson Noise:
The appearance of this noise is seen due to the statistical nature of electromagnetic waves
such as x-rays, visible lights and gamma rays. The x-ray and gamma ray sources emitted
number of photons per unit time. These rays are injected in patient’s body from its source,
in medical x rays and gamma rays imaging systems. These sources are having random
fluctuation of photons. Result gathered image has spatial and temporal randomness. This
noise is also called as quantum (photon) noise or shot noise.
3.1 Implementation of Poisson Noise Noise with OpenCV-Python:

4. Speckle Noise
A fundamental problem in optical and digital holography is the presence of speckle noise
in the image reconstruction process. Speckle is a granular noise that inherently exists in an
image and degrades its quality. Speckle noise can be generated by multiplying random
pixel values with different pixels of an image.

4.1 Implementation of Poisson Noise Noise with OpenCV-Python:

Mean Filters
In image restoration, mean filters play a crucial role in improving the quality of images
that have been degraded by various types of noise and distortions. Image restoration
aims to reconstruct or recover an image that has been corrupted, often using a model of
the degradation and employing algorithms to reverse or mitigate its effects.
Role of Mean Filters in Image Restoration
1. Noise Reduction: Mean filters are particularly effective in reducing random noise, such
as Gaussian noise, which is commonly present in images. By averaging the pixel values
within a neighborhood, mean filters can smooth out the noise, leading to a cleaner image.
2. Preprocessing : Mean filters are often used as a preprocessing step in more complex
image restoration pipelines. They can reduce high-frequency noise, making it easier for
subsequent restoration techniques to focus on more significant image features.

Types of Mean Filters in Image Restoration

1. Arithmetic Mean Filter

Command
Image
Filters
Arithmetic Mean Filter
Description
Applies a arithmetic mean filter to an image.
An arithmetic mean filter operation on an image removes short tailed noise such as
uniform and Gaussian type noise from the image at the cost of blurring the image. The
arithmetic mean filter is defined as the average of all pixels within a local region of an
image.
The arithmetic mean is defined as:

Pixels that are included in the averaging operation are specified by a mask. The larger the
filtering mask becomes the more predominant the blurring becomes and less high spatial
frequency detail that remains in the image.
Dialog box

Filter Size: size of the filter mask; a larger filter yields stronger effect.
Click Preview to judge the results.
Click OK to proceed
 2. Geometric Mean Filter

Command
Image
Filters
Geometric Mean Filter
Description
Applies a geometric mean filter to an image.
In the geometric mean method, the color value of each pixel is replaced with the
geometric mean of color values of the pixels in a surrounding region. A larger region
(filter size) yields a stronger filter effect with the drawback of some blurring.
The geometric mean is defined as:

The geometric mean filter is better at removing Gaussian type noise and preserving edge
features than the arithmetic mean filter. The geometric mean filter is very susceptible to
negative outliers.
Dialog box

Filter Size: larger filter yields stronger effect.

Click Preview to judge the results.
Click OK to proceed.

3. Harmonic Mean Filter

Command
Image
Filters
Harmonic Mean Filter
Description
Applies a harmonic mean filter to an image.
In the harmonic mean method, the color value of each pixel is replaced with the harmonic
mean of color values of the pixels in a surrounding region.
The harmonic mean is defined as:
A larger region (filter size) yields a stronger filter effect with the drawback of some
blurring.
The harmonic mean filter is better at removing Gaussian type noise and preserving edge
features than the arithmetic mean filter. The harmonic mean filter is very good at
removing positive outliers.
Dialog box

Filter Size: larger filter yields stronger effect.

Click Preview to judge the results.
Click OK to proceed.

4. Contraharmonic Mean Filter

Command
Image
Filters
Contraharmonic Mean Filter
Description
Applies a contraharmonic mean filter to an image.
With a contraharmonic mean filter, the color value of each pixel is replaced with the
contraharmonic mean of color values of the pixels in a surrounding region.
The contraharmonic mean with order Q is defined as:

A contraharmonic mean filter reduces or virtually eliminates the effects of salt-and-

pepper noise. For positive values of Q, the filter eliminates pepper noise. For negative
values of Q it eliminates salt noise. It cannot do both simultaneously.
Note that the contraharmonic filter is simply the arithmetic mean filter if Q = 0, and the
harmonic mean filter if Q = −1.
A larger region (filter size) yields a stronger filter effect with the drawback of some
blurring.Dialog box

Filter Size: larger filter yields stronger effect.

Order: the order Q of the filter.
Click Preview to judge the results.
Click OK to proceed.

Order Statistics :-
Order statistics play a crucial role in image restoration, particularly for removing non-Gaussian
noise such as impulse noise (e.g., salt-and-pepper noise). Unlike mean filters, which compute the
average of pixel values, order-statistics filters sort the pixel values within a neighborhood and
then select a specific value based on their order. This approach can preserve edges and details
better than traditional mean filters.

Common Order-Statistics Filters in Image Restoration

1. Median Filter
 The median filter is the most commonly used order-statistics filter. It replaces each
pixel value with the median value of the pixels in its neighborhood.
 It is highly effective at removing salt-and-pepper noise while preserving edges and
fine details.
 Formula: I′(x,y)=median{I(x+i,y+j)}
where the median is taken over a neighborhood centered at \((x, y)\).
2. Minimum Filter (Min Filter):
 The min filter replaces each pixel value with the minimum value of the pixels in its
neighborhood.
 It can be useful for removing dark outliers (pepper noise) but may result in image
darkening.
 Formula: I′(x,y)=min{I(x+i,y+j)}
3. Maximum Filter (Max Filter)
 The max filter replaces each pixel value with the maximum value of the pixels in its
neighborhood.
  It is effective for removing bright outliers (salt noise) but may result in image
brightening.
 Formula: I′(x,y)=max{I(x+i,y+j)}
4. Alpha-Trimmed Mean Filter
 The alpha-trimmed mean filter is a hybrid approach that removes a certain number
of the highest and lowest pixel values before computing the mean of the remaining
values.
 It provides a balance between noise reduction and detail preservation.
 Formula: i ′(x,y)=(mn−2d)1i=−(m−1)/2∑(m−1)/2j=−(n−1)/2∑(n−1)/2I(x+i,y+j)

Adaptive filters:-
Adaptive filters in image restoration are techniques used to enhance images by
dynamically adjusting their parameters based on the local characteristics of the image.
Unlike fixed filters that apply the same transformation to every pixel, adaptive filters tailor
their processing to the specific conditions around each pixel. This adaptability makes them
particularly effective in dealing with varying noise and detail levels across an image. Here’s
a more detailed look at adaptive filters in image restoration:

Key Concepts
1. Local Characteristics:
Noise Level: Adaptive filters assess the noise level in different parts of the image and
adjust their filtering strength accordingly. Areas with higher noise might receive stronger
filtering, while areas with less noise get lighter filtering to preserve details.
Edge Preservation: Edges are important features in images. Adaptive filters detect edges
and adapt their behavior to avoid blurring these crucial structures. This ensures that the
important details and boundaries in the image are maintained.
2. Types of Adaptive Filters:
Adaptive Mean Filter: This filter adapts the window size used for averaging based on
local variance. In smooth areas, a larger window might be used to smooth out noise, while
in areas with high detail, a smaller window preserves those details.
Adaptive Median Filter: Similar to the mean filter but uses the median value within the
window. This filter is particularly effective in removing salt-and-pepper noise while
preserving edges.
Wiener Filter: This filter minimizes the mean square error between the restored and the
original image. It adapts based on local mean and variance estimates, making it effective in
balancing noise reduction and detail preservation.
Bilateral Filter: This filter smooths images while preserving edges by considering both
spatial proximity and intensity similarity. It adapts to the image content by using a
weighted average that depends on these factors.
Applications
Noise Reduction: Adaptive filters are widely used to reduce various types of noise
(Gaussian, salt-and-pepper, etc.) in images while preserving important details.
Image Enhancement: These filters can enhance the visibility of details and features in an
image, making them useful in applications like medical imaging, satellite imaging, and
digital photography.
-Edge Detection and Preservation: In applications where edges are critical, adaptive
filters help in detecting and preserving edges more effectively than fixed filters.

Implementation Steps
1. Local Statistics Calculation: Compute local statistics such as mean, variance, and
median within a moving window around each pixel.
2. Parameter Adjustment: Adjust the filter parameters based on the local statistics. For
example, in a Wiener filter, the local variance is used to determine the filtering strength.
3. Filtering Operation: Apply the adaptive filter to each pixel using the adjusted
parameters, resulting in a processed image where noise is reduced, and important details
are preserved.
Advantages
Detail Preservation: Adaptive filters are better at preserving edges and fine details
compared to non-adaptive filters.
Noise Adaptation: They can effectively handle images with non-uniform noise levels,
adjusting the filtering strength as needed.
Versatility: Adaptive filters can be tailored for specific types of noise and image
characteristics, making them highly versatile.