CN106934806A - It is a kind of based on text structure without with reference to figure fuzzy region dividing method out of focus - Google Patents

Abstract

The invention discloses a method for segmenting an out-of-focus blurred area without a reference image based on structural definition, which comprises the following steps: (1) zooming the image to about 1/4 times the area of the original image; (2) calculating the definition Difference, respectively calculate the structural clarity of the image block corresponding to the original image and the zoomed image, and calculate the difference between the two; (3) Extract the blurred area, filter out the noise of the difference image, and use the image segmentation algorithm to segment the blurred area , and upsample the segmented result. For the segmentation of out-of-focus and blurred areas without reference images, the present invention uses the original image to construct a zoomed image, calculates the sharpness of the zoomed image and the original image respectively, and then obtains a blur degree distribution image, and finally quickly and effectively segments the out-of-focus blurred area of the image.

Description

A Segmentation Method for Out-of-focus Blurred Regions without Reference Image Based on Structural Definition

technical field

The invention relates to digital image technology, in particular to a method for segmenting out-of-focus fuzzy regions without a reference image based on structure definition.

Background technique

Image blur is a common image degradation process, usually caused by camera movement, object movement or inaccurate focus during the exposure process, and in some cases it is artificially formed for the pursuit of a certain artistic effect, including in During photography or post-image processing. Out-of-focus blur is a common image blur that is mainly caused by poor focus. Image blurring leads to the loss of information in the image, which makes it difficult to further process the image. Accurate and efficient detection of blurred pixels has important and practical applications in image segmentation, object detection, scene classification, image editing and other fields. In the application of fuzzy region segmentation without a reference image, there is only one input image. Currently, methods for segmentation of out-of-focus blurred regions without a reference image mainly include image frequency domain methods, space domain methods, and some methods combined with machine learning algorithms. However, these algorithms mainly have two problems. First, the processing speed is slow and the practicability is not strong. Second, the segmentation effect is not good.

An ideal image quality evaluation index can effectively distinguish blurred and clear images, so it can be used for image blurred area segmentation. The No-Reference Structural Sharpness Index (NRSS) is a good image quality evaluation index, which realizes the no-reference image quality evaluation on the basis of the Structural Sharpness (SSIM) index. However, there are still relatively few methods for segmenting blurred areas of images using NRSS at present, and the final segmentation effect is poor even if there is one.

Contents of the invention

Purpose of the invention: The purpose of the present invention is to solve the deficiencies in the prior art, and provide a method for segmenting out-of-focus blurred areas without reference images based on structure definition.

Technical solution: A method for segmenting out-of-focus blurred areas without reference images based on structure definition according to the present invention comprises the following steps:

(1) Scale the original image: the original image is proportionally scaled to about 0.25 times the area of the original image, that is, the size of the original image is M×N, and the size of the zoomed image is The scaled image is interpolated by bilinear interpolation to obtain the scaled image;

(2) Calculate the image definition difference:

(2.1) block the original image and the scaled image in step (1), thereby obtaining the original image block set R and the scaled image block set S;

(2.2) Calculate the non-reference graph structural similarity of each image block separately to measure the clarity of each image block;

(2.3) Calculate the difference of clarity of each image block respectively to obtain a difference matrix;

(3) Segment the fuzzy area:

(3.1) Use guided filtering to filter the difference matrix obtained in step (2) to filter out noise;

(3.2) Use the Otsu threshold method to segment the image after denoising, that is, first find the gray value that can maximize the variance between classes, and use this gray value to binarize;

(3.3) Upsample the segmented image and restore it to the original image scale.

Further, the concrete process of described step (2.1) is:

First, select an image block from the original image with a step size of (2,2), that is: start from the upper left corner of the image, first move 2 pixels along the X axis, and then move 2 pixels along the Y, repeat the above step, until all image blocks are obtained, the selected image block set is R, and each image block is recorded as R _i,j ;

Among them, i represents the row mark, j represents the column mark, assuming that the size of the image block is 2m×2n, then

Next, select an image block from the zoomed image with a step size of (1,1), that is, starting from the upper left corner of the image, first move 1 pixel along the X axis each time, and then move 1 pixel along the Y axis pixel, repeat the above steps until all the image blocks are obtained; the selected image block sequence is denoted as S, and each image block is denoted as S _i,j ;

Among them, i represents the row mark, j represents the column mark, assuming that the size of the image block is m×n, then

The number of image blocks in sets R and S should be consistent.

Further, the concrete process of described step (2.2) is:

Assuming a given image block P, the calculation steps of structural similarity without a reference image are as follows: blur image block P, use Gaussian blur to blur the image block to obtain image P _b ; extract gradient, use Sobel operator to extract the horizontal and vertical direction, the gradient images of the image blocks P and P _b are denoted as G and G _b respectively; find the N image blocks with the most information in the gradient image, and the richness of the gradient information is measured by the variance of the image, that is, find out the The N image blocks with the largest variance; calculate the no-reference image structure definition NRSS of the image block P, and the calculation formula of NRSS is:

Among them, the image block in G is recorded as G ⁱ , and the image block in G _b is recorded as G ⁱ _b ; the SSIM function is used to calculate the structural similarity of two image blocks, and this function considers the brightness, contrast and structure between image blocks at the same time Relevance of information. Given two image blocks a, b, the SSIM function can be expressed as:

SSIM(a,b)=[l(a,b)] ^α [c(a,b)] ^β [s(a,b)] ^γ

in,

s(a,b)=(σ _ab +C ₃ )/(σ _a σ _b +C ₃ ),

u _a , u _b represent the image gray mean values of image blocks a and b respectively, σ _a , σ _b represent the image gray standard deviations of image blocks a and b respectively, and σ _ab represents the image gray value of image blocks a and b. Variance, α, β, γ are parameter items, and C ₁ , C ₂ , C ₃ are constant items, which are used to prevent unstable operation when the denominator is close to zero.

Further, NRSS(S _i,j ) and NRSS(R _i,j ) are obtained respectively.

Further, the method of the step (2.3) is as follows:

Calculate the difference M′ _i,j of the sharpness of corresponding image blocks in the image block sets R and S, so as to obtain the difference matrix M′={M′ _i,j }:

M′ _i,j = NRSS(S _i,j )-NRSS(R _i,j );

Normalize the matrix M′ to get the matrix M={M _i,j }, and use the maximum and minimum values to normalize:

M _i,j =(M′ _i,j -min(M′))/(max(M′)-min(M′)), where max(M′) represents the maximum value of elements in the matrix, min(M ') represents the minimum value of the elements in the matrix.

Beneficial effects: the present invention does not lose generality. After the image is shrunk, the original blurred image will become clear, so the sharpness difference between the shrunk image and the original image can be used as a new discriminant index. Based on this method, the blur degree distribution of the image can be obtained, combined with the image segmentation algorithm, the blurred area of the image can be effectively segmented finally. Through image scale conversion, the amount of data to be processed is greatly reduced, so the algorithm of the invention has a fast processing speed, and meanwhile, the segmentation effect is better than the existing algorithm.

To sum up, the present invention has the advantages of fast segmentation speed, good segmentation effect, and being applicable to no reference images.

Description of drawings

Fig. 1 is the original grayscale image in the embodiment;

Fig. 2 is the image that adopts other methods artificial segmentation in the embodiment;

Fig. 3 uses the difference matrix after normalization that the present invention draws in the embodiment;

Fig. 4 is the segmentation result that uses the present invention to obtain in the embodiment;

Fig. 5 is the comparison chart of effect in the embodiment.

detailed description

The technical solutions of the present invention will be described in detail below, but the protection scope of the present invention is not limited to the embodiments.

Example:

Step 1: read the original color image and obtain the color image matrix;

Step 2: convert the color image matrix into a grayscale image matrix, so as to obtain the grayscale image as shown in Figure 1, at this time, determine the size of the image to be 640x621;

Step 3: Scale the image, determine that the size of the zoomed image is 320x310, and use linear interpolation to scale the image shown in Figure 1 to the image of the above size;

Step 4: Block the image and calculate the sharpness of each image block. The original image and the scaled image are divided into blocks. In this process, the moving step of the original image is (2,2), the moving step of the zoomed image is (1,1), and the image block size of the original image is is 32x32, and the image block size of the scaled image is 16x16. After this process is completed, the original image block set R and the scaled image block set S are obtained. pair collection Each image block in calculates the no-reference image structure sharpness (NRSS) to measure the sharpness of each image block. After this process is completed, the original image sharpness matrix R ^c and the scaled image sharpness matrix S ^c are obtained;

Step 5: Calculate the difference in sharpness. According to the sharpness matrices R ^c and S ^c calculated in step 4, the difference matrix M'=S ^c -R ^c can be obtained;

Step 6: Normalize the difference matrix M′ to obtain M. The specific normalization method can use the maximum value and minimum value normalization method, that is, M _i,j =(M′ _i,j −min(M′ ))/(max(M′)-min(M′)), the resulting normalized image is shown in Figure 3.

Step 7: Perform guided filtering on the matrix M, and use M itself as a reference image;

Step 8: segment the difference matrix M using the Otsu method;

Step 9: Enlarge the image, upsample the segmented result and perform interpolation using a linear interpolation algorithm to obtain the final segmented result, as shown in Figure 4.

In addition, the original grayscale image in Figure 1 is artificially segmented to obtain the image shown in Figure 2, and then the effect of the structure segmented by the two methods is compared, as shown in Figure 5, and the white color in the figure is clearly overlapped The black area is the overlapping fuzzy area, and the gray area is the misjudgment area.

It can be seen from this that the segmentation effect of the present invention is good and more accurate.

Claims (4)

1. A reference-image-free out-of-focus fuzzy region segmentation method based on structural definition comprises the following steps:

(1) scaling the original image by scaling the original image to about 0.25 times the original image area, i.e. the size of the original image is M × N and the size of the scaled image isInterpolating the zoomed image by bilinear interpolation to obtain a zoomed image;

(2) calculating an image sharpness difference:

(2.1) partitioning the original image and the zoomed image in the step (1) to obtain an original image block set R and a zoomed image block set S;

(2.2) respectively calculating the structure similarity of the reference-free image of each image block for measuring the definition degree of each image block;

(2.3) respectively calculating the difference of the definition degree of each image block to obtain a difference matrix;

(3) and (3) dividing fuzzy areas:

(3.1) filtering the difference matrix obtained in the step (2) by using guide filtering to filter noise;

(3.2) segmenting the denoised image by using an Otsu threshold method, namely finding out a gray value capable of maximizing the inter-class variance, and carrying out binarization on the gray value;

and (3.3) up-sampling the segmented image and restoring the segmented image to the original image scale.

2. The method for dividing the out-of-focus blurred region without the reference image based on the structural definition according to claim 1, wherein: the specific process of the step (2.1) is as follows:

first, an image block is selected from an original image, with a step size of (2,2), that is: starting from the upper left corner of the image, firstly moving 2 pixels along the X axis each time, then moving 2 pixels along the Y axis, repeating the steps until all image blocks are obtained, wherein the selected image block set is R, and each image block is marked as R_i,j；

Wherein i denotes a row label and j denotes a column label, assuming that the size of the image block is 2m × 2n

Next, an image block is selected from the scaled image, with a step size of (1,1), that is: starting from the upper left corner of the image, firstly moving 1 pixel along the X axis each time, then moving 1 pixel along the Y axis, and repeating the steps until all image blocks are obtained; selected image blockThe sequence is denoted S, each image block is denoted S_i,j；

Wherein i denotes a row label and j denotes a column label, assuming that the size of the image block is m × n

The number of image blocks in the sets R and S should be kept the same.

3. The method for dividing the out-of-focus blurred region without the reference image based on the structural definition according to claim 1, wherein: the specific process of the step (2.2) is as follows:

assuming a given image block P, the computation steps for the structure similarity of the no-reference pictures are as follows: blurring the image block P by Gaussian blur to obtain an image P_b(ii) a Extracting gradients, namely extracting gradients in horizontal and vertical directions of image blocks, image blocks P and P by using Sobel operator_bRespectively denoted as G and G_b(ii) a Finding out N image blocks with most abundant information in the gradient image, wherein the abundance of the gradient information is measured by using the variance of the image, namely finding out the N image blocks with the largest variance; calculating the non-reference picture structure definition NRSS of the image block P, wherein the calculation formula of the NRSS is as follows:

$NRSS\left(P\right)=1/N\underset{1}{\overset{N}{\Σ}}SSIM(G^i,G_b^i);$

wherein, the image block in G is marked as Gⁱ,G_bMiddle image block markIs composed ofThe SSIM function is used to compute the structural similarity of two picture blocks, which takes into account the correlation of luminance, contrast and structural information between the picture blocks at the same time. Given two image blocks a, b, the SSIM function can be expressed as:

SSIM(a,b)＝[l(a,b)]^α[c(a,b)]^β[s(a,b)]^γ

wherein,

$l(a,b)=(2u_a{u}_b+C_1)/(u_a^2+u_b^2+C_1),$

$c(a,b)=(2{\mathrm{\σ}}_a{\mathrm{\σ}}_b+C_2)/({\mathrm{\σ}}_a^2+{\mathrm{\σ}}_b^2+C_2),$

s(a,b)＝(σ_ab+C₃)/(σ_aσ_b+C₃)，

u_a,u_brepresenting the image mean values, σ, of the image blocks a, b, respectively_a,σ_bRepresenting the image grey scale standard deviation, σ, of the image blocks a, b, respectively_abRepresenting the image block a, b image gray-scale covariance α, gamma is a parameter term, C₁,C₂,C₃The constant term is used for preventing unstable operation when the denominator is close to zero;

further, NRSS (S) is obtained_i,j) And NRSS (R)_i,j)。

4. The method for dividing the out-of-focus blurred region without the reference image based on the structural definition according to claim 1, wherein: the method of the step (2.3) is as follows:

calculating difference value M 'of corresponding image block definition in image block sets R and S'_i,jThereby obtaining a difference matrix M '═ M'_i,j}：

M′_i,j＝NRSS(S_i,j)-NRSS(R_i,j)；

Normalizing the matrix M' to obtain a matrix M ═ M_i,jNormalization with maximum and minimum values:

M_i,j＝(M'_i,j-min (M '))/(max (M ') -min (M ')), where max (M ') denotes the maximum value of an element in the matrix and min (M ') denotes the minimum value of an element in the matrix.