CN112215773B - Local motion deblurring method, device and storage medium based on visual saliency - Google Patents

Abstract

The invention discloses a local motion deblurring method, a device and a storage medium based on visual saliency, wherein the method comprises the following steps: detecting a saliency map of the fuzzy region by adopting the popular regularization random walk saliency, and using a detection result for marking the fuzzy region in the image; performing binarization operation on the saliency map by adopting an Otsu method based on heredity, segmenting a fuzzy foreground and a clear background into different map layers, and acquiring a foreground and background segmentation binarization image; substituting the obtained binary image into an image deblurring model based on an MAP framework for optimization, and iteratively estimating an initial fuzzy kernel and a potential image; listing the priors of the initial fuzzy kernel and the potential image, and solving the initial fuzzy kernel and the potential image based on iterative weighted least square; and further optimizing a solution result by adopting improved self-adaptive guide filtering, and keeping the edge information of the image.

Description

Local motion deblurring method, device and storage medium based on visual saliency

technical field

The invention relates to image motion deblurring technology, belongs to the field of image processing, in particular to a visual saliency-based partial motion deblurring method, device and storage medium.

Background technique

Motion blurred images are blurred due to the relative motion between the camera and the moving object within the exposure time. The blurred image makes it impossible to see the target object you want to observe, or to obtain valuable information from the image. Moreover, the shooting process is short and difficult to reproduce. In many cases, it is impossible to re-shoot to obtain a clear image. Therefore, the research on motion blur restoration technology has become particularly important. It is used in road video surveillance, industrial production, and criminal investigation. , astronomical observation, and military satellite tracking and other fields.

In recent years, motion blurred image restoration methods have made great progress, but such methods still have certain limitations for the local blurred image restoration problem. Due to the difference between the foreground and the background imaging process, the globally consistent blur model cannot model the image formation process well, and due to the fast motion of the object, the blur will change suddenly, so some non-uniform blur models based on camera shake modeling Blur methods can't handle this kind of local motion blur well. There are also some existing methods to obtain additional information of moving objects by building a hybrid camera system, but such methods require well-designed hardware support.

In addition, there are some local motion blurred image restoration methods based on image segmentation. These methods depend on the segmentation quality to a large extent. If the segmentation is not accurate, the image restoration effect will not be very good, and it cannot meet various needs in practical applications. .

Contents of the invention

The invention provides a local motion deblurring method, device and storage medium based on visual salience. The invention improves the definition of the image, restores a higher-quality clear image, and effectively enhances the contrast and edge of the image. Details, and solve the problem of texture information loss in blurred areas in the image, see the description below for details:

In the first aspect, a local motion deblurring method based on visual saliency, the method comprising:

Using popular regularized random walk saliency to detect the saliency map of the blurred area, and use the detection result to mark the blurred area in the image;

The genetic-based Otsu method is used to binarize the saliency map, and the blurred foreground and clear background are divided into different layers, and the foreground and background segmentation binarized images are obtained;

Bring the obtained binarized image into the image deblurring model based on the MAP framework for optimization, and iteratively estimate the initial blur kernel and potential image;

List the priors for the initial blur kernel and latent image, and solve for the initial blur kernel and latent image based on iteratively weighted least squares;

The improved adaptive guided filtering is used to further optimize the solution results and keep the edge information of the image.

In an implementation manner, the saliency map of the fuzzy region detected using the popularity regularized random walk saliency is specifically:

Optimize the impact of boundaries according to the means of locating and eliminating false boundaries, and obtain background saliency values;

Optimizing pure background queries based on saliency estimates for foreground queries;

Based on the random walk model, a fitting constraint is proposed to inherit the saliency value of the foreground query to obtain the final saliency map.

In one implementation, the optimization of the obtained binarized image into the image deblurring model based on the MAP framework is as follows:

The image segmentation item l _i is introduced, the image is divided into different layers, the blur kernel is estimated and the clear latent image is recovered for each layer.

In an implementation manner, the improved adaptive guided filtering is specifically:

The guided image is constructed using a weighted least squares filter WLS:

和

分别是G在x和y方向上的一阶偏导数，表示图像的陡峭程度；τ_x,v和τ_y,v为细化的权重系数；Among them, WLS(G) _v is the guide image constructed by the weighted least squares filter WLS, G _v is the guide image corresponding to the spatial position v of the image pixel, I _v is the input image corresponding to the spatial position v of the image pixel, and v represents The spatial position of the image pixel point (x, y), the index τ determines the sensitivity of the input image I to the gradient change of the pixel point v(x, y), η is the smoothing parameter,

and

They are the first-order partial derivatives of G in the x and y directions, indicating the steepness of the image; τ _{x, v} and τ _{y, v} are the refined weight coefficients;

According to the local linear model between the guide image and the output image, the average value of the local variance of all pixels is incorporated into the cost function of the guide filter; an adaptive amplification factor β is introduced to suppress noise.

In the second aspect, a local motion deblurring device based on visual saliency, the device comprising:

The detection and labeling module is used to detect the saliency map of the fuzzy region using the popular regularized random walk saliency, and uses the detection result to mark the fuzzy region in the image;

The first acquisition module is used to perform a binarization operation on the saliency map by using the Otsu method based on genetics, divide the fuzzy foreground and the clear background into different layers, and obtain the foreground and background segmentation binarized images;

The first optimization module is used to bring the obtained binary image into the image deblurring model based on the MAP framework for optimization, and iteratively estimates the initial blurring kernel and potential image;

The second acquisition module is used to list the priors of the initial blur kernel and the potential image, and solve the initial blur kernel and the potential image based on iterative weighted least squares;

The second optimization module is used to further optimize the solution result by adopting the improved self-adaptive guiding filter, and keep the edge information of the image.

In an implementation manner, the detection and marking module includes:

The detection unit is used to optimize the influence of the boundary according to the means of locating and eliminating the wrong boundary, and obtain the background saliency value; optimize the pure background query according to the saliency estimation of the foreground query;

The fitting constraint unit is used to propose a fitting constraint based on the random walk model, and is used to inherit the saliency value of the foreground query to obtain the final saliency map.

In an implementation manner, the first optimization module includes:

The segmentation and restoration unit is used to introduce the image segmentation item l _i , divide the image into different layers, perform blur kernel estimation on each layer and restore the clear latent image.

In a third aspect, a local motion deblurring device based on visual saliency, the device includes: a processor and a memory, and program instructions are stored in the memory, and the processor invokes the program instructions stored in the memory to make the device Execute the method steps described in the first aspect.

In a fourth aspect, a computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the first aspect The method steps described.

The beneficial effects of the technical solution provided by the invention are:

1. Aiming at the problem of solving the blur kernel and potential image in the maximum posterior deblurring framework, the present invention adopts an effective iterative weighted least squares algorithm to optimize the solution of the object motion blur estimation model, that is, using Laplace first On the basis of solving the fuzzy kernel empirically and solving the clear latent image priori with the sparse image gradient, it further solves the problem of inaccurate optimal solution and low algorithm efficiency in the original algorithm;

2. The present invention uses an improved self-adaptive guided filtering algorithm to solve the problem of difficult recovery of texture edge details in severe motion blur. By maintaining the edge information of the image and adaptively suppressing noise, the definition of the image is further improved and the image is enriched. details.

Description of drawings

Fig. 1 is a flow chart of a local motion deblurring method based on visual saliency provided by the present invention;

Fig. 2 is another flowchart of a local motion deblurring method based on visual saliency provided by the present invention;

3 is a schematic diagram of a local motion blurred image;

Fig. 4 is a schematic diagram of the target image after deblurring processing in Fig. 3;

Fig. 5 is a schematic diagram of another partial motion blurred image;

Fig. 6 is a schematic diagram of the target image after deblurring processing in Fig. 5;

Fig. 7 is a schematic diagram of another partial motion blurred image;

Fig. 8 is a schematic diagram of the target image after deblurring processing in Fig. 7;

9 is a schematic structural diagram of a local motion deblurring device based on visual saliency provided by the present invention;

Fig. 10 is a schematic structural diagram of a detection and marking module;

FIG. 11 is another structural schematic diagram of a local motion deblurring device based on visual saliency provided by the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation manners of the present invention will be further described in detail below.

Referring to FIG. 1 , an embodiment of the present invention proposes a local motion deblurring method based on visual saliency, which includes the following steps:

Step 101: using popular regularized random walk saliency to detect the saliency map of the fuzzy region, and using the detection result to mark the fuzzy region in the image;

In the local blurred image caused by object movement, the blur kernel of each region is usually inconsistent, so it is necessary to solve the blur kernel layer by layer, and the solved blur kernel is used to restore the clear image, so the embodiment of the present invention adopts popular regularization Random walk saliency detection saliency map of blurred regions, used to mark blurred regions in images.

Step 102: Use the Ostu (Otsu method) threshold segmentation based on genetic algorithm to perform binarization operation on the saliency map, divide the fuzzy foreground and clear background into different layers, and then obtain the foreground segmentation binarized image and background segmentation two value image;

The subsequent steps estimate the blur kernel through different layers, that is, the accuracy of blur kernel estimation is improved through the above operations.

Wherein, Ostu is an algorithm for determining the image binarization segmentation threshold, also known as the maximum inter-class variance method, which is well known to those skilled in the art, and will not be described in detail in the embodiments of the present invention.

Step 103: Bring the obtained binarized image into the image deblurring model based on the MAP framework for optimization, that is, iteratively estimate the blur kernel and potential image from the blurred image;

In this image deblurring model based on the MAP framework, the segmentation algorithm can effectively guide the motion blur estimation, and the motion blur estimation in turn helps the segmentation estimation, so that the local blur in the image can be better removed. The potential image and blur kernel obtained in this step are used as intermediate results, and the final accurate blur kernel is further obtained through subsequent steps, and then the restored clear image is output.

Wherein, the optimized maximum a posteriori deblurring framework is well known to those skilled in the art, and this embodiment of the present invention will not describe it in detail.

Step 104: using Laplacian prior and sparse image gradient prior to list the priors of the initial blur kernel and latent image respectively, and further solving the initial blur kernel and latent image based on the prior value and iterative weighted least squares algorithm;

Aiming at the problem of solving the blur kernel and latent image in the maximum a posteriori deblurring framework, an effective iterative weighted least squares algorithm is used to solve the image deblurring model, that is, the Laplacian prior and the sparse image gradient prior are used to solve the image deblurring model. Define and solve the prior of the initial blur kernel and potential image respectively, and then use the iterative weighted least square algorithm to further optimize and solve the image deblurring model, and further solve the problem of inaccurate optimal solution and low efficiency of the original algorithm .

Step 105: Use the improved adaptive guided filtering to further optimize the solution result of step 104, which is used to solve the problem of blurred texture edge details when the motion blur is relatively serious. By maintaining the edge information of the image, adaptively suppressing noise, and further improving It improves the clarity of the image and enriches the details of the image.

In summary, the embodiment of the present invention improves the definition of the image through the above steps 101 to 105, restores a clear image of higher quality, effectively enhances the contrast and edge details of the image, and solves the problem of blurring in the image The problem of loss of area texture information.

In the following, in conjunction with Fig. 2 and the specific calculation formula, a saliency-based local motion blur recovery method in the above embodiment is detailed and expanded. The experimental objects used in the present invention are local motion blur images with different degrees of blur , aiming at the problem of information loss in blurred images, the method includes the following steps:

Step 201: using popular regularized random walk saliency to detect the saliency map of the fuzzy region, which is used to mark the fuzzy region in the image;

In the local blurred image caused by object motion, the blur kernel is usually inconsistent, and the blur kernel needs to be solved hierarchically. Therefore, the embodiment of the present invention uses the popular regularized random walk saliency to detect the saliency map of the blur region to mark the image The algorithm uses both local features and image detail information to provide more accurate saliency estimation.

The impact of boundaries is optimized by locating and eliminating false boundaries before performing background saliency detection. Based on the popular ranking regularization framework, those boundaries with the smallest probability of belonging to the background are deleted, and the saliency estimation is generated through the background query. The popular ranking function is as follows:

Among them, f ^* is the value of the popular ranking function, d _ii and d _jj are the degree matrix of the local graph (technical terms in this field, so I won’t go into details here), and the diagonal elements of the i and j rows, and the parameter α is the control smoothing The balance between constraints (the first item before the plus sign) and fitting constraints (the second item after the plus sign); f _i is the ranking value of node i, f _j is the ranking value of node j, and ω _ij is the graph The weight matrix of the edge, n is the number of elements, y is the indicator vector, defined as y=[y ₁ ,…,y _n ] ^T , then the background saliency can be obtained by pixel-by-pixel multiplication:

where l is the total number of pixels in the target region, n is the total number of superpixels, S _l (i) is the foreground-based saliency value, f _l ^* (i) is calculated by formula (1), where l corresponds to At the three boundary positions after the error boundary is eliminated, S _step1 (i) is the background saliency estimation value of a ranking.

可以直接从方程(1)计算，并将其作为前景显着性估计处理如下：But pure background query is sometimes inaccurate for fully describing foreground information, especially when the saliency objects are complex in structure and similar in layout aspect to the background. In view of this, the next foreground query based saliency estimation, ranking function

can be computed directly from Equation (1) and treated as a foreground saliency estimate as follows:

S _step2 (i)=f(i),i=1,...,n, (3)

Among them, S _step2 (i) is the estimated value of the foreground saliency of the secondary ranking, and f(i) is the estimated value of the ranking function of the i superpixel.

At the same time, the popular regularized random walk algorithm is used to generate a pixel-level saliency map, and the pixel-level saliency map comes from the background and foreground saliency estimation based on superpixels.

The embodiment of the present invention proposes a fitting constraint based on the random walk model:

Among them, Dir is the Dirichlet integral, Y is a pixel-by-pixel indicator vector, and inherits the value of step S _step2 . The regularized random walk ordering is computed at the pixel level, so both p ^k and Y are N×1 vectors, L is an N×N matrix, where N is the total number of pixels in the image, and T is the transformation of the matrix set, k is 1 or 2, where k=1 corresponds to the background label, and k=2 corresponds to the foreground label.

Step 202: Binarize the saliency map by using the Ostu threshold segmentation algorithm based on the genetic algorithm;

In the specific implementation, after the saliency map is obtained, the Ostu threshold segmentation algorithm based on genetic algorithm is used to perform binarization operation on the saliency map, in which white represents the salient area part is represented by 1, and the black corresponding non-salient area is represented by 0.

The traditional Ostu algorithm processes the grayscale information of the image itself, but does not process the spatial information between pixels and neighborhoods. Therefore, when the image is disturbed by noise or other external factors, it is easy to cause segmentation errors, and the calculation steps are cumbersome. The genetic algorithm has the characteristics of global search and parallelism in the running process, and it can intelligently select the optimal threshold value, which greatly improves the processing efficiency of image segmentation. Therefore, the embodiment of the present invention combines the genetic algorithm to effectively improve the image segmentation efficiency by seeking the optimal Ostu threshold, and uses morphological operations to eliminate noise and optimize the segmentation result.

Step 203: Estimate the latent image X and blur kernel K from the blurred image B, optimize the image deblurring model based on the MAP (Maximum A Posteriori) framework, introduce the image segmentation item l _i , and divide the image into different layers , and perform blur kernel estimation and latent image X recovery on each layer;

The optimized image deblurring model proposed above is:

p为概率符号，p(X)为潜像X的先验概率，p(K)为模糊核K的先验概率，p(l_i|K_i,X)为引入分割项l_i推导得到的先验概率， p(K_i)为第i个图层对应的模糊核的先验概率，p(B|K,X)为从模糊图像B中估计潜在图像X和模糊核K的概率，p(B|l_i,K_i,X)为似然概率。Among them, m represents the number of segmentation layers, l _i represents the binary image of the i-th layer, and has the same size as the input image, K _i is the blur kernel corresponding to the i-th layer, and satisfies

p is the probability symbol, p(X) is the prior probability of the latent image X, p(K) is the prior probability of the blur kernel K, p(l _i |K _i ,X) is derived by introducing the segmentation item l _i Prior probability, p(K _i ) is the prior probability of the blur kernel corresponding to the i-th layer, p(B|K,X) is the probability of estimating the potential image X and the blur kernel K from the blurred image B, p (B|l _i ,K _i ,X) is the likelihood probability.

Among them, the key to the optimized image deblurring model is to solve the latent image X and blur kernel K from it.

Step 204: iteratively solve the potential clear image X and the blur kernel K _i through the alternate minimization formula, and introduce an effective iterative weighted least squares (IRLS) algorithm to optimize the image deblurring model;

Based on the above discussion, the embodiments of the present invention need to define the priors p(X) and p(K _i ) of the clear latent image X and the blur kernel K _i . That is, use the sparse image gradient prior to define the prior p(X) of the clear latent image X, and use the Laplace prior to define the prior p(K _i ) of the blur kernel K _i :

和

分别表示x,y方向上微分算子；λ和γ是权重参数；Z_X和Z_K是正则项，X_u为像素空间位置u 的清晰潜像，K_iu为第i个图层像素空间位置u对应的模糊核，u为一个像素的空间位置。in,

and

Denote the differential operator in the x and y directions respectively; λ and γ are weight parameters; Z _X and Z _K are regular terms, X _u is the clear latent image of the pixel space position u, and K _iu is the pixel space position of the i-th layer The blur kernel corresponding to u, where u is the spatial position of a pixel.

Using the blur kernel K from the previous iteration, the intermediate latent image X is estimated:

Among them, l _iu is the binary image corresponding to the pixel space position u of the i-th layer, and X _u is the clear latent image of the pixel space position u.

The IRLS algorithm is used to solve the following formula:

t表示迭代索引，u表示每个像素的空间位置，l_iu为第i个图层像素空间位置u对应的二值图。Among them, ω _du =|(x*K _i -B) _u | ^-1 ,

t represents the iteration index, u represents the spatial position of each pixel, l _iu is the binary image corresponding to the i-th layer pixel spatial position u.

Given an intermediate latent image, estimate the blur kernel. Since the kernel estimation based on the image gradient can achieve better results, the image derivative in the data fitting item is used to replace the image intensity, and the small gradient value is removed to estimate the blur kernel, then the blur kernel K _i is estimated as:

为梯度算子。in,

is the gradient operator.

Similar to the solution of the latent image X, the IRLS algorithm is used to solve the following formula:

ω_u＝|K_iu|^-1。in,

ω _u = |K _iu | ⁻¹ .

Step 205: When the motion blur is serious, use an improved adaptive guided filtering algorithm to further enhance the image, improve the definition of the image, and enrich the detail information of the image.

A guided filter is a filter that enhances image detail. The filter filters the input image through a guide image. The guide image in the filtering process is denoted as G, the input image is denoted as I, and the filtered output image is denoted as Q. Assume that in the window ω _{r centered on the pixel r} There is a local linear model between the guide image G and the output image Q as follows:

Among them, a _r and b _r are the linear coefficients in the window ω _r .

First, in order to further enhance the details of the image and fully obtain the edge features of the guide image, the embodiment of the present invention uses a weighted least squares filter WLS to construct the guide image of the image, as shown below:

和

分别是G在x和y方向上的一阶偏导数，表示图像的陡峭程度；τ_x,v和τ_y,_v细化权重系数，可确定图像I的边缘，不同陡峭程度边缘具有不同的权重系数。Among them, WLS(G) _v is the guide image constructed by the weighted least squares filter WLS, G _v is the guide image corresponding to the spatial position v of the image pixel, I _v is the input image corresponding to the spatial position v of the image pixel, and v represents The spatial position of the image pixel point (x, y), the index τ determines the sensitivity of the input image I to the gradient change of the pixel point v(x, y), η is the smoothing parameter,

and

They are the first-order partial derivatives of G in the x and y directions, indicating the steepness of the image; τ _x,v and τ _y , _v refine the weight coefficients, which can determine the edge of the image I, and the edges with different steepness have different weights coefficient.

Second, guided image filtering, although good at preserving edges, is susceptible to halo artifacts near edges. Therefore, the average of the local variances of all pixels is incorporated into the cost function of guided filtering to preserve edges precisely. Then the mean value of the local variance of all pixels is defined as:

为标准差的平均值，

是引导图像在窗口ω_r中的局部方差。因此在窗口中对代价函数求最小解为：Wherein, N is the number of pixels of the guide image;

is the mean of the standard deviation,

is the local variance of the guided image in the window _ωr . Therefore, the minimum solution to the cost function in the window is:

Among them, ε is the smoothing parameter of the regular term, which is used to prevent a _r from being too large.

Finally, the noise in the background is often amplified due to the amplification factor of the detail layer J. In general, the magnification factor of the detail layer is set to a fixed value, but the noise will be amplified while enhancing the image, so an adaptive magnification factor β is introduced in the detail layer to suppress noise and improve details. Multiply the detail layer by the magnification factor β:

J'=β·J=β·(G-Q) (15)

where J′ is the enhanced detail layer. When the value of β is small, details will be suppressed. On the other hand, when the value of β is large, the noise will be amplified. Therefore, to enhance the details while suppressing the noise, set β as:

为窗口ω_r中的线性系数a的均值。Therefore, the final output image is: F=Q+J',

is the mean value of the linear coefficient a in the window ω _r .

Based on the same inventive concept, as an implementation of the above method, referring to FIG. 9, an embodiment of the present invention also provides a local motion deblurring device based on visual saliency, which includes:

The detection and marking module 1 is used to detect the saliency map of the fuzzy region by using the popular regularized random walk saliency, and use the detection result to mark the fuzzy region in the image;

The first acquisition module 2 is used to perform a binarization operation on the saliency map using the Otsu method based on genetics, divide the fuzzy foreground and the clear background into different layers, and obtain the foreground and background segmentation binarized images;

The first optimization module 3 is used to bring the obtained binary image into the image deblurring model based on the MAP framework for optimization, iteratively estimating the initial blur kernel and potential image;

The second acquisition module 4 is used to list the priors of the initial blur kernel and the potential image, and solve the initial blur kernel and the potential image based on iterative weighted least squares;

The second optimization module 5 is used to further optimize the solution result by adopting the improved self-adaptive guided filtering, and keep the edge information of the image.

During specific implementation, referring to Fig. 10, the detection and marking module 1 includes:

The detection unit 11 is used to optimize the influence of the boundary according to the means of locating and eliminating the wrong boundary, and obtain the background saliency value; optimize the pure background query according to the saliency estimation of the foreground query;

The fitting constraint unit 12 is used to propose a fitting constraint based on the random walk model, and is used to inherit the saliency value of the foreground query to obtain the final saliency map;

The marking unit 13 is configured to use the detection result to mark blurred areas in the image.

In one implementation, the first optimization module 2 includes:

The segmentation and restoration unit is used to introduce the image segmentation item l _i , divide the image into different layers, perform blur kernel estimation on each layer and restore the clear latent image.

It should be pointed out here that the device descriptions in the above embodiments correspond to the descriptions of the above method embodiments, and details are not described in this embodiment of the present invention.

The execution subjects of the above-mentioned modules and units may be devices with computing functions such as computers, single-chip microcomputers, microcontrollers, etc. During specific implementation, the embodiment of the present invention does not limit the execution subject, and the execution subjects are selected according to the needs of practical applications.

Based on the same inventive concept, an embodiment of the present invention also provides a local motion deblurring device based on visual saliency, as shown in FIG. 6 calling the program instructions stored in the memory 7 to make the device perform the following method steps in the embodiment:

Using popular regularized random walk saliency to detect the saliency map of the blurred area, and use the detection result to mark the blurred area in the image;

The genetic-based Otsu method is used to binarize the saliency map, and the blurred foreground and clear background are divided into different layers, and the foreground and background segmentation binarized images are obtained;

Bring the obtained binarized image into the image deblurring model based on the MAP framework for optimization, and iteratively estimate the initial blur kernel and potential image;

List the priors for the initial blur kernel and latent image, and solve for the initial blur kernel and latent image based on iteratively weighted least squares;

The improved adaptive guided filtering is used to further optimize the solution results and keep the edge information of the image.

It should be pointed out here that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

The execution subject of the above-mentioned processor and memory may be a device with computing functions such as a computer, a single-chip microcomputer, and a microcontroller. When implementing it, the embodiment of the present invention does not limit the execution subject, and it can be selected according to the needs of practical applications.

Data signals are transmitted between the memory 7 and the processor 6 through the bus 8, which will not be described in detail in this embodiment of the present invention.

Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiments.

The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state hard disk, and the like.

It should be pointed out here that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present invention will be generated in whole or in part.

A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted over computer-readable storage media. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media. Available media can be magnetic media or semiconductor media, etc.

In the embodiments of the present invention, unless otherwise specified, the models of the devices are not limited, as long as they can complete the above functions.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (6)

1. A local motion deblurring method based on visual saliency, characterized in that the method comprises: Using popular regularized random walk saliency to detect the saliency map of the blurred area, and use the detection result to mark the blurred area in the image; The genetic-based Otsu method is used to binarize the saliency map, and the fuzzy foreground and clear background are divided into different layers, and the foreground and background segmentation binarized images are obtained; Bring the obtained binarized image into the image deblurring model based on the MAP framework for optimization, and iteratively estimate the initial blur kernel and potential image; List the priors for the initial blur kernel and latent image, and solve for the initial blur kernel and latent image based on iteratively weighted least squares; The improved adaptive guided filtering is used to further optimize the solution results and maintain the edge information of the image; Wherein, the saliency map of the fuzzy region detected by the popularity regularized random walk saliency is specifically: Optimize the impact of boundaries according to the means of locating and eliminating false boundaries, and obtain background saliency values; Optimizing pure background queries based on saliency estimates for foreground queries; Based on the random walk model, a fitting constraint is proposed to inherit the saliency value of the foreground query to obtain the final saliency map; Background significance value:

where l is the total number of pixels in the target region, n is the total number of superpixels, S _l (i) is the foreground-based saliency value, f _l ^* (i) is the value of the popular ranking function, S _step1 (i) is the background saliency estimate for a ranking; Saliency estimation for foreground queries: S _step2 (i)=f(i),i=1,...,n, Among them, S _step2 (i) is the estimated value of the foreground saliency of the secondary ranking, and f(i) is the estimated value of the ranking function of the i superpixel; The fit constraints:

Among them, Dir is the Dirichlet integral, Y is a pixel-by-pixel indicator vector, inherits the value of step S _step2 , p ^k and Y are both N×1 vectors, L is an N×N matrix, and N is the image in The total number of pixels, T is the transpose of the matrix, k is 1 or 2; Wherein, the described binarized image that will obtain is brought into the image deblurring model based on MAP framework to optimize specifically as follows: Introduce the image segmentation item l _i , divide the image into different layers, estimate the blur kernel and recover the clear latent image for each layer; The improved adaptive guided filtering is specifically: The guided image is constructed using a weighted least squares filter WLS:

Among them, WLS(G) _v is the guide image constructed by the weighted least squares filter WLS, G _v is the guide image corresponding to the spatial position v of the image pixel, I _v is the input image corresponding to the spatial position v of the image pixel, and v represents The spatial position of the image pixel point (x, y), the index τ determines the sensitivity of the input image I to the gradient change of the pixel point v(x, y), η is the smoothing parameter,

and

are the first-order partial derivatives of G in the x and y directions, indicating the steepness of the image; τ _x,v and τ _y,v are the weight coefficients for refinement; According to the local linear model between the guide image and the output image, the average value of the local variance of all pixels is incorporated into the cost function of the guide filter, and an adaptive amplification factor β is introduced to suppress noise. 2. A local motion deblurring device based on visual saliency, wherein the device is used to implement the local motion deblurring method in claim 1, the device comprising: The detection and labeling module is used to detect the saliency map of the fuzzy region using the popular regularized random walk saliency, and uses the detection result to mark the fuzzy region in the image; The first acquisition module is used to perform a binarization operation on the saliency map by using the Otsu method based on genetics, divide the fuzzy foreground and the clear background into different layers, and obtain the foreground and background segmentation binarized images; The first optimization module is used to bring the obtained binary image into the image deblurring model based on the MAP framework for optimization, and iteratively estimates the initial blurring kernel and potential image; The second acquisition module is used to list the priors of the initial blur kernel and the potential image, and solve the initial blur kernel and the potential image based on iterative weighted least squares; The second optimization module is used to further optimize the solution result by adopting the improved self-adaptive guidance filter, and keep the edge information of the image. 3. A local motion deblurring device based on visual saliency according to claim 2, wherein the detection and marking module comprises: The detection unit is used to optimize the influence of the boundary according to the means of locating and eliminating the wrong boundary, and obtain the background saliency value; optimize the pure background query according to the saliency estimation of the foreground query; The fitting constraint unit is used to propose a fitting constraint based on the random walk model, and is used to inherit the saliency value of the foreground query to obtain the final saliency map. 4. A kind of local motion deblurring device based on visual saliency according to claim 2, characterized in that, the first optimization module comprises: The segmentation and restoration unit is used to introduce the image segmentation item l _i , divide the image into different layers, perform blur kernel estimation on each layer and restore the clear latent image. 5. A local motion deblurring device based on visual saliency, characterized in that the device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to causing the device to perform the method steps of claim 1. 6. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the rights The method steps described in claim 1.

Images