CN107220945A - The restored method of the pole blurred picture of multiple degeneration - Google Patents

Abstract

The invention discloses a method for restoring a multi-degraded extremely blurred image, which is used to solve the problem that the existing criminal investigation software cannot effectively restore the blurred image. This method mainly takes into account multiple blur factors caused by optical diffraction, quantization, defocus and relative motion, initializes each sub-degeneration function according to the measured physical parameters, and uses the variational Bayesian algorithm to solve each sub-degradation function in an alternate iterative manner. The optimal distribution of the degeneracy function is used, and the relationship between the degeneracy sub-functions and the degeneracy function of the system is used to synthesize the degeneracy sub-functions into the point spread function of the system. Finally, the image is restored using the L-R algorithm.

Description

Restoration method of extremely blurry image with multiple degradations

technical field

The invention belongs to the field of image restoration, relates to a method for restoring multiple degraded extremely blurred images, and is particularly suitable for restoring extremely blurred criminal investigation images.

Background technique

With the gradual expansion of the scope of use, video surveillance technology has become an important means of collecting criminal evidence and extracting criminal clues in the process of investigating various cases [1-4]. When encountering blurry images, investigators often use software such as Photoshop, Jingshitong image analysis platform, Video Investigator, "Doctor Shadow", "VCS", vReveal Premium, and "Sobey" to deal with it [1]. Generally, these softwares are effective for blurred images caused by a single degradation, but it is difficult to achieve ideal results for extremely blurred images caused by multiple degradations.

Early methods typically assumed that the PSF has a simple parametric form. In reality, PSF is much more complex and should be described by a complex parametric model. Dilip G.Warrier and Uday B.Desai regarded the PSF as a matrix, called the fuzzy matrix. The elements of the matrix adopt a random model, and use the mean field approximation to obtain a closed-form fuzzy element mean value expression. Therefore, the PSF estimation algorithm using the non-functional model has been developed rapidly: the early estimation method based on image complex cepstrum, the estimation method based on two-dimensional ARMA model, the Wiener filtering method using multi-frame image sequence and the recent emergence based on singular value decomposition estimation method, etc.

Recently, based on the statistical properties of natural images, many good algorithms for estimating PSF have been proposed. Fergus et al. used a mixed Gaussian model for the gray gradient of natural images, combined with variational Bayesian estimation PSF, and then used the L-R algorithm to deconvolute. This method works well for smaller PSFs. Later, Krishan and Fergus used the super-Laplace prior model for the heavy-tailed distribution of the gray gradient of natural images, and applied a cross-iteration scheme. This scheme uses a look-up table algorithm and can be quickly optimized. However, the restored image There is still an obvious step effect. Levin et al. summarized these methods and proposed an optimization method based on the marginal likelihood based on the maximum posterior probability, which proved to be more robust than the usual posterior probability. While these methods work well, they are computationally expensive. Because the latent image needs to be marginalized, the optimization of the energy function usually requires a rather complex iterative numerical algorithm. For example, in the optimization scheme, alternate iterations of PSF estimation and image restoration are required. However, if the initial blur kernel is not set well in a matching way or with an appropriate size, it will usually fail to converge to the true global minimum.

Contents of the invention

This patent uses the test value of the field scene as the initial value, and proposes a restoration method for a multi-degraded extremely blurred image, and uses the variational Bayesian theory to accurately estimate each sub-degeneration function, thereby accurately estimating the PSF of the imaging system. The recovery of extremely blurred criminal investigation images is realized. Due to the careful handling of multiple degradation factors, the restoration effect of the image is good.

A restoration method for multiple degraded extremely blurred images, S1, acquires data;

S01, get external parameters:

S02, obtain the blurred image matrix g(x,y) and find its gradient cepstrum

S2, establishing the system degradation function h(x, y) and each sub-degeneration function;

S3, initialize each sub-degradation function, and solve the initial gradient cepstrum of each sub-degeneration function;

S4, using the initial gradient cepstrum of each sub-degradation function and the gradient cepstrum of the blurred image as the input of the variational Bayesian, and solving the optimal distribution of each sub-degeneration function;

S5, using the relationship between each sub-degenerate function and the system point spread function, to solve the system point spread function h(x, y).

S6, using h(x, y) and g(x, y) as input of the L-R algorithm to obtain a clear image matrix f(x, y).

Further defined, the acquisition of external parameters in S01 specifically includes:

S01, obtain external parameters: obtain camera working parameters focal length f, lens diameter D, aperture coefficient F, CCD imaging pixel size w _x , w _y and object distance z, image distance v, number of pixels K, P, photosensitive element The length l of each pixel point, the moving distance d of the object within the exposure time τ, and the direction θ of the target moving.

Further defined, the respective degradation functions of the S2 specifically include:

The sub-degeneration function of the Airy disk mode caused by optical diffraction is h1(x,y);

The sub-degeneration function of the image sensor during quantization is h2(x,y);

The sub-degeneration function of camera defocus is h3(x,y);

The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x,y).

Further defined, the sub-degeneration function h1(x,y) of the Airy disk mode caused by optical diffraction is specifically:

The sub-degradation function of the image sensor during quantization is h2(x, y), specifically:

The sub-degradation function of camera defocus is h3(x,y) specifically:

The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x, y), specifically:

Further defined, the S3 specifically includes:

Substituting the parameter values obtained in S01 into the corresponding sub-degeneration functions in S2 to initialize each sub-degeneration function, and obtain initial values h10, h20, h30, and h40 of each sub-degeneration function;

Solve the initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 of each sub-degradation function, and set the gradient cepstrum of the system degradation function h(x,y) as C _h .

Further defined, the S4 specifically includes:

The initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 C _h40 , Get C _h1 , C _h2 , C _h3 , C _h4 as input to Variational Bayesian;

They satisfy the linear relationship:

C _h =k1C _h1 +k2C _h2 +k3C _h3 +k4C _h4 , wherein k ₁ , k ₂ , k ₃ , and k ₄ are constants, and at least one of k ₁ , k ₂ , k ₃ , and k ₄ is not 0.

It is further defined that the typical value of the k parameters k1, k2, k3, and k4 is 1, and the value of the k parameter varies around 1 in view of the degradation of a certain imaging system, and the specific value is determined by experiments. This patent is not limited to four degradation factors, if there are more than four degradation factors, the number of k parameters should be increased.

The beneficial effects of the present invention are: by considering multiple degradation factors of image blur and accurately estimating each sub-degeneration function through variational Bayesian theory, thereby accurately estimating the PSF of the imaging system and realizing the restoration of extremely blurred images.

detailed description

Description of drawings:

Fig. 1 is a schematic flow chart of the present invention.

Figure 2 The imaging optical path diagram of the camera.

Fig. 3 is a camera picture when the camera is clearly shooting.

Figure 4 is a frame of a blurred video from the camera.

Fig. 5 is the clear picture processed by the method of the present invention

As shown in Figure 1, a restoration method for an extremely blurred image with multiple degradation factors includes the following steps:

S1, get data;

S01, obtain external parameters: obtain camera working parameters focal length f, lens diameter D, aperture coefficient F, CCD imaging pixel size ω _x , ω _y and object distance z, image distance v, number of pixels K, P, photosensitive element The length of each pixel point l, the moving distance d of the object within the exposure time τ, and the direction of the target moving θ;

S02, obtain the fuzzy image matrix g(x,y);

S2, establish the degradation function,

The system degradation function is h(x,y);

The sub-degeneration function of the Airy disk mode caused by optical diffraction is h1(x,y);

The sub-degeneration function of the image sensor during quantization is h2(x,y);

The sub-degeneration function of camera defocus is h3(x,y);

The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x,y);

The sub-degeneration function h1(x,y) of the Airy disk mode caused by optical diffraction is specifically:

The sub-degradation function of the image sensor during quantization is h2(x, y), specifically:

The sub-degradation function of camera defocus is h3(x,y) specifically:

The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x, y), specifically:

S3. Substituting the parameter values obtained in S01 into the corresponding sub-degradation functions in S2 to initialize each sub-degeneration function, and obtain initial values h10, h20, h30, and h40 of each sub-degeneration function;

Solve the initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 of each sub-degradation function, and the gradient cepstrum of the system degradation function h(x, y) is C _h ;

The initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 , and C _g of each sub-degradation function are used as the input of the variational Bayesian to obtain C _h1 , C _h2 , C _h3 , and C _h4 ;

They satisfy the linear relationship:

C _h =k1C _h1 +k2C _h2 +k3C _h3 +k4C _h4 , wherein k ₁ , k ₂ , k ₃ , and k ₄ are constants, and at least two of k ₁ , k ₂ , k ₃ , and k ₄ are not 0, its value is obtained by experiment.

S4, using h(x, y) and g(x, y) as input of the L-R algorithm to obtain a clear image matrix f(x, y).

The specific principles are as follows:

The degraded image g(x,y) of the clear image f(x,y) can be expressed as:

in Indicates the convolution operation, h(x, y) is the system point spread function (Point Spread Function, PSF) υ(x, y) is the system noise. The gradient of a natural image f(x,y) obeys a heavy-tailed distribution, and a mixed Gaussian distribution is used to represent its distribution:

Among them, C represents the number of mixed Gaussian distributions, π _c represents the weight of the c-th distribution, and v _c is the variance of the c-th distribution. For the following calculations, the image is subjected to gradient cepstrum processing. According to the definition of cepstrum, the cepstrum of the fuzzy image g(x, y) is:

C _g (p,q)=FFT ^-1 {log[1+|G(u,v)|]} (3)

Gradient processing is performed on the formula (3) in the x direction and y direction respectively, and the gradient cepstrum of the blurred image can be obtained.

The system takes the image degradation caused by optical diffraction, quantization, defocus and relative motion into consideration, and constructs the model of point spread function caused by multiple degradation factors. Assuming that each sub-degradation function in the imaging process is expressed as h1, h2, h3, h4, the degraded image g(x,y) under the joint action of multiple degradation and noise v(x,y) is:

Here, v(x,y) is zero-mean Gaussian white noise, and h1(x,y) represents the sub-degeneration function of the Airy disk mode caused by optical diffraction:

in, J1() is the first-order Bessel function of the first kind; r ₀ is the distance from the center of the Airy disk to the main lobe; B=πf/F (f is the focal length of the lens, F is the aperture coefficient).

h2(x,y) represents the sub-degradation function of the rectangular sensing unit during quantization:

Among them, ωx and ωy represent the length and width of a rectangular pixel, respectively.

h3(x,y) represents the sub-degradation function when defocusing:

Among them, R is the radius of the defocus spot, expressed as:

R＝(1/f-1/v-1/z)Dv/2 (8)

In the formula, f is the focal length of the lens, z is the object distance, D is the radius of the convex lens, and v is the image distance.

h4(x,y) represents the sub-degradation function when the imaging system and the target move in a straight line at a constant speed:

where θ is the blur angle and L is the length of the degradation function. θ is estimated by analyzing the target motion of the video frame, and L can be measured on the spot according to the imaging ratio.

In a single frame image, the motion blur length is the moving distance of the vehicle during exposure, and the moving time is the exposure time. Calculate the corresponding length of the motion blur length on the photosensitive element according to the method of imaging proportional relationship, the principle is as follows:

As shown in the imaging optical path diagram in Figure 2, d is the distance of the vehicle moving when it is photographed—the length of the motion blur, and the angle between it and the focal plane is α; k is the corresponding length of the motion blur length d on the photosensitive element. On the photosensitive element, the distance between the motion blur edge and the normal is p, the focal length is f, and the object distance is z. According to the proportional relationship of similar triangles, we can get:

Since k and p are length units, but actually we calculate the length by counting the number of pixels, so the actual number of pixels is K, P, and the length of each pixel of the photosensitive element is l, so k=kl, p=Pl.

Assuming the cepstrum C _h , C _h1 , C _h2 , C _h3 , C _h4 of h, h1, h2, h3, h4, then the following relationship exists:

C _h ＝k1C _h1 +k2C _h2 +k3C _h3 +k4C _h4 (11)

Among them, k1, k2, k3, and k4 are constants. For blurred images caused by the four degradation factors, the typical values of the k parameters k1, k2, k3, and k4 are 1. For the degradation of a certain imaging system, the value of the k parameter changes around 1, and the specific value is determined by the experiment. . This patent is not limited to four degradation factors, if there are more than four degradation factors, the number of k parameters should be increased, and the estimation method is still valid.

It can be seen from formula (1) that the process of recovering a clear image when the point spread function of the system is unknown is blind deconvolution. Therefore, image restoration is performed in two steps. First, the point spread function of the system is estimated. Secondly, the L-R algorithm is used to restore the image. The variational Bayesian theory is used to estimate the fuzzy kernel with high precision.

Interpretation of the point spread function of the variational Bayesian estimation system:

Given a blurred image g(x,y), estimate the PSF (systematic point spread function) and the sharp image f(x,y) by finding the maximum posterior probability given the prior information of f(x,y) ).

For formula (4), assuming hidden variables Θ = {f, h1, h2, h3, h4}, it is troublesome to directly calculate the posterior distribution p(Θ|g), VB uses a simpler distribution q(Θ|g) to Approximate posterior distributions p(Θ|g), their KL divergence is:

Make formula (12) minimum to solve q(Θ|g). Since the integral variable is Θ and p(g) is a constant, formula (12) can be expressed as:

Define C _KL as the cost function, which can be expressed as:

Assuming that q(Θ) and q(g) are independent of each other q(Θ, g)=q(Θ)q(g), that is, formula (14) can be rewritten as:

Now put Θ={f, h1, h2, h3, h4} into (15) formula, can get:

Assuming that each sub-degenerate function is independent of each other, then

q(f,h1.h2,h3,h4)=q(f)q(h1)q(h2)q(h3)q(h4) (17)

Substitute (17) into (16) to get:

Further derivation of formula (18), we get:

C _KL ＝∫q(f)q(h1)q(h2)q(h3)q(h4)[lnq(f)+lnq(h1)+lnq(h2)+lnq(h3)+lnq(h4)

-lnp(f)-lnp(h1)-lnp(h2)-lnp(h3)-lnp(h4)-lnp(g|f,h1.h2,h3,h4)]dΘ (19)

When the cost function (19) is minimized, alternate iterations are used to solve it. When solving a certain variable, it is assumed that the remaining variables are constant.

When solving q(h1), assuming q(f), q(h2), q(h3), q(h4) are known constants, let: Assuming that the noise in the degradation model is Gaussian white noise with intensity σ2, then:

Among them, N() represents a Gaussian distribution, which can be obtained:

Then formula (19) only integrates h1, and the cost function can be rewritten as follows:

According to the constraint condition ∫h1dh1=1, add the Lagrangian multiplier λ _h1 to (22), find the extremum of the cost function, and obtain q(h1):

It can be seen that q(h1) is a function of f, h2, h3, and h4, so it needs to be updated alternately to iterate. In the same way:

in,

in,

in,

Therefore, the values of each sub-degeneration function obtained through the field test are used as initial values, and the optimal distribution of each sub-degeneration function can be obtained by using formulas (23)-(26) by using alternate iterations, so that each sub-degeneration function can be obtained by calculating the mathematical expectation. The exact value of the subdegenerate function.

It is precisely because of the above relationship that the variational Bayesian sub-degradation function estimation method must be performed in the gradient cepstrum domain. This method first estimates the sub-degeneration functions C _h1 , C _h2 , C _h3 , and C _h4 , and then uses (11) Synthesize the point spread function C _h of the system with the formula, so as to obtain the system point spread function h. Finally, the clear image f(x,y) can be obtained by using the LR algorithm.

Specific implementation steps:

The first step is to obtain the parameters focal length f, lens diameter D, aperture factor F, CCD imaging pixel size w _x , w _y and object distance z, image distance v, number of pixels K, P, and each pixel of the photosensitive element The length l, the moving distance d of the object within the exposure time τ, and the moving direction θ of the target.

In the second step, the initial values h10, h20, h30, and h40 of each sub-degeneration function are obtained by using the function mode of formulas (5)-(10).

The third step is to solve the initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 of each sub-degradation function and the gradient cepstrum of the blurred image

In the fourth step, C _h10 , C _h20 , C _h30 , C _h40 , C _h1 , C _h2 , C _h3 , and C _h4 can be obtained by variational Bayesian using the super-Laplace distribution of the natural image gray gradient as the initial value.

The fifth step is to use formula (11) to obtain the point spread function of the system.

The sixth step is to solve the distribution of the point spread function of the system in the time domain, and calculate the expected point spread function of the system.

In the seventh step, the L-R algorithm is used for restoration to obtain a clear image f(x,y).

For investigators, it is not necessary to understand the principle of variational Bayesian estimation point spread function and L-R restoration algorithm, and only need to use the following steps to use the extremely blurred image restoration system:

(1) For a very blurred image obtained by monitoring, input parameters focal length f, lens diameter D, aperture coefficient F, CCD imaging pixel size ω _x , ω _y and object distance z, image distance v, number of pixels Mesh K, P, the length l of each pixel of the photosensitive element, the movement distance d of the object within the exposure time τ, and the direction θ of the target movement.

(2) Input these parameters and the blurred image into the system, and obtain the restored image through the restoration system.

During this process, the accuracy of the test parameters f, D, F, z, v, and d must reach 0.1mm, the accuracy of τ must reach 0.001s, and the accuracy of θ must reach 0.1 degrees. Parametric testing must have strict steps. As long as the steps are followed, different people can run the software and get the same results.

As shown in Figure 4, a fuzzy video was read from the hard disk of the monitoring system at around 5:46 pm on March 15, 2017, including a frame.

We know from the product manual that the focal length of the monitoring system lens is F=6mm, the aperture coefficient is f=1.4, the imaging pixel size ωx=ωy=6.4μm, and the exposure time of the image sensor τ=0.01s. The distance between these vehicles and the lens was measured on site to be about 108m. The k parameters tested by experiments are specifically: k1=1.2; k2=0.9; k3=0.8; k4=1.3.

The result obtained by the patented method is shown in Figure 5, and the signal-to-noise ratio is increased by about 9dB.

Claims (7)

1. A restoration method of the extremely blurred image of multiple degradations, characterized in that: S1, get data; S01, get external parameters: S02, obtain the blurred image matrix g(x, y), and solve its gradient cepstrum; S2, establishing the system degradation function h(x, y) and each sub-degeneration function; S3, initialize each sub-degradation function, and obtain the initial value of each sub-degeneration function; and solve the initial gradient cepstrum of each sub-degeneration function; S4, using the initial gradient cepstrum of each sub-degradation function and the gradient cepstrum of the blurred image as the input of the variational Bayesian, and solving the optimal distribution of each sub-degeneration function; S5, using the relationship between each sub-degenerate function and the system point spread function, to solve the system point spread function h(x, y). S6, using h(x, y) and g(x, y) as input of the L-R algorithm to obtain a clear image matrix f(x, y). 2. the restoration method of a kind of extremely blurry image of multiple degeneration according to claim 1, is characterized in that: The acquisition of external parameters in S01 specifically includes: S01, obtain external parameters: obtain camera working parameters focal length f, lens diameter D, aperture coefficient F, CCD imaging pixel size w _x , w _y and object distance z, image distance v, number of pixels K, P, photosensitive element The length l of each pixel point, the moving distance d of the object within the exposure time τ, and the direction θ of the target moving. 3. the restoration method of a kind of extremely blurry image of multiple degeneration according to claim 2, is characterized in that: The respective degradation functions of the S2 specifically include: The sub-degeneration function of the Airy disk mode caused by optical diffraction is h1(x,y); The sub-degeneration function of the image sensor during quantization is h2(x,y); The sub-degeneration function of camera defocus is h3(x,y); The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x,y). 4. the restoration method of a kind of extremely blurry image of multiple degeneration according to claim 3, is characterized in that: The sub-degeneration function h1(x,y) of the Airy disk mode caused by optical diffraction is specifically: The sub-degradation function of the image sensor during quantization is h2(x, y), specifically: The sub-degradation function of camera defocus is h3(x,y) specifically: The sub-degradation function of the relative uniform linear motion between the camera and the target is h4(x, y), specifically: 5. the restoration method of a kind of extremely blurry image of multiple degeneration according to claim 4, is characterized in that: The S3 specifically includes: Bringing the parameter values obtained in S01 into the corresponding sub-degeneration functions in S2 to initialize each sub-degeneration function, and obtain initial values h10, h20, h30, and h40 of each sub-degeneration function; Solve the initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 of each sub-degradation function, and set the gradient cepstrum of the system degradation function h(x,y) as C _h . 6. a kind of restoration method of the extremely blurred image of multiple degeneration according to claim 5, is characterized in that: The S4 specifically includes: The initial gradient cepstrum C _h10 , C _h20 , C _h30 , C _h40 , C _▽g of each sub-degradation function are used as the input of the variational Bayesian to obtain C _h1 , C _h2 , C _h3 , C _h4 ; They satisfy the linear relationship: C _h =k1C _h1 +k2C _h2 +k3C _h3 +k4C _h4 , wherein k ₁ , k ₂ , k ₃ , and k ₄ are constants, and at least one of k ₁ , k ₂ , k ₃ , and k ₄ is not 0. 7. a kind of restoration method of the extremely blurred image of multiple degeneration according to claim 5, is characterized in that: For blurred images caused by the four degradation factors, the typical values of the k parameters k1, k2, k3, and k4 are 1. For the degradation of a certain imaging system, the value of the k parameter changes around 1, and the specific value is determined by the experiment. . This patent is not limited to four degradation factors, if there are more than four degradation factors, the number of k parameters should be increased.