CN107154064A - Natural image compressed sensing method for reconstructing based on depth sparse coding - Google Patents

Abstract

The invention discloses a natural image compressive perception reconstruction method based on deep sparse coding, which mainly solves the problem that the existing method is difficult to quickly and accurately reconstruct the natural image with coefficients. The implementation plan is: 1) divide the image into blocks and transform in the orthogonal transform domain, and calculate the observation vector of the transformation coefficient; 2) use the iterative threshold method to find the restored transformation coefficient of the observation vector, and update the calculation parameters; 3) calculate 2) Observation vectors of transformation coefficients in 1) to obtain the residual amount of the observation vectors; 4) Repeat steps 1)-3) to obtain the trained model, and save the model parameters; 5) Combine the test observation data with Input the model parameters into the trained model to obtain the image transformation coefficients corresponding to the test observation data; 6) inversely transform the transformation coefficients in 5) to obtain the final reconstructed natural image. The natural image reconstructed by the invention is clear and the reconstruction speed is very fast, and can be used for restoring the natural image.

Description

Compressive Sensing Reconstruction Method for Natural Images Based on Deep Sparse Coding

technical field

The invention belongs to the field of image processing, and in particular relates to a natural image compression sensing reconstruction method, which can be used for sampling natural image restoration.

Background technique

With the development of media technology, massive image data is facing huge challenges in real-time transmission and storage. The introduction of compressed sensing technology has opened up new ideas for these problems in theory, and effectively solved the problems. Compressed sensing theory believes that if the signal has sparsity under a certain transformation base, the signal can be randomly projected and observed, and it can be accurately reconstructed through the prior information of the signal with fewer observations. The model is a norm optimization problem under the fidelity constraints of solving observed data.

For the above-mentioned compressed sensing model, different norm constraints represent different reconstruction algorithms and reconstruction performance. If the norm is used, the orthogonal matching pursuit OMP algorithm is usually used for reconstruction. Although the norm satisfies the original idea of the compressed sensing model, which is to find the solution of the optimization problem with the minimum sparsity, but because it is an NP-Hard problem, the accuracy of the solution is often not high. Therefore, some scholars proposed to use norm instead of norm to make it a convex optimization problem, and proposed a series of iteration-based reconstruction algorithms: iterative threshold shrinkage algorithm ISTA, fast iterative threshold shrinkage algorithm FISTA, approximate information transfer algorithm AMP, etc. Although the above algorithms have further solved the problem, they are all based on optimization theory, and there are still many problems such as limited reconstruction accuracy, complex iterative process, and slow convergence speed.

In recent years, with the development of deep learning technology, some scholars began to propose Recon-Net, a compressed sensing reconstruction method based on convolutional neural network. Although this method solves the problem of complex iterative process, there are still two problems: 1) The reconstructed image obtained by directly applying the model is noisy, and the reconstructed image with better quality must be denoised; 2) The convergence speed of the model is slow during training. Even on a high-performance computer, it takes a day to train the model. time.

Contents of the invention

The purpose of the present invention is to solve the problems existing in the traditional reconstruction method based on optimization and the current reconstruction method based on deep learning, and propose a natural image compression sensing reconstruction method based on deep sparse coding, so as to simplify the model complexity and reduce the training time of the model , to improve the reconstruction speed and reconstruction effect of the image.

The technical solution of the present invention is: by using the sparse prior information of the natural image in the transform domain to perform compressive sensing encoding on it, and then combining the learning-based approximate information transfer algorithm LAMP and the recurrent neural network RNN model, the image encoding information is realized. Restoration and reconstruction, its implementation steps include the following:

(1) Model training steps:

(1a) Input multiple pictures, and take n training image blocks X from these pictures;

(1b) Perform compressed sensing observation on the n training image blocks X in (1a), obtain n observation coefficients Y, and use these training image blocks and corresponding observation coefficients to form n training sample pairs: {(X=x ₁ ,x ₂ ,...,x _n-1 ,x _n ),(Y=y ₁ ,y ₂ ,...,y _n-1 ,y _n )};

(1c) Set the number of model training K=100, randomly select r training samples y _r , x _r from Y and X each time, and use the gradient descent method for training, the cut-off condition flag for each training is to iterate the model for 50 times The error value does not decay;

(1d) Set the parameters of the sparse coding algorithm, initialize the number of iterations T=10, and make the initial transformation system Observed data residual v ^t =y, where is the training image block transformation coefficient calculated for the t-th iteration, v ^t is the observation residual of the t-th iteration;

(1e) Calculate the training image block transformation coefficient of the t+1 iteration: in is the transformation coefficient of the t-th iteration, A ^T C ^t v ^t is the transformation coefficient of the observation residual of the t-th iteration, ^AT is the transpose of the observation matrix A, C ^t is the parameter matrix to be optimized for the t-th time, is the threshold of the t-th iteration, α ^t is the scalar parameter value of the t-th update, M is the dimension of the observed data y, ||v ^t || ₂ represents the two-norm of v ^t , and η _st (·) is the threshold contraction function;

(1f) Calculate the observation residuals of the t+1th iteration: Among them, b ^t+1 v ^t is the Onsagercorrection item;

(1g) Loop execution T times (1e)—(1f) to get the transformation coefficient

(1h) When (1g) obtained transformation coefficient When the iteration cut-off condition flag is met, save the model parameters of this training;

(1i) cyclically execute K times (1d)—(1h) to complete model training;

(2) Test steps:

(2a) The observed data y _test and T parameter matrices obtained through model training and scalar parameter values Input it into the trained model to get the test image transformation coefficient corresponding to the input observation data y _test

(2b) For transform coefficients Do PCA inverse transformation to get the image Img corresponding to the observed data y _test : Among them, Ψ ^-1 represents the PCA inverse transformation matrix;

(2c) According to the ΨΨ ^-1 = ΨΨ ^T relationship that exists due to the orthogonality of the PCA transformation, the image Img corresponding to the observed data y _test is rewritten as: Complete the natural image reconstruction of the observed data y _test , where Ψ ^T represents the transpose of the PCA forward transformation matrix Ψ.

Compared with other prior art, the present invention has the following advantages:

First, the present invention introduces the sparse prior information of natural images, combines the advantages of deep neural network and sparse coding, reduces the complexity of the model, thereby reduces the reconstruction time of images, realizes the rapid reconstruction of compressed sensing, and improves the natural Image reconstruction effect;

Second, the present invention adopts the model training method of transfer learning, which improves the speed of model training, thereby realizing fast training of the model.

Description of drawings

Fig. 1 is the realization overall flowchart of the present invention;

Fig. 2 is the sub-flow chart of model training in the present invention;

Fig. 3 is a sub-flow chart of image reconstruction in the present invention;

Fig. 4 is the original natural image of Barbara used in the simulation experiment of the present invention;

Fig. 5 is the reconstruction effect diagram when the compression rate is 0.25 to the Barbara image with the existing TVAL3 method;

Fig. 6 is the reconstructed rendering of the Barbara image when the compression rate is 0.25 with the existing Recon-Net method;

Fig. 7 is a reconstruction effect diagram of the Barbara image when the compression rate is 0.25 using the present invention.

detailed description:

Embodiment of the present invention and effect are described in detail below in conjunction with accompanying drawing:

Referring to Fig. 1, the natural image reconstruction method based on deep sparse coding of the present invention includes two parts of model training and testing. First, input n training image blocks X and construct training sample pairs, then perform model training to obtain a trained model, and then input test observation data into the trained model for testing to obtain reconstructed natural images.

The following two parts of model training and testing of the present invention are described in detail:

1. Model training part

Referring to Figure 2, the implementation steps of this part are as follows:

Step 1: Input n training image blocks X to obtain training sample pairs,

(1a) Input n training image blocks X, perform principal component analysis PCA transformation on each input training image block data x _i , and obtain transformation coefficients Among them, Ψ represents the principal component analysis PCA forward transformation matrix, according to the orthogonality of the principal component analysis PCA transformation, the relational expression of ΨΨ ^-1 = ΨΨ ^T is obtained, according to the relational expression Among them, Ψ ^-1 and Ψ ^T denote the transposition of PCA inverse transformation matrix and forward transformation matrix Ψ respectively;

(1b) Observe each training image block according to the compressed sensing observation model, and obtain the observation data y _i : Among them, A=ΦΨ ^T , Φ is an undersampled Gaussian random observation matrix, and the vector w is Gaussian white noise with zero mean;

(1c) Form n training sample pairs with n training image blocks X and corresponding n observation coefficients Y: {(X=x ₁ , x ₂ ,..., x _n-1 , x _n ),(Y =y ₁ ,y ₂ ,...,y _n-1 ,y _n )}.

Step 2: Set the number of model training times, randomly select r training sample pairs,

(2a) Set the number of times of model training K=100;

(2b) Randomly select r training sample pairs {x _r , y _r } from Y and X during each training.

Step 3: Set the parameters of the sparse coding algorithm and initialize the relevant variables in the algorithm.

(3a) The number of initialization iterations T=10;

(3b) Let the initial transformation coefficient Observed data residual v ^t =y, where is the image block transformation coefficient calculated by the t-th iteration, v ^t is the observation residual of the t-th iteration.

Step 4: Calculate the training image block transformation coefficient of the t+1th iteration

(4a) Calculate the sparse coefficient A ^T C ^t v ^t of the observation residual v ^t of the t-th iteration, where A ^T is the transpose of the observation matrix A, and C ^t is the parameter matrix to be optimized for the t-th iteration and is initialized to unit matrix;

(4b) Calculate the training image block sparse coefficient of the t+1th iteration, in, is the transformation coefficient of the t-th iteration, is the threshold of the t-th iteration, α ^t is the scalar parameter value updated at the t-th time, M is the dimension of the observed data y, and ||v ^t || ₂ represents the second norm of v ^t ;

(4c) Calculate the training image block transformation coefficient of the t+1 iteration,

Among them, η _st (·) is the threshold contraction function, which is used to compare the threshold λ ^t with the sparse coefficient μ ^t+1 . The specific method is to set the sparse coefficient μ ^t+1 smaller than the threshold λ ^t to zero, and set The sparse coefficient μ ^{t +1} of t is set as the absolute value of the difference between μ ^t+1 and the threshold λ ^t .

Step 5: Calculate the observation residual v ^t+1 of the t+1th iteration.

(5a) first obtain the step 4 The coefficients greater than zero are set to 1, and then averaged by column to get The zero norm of Then calculate the weight b ^t+1 :

where N is the transform coefficient The dimension, M is the dimension of the observation residual v ^t ;

(5b) Dot product the weight b ^t+1 with the observation residual v ^t to obtain the matrix b ^t+1 v ^t of the Onsager correction item;

(5c) Substitute the matrix b ^t+1 v ^t obtained in (5b) into the formula Calculate the observation residual v ^t+1 of the t+1th iteration, where y is the observation data of the training image block, and A is the observation matrix.

Step 6: Perform step 4-step 5 cyclically T=10 times to obtain the transformation coefficient

Step 7: Save the model parameters of this training.

When the transformation coefficient obtained in step 6 When the iteration cut-off condition flag is met, the model parameters of this training are saved. The cut-off condition flag of each training is that the model error value does not decay after 50 iterations.

Step 8: Perform step 3-step 7 cyclically K times to complete the model training.

2. Test part

Referring to Figure 3, the implementation steps of this part are as follows:

Step 9: Input the test observation data and the parameters saved in the model training part to obtain the image transformation coefficient corresponding to the input observation data y _test

Take out the saved T matrix C ^t and scalar α ^t from the model training part and write them as with And these two parameters and the test observation data y _test collected in the real scene are input into the trained model, and the image transformation coefficient corresponding to the observation data y _test is output

Step 10: Obtain the natural image Img corresponding to the observation data y _test through principal component analysis PCA inverse transformation.

(10a) for transform coefficients Perform principal component analysis PCA inverse transformation to obtain the natural image corresponding to the observed data y _test : Among them, Ψ ^-1 represents the principal component analysis PCA inverse transformation matrix;

(10b) According to the relational expression of ΨΨ ^-1 = ΨΨ ^T of principal component analysis PCA transformation, the natural image Img corresponding to the observed data y _test is rewritten as: Complete the natural image reconstruction of the observed data y _test , where Ψ ^T represents the transpose of the principal component analysis PCA forward transformation matrix Ψ.

Effect of the present invention can be specified by following simulation experiments:

1. Simulation conditions:

1) The programming platform used in the simulation experiment is Pycharm v2016;

2) The natural image data used in the simulation experiment comes from the standard training and testing data sets;

3) The size of the training image block used in the simulation experiment is 25×25, and the number of training samples n is 52650;

4) In the simulation experiment, the peak signal-to-noise ratio PSNR index is used to evaluate the compressive sensing experiment results, and the peak signal-to-noise ratio PSNR is defined as:

Among them, MAX _i and MSE _i are the maximum pixel value and mean square error of the reconstructed high-resolution natural image Img, and N is the number of pixels.

2. Simulation content:

In simulation 1, the TVAL3 method is used to reconstruct the natural image Barbara when the compression rate is 0.25, and the reconstruction results are shown in Figure 5.

In simulation 2, the Recon-Net method is used to reconstruct the natural image Barbara at a compression rate of 0.25, and the reconstruction results are shown in Figure 6.

In simulation 3, the method of the present invention is used to reconstruct the natural image Barbara when the compression ratio is 0.25, and the reconstruction result is shown in FIG. 7 .

It can be seen from the reconstruction results of the natural image Barbara shown in Fig. 5-Fig. 7 that the image reconstructed by the present invention is clearer, the edges of the image are sharper, and the visual effect is better than those reconstructed by other methods.

The existing TVAL3 method, NLR-CS method, Recon-Net method and the method of the present invention are respectively used to reconstruct and simulate the natural image Barbara, and the obtained peak signal-to-noise ratio PSNR is shown in Table 1.

Table 1 PSNR values of different reconstruction methods

It can be seen from Table 1 that the PSNR of the present invention is 3.73dB higher than that of the existing TVAL3 method when the compression rate is 0.25, and 2.00dB higher than that of the existing Recon-Net.

Claims (3)

1. Natural image reconstruction method based on deep sparse coding, including: (1) Model training steps: (1a) Input multiple pictures, and take n training image blocks X from these pictures; (1b) Perform compressed sensing observation on the n training image blocks X in (1a), obtain n observation coefficients Y, and use these training image blocks and corresponding observation coefficients to form n training sample pairs: {(X=x ₁ ,x ₂ ,...,x _n-1 ,x _n ),(Y=y ₁ ,y ₂ ,...,y _n-1 ,y _n )}; (1c) Set the number of model training K=100, randomly select r training samples y _r , x _r from Y and X each time, and use the gradient descent method for training, the cut-off condition flag for each training is to iterate the model for 50 times The error value does not decay; (1d) Set the parameters of the sparse coding algorithm, initialize the number of iterations T=10, and make the initial transformation system Observed data residual v ^t =y, where is the training image block transformation coefficient calculated for the t-th iteration, v ^t is the observation residual of the t-th iteration; (1e) Calculate the training image block transformation coefficient of the t+1 iteration: in is the transformation coefficient of the t-th iteration, A ^T C ^t v ^t is the transformation coefficient of the observation residual of the t-th iteration, ^AT is the transpose of the observation matrix A, C ^t is the parameter matrix to be optimized for the t-th time, is the threshold of the t-th iteration, α ^t is the scalar parameter value of the t-th update, M is the dimension of the observed data y, ||v ^t || ₂ represents the two-norm of v ^t , and η _st (·) is the threshold contraction function; (1f) Calculate the observation residuals of the t+1th iteration: Among them, b ^t+1 v ^t is the Onsagercorrection item; (1g) Loop execution T times (1e)—(1f) to get the transformation coefficient (1h) When (1g) obtained transformation coefficient When the iteration cut-off condition flag is met, save the model parameters of this training; (1i) cyclically execute K times (1d)—(1h) to complete model training; (2) Test steps: (2a) The observed data y _test and T parameter matrices obtained through model training and scalar parameter values Input it into the trained model to get the test image transformation coefficient corresponding to the input observation data y _test (2b) For transform coefficients Do PCA inverse transformation to get the image Img corresponding to the observed data y _test : Among them, Ψ ^-1 represents the PCA inverse transformation matrix; (2c) According to the ΨΨ ^-1 = ΨΨ ^T relationship that exists due to the orthogonality of the PCA transformation, the image Img corresponding to the observed data y _test is rewritten as: Complete the natural image reconstruction of the observed data y _test , where Ψ ^T represents the transpose of the PCA forward transformation matrix Ψ. 2. according to the method for claim 1, wherein in the step (1b), carry out compressed sensing observation to n image blocks X in (1a), carry out as follows: (1b1) Perform PCA transformation on each image block data x _i to obtain the transformation coefficient: Among them, Ψ and Ψ ^-1 represent PCA forward transformation and its inverse transformation respectively, and due to the orthogonality of PCA transformation, satisfying ΨΨ ^-1 = ΨΨ ^T , then, Ψ ^T represents the transpose of the PCA forward transformation matrix Ψ; (1b2) Observe each image block according to the compressed sensing observation model to obtain observation data: Where A=ΦΨ ^T , the matrix Φ is an undersampled Gaussian random observation matrix, and the vector w is Gaussian white noise with zero mean. 3. according to the method described in claim (1), wherein the Onsager correction item b ^t+1 v ^t in the step (1f) is calculated as follows: (1f1) put The coefficients greater than zero are set to 1, and then averaged by column to get The zero norm of Finally calculate the weight b ^t+1 : <mrow> <msup> <mi>b</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <mfrac> <mi>N</mi> <mi>M</mi> </mfrac> <mo>|</mo> <mo>|</mo> <msup> <mover> <mi>x</mi> <mo>^</mo> </mover> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>|</mo> <msub> <mo>|</mo> <mn>0</mn> </msub> <mo>,</mo> </mrow> where N is the transform coefficient The dimension, M is the dimension of the observation residual v ^t ; (1f2) Dot-multiply the weight b ^t+1 and the observation residual v ^t to obtain the matrix b ^t+1 v ^t of the Onsager correction item.