CN106157251A - A kind of face super-resolution method based on Cauchy's regularization - Google Patents

Abstract

A kind of face super-resolution method based on Cauchy's regularization, divides overlapped image block including to the low resolution face sample image in the low-resolution face image inputted, low resolution training set and the high-resolution human face sample image in high-resolution training set；Set up high-resolution and low-resolution image block collection respectively as high-resolution and low-resolution image block dictionary；Represent this image block with low-resolution image block dictionary, utilize Cauchy's bound term to solve reconstructed coefficients, then utilize this reconstructed coefficients and high-definition picture block dictionary to rebuild the high-definition picture block made new advances；The high-definition picture block of gained is fused into high-definition picture by the position relationship according to low-resolution image block.The present invention on the basis of method of least square to reconstructed coefficients plus Cauchy retrain, the present invention can get more high-quality and truth closer to reconstruction effect.

Description

A face super-resolution method based on Cauchy regularization

technical field

The invention relates to the field of image super-resolution, in particular to a Cauchy regularization-based face super-resolution method.

Background technique

In the past decade, people have witnessed the rapid development of video applications, such as video chat, video surveillance, video retrieval and so on. At the same time, people often face unsatisfactory video quality, especially the face image in the video has higher requirements. As a result, a technique called face super-resolution was developed and attracted a lot of attention in the fields of image processing and computer vision. Face super-resolution technology is a technology that can reconstruct one or more high-resolution images by using a single frame or continuous multiple frames of low-resolution face images, which can effectively enhance the resolution of low-quality images.

The face super-resolution technology restores the input low-resolution image to a high-resolution image, which is an inverse problem. In order to restore visually satisfying high-resolution images, researchers have proposed many regularization methods using image prior information, the purpose of which is to make this reverse process a stable result.

Recently, as a powerful tool for statistical signal modeling, sparse representation as a regularization term has been widely used in this inverse problem. Yang et al. introduced the sparse representation of the L ₁ norm in the phantom face for the first time in the literature [1], so that the corresponding low-resolution image blocks and high-resolution image blocks have the same sparse representation. This method enhances the face details. On the basis of the face super-resolution method based on the location image block, Jung et al. regarded the face super-resolution problem as a least squares problem with sparse constraints [2], and obtained a good super-resolution effect. Recently, Dong et al. took advantage of the non-local self-similarity of images to propose a non-local centralized sparse representation algorithm (NCSR) to better predict sparse reconstruction coefficients [3].

The sparse representation method assumes that the coefficient vector conforms to a multivariate Laplacian distribution with zero mean. This method adds a 1-norm penalty (L ₁ -penalty) to the reconstruction coefficient, and the reconstruction coefficient of the sparse representation is at the zero value. Spike distribution, reflecting strong sparsity. Since the face super-resolution method based on sparse representation emphasizes sparsity, it may choose a base image that is very different from the input image to reconstruct the input image, which makes the reconstructed high-resolution face image have a lot of noise, especially In parts such as eyes and mouth with rich edges, therefore, unsatisfactory reconstructed images may be obtained. Methods based on sparse representation assume that the reconstruction coefficients follow a Laplace distribution, however, this assumption may not be consistent with the real distribution.

Relevant references in the prior art are as follows:

Document 1: J. Yang, H. Tang, Y. Ma, and T. Huang, "Face hallucination via sparse coding," inProc. IEEE Conf. on Image Processing (ICIP), 2008, pp.1264–1267.

Literature 2: C.Jung, L.Jiao, B.Liu, and M.Gong, "Position-Patch Based Face Hallucination Using Convex Optimization," IEEE Signal Process. Lett., vol.18, no.6, pp.367– 370, 2011.

Document 3: Guangming Shi Weisheng Dong, Lei Zhang and Xin Li, "Nonlocally centralized sparse representation for image restoration," IEEE Trans. on Image Processing, vol.22, no.4, pp.1620–1630, 2013.

Document 4: C.Thomaz and G.Giraldi, "A new ranking method for principal components analysis and its application to face image analysis," Image and Vision Computing, vol.28, no.6, pp.902–913, 2010.

Document 5: H.Chang, D.Y.Yeung, and Y.M.Xiong. Super-resolution through neighbor embedding. In CVPR, pp.275–282, 2004.

Document 6: X.Ma, J.P Zhang, and C.Qi. Hallucinating face by position-patch. Pattern Recognition, 43(6):3178–3194, 2010.

Contents of the invention

The purpose of the present invention is to overcome the defects of the prior art and provide a method for super-resolution of human faces based on Cauchy regularization.

In order to achieve the above object, the technical solution adopted in the present invention is a method for super-resolution of human faces based on Cauchy regularization, comprising the following steps:

Step 1, dividing the image blocks overlapping each other for the input low-resolution face image, the low-resolution face sample image in the low-resolution training set, and the high-resolution face sample image in the high-resolution training set;

Step 2, for each image block in the input low-resolution face image, take the image block at the corresponding position of each low-resolution face sample image in the low-resolution training set as the sample point, and obtain the corresponding low-resolution image block dictionary , establish a low-resolution face sample block space, take the image block at the corresponding position of each high-resolution face sample image in the high-resolution training set as a sample point, obtain the corresponding high-resolution image block dictionary, and establish a high-resolution face Sample block space; the implementation is as follows, suppose the image block set formed after dividing the input low-resolution face image X ^t into M overlapping image blocks is Divide overlapping image blocks for the high-resolution and low-resolution face image training sets, and then obtain M high-resolution image block dictionaries corresponding to the M image blocks of the input low-resolution face image and a dictionary of low-resolution image patches Among them, the identifier i represents the sequence number of the high-resolution face sample image in the high-resolution training set and the sequence number of the low-resolution face sample image in the low-resolution training set, and the identifier j represents the block position sequence number on the image, For the number of low-resolution face sample images in the low-resolution training set and the number of high-resolution face sample images in the high-resolution training set, M is the number of blocks divided into image blocks for each image;

Step 3, for a certain image block in the input low-resolution face image, use the low-resolution face sample block space obtained in step 2 respectively The sparse representation of Cauchy regularization is performed to obtain the optimal weight coefficient α for linear reconstruction, which is implemented as follows,

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; ln</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\frac{{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

Among them, J(α) returns the value of α when the function of variable α gets the minimum value, α is dimension vector, by A linear reconstruction coefficient α _i constitutes, σ _α is the standard deviation of , is the Cauchy regular term, Represents the square of the result of the two-norm ||·|| ₂ , and λ is a regularization parameter to balance the reconstruction error and sparsity;

Step 4, using the optimal weight coefficient obtained in step 3 and the high-resolution face sample block space obtained in step 2, linearly reconstructing to obtain a new high-resolution image block;

Step 5: Superimpose all weighted and reconstructed high-resolution face image blocks according to their positions on the face, and then divide by the number of times each pixel position overlaps to obtain a high-resolution face image.

In the present invention, a more suitable prior model is used for face super-resolution reconstruction, and a model called Cauchy regularization is proposed to improve the performance of face super-resolution. Unlike the reconstruction coefficients of the Laplace distribution, which are spiky at zero values, the reconstruction coefficients corresponding to the Cauchy prior exhibit milder sparsity at zero values. By adding a Cauchy prior to the solution, a moderately sparse regularization method can be obtained that outperforms sparse algorithms based on the L1 norm.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention.

detailed description

The technical scheme of the present invention can adopt software technology to realize automatic flow operation. The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Referring to Fig. 1, the specific steps of the embodiment of the present invention are:

Step 1, dividing the image blocks overlapping each other for the input low-resolution face image, the low-resolution face sample image in the low-resolution training set, and the high-resolution face sample image in the high-resolution training set;

The low-resolution training set and the high-resolution training set provide preset training sample pairs, the low-resolution training set contains low-resolution face sample images, and the high-resolution training set contains high-resolution face sample images. In the embodiment, all high-resolution images are face images that have been manually aligned and registered, and the pixel size is 120×100. Each low-resolution face sample image in the low-resolution training set is obtained from a high-resolution face sample image in the high-resolution training set with 4×4 smooth filtering and 4 times downsampling, and the pixel size of the low-resolution image is 30 ×25, the high-resolution image block size is set to 12×12, the overlapping pixel value is set to 4, the low-resolution image block size is 3×3, and the overlapping pixel value is 1.

In an embodiment, the image block set formed after dividing the input low-resolution face image X ^t into overlapping image blocks is The high-resolution and low-resolution face image training sets are respectively divided into overlapping image blocks, and M high-resolution image blocks corresponding to the M image blocks of the input low-resolution face image are respectively obtained A dictionary of M low-resolution image blocks corresponding to the M image blocks of the input low-resolution face image A dictionary of Among them, the identifier i represents the sequence number of the high-resolution face sample image in the high-resolution training set and the sequence number of the low-resolution face sample image in the low-resolution training set, and the identifier j represents the block position sequence number on the image, For the number of low-resolution face sample images in the low-resolution training set and the number of high-resolution face sample images in the high-resolution training set, M is the number of blocks divided into image blocks for each image;

Step 2, for each face image block in the input low-resolution face image, take the image block at the corresponding position of each low-resolution face sample image in the low-resolution training set as a sample point, and establish a low-resolution face sample block space, take the image block at the corresponding position of each high-resolution face sample image in the high-resolution training set as a sample point, and establish a high-resolution face sample block space;

In an embodiment, for a certain position image block in the input low-resolution face image The corresponding low-resolution image block dictionary and high-resolution image block dictionary can be obtained, so as to establish a low-resolution face sample block space and high-resolution face sample block space ${\stackrel{\&OverBar;}{Y}}_j={\left\{\stackrel{\&OverBar;}{{\mathrm{the\; y}}_i}\right\}}_{i=1}^{\stackrel{\&OverBar;}{N}};$

In step 3, for each image block in the input low-resolution face image Use the low-resolution face sample block space obtained in step 2 respectively Perform Cauchy regularized sparse representation to obtain the optimal weight coefficient α for linear reconstruction;

The linear reconstruction of Cauchy regularization refers to:

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; ln</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\frac{{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

Among them, J(α) returns the value of α when the function of variable α gets the minimum value, α is dimension vector, by A linear reconstruction coefficient α _i is formed, referred to as a coefficient vector. σ _α is the standard deviation of the coefficient α, is the Cauchy regular term, Represents the square of the result of the two-norm ||·|| ₂ , λ is a regularization parameter to balance the reconstruction error and sparsity, and λ is recommended to take a value of 1e-2.

The method of Cauchy regularization is different from the existing sparse representation method, and the solution of sparse representation is equivalent to the following optimization problem:

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; ,</span>$

where Σ _i |α _i | is a sparse regular term.

Step 4, using the optimal weight coefficient α obtained in step 3 and the high-resolution face sample block space obtained in step 2 A new high-resolution image patch can be reconstructed by

${\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; Y</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; .</span>$

can pass linear reconstruction coefficients Reconstruction results in high-resolution image patches.

Step 5, all weighted reconstructed high-resolution image blocks Superimpose according to the position, and then divide by the number of overlaps of each pixel position to reconstruct a high-resolution face image.

In order to verify the effectiveness of the present invention, the FEI face database [4] is used for experiments, and 400 frontal, pre-aligned face images of all 200 individuals are selected. The original high-resolution face images are 120×100 pixels. The low-resolution face image is obtained by 4 times Bicubic downsampling of the high-resolution face image. 360 images are randomly selected as training samples, and the remaining 40 images are used as test images. We compare the reconstruction effect obtained by the present invention with some methods based on block positions, such as nearest neighbor embedding method (NE, literature 5), least square method (LSR, literature 6), sparse representation (SR, literature 2) And non-local centralized sparse representation (NCSR, literature 3) and so on.

The experiment uses Peak Signal to Noise Ratio (PSNR) to measure the pros and cons of the comparison algorithm, and SSIM is an index to measure the similarity between two images. The closer the value is to 1, the better the image reconstruction effect is. Compare the average PSNR and SSIM values obtained by processing all 40 test images by the above methods, see Table 1 for details.

As can be seen from Table 1, the PSNR values of the comparison method and the method of the present invention are respectively 31.75, 31.90, 32.11, 31.30 and 32.51, and the SSIM values are respectively 0.894, 0.903, 0.905, 0.906 and 0.910, that is, the comparison method of the present invention The PSNR value and SSIM value of the best algorithm in the method are improved by 0.4dB and 0.004, respectively. It can be seen that, compared with other existing methods, the effect of the method of the present invention has been significantly improved.

Table 1 PSNR value and SSIM value of the inventive method and existing method

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (1)

1. a face super-resolution method based on Cauchy's regularization, it is characterised in that comprise the steps:

Step 1, to the low resolution face sample image in the low-resolution face image inputted, low resolution training set and height High-resolution human face sample image in resolution training set divides overlapped image block；

Step 2, for each image block in the low-resolution face image of input, takes each low resolution in low resolution training set The image block of face sample image relevant position, as sample point, obtains corresponding low-resolution image block dictionary, sets up low resolution Face sample block space, takes in high-resolution training set the image block of each high-resolution human face sample image relevant position as sample This point, obtains corresponding high-definition picture block dictionary, sets up high-resolution human face sample block space；Realize as follows,

If low-resolution face image X to input^tThe image block collection constituted after being divided into M overlapped image block isHigh-resolution and low-resolution facial image training set is respectively divided overlapped image block, then respectively obtains The high-definition picture block dictionary of individual low-resolution face image M the image block correspondence position with input of MWith low point Resolution image block dictionaryWherein, mark i represents the sequence of high-resolution training set middle high-resolution face sample image Number and low resolution training set in the sequence number of low resolution face sample image, mark j represents the block position number on image,For The number of low resolution face sample image and high-resolution training set middle high-resolution face sample image in low resolution training set Number, M be each image divide image block block number；

Step 3, to some image block in input low-resolution face image, uses step 2 gained low resolution face sample respectively This block spaceCarry out the rarefaction representation of Cauchy's regularization, obtain the optimal weights factor alpha of linear reconstruction, it is achieved as Under,

$J\left(\mathrm{\&alpha;}\right)={\left|\right|x_j^t-{\stackrel{\&OverBar;}{X}}_j\mathrm{\&alpha;}\left|\right|}_2^2+{\mathrm{\&lambda;\&Sigma;}}_i\ln (1+\frac{{\mathrm{\&alpha;}}_i^2}{{\mathrm{\&sigma;}}_{\mathrm{\&alpha;}}^2}),$

Wherein, J (α) returns the function value of α when obtaining minima about variable α, and α isDimensional vector, byIndividual line Property reconstruction coefficients α_iConstitute, σ_αThe standard variance being,It is Cauchy's regular terms,Represent two norms | | | |₂'s Result is squared, and λ is Equilibrium fitting error and openness regularization parameter；

Step 4, utilizes optimal weights coefficient and step 2 gained high-resolution human face sample block space that step 3 obtains, linearly Reconstruct obtains new high-definition picture block；

Step 5, the high-resolution human face image block that all weightings are reconstructed according to the position superposition on face, then divided by The number of times that each location of pixels is overlapping, obtains a high-resolution human face image.