CN108090873A - Pyramid face image super-resolution reconstruction method based on regression model - Google Patents

Abstract

The method for super-resolution reconstruction of pyramidal face images based on a regression model in the present invention involves image enhancement or restoration, using the feature of non-local similarity of images to search and reconstruct low-resolution face images in the test set in their corresponding feature images The similar blocks of the image block, get the position set of all similar blocks, and use the face image blocks of all the low-resolution images in the training set in the position set as the low-resolution training set corresponding to the low-resolution face image blocks in the test set , using the distance between the feature image blocks corresponding to the low-resolution face image blocks in the test set and the feature image blocks corresponding to the low-resolution face image blocks in the training set and the interpolated and enlarged low-resolution images in the test set The sum of the distances between the feature image block corresponding to the face image block and the feature image block corresponding to the high-resolution face image block in the training set constructs a constraint condition; it overcomes many defects in the process of face image reconstruction in the prior art.

Description

Pyramid Face Image Super-resolution Reconstruction Method Based on Regression Model

technical field

The technical solution of the present invention relates to image enhancement or restoration, in particular to a method for super-resolution reconstruction of pyramid face images based on a regression model.

Background technique

During the image acquisition process, due to the limitations of the imaging system and the influence of environmental factors, the acquired images often deviate from the real scene. How to improve the spatial resolution of the image and improve the image quality has always been an important problem that the image acquisition technology is dedicated to solving. With the development of science and technology, the performance of the hardware equipment of the imaging system is getting better and better, but the method of improving the image quality by upgrading the hardware system requires a high cost. On the basis that the hardware level has reached a certain height, it becomes a cost-effective method to improve image quality through software technology, and image super-resolution reconstruction (Super Resolution, SR) is an effective method based on this.

Broadly speaking, image super-resolution reconstruction methods are mainly divided into super-resolution reconstruction methods based on multiple images and super-resolution reconstruction methods based on single images. Because the latter has a wide range of applications and good learning effects, it has become one of the focuses of many scholars' research in recent years. For example, Document 1 proposes a multi-layer face super-resolution reconstruction method based on position-restricted neighborhood embedding and intermediate dictionary, which uses the manifold restriction of the local geometric structure of the image block for super-resolution reconstruction, and captures the degradation of the image process, and enhance the consistency between the reconstructed high-resolution face image and the original high-resolution image through the method of constructing an intermediate dictionary, thereby reconstructing a high-quality face image, but directly converting the mapping obtained from the low-resolution image The relationship applied to high-resolution images is not completely suitable for high-resolution images, and the differences between low-resolution images and high-resolution images are likely to cause errors in image reconstruction. CN103824272B discloses a face super-resolution reconstruction method based on K-nearest neighbor re-identification, which uses the geometric information of low-resolution manifold and high-resolution manifold to update the identified neighbor image block, and the re-identified neighbor The image block calculates the weight coefficient, the information provided by the high-resolution image makes up for the lack of information provided by the low-resolution image, and the quality of the reconstructed image has been greatly improved. All the high-resolution image blocks are searched for adjacent blocks again, and then the image block with the most repeated times is selected as the training image block. Two search processes and one comparison process reduce the efficiency of the reconstruction method. Both of the above two face super-resolution reconstruction methods based on neighborhood embedding have the defect that images are prone to blurring due to over-fitting or under-fitting. In order to solve this problem, the sparse prior knowledge is introduced into face super-resolution reconstruction. CN103325104A proposes a method for face image super-resolution reconstruction based on iterative sparse expression. First, the high-resolution face estimation image is used A high-resolution face image dictionary is linearly expressed, and then the high-resolution face estimation result is obtained by using the local linear regression method, and finally iteratively converges to a stable value to obtain the final reconstructed face image. Face image super-resolution is inaccurately aligned to face images. However, the training process of the dictionary and the iterative convergence process will take a long time, resulting in poor real-time performance. Both the above-mentioned super-resolution reconstruction methods based on neighborhood embedding and sparse representation have the disadvantage that the reconstructed face images still cannot meet people's needs for high-quality images. In order to make full use of the similarity features of different face images, Literature 2 proposes a face image super-resolution reconstruction method based on position blocks, assuming that the image blocks at the same position in different face images have the same image structure, directly using training Concentrating the face image blocks at the same position to form a set to reconstruct the face image, however, this method directly uses all the blocks at the same position for reconstruction, ignoring the impact of a few blocks with large differences on the reconstruction, there is a reconstruction The image is prone to the defect of partial blur phenomenon. Document 3 proposes to select image blocks of the same category as the input block in the training set for reconstruction by adding low-rank constraints, but this method has the defect of relying too much on the training set and not utilizing the nature of the input image itself. In Document 4, it is proposed to construct a weight matrix according to the distance between the input block and the same position block in the training set to solve the mapping matrix. However, the method based on the position block is to train the mapping between face image blocks from low-resolution face image blocks. The relationship between the high-resolution face image blocks may affect the reconstruction effect of the face image, and the reconstruction process of the face image cannot reflect the attenuation process of the image, and the reconstructed high-resolution face image has local ghosting phenomenon of defects.

In short, the existing technology of face image super-resolution reconstruction method does not solve the problem that the difference between high-resolution images affects the quality of the reconstructed image, and there is a problem that the reconstruction process of the face image cannot truly reflect the face image. Defects in the degradation process and defects that cause partial ghosting in the reconstructed face image.

The literature sources of the prior art papers involved in the above-mentioned papers are as follows:

Document 1: Jiang, J., Hu, R., Wang, Z., & Han, Z. (2014). Face super-resolution viamultilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Transactions on Image Processing, 23(10 ), 4220-4231.

Document 2: Ma, X., Zhang, J., & Qi, C. (2010). Hallucinating face by position-patch. Pattern Recognition, 43(6), 2224-2236.

Document 3: Gao, G., Jing, X.Y., Huang, P., Zhou, Q., Wu, S., & Yue, D. (2016).Locality-Constrained Double Low-Rank Representation for Effective FaceHallucination.IEEE Access, 4, 8775-8786.

Document 4: Jiang, J., Chen, C., Ma, J., Wang, Z., Wang, Z., & Hu, R. (2017). SRLSP: Aface image super-resolution algorithm using smooth regression with local structure prior .IEEE Transactions on Multimedia, 19(1), 27-40.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for super-resolution reconstruction of pyramidal face images based on a regression model, to extract gradient features from high and low resolution face images in the training set to obtain corresponding gradient feature images, and to train The high- and low-resolution images in the set and their corresponding gradient feature images are divided into overlapping blocks; the gradient features are extracted from the low-resolution face images in the test set to obtain the feature image, and non-local similarity search is used to reconstruct the feature image Obtain similar blocks from the image blocks, and use the low-resolution face image blocks in the same position as the similar blocks in the training set to expand the training set of reconstructed face image blocks; use the feature image blocks corresponding to the low-resolution face image blocks in the test set and The distance between the feature image block corresponding to the enlarged face image block and the feature image block corresponding to the face image block in the training set is constructed by interpolation to make the regression process of reconstruction smoother; The high-resolution face images are divided into blocks of different scales to build a pyramid model to achieve super-resolution reconstruction of faces. The method of the present invention overcomes the problems in the prior art that the difference between the high-resolution images in the training set does not consider the influence of the difference between the high-resolution images in the training set on the quality of the reconstructed image in the prior art, and the face image reconstruction process cannot truly reflect the face image The defects of the degradation process and the defects that cause the reconstructed face image still have partial ghosting phenomenon.

The technical solution adopted by the present invention to solve the technical problem is: a method for super-resolution reconstruction of a pyramid human face image based on a regression model, and the specific steps are as follows:

A. The training process of the low-resolution face image set and the high-resolution face image set in the training set:

The first step is to expand the low-resolution face image set and high-resolution face image set in the training set:

According to the symmetrical characteristics of face images, the low-resolution face image set and high-resolution face image set in the training set are expanded by flipping left and right. The size of the image remains the same, and the number is doubled, respectively. A collection of low-resolution face images and an expanded set of high-resolution face images Among them, l represents a low-resolution image with a size of a*b pixels, h represents a high-resolution image with a size of (d*a)*(d*b) pixels, d is a multiple, and M represents the number of images;

In the second step, gradient features are extracted from the expanded low-resolution face image set P _l and high-resolution face image set P _h respectively:

For each face image in the expanded low-resolution face image set P _l and high-resolution face image set _Ph , extract the first-order gradient and second-order gradient as components to form a gradient feature, and obtain the low-resolution The low-resolution face gradient feature image set in the high-rate face image set P _l and the high-resolution face gradient feature image set in the high-resolution face image set _Ph

The third step is to block the expanded high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h respectively} :

For each face image in the expanded high-resolution face image set _Ph and the corresponding high-resolution face gradient feature image Carry out overlapped blocks respectively, each block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8-12. The overlapping method is to overlap K ₁ row of pixels between the current block and the upper and lower adjacent image blocks. , overlap K ₁ columns of pixels between the left and right adjacent image blocks, and 0≤K ₁ ≤R ₁ /2, and then use the order from top to bottom and from left to right for each high-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered, and the numbers are 1, 2,..., U, and U are the total number of blocks of each image. The image blocks with the same number are called the image blocks at the same position, thus completing the expansion of the high The high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h are} respectively divided into blocks;

The fourth step is to block the expanded low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l respectively:

In the same way as the above-mentioned high-resolution face image set P _h , for each low-resolution face image in the expanded low-resolution face image set P _l and the corresponding low-resolution face gradient feature image Carry out overlapped blocks respectively, the size of each block is (R ₁ /d)*(R ₁ /d) pixels, the value of R ₁ is 8~12, the way of overlapping is between the current image block and the upper and lower adjacent image blocks Overlap K ₁ /d row pixels between the left and right adjacent image blocks, and overlap K ₁ /d column pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right for each low-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered respectively, and the numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called the image blocks of the same position, thus completing the expansion of the The low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l are respectively divided into blocks;

So far, complete the training process of A. training set low-resolution face image set P _l and high-resolution face image set P _h ;

B. Reconstruction process of low-resolution face images in the test set:

The fifth step is to enlarge the low-resolution face images in the test set to obtain enlarged high-resolution face images:

Input the low-resolution face image to be tested into the computer to obtain the low-resolution face image I _tl in the test set, and use bicubic interpolation to enlarge a certain low-resolution face image in the test set to obtain the enlarged The image is used as the enlarged high-resolution face image I _th in the test set, so that the enlarged high-resolution face image I _th in the test set is the same as the high-resolution face image in the training set equal in size;

The sixth step is to extract gradient features from the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set:

Extract the first-order gradient and second-order gradient of the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set obtained in the fifth step above respectively as components to form their respective gradient features, and obtain their corresponding The low-resolution face gradient feature image g _tl and the high-resolution face gradient feature image g _th ;

The seventh step is to block the enlarged high-resolution face image I _th and its corresponding high-resolution face gradient feature image g _th in the test set:

The enlarged high-resolution face image I _th in the test set obtained in the fifth step above and the corresponding high-resolution face gradient feature image g _th in the sixth step above are respectively divided into overlapping blocks, each The block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8 to 12, so that the block size is the same as the block size of the high-resolution face image in the training set, and the overlapping method is that the current image block and the upper and lower adjacent images Overlap K ₁ row of pixels between the blocks, and overlap K ₁ column of pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right to number all the blocks of each face image respectively, The numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called image blocks at the same position;

The eighth step is to block the low-resolution face image I _tl and its corresponding low-resolution face gradient feature image g _tl in the test set:

The low-resolution face image I _tl in the test set obtained in the fifth step above and the corresponding low-resolution face gradient feature image g _tl in the above-mentioned sixth step are divided into overlapping blocks, each block size is (R ₁ /d)*(R ₁ /d), the value of R ₁ is 8 to 12, so that the block size is the same as the block size of the low-resolution face image in the training set, and the overlapping method is that the current image block and Overlap K ₁ /d rows of pixels between the upper and lower adjacent image blocks, and overlap K ₁ /d columns of pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right for each face image All the blocks of are numbered respectively, the numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called image blocks at the same position;

The ninth step is to use the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set to find the number of similar blocks:

According to the order from top to bottom and from left to right, the image blocks of the low-resolution face image I _t1 in the test set obtained in the above eighth step are reconstructed, taking the reconstruction of the jth image block as an example, using The non-local similarity of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set, and find the similar block of the jth image block in the low-resolution face image I _tl in the test set , let the jth face gradient feature image block of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set be g _tl,j , for the low-resolution face gradient feature image All face image blocks in g _tl are scanned from top to bottom and from left to right, the scanned image block is not repeated with the jth image block, and the scanned face gradient feature image block and the jth image block are calculated The Euclidean distance of the face gradient feature image block, and then sort the distances of all low-resolution face gradient feature image blocks in the order of distance from small to large, and take the first n blocks with the smallest distance as the jth low-resolution face block Similar image blocks of face gradient feature image block g _tl,j , set the number set of similar low-resolution face gradient feature image blocks as [v ₁ ,v ₂ ,...,v _n ], the corresponding low The set of resolution face gradient feature image blocks is This completes the process of seeking the numbering of similar blocks using the low-resolution face gradient feature image g _t1 corresponding to the low-resolution face image I _t1 in the test set;

In the tenth step, use the position numbers of the similar blocks to obtain the set of image blocks at the same number for all images in the expanded low-resolution face gradient feature image set G _l in the training set:

For the i, i=1, 2, ..., M face images in the low-resolution face gradient feature image set G _l expanded in the training set in the second step above The numbered set of the face feature image block numbered j and the similar low-resolution face gradient feature image block in the ninth step above is the same image block in [v ₁ ,v ₂ ,...,v _n ] Form a collection Then in the expanded low-resolution face gradient feature image set Gl in the training set, the image block numbered j in all images and the numbered set of similar low _{-resolution face gradient feature image blocks [v 1 ,v 2 , ..} .,v _n ] set of image blocks for:

For convenience of writing, the Recorded as:

Wherein M*(1+n) represents that there are M face images, and each face image has 1+n image blocks;

In the eleventh step, use the position number of the similar block to obtain the set of image blocks at the same number for all images in the expanded high-resolution face gradient feature image set G _h in the training set:

For the i-th, i=1, 2,...,M images in the high-resolution face gradient feature image set G _h expanded in the training set in the second step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then in the expanded high-resolution face gradient feature image set G _h in the training set, it is a set composed of all image numbers j and [v ₁ ,v ₂ ,...,v _n ] image blocks for:

For convenience of writing, the Recorded as:

In the twelfth step, use the position numbers of the similar blocks to find the set of all face images in the image blocks at the same number in the expanded low-resolution face image set _P1 :

For the first i, i= ₁ , 2,..., M pieces of face images in the expanded low-resolution face image set P1 in the above first step A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image numbers in P _l are j and the set composed of image blocks [v ₁ ,v ₂ ,...,v _n ] for:

For convenience of writing, the Recorded as:

The thirteenth step, use the position numbers of similar blocks to find the set of image blocks at the same number of all face images in the expanded high-resolution face image set _Ph :

For the first i, _i =1, 2,..., M face images in the expanded high-resolution face image set Ph in the first step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image blocks in P _h whose number is j and [v ₁ ,v ₂ ,...,v _n ] form a set for:

For convenience of writing, the Recorded as:

In the fourteenth step, calculate the weight matrix corresponding to the jth face image block:

First use the following formula (9) to calculate the jth face image block g _tl,j of the gradient feature image corresponding to the low-resolution face image I _tl in the above-mentioned eighth step test set and the above-mentioned tenth step obtained The Euclidean distance set of all face image blocks in Then use the following formula (10) to calculate the jth block image block _{gth, j} of the high-resolution face gradient feature image _gth corresponding to the high-resolution face image _Ith enlarged in the test set of the above-mentioned seventh step. in eleven steps The set of Euclidean distances of all image blocks in

After obtaining the above distances, the weight matrix W _j of the jth block is obtained by formula (11):

where α is the smoothing factor;

In the fifteenth step, calculate the mapping matrix corresponding to the jth face image block:

The mapping process of obtaining the corresponding jth high-resolution face image block from the jth low-resolution face image block in the training set is recorded as a simple mapping relationship, and the formula is obtained:

in Represents the mapping matrix of the jth face image block, T represents the transposition of the matrix, and the optimal mapping matrix is obtained by the following formula (13):

Since there is not a simple mapping relationship between high-resolution face image blocks and low-resolution face image blocks, the distance matrix obtained in the fourteenth step is used to perform smooth constraints on formula (13) to obtain the following smooth regression formula (14) :

in Where tr(.) is the trace of the matrix. In order to make the mapping process smoother, a regularization term is added to obtain the following formula (15):

in F represents the Frobenius norm, λ is used to weigh the reconstruction error and the sparsity of A _j , and the mapping matrix corresponding to the j-th block image is obtained by simplification:

where E represents the identity matrix;

In the sixteenth step, reconstruct the low-resolution face image blocks in the test set to obtain high-resolution face image blocks:

pass Obtain the high-frequency information of the high-resolution face image block corresponding to the face image block I _tl,j in the low-resolution face image I _tl in the test set, and then interpolate the high-frequency information into I _tl,j to obtain the reconstructed Face image block I _t '_h,j;

In the seventeenth step, combine all reconstructed image blocks into the reconstructed high-resolution face image:

According to the order from top to bottom and from left to right, all the reconstructed face image blocks are combined according to the numbers, and the overlapping parts are averaged during the combination process to obtain the reconstructed high-resolution face image I _t '_h;

The eighteenth step is to build a pyramidal face super-resolution reconstruction model:

(18.1) Use the nearest neighbor interpolation method to carry out dimensionality reduction on the I _t ' _h obtained in the seventeenth step above, and obtain a low-resolution face image I _t ' _l after dimension reduction, so that the face image after dimension reduction is consistent with I _tl is the same size;

(18.2) Reconstruct all low-resolution face images in the training set with the steps from the first step to the seventeenth step above, and reconstruct the i-th low-resolution face image in the training set The process of rebuilding is: As the low-resolution face images in the test set, the training set and As a training set, use the above steps 1 to 17 to reconstruct high-resolution images Then use the nearest neighbor interpolation method to Perform dimensionality reduction, get

(18.3) Take the block size of the high-resolution face image as R ₂ *R ₂ pixels, the value of R ₂ is 6 to 10, and R ₂ ≠ R ₁ , the number of overlapping pixels between high-resolution image blocks is K ₂ , the block size of the low-resolution face image is (R ₂ /d)*(R ₂ /d) pixels, d is the reduction factor and the value of d in the first step is the same, the low-resolution image The number of overlapping pixels between blocks is K ₂ /d, and I _t ' _l obtained in (18.1) is used as the low-resolution face image in the test set, and the obtained in (18.2) and As a training set, perform a face image super-resolution reconstruction process again to obtain the final reconstructed face image;

So far, the reconstruction process of low-resolution face images in the B. test set is completed, and the super-resolution reconstruction of pyramidal face images based on the regression model is finally completed.

The above-mentioned regression model-based pyramidal face image super-resolution reconstruction method, the first step, expands the size of the low-resolution face image set and the high-resolution face image set in the training set to be (d*a)*(d *b) pixel, d is a multiple, and the value of d is 2; the third step, for the expanded high-resolution face image set _Ph and its corresponding high-resolution face gradient feature image set G _h Carry out K ₁ columns of pixels overlapping between the left and right adjacent image blocks in the block respectively, and the value of this K ₁ is 4; The fourth step is to expand the low-resolution face image set P ₁ and its corresponding The low-resolution face gradient feature image set G _l is divided into blocks, each block size is (R ₁ /d)*(R ₁ /d) pixels, and the value of d is 2; Between overlapping K ₁ /d column pixels, the value of the K ₁ is 4; the seventh step, for the enlarged high-resolution face image I _th in the test set and its corresponding high-resolution face gradient feature image The way g _th overlaps in the block is that the current image block overlaps K ₁ row of pixels with the upper and lower adjacent image blocks, and overlaps K ₁ column of pixels with the left and right adjacent image blocks, and the value of K ₁ is 4; The eighth step, the low-resolution face image I _tl and its corresponding low-resolution face gradient feature image g _tl in the test set are divided into blocks. The size of each block is (R ₁ /d)*(R ₁ /d), the value of this d is 2; overlap K ₁ /d columns of pixels between the left and right adjacent image blocks, and the value of this K ₁ is 4; the eighteenth step is to construct a pyramid face super-resolution The number of overlapping pixels between the high-resolution image blocks in (18.3) of the reconstruction model is K ₂ , and the value of K ₂ is 4; the block size of the low-resolution face image is (R ₂ /d)*(R ₂ /d) pixels, d is the reduction factor and has the same value as d in the first step, and the value of d is 2.

The known techniques used in the present invention are: gradient feature, non-local similarity and linear regression.

The beneficial effects of the present invention are: compared with the prior art, the outstanding substantive features and remarkable progress of the present invention are as follows:

(1) The present invention utilizes the feature that the image has non-local similarity, searches for the similar blocks of the reconstructed image block in the low-resolution face image in the test set in its corresponding feature image, obtains the position set of all similar blocks, and trains Concentrate the face image blocks of all low-resolution images in this location set as the low-resolution training set corresponding to the low-resolution face image blocks in the test set, rather than the ones described in the above documents 1, 2, 3 and 4 method, only use a set of all face image blocks at a certain position in the low-resolution face image in the training set, or compare all face image blocks in the low-resolution training set with the low-resolution face image in the test set The distance of the image block, the set of some face image blocks with the closest distance is used as a low-resolution training set, and the method of the present invention can more accurately and efficiently obtain the low-resolution training set corresponding to the low-resolution face image block in the test set. set.

(2) The present invention uses the distance between the feature image block corresponding to the low-resolution face image block in the test set and the feature image block corresponding to the low-resolution face image block in the training set and the low-resolution image block in the test set The feature image block corresponding to the face image block after interpolation amplification, and the sum of the distances between the feature image blocks corresponding to the high-resolution face image block in the training set construct constraint conditions, compared with the method described in prior art CN103824272B , the present invention directly combines the distance between low-resolution face image blocks and the distance between high-resolution face image blocks in the test set, and only needs to search for similar blocks once in a feature image to obtain the positions of similar blocks , instead of sorting the distance between the low-resolution face image blocks in the test set, and the distance between the interpolated and enlarged high-resolution face image blocks in the test set and the high-resolution face image blocks in the training set The distance is sorted, and there is no need to search all the low-resolution face image blocks and high-resolution face image blocks in the training set when calculating the distance. It has higher search efficiency while ensuring accurate constraints. substantive features.

(3) The present invention builds the pyramid model of face image reconstruction according to the different block sizes, which ensures that the reconstruction process of the face image covers a plurality of different scales, thereby effectively merging the features of different scales of face images, image details The restoration is clearer, and the pyramid model overcomes the problem that the existing face super-resolution reconstruction method cannot truly reflect the image degradation process, ensures that the reconstructed face image is closer to the real face image, and also overcomes the existing technology. The problem that the difference between high-resolution images affects the quality of the reconstructed image is not considered in the reconstruction of the face image, and the defect that the reconstruction process of the face image cannot truly reflect the degradation process of the face image.

(4) According to the left-right symmetrical feature of the face image, the present invention expands the data set by flipping left and right, obtains a training set with richer information, and ensures that there are enough similar image blocks to reconstruct in the case of small samples The purpose of the input block.

(5) The present invention utilizes the characteristics of non-local similarity of images, and constructs a set of image blocks at the same number of input blocks and similar blocks in the training set through the non-local similarity of images, which enriches the face image superposition based on position blocks. The set of blocks at the same position in the resolution reconstruction method ensures the effect of face image reconstruction.

(6) The present invention constructs weight matrix by the sum of the distance between the low-resolution image block in the test set and the low-resolution image block in the training set and the distance between the interpolation enlarged image block in the test set and the high-resolution image block in the training set, It can use low-resolution image information and high-resolution image information at the same time, avoiding the shortcomings of inaccurate reconstruction of images when the difference between low-resolution images is large, and through weight constraints, the process of reconstructing images is smoother and image details are restored more easily. precise.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Fig. 1 is a schematic flow diagram of the method of the present invention.

Fig. 2 is a schematic diagram of the block process of a high-resolution face image in the method of the present invention.

Fig. 3 is a schematic diagram of the interpolation process in the method of the present invention.

Figure 4 is an example of samples in the FERET database and CAS-PEAL-R1 database, the first row is a sample example in the FERET database, and the second row is a sample diagram of the CAS-PEAL-R1 database.

Fig. 5 is an effect diagram of image reconstruction using six different methods of Bicubic, ANR, A+, LINE, SRLSP, and the method of the present invention in the FERET database.

Fig. 6 is an effect diagram of image reconstruction using six different methods of Bicubic, ANR, A+, LINE, SRLSP, and the method of the present invention in the CAS-PEAL-R1 database.

Detailed ways

The embodiment shown in Fig. 1 shows that the flow process of the inventive method is: That is, A. The training process of the low-resolution face image set and the high-resolution face image set in the training set: expanding the low-resolution face image set and the high-resolution face image set in the training set → for the expanded low-resolution face image set Gradient features are extracted from the face image set P _l and the high-resolution face image set P _h respectively → for the expanded high-resolution face image set P _h and its corresponding high-resolution face gradient feature image set G _h Divide into blocks → respectively divide the expanded low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l into blocks → That is, B. The reconstruction process of low-resolution face images in the test set: Enlarge the low-resolution face images in the test set to obtain enlarged high-resolution face images → pair the low-resolution face images I _tl and the enlarged high-resolution face images in the test set Extract the gradient features from the resolution face image I _th respectively → divide the enlarged high-resolution face image I _th in the test set and its corresponding high-resolution face gradient feature image g _th into blocks → divide the low-resolution face image in the test set The rate face image I _tl and its corresponding low-resolution face gradient feature image g _tl are divided into blocks → use the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set to find similarity The number of the block → use the position number of the similar block to find the set of image blocks at the same number of all images in the expanded low-resolution face gradient feature image set G _l in the training set → use the position number of the similar block to find the training set Concentrate all the images in the expanded high-resolution face gradient feature image set G _h as a collection of image blocks at the same number → use the position numbers of similar blocks to find all the expanded low-resolution face image sets P _l A set of image blocks at the same number of face images → use the position numbers of similar blocks to find the set of image blocks at the same number of all face images in the expanded high-resolution face image set P _h → calculate the first The weight matrix corresponding to j face image blocks -> calculate the mapping matrix corresponding to the jth face image block → reconstruct the low-resolution face image blocks in the test set to obtain high-resolution face image blocks → combine all reconstructed image blocks Obtain a reconstructed high-resolution face image → construct a pyramidal face super-resolution reconstruction model.

The embodiment shown in Fig. 2 shows, among the figure, R ₁ is block size, and K ₁ is block overlap pixel, and the block process of high-resolution face image in the inventive method is: the block of high-resolution face image block The size is R ₁ *R ₁ , and the current image block overlaps K _{1 rows of pixels with the upper and lower adjacent image blocks, and overlaps K 1 columns} of pixels with the left and right adjacent image blocks. The block process for low-resolution face images is similar.

The embodiment shown in Fig. 3 shows that in the figure, LR represents low resolution, HR represents high resolution, and each black dot represents a low-resolution pixel in the low-resolution face image block, and each white dot Represents a high-resolution pixel in the high-frequency information obtained by reconstruction; the interpolation process in the method of the present invention is: input LR image → LR image block → add HR information → output HR image block, the high-frequency information to be obtained is according to The order from top to bottom and from left to right is sequentially interpolated into low-resolution face image blocks to obtain reconstructed high-resolution face image blocks.

The embodiment shown in FIG. 4 shows that the sample examples in the FERET database and the CAS-PEAL-R1 database, the first row is a sample example in the FERET database, and the second row is a sample example in the CAS-PEAL-R1 database. The FERET database contains 200 people. In this embodiment, 80 males are randomly selected from the FERET database with one frontal face image per person and 70 females with one frontal face image per person to form a training set. One frontal face image for each of 28 men and one frontal face image for each of 22 women are used for testing; the CAS-PEAL-R1 database contains 1040 individuals, and this implementation example is randomly selected on the CAS-PEAL-R1 database. A frontal face image for each of 103 men and a frontal face image for each of 97 women were selected to form the training set. In addition, a frontal face image for each of 57 men and one frontal face image for each of 43 women were randomly selected. A frontal face image is used for testing.

The embodiment shown in Fig. 5 shows, in the FERET database, apply Bicubic, ANR, A+, LINE, SRLSP, the effect figure that the six different methods of the present invention method are reconstructed to the image, each row represents the same face image in the FERET database , each line from top to bottom is the selected 5 face images. For each row, from left to right, it represents the high-resolution face image and the original high-resolution face image reconstructed using Bicubic, ANR, A+, LINE, SRLSP, and the method of the present invention, where the Bicubic method is used as a basic comparison, It can be seen that the face image obtained by the Bicubic method is the most blurred. The details of the eyes and mouth are restored by the ANR and A+ methods. Although the details of the LINE and SRLSP methods are better restored, the partial ghosting of the image is more serious. The method of the invention overcomes the ghosting phenomenon of the image while ensuring the restoration of details, and obtains the optimal reconstructed face image.

The embodiment shown in Fig. 6 shows, in CAS-PEAL-R1 database, apply Bicubic, ANR, A+, LINE, SRLSP, the effect figure that six different methods of the method of the present invention carry out image reconstruction, and each row represents CAS-PEAL-R1 For the same face image in the database, each row from top to bottom is the selected 5 face images. For each row, from left to right, it represents the high-resolution face image and the original high-resolution face image reconstructed using Bicubic, ANR, A+, LINE, SRLSP, and the method of the present invention, where the Bicubic method is used as a basic comparison, It can be seen that the face image obtained by the Bicubic method is the most blurred. The details of the eyes and mouth are restored by the ANR and A+ methods. Although the details of the LINE and SRLSP methods are better restored, there is a partial ghosting phenomenon, and some images are still blurred. There is jagged edges. The method of the invention not only recovers the details of the image most clearly, but also overcomes the local ghosting phenomenon and the edge sawtooth phenomenon existing in other methods, and obtains the optimal reconstructed face image.

Example 1

The method for super-resolution reconstruction of pyramid face image based on regression model of the present embodiment, concrete steps are as follows:

A. The training process of the low-resolution face image set and the high-resolution face image set in the training set:

The first step is to expand the low-resolution face image set and high-resolution face image set in the training set:

According to the symmetrical characteristics of face images, the low-resolution face image set and high-resolution face image set in the training set are expanded by flipping left and right. The size of the image remains the same, and the number is doubled, respectively. A collection of low-resolution face images and an expanded set of high-resolution face images Where l represents a low-resolution image with a size of a*b pixels, h represents a high-resolution image with a size of (d*a)*(d*b) pixels, d is a multiple, the value of d is 2, and M represents an image quantity;

In the second step, gradient features are extracted from the expanded low-resolution face image set P _l and high-resolution face image set P _h respectively:

For each face image in the expanded low-resolution face image set P _l and high-resolution face image set _Ph , extract the first-order gradient and second-order gradient as components to form a gradient feature, and obtain the low-resolution The low-resolution face gradient feature image set in the high-rate face image set P _l and the high-resolution face gradient feature image set in the high-resolution face image set _Ph

The third step is to block the expanded high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h respectively} :

For each face image in the expanded high-resolution face image set _Ph and the corresponding high-resolution face gradient feature image Carry out overlapped blocks respectively, each block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8. The overlapping method is to overlap K ₁ row of pixels between the current block and the upper and lower adjacent image blocks, and Overlap K ₁ columns of pixels between the left and right adjacent image blocks, the value of K ₁ is 4, and then use the order from top to bottom and from left to right for each high-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered, and the numbers are 1, 2,..., U, and U are the total number of blocks of each image. The image blocks with the same number are called the image blocks at the same position, thus completing the expansion of the high The high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h are} respectively divided into blocks;

The fourth step is to block the expanded low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l respectively:

In the same way as the above-mentioned high-resolution face image set P _h , for each low-resolution face image in the expanded low-resolution face image set P _l and the corresponding low-resolution face gradient feature image Carry out overlapped blocks separately, each block size is (R ₁ /d)*(R ₁ /d) pixels, the value of R ₁ is 8, and the value of d is 2. The way of overlapping is that the current image block is the same as the upper and lower Overlap K ₁ /d rows of pixels between adjacent image blocks, and overlap K ₁ /d columns of pixels between left and right adjacent image blocks, the value of K ₁ is 4, and then use the order from top to bottom and from left to right to Each low-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered respectively, and the numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called the image blocks of the same position, thus completing the expansion of the The low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l are respectively divided into blocks;

So far, complete the training process of A. training set low-resolution face image set P _l and high-resolution face image set P _h ;

B. Reconstruction process of low-resolution face images in the test set:

The fifth step is to enlarge the low-resolution face images in the test set to obtain enlarged high-resolution face images:

Input the low-resolution face image to be tested into the computer to obtain the low-resolution face image I _tl in the test set, and use bicubic interpolation to enlarge a certain low-resolution face image in the test set to obtain the enlarged The image is used as the enlarged high-resolution face image I _th in the test set, so that the enlarged high-resolution face image I _th in the test set is the same as the high-resolution face image in the training set equal in size;

The sixth step is to extract gradient features from the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set:

Extract the first-order gradient and second-order gradient of the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set obtained in the fifth step above respectively as components to form their respective gradient features, and obtain their corresponding The low-resolution face gradient feature image g _tl and the high-resolution face gradient feature image g _th ;

The seventh step is to block the enlarged high-resolution face image I _th and its corresponding high-resolution face gradient feature image g _th in the test set:

The enlarged high-resolution face image I _th in the test set obtained in the fifth step above and the corresponding high-resolution face gradient feature image g _th in the sixth step above are respectively divided into overlapping blocks, each The block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8, so that the block size is the same as the block size of the high-resolution face image in the training set, and the overlapping method is between the current image block and the upper and lower adjacent image blocks. overlap K ₁ row of pixels between the left and right adjacent image blocks, and overlap K ₁ column pixels between the left and right adjacent image blocks. The value of K ₁ is 4. The blocks are numbered respectively, and the numbers are 1, 2,..., U, U is the total number of blocks for each image, and the image blocks with the same number are called image blocks at the same position;

The eighth step is to block the low-resolution face image I _tl and its corresponding low-resolution face gradient feature image g _tl in the test set:

The low-resolution face image I _tl in the test set obtained in the fifth step above and the corresponding low-resolution face gradient feature image g _tl in the above-mentioned sixth step are divided into overlapping blocks, each block size is (R ₁ /d)*(R ₁ /d), the value of R ₁ is 8, and the value of d is 2, so that the block size is the same as the block size of the low-resolution face image in the training set, and the overlapping method is The current image block overlaps K ₁ /d row pixels with the upper and lower adjacent image blocks, and overlaps K ₁ /d column pixels with the left and right adjacent image blocks. The value of K ₁ is 4. Number all blocks of each face image in order from left to right, the numbers are 1, 2,..., U, U is the total number of blocks in each image, and the image blocks with the same number are called the same position the image block;

The ninth step is to use the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set to find the number of similar blocks:

According to the order from top to bottom and from left to right, the image blocks of the low-resolution face image I _t1 in the test set obtained in the above eighth step are reconstructed, taking the reconstruction of the jth image block as an example, using The non-local similarity of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set, and find the similar block of the jth image block in the low-resolution face image I _tl in the test set , let the jth face gradient feature image block of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set be g _tl,j , for the low-resolution face gradient feature image All face image blocks in g _tl are scanned from top to bottom and from left to right, the scanned image block is not repeated with the jth image block, and the scanned face gradient feature image block and the jth image block are calculated The Euclidean distance of the face gradient feature image block, and then sort the distances of all low-resolution face gradient feature image blocks in the order of distance from small to large, and take the first n blocks with the smallest distance as the jth low-resolution face block Similar image blocks of face gradient feature image block g _tl,j , set the number set of similar low-resolution face gradient feature image blocks as [v ₁ ,v ₂ ,...,v _n ], the corresponding low The set of resolution face gradient feature image blocks is This completes the process of seeking the numbering of similar blocks using the low-resolution face gradient feature image g _t1 corresponding to the low-resolution face image I _t1 in the test set;

In the tenth step, use the position numbers of the similar blocks to obtain the set of image blocks at the same number for all images in the expanded low-resolution face gradient feature image set G _l in the training set:

For the i, i=1, 2, ..., M face images in the low-resolution face gradient feature image set G _l expanded in the training set in the second step above The numbered set of the face feature image block numbered j and the similar low-resolution face gradient feature image block in the ninth step above is the same image block in [v ₁ ,v ₂ ,...,v _n ] Form a set Then in the expanded low-resolution face gradient feature image set Gl in the training set, the image block numbered j in all images and the numbered set of similar low _{-resolution face gradient feature image blocks [v 1 ,v 2 , ..} .,v _n ] set of image blocks for:

For convenience of writing, the Recorded as:

Wherein M*(1+n) represents that there are M face images, and each face image has 1+n image blocks;

In the eleventh step, use the position number of the similar block to obtain the set of image blocks at the same number for all images in the expanded high-resolution face gradient feature image set G _h in the training set:

For the i-th, i=1, 2,...,M images in the high-resolution face gradient feature image set G _h expanded in the training set in the second step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then in the expanded high-resolution face gradient feature image set G _h in the training set, it is a set composed of all image numbers j and [v ₁ ,v ₂ ,...,v _n ] image blocks for:

For convenience of writing, the Recorded as:

In the twelfth step, use the position numbers of the similar blocks to find the set of all face images in the image blocks at the same number in the expanded low-resolution face image set _P1 :

For the first i, i= ₁ , 2,..., M pieces of face images in the expanded low-resolution face image set P1 in the above first step A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image numbers in P _l are j and the set composed of image blocks [v ₁ ,v ₂ ,...,v _n ] for:

For convenience of writing, the Recorded as:

The thirteenth step, use the position numbers of similar blocks to find the set of image blocks at the same number of all face images in the expanded high-resolution face image set _Ph :

For the first i, _i =1, 2,..., M face images in the expanded high-resolution face image set Ph in the first step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image blocks in P _h whose number is j and [v ₁ ,v ₂ ,...,v _n ] form a set for:

For convenience of writing, the Recorded as:

In the fourteenth step, calculate the weight matrix corresponding to the jth face image block:

First use the following formula (9) to calculate the jth face image block g tl of the gradient feature image corresponding to the low-resolution face image I _tl in the above-mentioned eighth step test set _{, j} and the above-mentioned tenth step obtain The Euclidean distance of all face image blocks in Then use the following formula (10) to calculate the jth block image block _{gth, j} of the high-resolution face gradient feature image _gth corresponding to the high-resolution face image _Ith enlarged in the test set of the above-mentioned seventh step. in eleven steps The set of Euclidean distances of all image blocks in

After obtaining the above distances, the weight matrix W _j of the jth block is obtained by formula (11):

where α is the smoothing factor;

In the fifteenth step, calculate the mapping matrix corresponding to the jth face image block:

The mapping process of obtaining the corresponding jth high-resolution face image block from the jth low-resolution face image block in the training set is recorded as a simple mapping relationship, and the formula is obtained:

in Represents the mapping matrix of the jth face image block, T represents the transposition of the matrix, and the optimal mapping matrix is obtained by the following formula (13):

Since there is not a simple mapping relationship between high-resolution face image blocks and low-resolution face image blocks, the distance matrix obtained in the fourteenth step is used to perform smooth constraints on formula (13) to obtain the following smooth regression formula (14) :

in Where tr(.) is the trace of the matrix. In order to make the mapping process smoother, a regularization term is added to obtain the following formula (15):

in F represents the Frobenius norm, λ is used to weigh the reconstruction error and the sparsity of A _j , and the mapping matrix corresponding to the j-th block image is obtained by simplification:

where E represents the identity matrix;

In the sixteenth step, reconstruct the low-resolution face image blocks in the test set to obtain high-resolution face image blocks:

pass Obtain the high-frequency information of the high-resolution face image block corresponding to the face image block I _tl,j in the low-resolution face image I _tl in the test set, and then interpolate the high-frequency information into I _tl,j to obtain the reconstructed face image block I'_th,j;

In the seventeenth step, combine all reconstructed image blocks into the reconstructed high-resolution face image:

According to the order from top to bottom and from left to right, all the reconstructed face image blocks are combined according to the numbers, and the overlapping parts are averaged during the combination process to obtain the reconstructed high-resolution face image I _t '_h;

The eighteenth step is to build a pyramidal face super-resolution reconstruction model:

(18.1) Use the nearest neighbor interpolation method to carry out dimensionality reduction to the _I'th that above-mentioned seventeenth step obtains, obtain the low-resolution face image I' _tl after dimensionality reduction, make the face image after dimensionality reduction and I _tl the same size;

(18.2) Reconstruct all low-resolution face images in the training set with the steps from the first step to the seventeenth step above, and reconstruct the i-th low-resolution face image in the training set The process of rebuilding is: As the low-resolution face images in the test set, the training set and As a training set, use the high-resolution images reconstructed from the first step to the seventeenth step above Then use the nearest neighbor interpolation method to Perform dimensionality reduction, get

(18.3) Take the block size of the high-resolution face image as R ₂ *R ₂ pixels, the value of R ₂ is 6, the number of overlapping pixels between high-resolution image blocks is K ₂ , and the value of K ₂ is 4. The block size of the low-resolution face image is (R ₂ /d)*(R ₂ /d) pixels, d is the reduction factor and the value of d in the first step is the same, the value is 2, low The number of overlapping pixels between the resolution image blocks is K ₂ /d, and the I _t ' _l obtained in (18.1) is used as the low-resolution face image in the test set, and the obtained in (18.2) and As a training set, perform a face image super-resolution reconstruction process again to obtain the final reconstructed face image;

So far, the reconstruction process of low-resolution face images in the B. test set is completed, and the super-resolution reconstruction of pyramidal face images based on the regression model is finally completed.

Example 2

Except that the value of R 1 in the third step is 10, the value of R ₁ in the fourth step is 10, the value of R ₁ in the seventh step is 10, the value of R ₁ in the eighth step is 10, and the value of R ₁ in the eighth step is 10. _The value of R in (18.3) of eighteen steps is 8, other is with embodiment 1.

Example 3

Except that the value of R 1 in the third step is 12, the value of R ₁ in the fourth step is 12, the value of R ₁ in the seventh step is 12, the value of R ₁ in the eighth step is 12, and the value of R ₁ in the eighth step is 12. _The value of R in (18.3) of 18 steps is outside 10, and other is with embodiment 1.

The known techniques used in the above embodiments include: gradient features, non-local similarity and linear regression.

Claims (2)

1. the super-resolution reconstruction method of pyramid face image based on regression model, it is characterized in that concrete steps are as follows: A. The training process of the low-resolution face image set and the high-resolution face image set in the training set: The first step is to expand the low-resolution face image set and high-resolution face image set in the training set: According to the symmetrical characteristics of face images, the low-resolution face image set and high-resolution face image set in the training set are expanded by flipping left and right. The size of the image remains the same, and the number is doubled, respectively. A collection of low-resolution face images and an expanded set of high-resolution face images Among them, l represents a low-resolution image with a size of a*b pixels, h represents a high-resolution image with a size of (d*a)*(d*b) pixels, d is a multiple, and M represents the number of images; In the second step, gradient features are extracted from the expanded low-resolution face image set P _l and high-resolution face image set P _h respectively: For each face image in the expanded low-resolution face image set P _l and high-resolution face image set _Ph , extract the first-order gradient and second-order gradient as components to form a gradient feature, and obtain the low-resolution The low-resolution face gradient feature image set in the high-rate face image set P _l and the high-resolution face gradient feature image set in the high-resolution face image set _Ph The third step is to block the expanded high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h respectively} : For each face image in the expanded high-resolution face image set _Ph and the corresponding high-resolution face gradient feature image Carry out overlapped blocks respectively, each block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8-12. The overlapping method is to overlap K ₁ row of pixels between the current block and the upper and lower adjacent image blocks. , overlap K ₁ columns of pixels between the left and right adjacent image blocks, and 0≤K ₁ ≤R ₁ /2, and then use the order from top to bottom and from left to right for each high-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered, and the numbers are 1, 2,..., U, and U are the total number of blocks of each image. The image blocks with the same number are called the image blocks at the same position, thus completing the expansion of the high The high-resolution face image set _{Ph and its corresponding high-resolution face gradient feature image set G h are} respectively divided into blocks; The fourth step is to block the expanded low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l respectively: In the same way as the above-mentioned high-resolution face image set P _h , for each low-resolution face image in the expanded low-resolution face image set P _l and the corresponding low-resolution face gradient feature image Carry out overlapped blocks respectively, the size of each block is (R ₁ /d)*(R ₁ /d) pixels, the value of R ₁ is 8~12, the way of overlapping is between the current image block and the upper and lower adjacent image blocks Overlap K ₁ /d row pixels between the left and right adjacent image blocks, and overlap K ₁ /d column pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right for each low-resolution face image and its corresponding gradient feature image All the blocks of each image are numbered respectively, and the numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called the image blocks of the same position, thus completing the expansion of the The low-resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l are respectively divided into blocks; So far, complete the training process of A. training set low-resolution face image set P _l and high-resolution face image set P _h ; B. Reconstruction process of low-resolution face images in the test set: The fifth step is to enlarge the low-resolution face images in the test set to obtain enlarged high-resolution face images: Input the low-resolution face image to be tested into the computer to obtain the low-resolution face image I _tl in the test set, and use bicubic interpolation to enlarge a certain low-resolution face image in the test set to obtain the enlarged The image is used as the enlarged high-resolution face image I _th in the test set, so that the enlarged high-resolution face image I _th in the test set is the same as the high-resolution face image in the training set equal in size; The sixth step is to extract gradient features from the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set: Extract the first-order gradient and second-order gradient of the low-resolution face image I _tl and the enlarged high-resolution face image I _th in the test set obtained in the fifth step above respectively as components to form their respective gradient features, and obtain their corresponding The low-resolution face gradient feature image g _tl and the high-resolution face gradient feature image g _th ; The seventh step is to block the enlarged high-resolution face image I _th and its corresponding high-resolution face gradient feature image g _th in the test set: The enlarged high-resolution face image I _th in the test set obtained in the fifth step above and the corresponding high-resolution face gradient feature image g _th in the sixth step above are respectively divided into overlapping blocks, each The block size is R ₁ * R ₁ pixels, and the value of R ₁ is 8 to 12, so that the block size is the same as the block size of the high-resolution face image in the training set, and the overlapping method is that the current image block and the upper and lower adjacent images Overlap K ₁ row of pixels between the blocks, and overlap K ₁ column of pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right to number all the blocks of each face image respectively, The numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called image blocks at the same position; The eighth step is to block the low-resolution face image I _tl and its corresponding low-resolution face gradient feature image g _tl in the test set: The low-resolution face image I _tl in the test set obtained in the fifth step above and the corresponding low-resolution face gradient feature image g _tl in the above-mentioned sixth step are divided into overlapping blocks, each block size is (R ₁ /d)*(R ₁ /d), the value of R ₁ is 8 to 12, so that the block size is the same as the block size of the low-resolution face image in the training set, and the overlapping method is that the current image block and Overlap K ₁ /d rows of pixels between the upper and lower adjacent image blocks, and overlap K ₁ /d columns of pixels between the left and right adjacent image blocks, and then use the order from top to bottom and from left to right for each face image All the blocks of are numbered respectively, the numbers are 1, 2,..., U, U is the total number of blocks of each image, and the image blocks with the same number are called image blocks at the same position; The ninth step is to use the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set to find the number of similar blocks: According to the order from top to bottom and from left to right, the image blocks of the low-resolution face image I _t1 in the test set obtained in the above eighth step are reconstructed, taking the reconstruction of the jth image block as an example, using The non-local similarity of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set, and find the similar block of the jth image block in the low-resolution face image I _tl in the test set , let the jth face gradient feature image block of the low-resolution face gradient feature image g _tl corresponding to the low-resolution face image I _tl in the test set be g _tl,j , for the low-resolution face gradient feature image All face image blocks in g _tl are scanned from top to bottom and from left to right, the scanned image block is not repeated with the jth image block, and the scanned face gradient feature image block and the jth image block are calculated The Euclidean distance of the face gradient feature image block, and then sort the distances of all low-resolution face gradient feature image blocks in the order of distance from small to large, and take the first n blocks with the smallest distance as the jth low-resolution face block Similar image blocks of face gradient feature image block g _tl,j , set the number set of similar low-resolution face gradient feature image blocks as [v ₁ ,v ₂ ,...,v _n ], the corresponding low The set of resolution face gradient feature image blocks is This completes the process of seeking the numbering of similar blocks using the low-resolution face gradient feature image g _t1 corresponding to the low-resolution face image I _t1 in the test set; In the tenth step, use the position numbers of the similar blocks to obtain the set of image blocks at the same number for all images in the expanded low-resolution face gradient feature image set G _l in the training set: For the i, i=1, 2, ..., M face images in the low-resolution face gradient feature image set G _l expanded in the training set in the second step above The numbered set of the face feature image block numbered j and the similar low-resolution face gradient feature image block in the ninth step above is the same image block in [v ₁ ,v ₂ ,...,v _n ] Form a collection Then in the expanded low-resolution face gradient feature image set Gl in the training set, the image block numbered j in all images and the numbered set of similar low _{-resolution face gradient feature image blocks [v 1 ,v 2 , ..} .,v _n ] set of image blocks for: <mrow><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mn>1</mn></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mn>1</mn></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mi>M</mi></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mi>M</mi></msubsup><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow><mo>,</mo></mrow> For convenience of writing, the Recorded as: <mrow><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow><mo>,</mo></mrow> Wherein M*(1+n) represents that there are M face images, and each face image has 1+n image blocks; In the eleventh step, use the position number of the similar block to obtain the set of image blocks at the same number for all images in the expanded high-resolution face gradient feature image set G _h in the training set: For the i-th, i=1, 2,...,M images in the high-resolution face gradient feature image set G _h expanded in the training set in the second step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then in the expanded high-resolution face gradient feature image set G _h in the training set, it is a set composed of all image numbers j and [v ₁ ,v ₂ ,...,v _n ] image blocks for: <mrow><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mn>1</mn></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mn>1</mn></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mi>M</mi></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>g</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mi>M</mi></msubsup><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow><mo>,</mo></mrow> For convenience of writing, the Recorded as: <mrow><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>4</mn><mo>)</mo></mrow><mo>,</mo></mrow> In the twelfth step, use the position numbers of the similar blocks to find the set of all face images in the image blocks at the same number in the expanded low-resolution face image set _P1 : For the first i, i= ₁ , 2,..., M pieces of face images in the expanded low-resolution face image set P1 in the above first step A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image numbers in P _l are j and the set composed of image blocks [v ₁ ,v ₂ ,...,v _n ] for: <mrow><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mn>1</mn></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mn>1</mn><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mn>1</mn></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mi>M</mi></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mi>l</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mi>M</mi></msubsup><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>5</mn><mo>)</mo></mrow><mo>,</mo></mrow> For convenience of writing, the Recorded as: <mrow><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msub><mi>p</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>p</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>p</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>6</mn><mo>)</mo></mrow><mo>,</mo></mrow> The thirteenth step, use the position numbers of similar blocks to find the set of image blocks at the same number of all face images in the expanded high-resolution face image set _Ph : For the first i, _i =1, 2,..., M face images in the expanded high-resolution face image set Ph in the first step above A set of image blocks whose numbers are j and similar low-resolution face gradient feature image blocks in the ninth step above are [v ₁ ,v ₂ ,...,v _n ] Then all the image blocks in P _h whose number is j and [v ₁ ,v ₂ ,...,v _n ] form a set for: <mrow><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mn>1</mn></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mn>1</mn></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mn>1</mn></msubsup><mo>,</mo><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>1</mn></msub></mrow><mi>M</mi></msubsup><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mn>2</mn></msub></mrow><mi>M</mi></msubsup><mn>...</mn><mo>,</mo><msubsup><mi>p</mi><mrow><mi>h</mi><mo>,</mo><msub><mi>v</mi><mi>n</mi></msub></mrow><mi>M</mi></msubsup><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>7</mn><mo>)</mo></mrow><mo>,</mo></mrow> For convenience of writing, the Recorded as: <mrow><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><msub><mi>p</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>p</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>p</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>8</mn><mo>)</mo></mrow><mo>,</mo></mrow> In the fourteenth step, calculate the weight matrix corresponding to the jth face image block: First use the following formula (9) to calculate the jth face image block g _tl,j of the gradient feature image corresponding to the low-resolution face image I _tl in the above-mentioned eighth step test set and the above-mentioned tenth step obtained The Euclidean distance set of all face image blocks in Then use the following formula (10) to calculate the jth block image block _{gth, j} of the high-resolution face gradient feature image _gth corresponding to the high-resolution face image _Ith enlarged in the test set of the above-mentioned seventh step. in eleven steps The set of Euclidean distances of all image blocks in <mrow><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mo>_</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>_</mo><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mn>1</mn><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mn>...</mn><mo>,</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>)</mo></mrow><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>9</mn><mo>)</mo></mrow><mo>,</mo></mrow> <mrow><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mo>_</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>_</mo><msub><mi>S</mi><msub><mi>G</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><mo>&amp;lsqb;</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>1</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mn>2</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mn>...</mn><mo>,</mo><mi>d</mi><mi>i</mi><mi>s</mi><mi>t</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>t</mi><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi><mi>M</mi><mo>*</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mi>n</mi><mo>)</mo></mrow></mrow></msub><mo>)</mo></mrow><mo>&amp;rsqb;</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>10</mn><mo>)</mo></mrow><mo>,</mo></mrow> After obtaining the above distances, the weight matrix W _j of the jth block is obtained by formula (11): where α is the smoothing factor; In the fifteenth step, calculate the mapping matrix corresponding to the jth face image block: The mapping process of obtaining the corresponding jth high-resolution face image block from the jth low-resolution face image block in the training set is recorded as a simple mapping relationship, and the formula is obtained: <mrow><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>=</mo><msubsup><mi>A</mi><mi>j</mi><mi>T</mi></msubsup><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>12</mn><mo>)</mo></mrow><mo>,</mo></mrow> in Represents the mapping matrix of the jth face image block, T represents the transposition of the matrix, and the optimal mapping matrix is obtained by the following formula (13): <mrow><msub><mi>A</mi><mi>j</mi></msub><mo>=</mo><munder><mi>min</mi><msub><mi>A</mi><mi>j</mi></msub></munder><mo>|</mo><mo>|</mo><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>-</mo><msubsup><mi>A</mi><mi>j</mi><mi>T</mi></msubsup><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>|</mo><mo>|</mo><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>13</mn><mo>)</mo></mrow><mo>,</mo></mrow> Since there is not a simple mapping relationship between high-resolution face image blocks and low-resolution face image blocks, the distance matrix obtained in the fourteenth step is used to perform smooth constraints on formula (13) to obtain the following smooth regression formula (14) : <mrow><msubsup><mi>A</mi><mi>j</mi><mo>&amp;prime;</mo></msubsup><mo>=</mo><munder><mi>min</mi><msub><mi>A</mi><mi>j</mi></msub></munder><mo>|</mo><mo>|</mo><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>-</mo><msubsup><mi>A</mi><mi>j</mi><mi>T</mi></msubsup><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>|</mo><msub><mo>|</mo><msub><mi>W</mi><mi>j</mi></msub></msub><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>14</mn><mo>)</mo></mrow><mo>,</mo></mrow> in Where tr(.) is the trace of the matrix. In order to make the mapping process smoother, a regularization term is added to obtain the following formula (15): <mrow><msubsup><mi>A</mi><mi>j</mi><mrow><mo>&amp;prime;</mo><mo>&amp;prime;</mo></mrow></msubsup><mo>=</mo><munder><mi>min</mi><msub><mi>A</mi><mi>j</mi></msub></munder><mo>|</mo><mo>|</mo><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>-</mo><msubsup><mi>A</mi><mi>j</mi><mi>T</mi></msubsup><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><mo>|</mo><msub><mo>|</mo><msub><mi>W</mi><mi>j</mi></msub></msub><mo>+</mo><mi>&amp;lambda;</mi><mo>|</mo><mo>|</mo><msub><mi>A</mi><mi>j</mi></msub><mo>|</mo><msubsup><mo>|</mo><mi>F</mi><mn>2</mn></msubsup><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>15</mn><mo>)</mo></mrow><mo>,</mo></mrow> in F represents the Frobenius norm, λ is used to weigh the reconstruction error and the sparsity of A _j , and the mapping matrix corresponding to the j-th block image is obtained by simplification: <mrow><msubsup><mi>A</mi><mi>j</mi><mrow><mo>&amp;prime;</mo><mo>&amp;prime;</mo></mrow></msubsup><mo>=</mo><msup><mrow><mo>(</mo><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><msub><mi>W</mi><mi>j</mi></msub><msubsup><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub><mi>T</mi></msubsup><mo>+</mo><mi>&amp;lambda;</mi><mi>E</mi><mo>)</mo></mrow><mrow><mo>-</mo><mn>1</mn></mrow></msup><msub><mi>S</mi><msub><mi>P</mi><mrow><mi>h</mi><mo>,</mo><mi>j</mi></mrow></msub></msub><msub><mi>W</mi><mi>j</mi></msub><msubsup><mi>S</mi><msub><mi>P</mi><mrow><mi>l</mi><mo>,</mo><mi>j</mi></mrow></msub><mi>T</mi></msubsup><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>16</mn><mo>)</mo></mrow><mo>,</mo></mrow> where E represents the identity matrix; In the sixteenth step, reconstruct the low-resolution face image blocks in the test set to obtain high-resolution face image blocks: pass Obtain the high-frequency information of the high-resolution face image block corresponding to the face image block I _tl,j in the low-resolution face image I _tl in the test set, and then interpolate the high-frequency information into I _tl,j to obtain the reconstructed Face image block I′ _th,j ; In the seventeenth step, combine all reconstructed image blocks into the reconstructed high-resolution face image: According to the order from top to bottom and from left to right, all the face image blocks that are reconstructed are combined according to the numbers, and the overlapping parts are averaged during the combination process to obtain the reconstructed high-resolution face image I′ _th ; The eighteenth step is to build a pyramidal face super-resolution reconstruction model: (18.1) Use the nearest neighbor interpolation method to carry out dimensionality reduction to the I′ _th that above-mentioned seventeenth step obtains, obtain the low-resolution face image I′ _tl after dimensionality reduction, make the face image after dimensionality reduction and I _tl the same size; (18.2) Reconstruct all low-resolution face images in the training set with the steps from the first step to the seventeenth step above, and reconstruct the i-th low-resolution face image in the training set The process of rebuilding is: As the low-resolution face images in the test set, the training set and As a training set, use the above steps 1 to 17 to reconstruct high-resolution images Then use the nearest neighbor interpolation method to Perform dimensionality reduction, get (18.3) Take the block size of the high-resolution face image as R ₂ *R ₂ pixels, the value of R ₂ is 6 to 10, and R ₂ ≠ R ₁ , the number of overlapping pixels between high-resolution image blocks is K ₂ , the block size of the low-resolution face image is (R ₂ /d)*(R ₂ /d) pixels, d is the reduction factor and the value of d in the first step is the same, the low-resolution image The number of overlapping pixels between blocks is K ₂ /d, the I′ _tl obtained in (18.1) is used as the low-resolution face image in the test set, and the I′ tl obtained in (18.2) and As a training set, perform a face image super-resolution reconstruction process again to obtain the final reconstructed face image; So far, the reconstruction process of low-resolution face images in the B. test set is completed, and the super-resolution reconstruction of pyramidal face images based on the regression model is finally completed. 2. according to the described method for super-resolution reconstruction of pyramid human face image based on regression model of claim 1, it is characterized in that: described the first step, expand low-resolution human face image set and high-resolution human face image set in the training The size is (d*a)*(d*b) pixels, d is a multiple, and the value of this d is 2; the third step, for the expanded high-resolution face image set _Ph and its corresponding The high-resolution face gradient feature image set G _h is divided into blocks and overlaps K ₁ columns of pixels between the left and right adjacent image blocks, and the value of this K ₁ is 4; the fourth step, for the expanded low The resolution face image set P _l and its corresponding low-resolution face gradient feature image set G _l are divided into blocks, each block size is (R ₁ /d)*(R ₁ /d) pixels, the d The numerical value of is 2; Overlap between the left and right adjacent image blocks K ₁ /d row of pixels, the numerical value of this K ₁ is 4; The seventh step, to the enlarged high-resolution face image I _th and The corresponding high-resolution face gradient feature image g _th is overlapped in the block division by overlapping K ₁ row of pixels between the current image block and the upper and lower adjacent image blocks, and overlapping K ₁ rows with the left and right adjacent image blocks Column pixel, the numerical value of this K ₁ is 4; Described eighth step, to the low-resolution face image I _tl and the corresponding low-resolution face gradient feature image g _tl in the test set, carry out each block in the block The size is (R ₁ /d)*(R ₁ /d), the value of d is 2; it overlaps K ₁ /d columns of pixels between the left and right adjacent image blocks, and the value of K ₁ is 4; the first Eighteen steps, the number of overlapping pixels between the high-resolution image blocks in (18.3) of the pyramidal face super-resolution reconstruction model is K ₂ , and the value of this K ₂ is 4; the block of the low-resolution face image The size is (R ₂ /d)*(R ₂ /d) pixels, d is the reduction factor and the value of d in the first step is the same, and the value of d is 2.