CN103020935A - Self-adaption online dictionary learning super-resolution method - Google Patents

Abstract

The invention discloses a self-adaption online dictionary learning super-resolution method. The method firstly selects one group of high-resolution training image set and one group of low-resolution training image set,then establishes a relationship on the high-resolution training image set and the low-resolution training image set, and provides more prior information for image reconstructing through high-resolution/low-resolution dictionary to obtain a better reconstructing effect under the condition of high amplification factor, at the same time, the method uses SP (Self Propelled) sparse coding algorithm, overcomes the defect of low reconstructing accuracy due to traditional greedy algorithm and keeps a lower computation complexity. The method has the advantages of overcoming the defects of complex computation and low training speed in the current mainstream dictionary learning algorithms such as MOD (Multimedia On Demand) and K-SVD(k-singular value decomposition) and shortening image reconstructing time in general. Compared with the prior art based on learning image super-resolution, the method of the invention has a higher reconstructing accuracy and a shorter algorithm time.

Description

An image super-resolution method for adaptive online dictionary learning

technical field

The invention relates to image super-resolution technology in the field of signal processing and multimedia data processing, in particular to an image super-resolution method for self-adaptive online dictionary learning.

Background technique

Image super-resolution (SR) refers to the process of restoring single or multiple low-resolution images into high-resolution (HR) images. With the increasing application of images in various industries, people have higher and higher requirements for image resolution, because HR images mean that images have high pixel density, which can provide people with more details, and these details are often in the Image plays a key role in practical applications. At present, image super-resolution technology has been widely used in medical imaging, remote sensing detection, public security and other fields. Even in some image application fields, image resolution has become an important indicator to measure image quality. Therefore, in-depth research on image super-resolution technology has very important practical significance.

The most direct way to obtain high-resolution images is to use high-resolution image sensors, but the production of high-resolution image sensors is often limited by the level of technology and cost, so without changing the sensor, use software to improve the image The resolution has become a popular research direction in the field of computer vision. Image super-resolution techniques can be divided into three main categories: interpolation-based, reconstruction-based, and learning-based methods, among which model-based reconstruction and learning-based methods are the main research directions in recent years. The reconstruction-based method is mainly to construct a mapping model from a low-resolution image to a high-resolution image, and the reconstruction efficiency is high, but it is difficult to find a unified mapping model, and the image super-resolution reconstruction with a high magnification factor The reconstructed image quality will drop sharply. The learning-based method is currently the most popular image super-resolution technology because of its advantages of accurate reconstruction and high robustness.

The idea of learning-based super-resolution algorithm was first proposed by Freeman et al. in 2002. They used Markov network to learn the relationship between high-frequency and low-frequency information. Compared with the previous methods based on interpolation and reconstruction, this A method using sample learning can obtain more high-frequency information, which solves the problem of serious distortion of the reconstructed image under high amplification factors due to insufficient prior information provision ability. However, the shortcomings of this method are also obvious, that is, the selection of training samples is relatively high, and it is extremely sensitive to the noise in the image. Subsequently, Chang et al. proposed an image super-resolution method based on neighborhood embedding. The basic idea of this method is to assume that the corresponding high and low resolution image blocks can form a manifold with the same local geometric structure in their feature space. The algorithm relatively requires fewer samples and has better anti-noise performance. However, the problem of the algorithm is that it is difficult to establish the neighborhood preservation relationship when the high and low resolution image blocks are embedded in the neighborhood. In order to improve this problem, the training sample selection algorithm based on histogram matching and the method based on local residual embedding have appeared. Karl et al. proposed to use Support Vector Regression (SVR) to achieve image super-resolution. SVR uses a smaller training set through automatic selection of samples, but the contrast of the image is relatively reduced. In 2010, under the framework of Compressed Sensing (CS), Yang et al. proposed a learning-based image super-resolution reconstruction method using sparse representation theory. They trained from a set of high-resolution images and their corresponding low-resolution images. The image learns a set of sparse representation bases. This set of bases constitutes an over-complete dictionary. Through linear programming, each image block in the training set can be sparsely represented by the over-complete dictionary. Then, in the process of super-resolution image reconstruction, the original The sparse representation coefficients of the low-resolution image under the over-complete dictionary, and then use this set of sparse representation coefficients to reconstruct the high-resolution image. This algorithm pioneered the use of CS theory for image super-resolution technology, which overcomes the problem of selecting the size of the neighborhood in the neighborhood embedding method, that is, when solving the sparse representation, there is no need to specify the number of bases required for reconstruction. It indicates that the coefficient and the number of bases are obtained by solving the linear programming at the same time. However, the algorithm has defects such as too large training samples, too complicated dictionary pair establishment process, and non-adaptive dictionary selection. In the same year, Pu Jian et al. proposed to use BP sparse coding and K-SVD dictionary update algorithm in dictionary learning, and achieved better super-resolution results than the former; however, this algorithm still has defects such as long training sample time .

Contents of the invention

The technical problem to be solved by the present invention is to provide an image super-resolution method for adaptive online dictionary learning with short training sample time and high image quality.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: an image super-resolution method for adaptive online dictionary learning, comprising the following steps:

① Select a set of high-resolution images and their corresponding set of low-resolution images as training samples;

②Extract the high-frequency features of the training samples, obtain a set of high-resolution sample feature images and its corresponding set of low-resolution sample feature images, and randomly extract Q high-resolution sample feature images from the high-resolution sample feature images. The block composition matrix M is to extract the low-resolution feature image small blocks corresponding to the extracted high-resolution sample feature image small blocks in the low-resolution sample feature image to form a matrix M′, where Q is greater than or equal to 30,000 and less than or equal to 100,000, Divide the matrix M and the matrix M′ into k categories, where k≥2, and obtain the k cluster centers m ₁ , m ₂ , m ₃ ,…,m _k of the matrix M′ and the initial sub-dictionary set {(D _l1 , D _h1 ),(D _l2 ,D _h2 ),(D _l3 ,D _h3 ),…,(D _lk ,D _hk )}, where D _l1 ,D _l2 ,D _l3 ,…,D _lk represent k initial low Resolution sub-dictionary, D _h1 , D _h2 , D _h3 ,..., D _hk represent k initial high-resolution sub-dictionaries;

③Use the online algorithm to optimize the initial sub-dictionary set to obtain the optimal target sub-dictionary set {(D _l1-best ,D _h1-best ),(D _l2-best ,D _h2-best ),(D _l3-best , D _h3-best ),…,(D _lk-best ,D _hk-best )}, where D _l1-best , D _l2-best , D _l3-best ,…, D _lk-best represent k optimal low resolution Rate target sub-dictionary, D _h1-best , D _h2-best , D _h3-best ,..., D _hk-best represent k optimal high-resolution target sub-dictionaries;

④ Input the low-resolution image X that needs to be super-resolution enlarged, extract the high-frequency features of the low-resolution image X, obtain the low-resolution feature image X' corresponding to the low-resolution image X, and then select the low-resolution feature image X 'The qth image block X _q , q≥1, the matrix corresponding to the image block X _q is denoted as x _q , and the matrix x _q and cluster centers m ₁ ,m ₂ ,m ₃ ,…,m _k are calculated respectively Euclidean distance, compare the Euclidean distance between x _q and each cluster center, get the cluster center m _n with the smallest Euclidean distance, 1≤n≤k, choose the initial low resolution with m _n as the cluster center The optimal low-resolution target sub-dictionary corresponding to the sub-dictionary, denoted as D _lq-best ;

⑤ Use the SP algorithm to calculate the sparse representation coefficient α _q of the matrix x _q under D _lq-best , α _q = argmin||x _q -D _lq-best α _q || ₂ + λ||α _q || ₁ , α _q =SP(D _lq-best ,x _q ), the specific steps are as follows:

表示D_lp-best里列号为

的原子组成的矩阵，

表示矩阵x_q在

上的投影， ${\hat{x}}_q=D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{\′}{x}_q,$ 其中 $D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{\′}={\left(D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{*}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}\right)}^{-1}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{*},$ *表示矩阵的转置；⑤-1 Record the column number index of the K column atom with the greatest correlation with the matrix x _q in D _lq-best as

Indicates that the column number in D _lp-best is

A matrix composed of atoms,

Indicates that the matrix x _q is in

projection on ${\hat{x}}_q={\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\′}{x}_q,$ in ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\′}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*},$ * represents the transpose of the matrix;

的初始残差向量为r₀，

初始状态下x'_q中元素最大值所对应的K个序列原子的列号索引记为I₀，

表示D_lp-best里列号为I₀的原子组成的矩阵；⑤-2 Initialization: Define the initial state projection

The initial residual vector is r ₀ ,

In the initial state, the column number index of the K sequence atoms corresponding to the maximum value of the element in x' _q is denoted as I ₀ ,

Represents the matrix composed of atoms whose column number is I ₀ in D _lp-best ;

的残差向量为r_ω，令 $x_q^{\′}=D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_{\mathrm{\ω}}}^{\′}{x}_q,$ $D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_{\mathrm{\ω}}}^{\′}={\left(D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_{\mathrm{\ω}}}^{*}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_{\mathrm{\ω}}}\right)}^{-1}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_{\mathrm{\ω}}}^{*},$ 当前迭代中x′_p中元素最大值所对应的K个序列原子的列号索引为I_ω，结合公式{D_lq-best中与r_ω-1相关性最大的K列原子的列号索引}和

更新r_ω、

和I_ω，r_ω-1表示投影

在第ω-1次迭代中的残差向量，

表示D_lp-best里列号为I_ω的原子组成的矩阵，表示D_lp-best里列号为

的原子组成的矩阵；⑤-3 Let the current iteration number be ω, ω≥1, and project in the current iteration

The residual vector of is r _ω , let $x_q^{\′}={\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\ω}}}^{\′}{x}_q,$ ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\ω}}}^{\′}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\ω}}}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\ω}}}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\ω}}}^{*},$ In the current iteration, the column number index of the K sequence atoms corresponding to the maximum value of the element in x′ _p is I _ω , combined with the formula {the column number index of the K column atom with the greatest correlation with rω _-1 in _Dlq-best } and

Update r _ω ,

and I _ω , r _ω-1 represents the projection

The residual vector at the ω-1th iteration,

Represents the matrix composed of atoms whose column number is I _ω in D _lp-best , Indicates that the column number in D _lp-best is

A matrix composed of atoms;

⑤-4 Compare r _ω and r _ω-1 , if ||r _ω || ₂ >r _ω-1 || ₂ , then the iteration ends, otherwise, add 1 to ω as the current number of iterations and return to step ⑤-3 Perform iterative updates;

⑤-5 After the iteration is completed, use the least squares method to obtain the optimal sparse representation coefficient α _q of x _q in the dictionary D _lq-best ;

⑥ Select the high-resolution target sub-dictionary D _{hq- best corresponding to D lq-best} , and use the sparse representation coefficient α _q and D _hq-best to solve the high-resolution image reconstruction block of x _q corresponding to D _hq-best y _q =D _hq-best α _q ;

⑦According to the method of steps ③～⑥, process all the small image blocks of the low-resolution feature image X' according to the "zigzag" shape, and obtain the initial high-resolution reconstructed image

进行去块处理得到最终高分辨率重构图像Y。⑧ Reconstruction of high-resolution images

Deblocking is performed to obtain the final high-resolution reconstructed image Y.

The concrete steps of described step 3. optimizing the initial sub-dictionary set are as follows:

③-1 Optimize the low-resolution sub-dictionary D _l1 , the specific steps are:

a. Assuming that the total number of optimization iterations is T, in the t-th iteration, input any feature image block f _l in the feature image of the low-resolution sample, 1≤t≤T;

使用子空间追踪法求解

λ表示正则化系数且0<λ<1，其中，argmin表示求解泛函的最小值，||||表示求解范数，得到第t次迭代中特征图像块f_l在目标子字典下的稀疏分解系数Λ_t；b. Fix the target sub-dictionary obtained by the t-1th iteration

Solving using Subspace Pursuit

λ represents the regularization coefficient and 0<λ<1, where argmin represents the minimum value of the solution functional, |||| represents the solution norm, and the feature image block f _l in the target subdictionary in the t-th iteration The sparse decomposition coefficient Λ _t under ;

c. Fixed sparse decomposition coefficient Λ _t , assuming that the low-resolution sub-dictionary D _l1 has p atoms, so that $A_t=[a_1,a_2,\&Center\; Dot;\&CenterDot;\&CenterDot;,a_j,\&CenterDot;\&Center\; Dot;\&Center\; Dot;,a_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{\mathrm{\&Lambda;}}_i{\mathrm{\&Lambda;}}_i^T,$ $B_t=[b_1,b_2,\&Center\; Dot;\&Center\; Dot;\&CenterDot;,b_j,\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,b_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{f}_l{\mathrm{\&Lambda;}}_i^T,$ i=1,2,...t, a _j represents the jth column in A _t , b _j represents the jth column in B _t , a _j and b _j are p×1 matrix, 1≤j≤p;

表示低分辨率子字典D_l1的正在更新的目标子字典，A_jj表示a_j中第j行的值，max表示求解函数的最大值；d. Let the j-th atom of the target sub-dictionary corresponding to the low-resolution sub-dictionary D _l1 be d _j , and optimize d _j , then $d_j=\frac{1}{\max ({\left|\right|u_j\left|\right|}_2,1)}{u}_j,$ $u_j=\frac{1}{A_{\mathrm{jj}}}(b_j-{\mathrm{D.}}_{{l1}_{t-j}}^{\&prime;}{a}_j)+d_j,$

Represents the target sub-dictionary being updated of the low-resolution sub-dictionary D _l1 , A _jj represents the value of the jth row in a _j , and max represents the maximum value of the solution function;

e. Update the p atoms of the low-resolution sub-dictionary D _l1 according to step d, complete the t-th iteration, and obtain the target sub-dictionary of the low-resolution sub-dictionary D _l1 after the t-th iteration

代入公式 $D_{{l1}_t}=\arg \min \frac{1}{t}\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}(\frac{1}{2}{\left|\right|f_l-D_{{l1}_t}^{\&prime;}{\mathrm{\&Lambda;}}_i\left|\right|}_2^2+\mathrm{\&lambda;}{\left|\right|{\mathrm{\&Lambda;}}_i\left|\right|}_1),$ 0<λ<1，得到第t次循环更新的目标子字典

f. Will

Into the formula ${\mathrm{D.}}_{{l1}_t}=\arg \min \frac{1}{t}\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}(\frac{1}{2}{\left|\right|f_l-{\mathrm{D.}}_{{l1}_t}^{\&prime;}{\mathrm{\&Lambda;}}_i\left|\right|}_2^2+\mathrm{\&lambda;}{\left|\right|{\mathrm{\&Lambda;}}_i\left|\right|}_1),$ 0<λ<1, get the target sub-dictionary updated in the t-th cycle

g. By analogy, when the input image is the feature image block f _l , T iterations of the low-resolution sub-dictionary D _l1 are completed to obtain the optimized target sub-dictionary corresponding to the feature image block f _l

h. Input all feature image blocks except f _l in the low-resolution sample feature image in sequence, and perform training according to the same principles as steps a~g, complete the training of the initial low-resolution sub-dictionary D _l1 , and convert the last round The updated target sub-dictionary is used as the optimal target sub-dictionary D _l1-best of the initial low-resolution sub-dictionary D _l1 .

③-2 Use the same principle of optimizing the low-resolution sub-dictionary D _l1 to optimize D _l2 , D _l3 , ..., D _lk , D _h1 , D _h2 , D _h3 , ..., D _hk to obtain D _l2 , D _l3 ,…,D _lk , D _h1 , D _h2 , D _h3 ,…,D _hk correspond to the optimal target sub-dictionary, and finally get the optimal target sub-dictionary set

{(D _l1-best ,D _h1-best ),(D _l2-best ,D _h2-best ),(D _l3-best ,D _h3-best ),…,(D _lk-best ,D _hk-best ) }.

The deblocking processing method in the step ⑧ is to take the mean value of the pixels in the overlapping part after each iteration.

Compared with the prior art, the present invention has the advantage of well overcoming the problems of traditional image super-resolution such as reconstruction quality degradation and long algorithm time under high magnification factors:

1) Compared with the existing interpolation-based and model-based image super-resolution techniques, the method proposed in the present invention overcomes the defect that the former cannot provide or cannot provide sufficient prior information. First, select a set of high-resolution and low-resolution resolution training image set, and then establish a corresponding relationship on the high/low training image set, and provide more prior information for the reconstructed image through the high/low resolution dictionary pair, which can also be obtained under high magnification factors Better reconstruction effect;

2) Compared with the existing learning-based image super-resolution technology, the method proposed by the present invention has higher reconstruction accuracy and shorter algorithm time. The present invention uses the SP sparse coding algorithm, which overcomes the traditional greedy algorithm. The disadvantage of low structural accuracy, while maintaining a low computational complexity; while the online dictionary learning algorithm overcomes the shortcomings of complex calculation and slow training speed in the current mainstream dictionary learning algorithms (such as MOD, K-SVD, etc.), Therefore, the image reconstruction time is further shortened overall.

Description of drawings

The flow chart of Fig. 1 inventive method;

Figure 2(a) Original high-resolution cloud image;

Fig. 2(b) Cloud image reconstructed by bilinear interpolation method;

Fig. 2(c) Cloud atlas image reconstructed by Yang method;

Fig. 2 (d) adopts the cloud atlas image reconstructed by the method of the present invention;

Figure 3(a) Original high-resolution Cat image;

Fig. 3(b) Cat image reconstructed by bilinear interpolation method;

Figure 3(c) Cat image reconstructed by Yang method;

Figure 3(d) Cat image reconstructed by the method of the present invention;

Figure 4(a) Original high-resolution Building image;

Figure 4(b) Building image reconstructed by bilinear interpolation method;

Figure 4(c) Building image reconstructed by Yang method;

Figure 4(d) is the Building image reconstructed by the method of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, an image super-resolution method for adaptive online dictionary learning is characterized in that it comprises the following steps:

① Select a set of high-resolution images and their corresponding set of low-resolution images as training samples;

②Extract the high-frequency features of the training samples, obtain a set of high-resolution sample feature images and its corresponding set of low-resolution sample feature images, and randomly extract Q high-resolution sample feature images from the high-resolution sample feature images. The block composition matrix M is to extract the low-resolution feature image small blocks corresponding to the extracted high-resolution sample feature image small blocks in the low-resolution sample feature image to form a matrix M′, where Q is greater than or equal to 30,000 and less than or equal to 100,000, Divide the matrix M and the matrix M′ into k categories, where k≥2, and obtain the k cluster centers m ₁ , m ₂ , m ₃ ,…,m _k of the matrix M′ and the initial sub-dictionary set {(D _l1 , D _h1 ),(D _l2 ,D _h2 ),(D _l3 ,D _h3 ),…,(D _lk ,D _hk )}, where D _l1 ,D _l2 ,D _l3 ,…,D _lk represent k initial low Resolution sub-dictionary, D _h1 , D _h2 , D _h3 ,..., D _hk represent k initial high-resolution sub-dictionaries;

③Use the online algorithm to optimize the initial sub-dictionary set to obtain the optimal target sub-dictionary set {(D _l1-best ,D _h1-best ),(D _l2-best ,D _h2-best ),(D _l3-best , D _h3-best ),…,(D _lk-best ,D _hk-best )}, where D _l1-best , D _l2-best , D _l3-best ,…, D _lk-best represent k optimal low resolution Rate target sub-dictionary, D _h1-best , D _h2-best , D _h3-best ,..., D _hk-best represent k optimal high-resolution target sub-dictionaries; the specific steps are as follows:

③-1 Optimize the low-resolution sub-dictionary D _l1 , the specific steps are:

a. Assuming that the total number of iterations of optimization is T, in the tth iteration, input any feature image block f _l in the feature image of the low-resolution sample, 1≤t≤T;

使用子空间追踪法求解

λ表示正则化系数且0<λ<1，其中，argmin表示求解泛函的最小值，||||表示求解范数，得到第t次迭代中特征图像块f_l在目标子字典

下的稀疏分解系数Λ_t；b. Fix the target sub-dictionary obtained by the t-1th iteration

Solving using Subspace Pursuit

λ represents the regularization coefficient and 0<λ<1, where argmin represents the minimum value of the solution functional, |||| represents the solution norm, and the feature image block f _l in the target subdictionary in the t-th iteration

The sparse decomposition coefficient Λ _t under ;

c. Fixed sparse decomposition coefficient Λ _t , assuming that the low-resolution sub-dictionary D _l1 has p atoms, so that $A_t=[a_1,a_2,\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,a_j,\&Center\; Dot;\&Center\; Dot;\&CenterDot;,a_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{\mathrm{\&Lambda;}}_i{\mathrm{\&Lambda;}}_i^T,$ $B_t=[b_1,b_2,\&CenterDot;\&CenterDot;\&CenterDot;,b_j,\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,b_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{f}_l{\mathrm{\&Lambda;}}_i^T,$ i=1,2,...t, a _j represents the jth column in A _j , b _j represents the jth column in B _t , both a _j and b _j are p×1 matrices, 1≤j≤p;

表示低分辨率子字典D_l1的正在更新的目标子字典，A_jj表示a_j中第j行的值，max表示求解函数的最大值；d. Let the j-th atom of the target sub-dictionary corresponding to the low-resolution sub-dictionary D _l1 be d _j , and optimize d _j , then $d_j=\frac{1}{\max ({\left|\right|u_j\left|\right|}_2,1)}{u}_j,$ $u_j=\frac{1}{A_{\mathrm{jj}}}(b_j-{\mathrm{D.}}_{{l1}_{t-j}}^{\&prime;}{a}_j)+d_j,$

Represents the target sub-dictionary being updated of the low-resolution sub-dictionary D _l1 , A _jj represents the value of the jth row in a _j , and max represents the maximum value of the solution function;

e. Update the p atoms of the low-resolution sub-dictionary D _l1 according to step d, complete the t-th iteration, and obtain the target sub-dictionary of the low-resolution sub-dictionary D _l1 after the t-th iteration

f. Will Into the formula ${\mathrm{D.}}_{{l1}_t}=\arg \min \frac{1}{t}\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}(\frac{1}{2}{\left|\right|f_l-{\mathrm{D.}}_{{l1}_t}^{\&prime;}{\mathrm{\&Lambda;}}_i\left|\right|}_2^2+\mathrm{\&lambda;}{\left|\right|{\mathrm{\&Lambda;}}_i\left|\right|}_1),$ 0<λ<1, get the target sub-dictionary updated in the t-th cycle

g. By analogy, when the input image is the feature image block f _l , T iterations of the low-resolution sub-dictionary D _l1 are completed to obtain the optimized target sub-dictionary corresponding to the feature image block f _l

h. Input all feature image blocks except f _l in the low-resolution sample feature image in sequence, and perform training according to the same principles as steps a~g, complete the training of the initial low-resolution sub-dictionary D _l1 , and convert the last round The updated target sub-dictionary is used as the optimal target sub-dictionary D _l1-best of the initial low-resolution sub-dictionary D _l1 .

③-2 Use the same principle of optimizing the low-resolution sub-dictionary D _l1 to optimize D _l2 , D _l3 ,...,D _lk , D _h1 , D _h2 ,D _h3 ,...,D _hk to obtain D _l2 , D _l3 ,…,D _lk , D _h1 , D _h2 , D _h3 ,…,D _hk correspond to the optimal target sub-dictionary, and finally get the optimal target sub-dictionary set {(D _l1-best ,D _h1-best ), (D _l2-best ,D _h2-best ),(D _l3-best ,D _h3-best ),…,(D _lk-best ,D _hk-best )};

④ Input the low-resolution image X that needs to be super-resolution enlarged, extract the high-frequency features of the low-resolution image X, obtain the low-resolution feature image X' corresponding to the low-resolution image X, and then select the low-resolution feature image X 'The qth image block X _q , q≥1, the matrix corresponding to the image block X _q is denoted as x _q , and the matrix x _q and cluster centers m ₁ ,m ₂ ,m ₃ ,…,m _k are calculated respectively Euclidean distance, compare the Euclidean distance between x _q and each cluster center, get the cluster center m _n with the smallest Euclidean distance, 1≤n≤k, choose the initial low resolution with m _n as the cluster center The optimal low-resolution target sub-dictionary corresponding to the sub-dictionary, denoted as D _lq-best ;

⑤ Use the SP algorithm to calculate the sparse representation coefficient α _q of the matrix x _q under D _lq-best , α _q = argmin||x _q -D _lq-best α _q || ₂ + λ||α _q || ₁ , α _q =SP(D _lq-best , x _q ), the specific steps are as follows:

表示D_lp-best里列号为的原子组成的矩阵，表示矩阵x_q在

上的投影， ${\hat{x}}_q=D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{\&prime;}{x}_q,$ 其中 $D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{\&prime;}={\left(D_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{*}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}\right)}^{-1}{D}_{\mathrm{lq}-\mathrm{best}-{\tilde{I}}_0}^{*},$ *表示矩阵的转置；⑤-1 Record the column number index of the K column atom with the greatest correlation with the matrix x _q in D _lq-best as

Indicates that the column number in D _lp-best is A matrix composed of atoms, Indicates that the matrix x _q is in

projection on ${\hat{x}}_q={\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\&prime;}{x}_q,$ in ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\&prime;}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*},$ * represents the transpose of the matrix;

的初始残差向量为r₀，

初始状态下x′_q中元素最大值所对应的K个序列原子的列号索引记为I₀，

表示D_lp-best里列号为I₀的原子组成的矩阵；⑤-2 Initialization: Define the initial state projection

The initial residual vector is r ₀ ,

In the initial state, the column number index of K sequence atoms corresponding to the maximum value of elements in x′ _q is denoted as I ₀ ,

Represents the matrix composed of atoms whose column number is I ₀ in D _lp-best ;

和I_ω，r_ω-1表示投影

在第ω-1次迭代中的残差向量，

表示D_lp-best里列号为I_ω的原子组成的矩阵，

表示D_lp-best里列号为的原子组成的矩阵；⑤-3 Let the current iteration number be ω, ω≥1, and project in the current iteration The residual vector of is r _ω , let $x_q^{\&prime;}={\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{\&prime;}{x}_q,$ ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{\&prime;}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{*},$ In the current iteration, the column number index of the K sequence atoms corresponding to the maximum value of the element in x′ _p is I _ω , combined with the formula {the column number index of the K column atom with the greatest correlation with rω _-1 in _Dlq-best } and Update r _ω ,

and I _ω , r _ω-1 represents the projection

The residual vector at the ω-1th iteration,

Represents the matrix composed of atoms whose column number is I _ω in D _lp-best ,

Indicates that the column number in D _lp-best is A matrix composed of atoms;

⑤-4 Compare r _ω and r _ω-1 , if ||r _ω || ₂ >||r _ω-1 || ₂ , then the iteration ends, otherwise, add 1 to ω as the current number of iterations and return to step ⑤- 3 for iterative update;

⑤-5 After the iteration is completed, use the least squares method to obtain the optimal sparse representation coefficient α _q of x _q in the dictionary D _lq-best ;

⑥ Select the high-resolution target sub-dictionary D _{hq- best corresponding to D lq-best} , and use the sparse representation coefficient α _q and D _hq-best to solve the high-resolution image reconstruction block of x _q corresponding to D _hq-best y _q =D _hq-best α _q ;

⑦According to the method of steps ③～⑥, process all the small image blocks of the low-resolution feature image X' according to the "zigzag" shape, and obtain the initial high-resolution reconstructed image

进行去块处理得到最终高分辨率重构图像Y，其中去块处理方法为对每次迭代后重叠部分的像素取均值。⑧ Reconstruction of high-resolution images

Perform deblocking processing to obtain the final high-resolution reconstructed image Y, wherein the deblocking processing method is to average the pixels in the overlapping part after each iteration.

The effectiveness of the method of the present invention is verified by experimental simulation. Experimental simulation is carried out on Matlab7.0 platform. Three high-resolution grayscale images of 512×512 were selected as the original images in the experiment, which are the original high-resolution remote sensing satellite cloud image shown in Figure 2(a), and the original high-resolution Cat image shown in Figure 3(a). image and the original high-resolution Building image shown in Figure 4(a). At first three pieces of original images are obtained corresponding low-resolution images through quadruple down-sampling, fuzzy and noise-added processing, and then the corresponding low-resolution images of three pieces of original images are obtained by bilinear interpolation method, Yang method and the method of the present invention respectively The image is enlarged and reconstructed by four times super-resolution, and the image reconstruction quality and algorithm time are compared to obtain the simulation results. 50,000 small feature image blocks are extracted from the high-resolution sample feature image and the low-resolution sample feature image to form the initial dictionary, that is, Q is equal to 50,000, and 3×3 image blocks are selected for image reconstruction, and the overlapping part is 1 pixel. Among them, the cloud image reconstructed by bilinear interpolation method is shown in Fig. 2(b); the cloud image reconstructed by Yang method is shown in Fig. 2(c); the cloud image reconstructed by the method of the present invention is shown in Fig. 2(d); the Cat image reconstructed by the bilinear interpolation method is shown in Figure 3(b); the Cat image reconstructed by the Yang method is shown in Figure 3(c); the reconstructed by the method of the present invention The Cat image is shown in Figure 3(d); the Building image reconstructed by the bilinear interpolation method is shown in Figure 4(b); the Building image reconstructed by the Yang method is shown in Figure 4(c); The Building image reconstructed by the method of the present invention is shown in Fig. 4(d). The quality of the reconstructed image is judged by the peak signal-to-noise ratio (PSNR) of the original high-resolution image:

$\mathrm{<span\; class="notranslate">\; PSNR</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; log</span>}\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; (</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In formula (1), f(i,j) represents the i-th row and j-column pixel of the original high-resolution image, and f′(i,j) represents the i-th row and j of the degraded super-resolution reconstructed image Column pixels, f _max indicates the maximum pixel value of the image, and M×N is the image size. All method times are in seconds (s). The simulation data results of the three images are shown in Table 1 and Table 2:

Table 1 PSNR (dB) of three reconstructed images under different methods

Table 2 Yang method and the time required by the method of the present invention (sample training time + reconstruction time (s))

Claims (3)

1. an image super-resolution method of adaptive online dictionary learning, is characterized in that comprising the following steps: ① Select a set of high-resolution images and their corresponding set of low-resolution images as training samples; ②Extract the high-frequency features of the training samples, obtain a set of high-resolution sample feature images and its corresponding set of low-resolution sample feature images, and randomly extract Q high-resolution sample feature images from the high-resolution sample feature images. The block composition matrix M is to extract the low-resolution feature image small blocks corresponding to the extracted high-resolution sample feature image small blocks in the low-resolution sample feature image to form a matrix M′, where Q is greater than or equal to 30,000 and less than or equal to 100,000, Divide the matrix M and the matrix M′ into k categories, where k≥2, and obtain the k cluster centers m ₁ , m ₂ , m ₃ ,…,m _k of the matrix M′ and the initial sub-dictionary set {(D _l1 , D _h1 ),(D _l2 ,D _h2 ),(D _l3 ,D _h3 ),…,(D _lk ,D _hk )}, where D _l1 ,D _l2 ,D _l3 ,…,D _lk represent k initial low Resolution sub-dictionary, D _h1 , D _h2 , D _h3 , ..., D _hk represent k initial high-resolution sub-dictionaries; ③Use the online algorithm to optimize the initial sub-dictionary set to obtain the optimal target sub-dictionary set {(D _l1-best ,D _h1-best ),(D _l2-best ,D _h2-best ),(D _l3-best , D _h3-best ),…,(D _lk-best ,D _hk-best )}, where D _l1-best , D _l2-best , D _l3-best ,…,D _lk-best represent k optimal low resolution Rate target sub-dictionary, D _h1-best , D _h2-best , D _h3-best ,..., D _hk-best represent k optimal high-resolution target sub-dictionaries; ④ Input the low-resolution image X that needs to be super-resolution enlarged, extract the high-frequency features of the low-resolution image X, obtain the low-resolution feature image X' corresponding to the low-resolution image X, and then select the low-resolution feature image X 'The qth image block X _q , q≥1, the matrix corresponding to the image block X _q is denoted as x _q , and the matrix x _q and cluster centers m ₁ ,m ₂ ,m ₃ ,…,m _k are calculated respectively Euclidean distance, compare the Euclidean distance between x _q and each cluster center, get the cluster center m _n with the smallest Euclidean distance, 1≤n≤k, choose the initial low resolution with m _n as the cluster center The optimal low-resolution target sub-dictionary corresponding to the sub-dictionary, denoted as D _lq-best ; ⑤ Use the SP algorithm to calculate the sparse representation coefficient α _q of the matrix x _q under D _lq-best , α _q = argmin||x _q -D _lq-best α _q || ₂ + λ||α _q || ₁ , α _q =SP(D _lq-best , x _q ), the specific steps are as follows: ⑤-1 Record the column number index of the K column atom with the greatest correlation with the matrix x _q in D _lq-best as

Indicates that the column number in D _lp-best is

A matrix composed of atoms, Indicates that the matrix x _q is in projection on ${\hat{x}}_q={\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\&prime;}{x}_q,$ in ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\&prime;}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{*},$ * represents the transpose of the matrix; ⑤-2 Initialization: Define the initial state projection

The initial residual vector is r ₀ ,

In the initial state, the column number index of K sequence atoms corresponding to the maximum value of elements in x′ _q is denoted as I ₀ ,

Represents the matrix composed of atoms whose column number is I ₀ in D _lp-best ; ⑤-3 Let the current iteration number be ω, ω≥1, and project in the current iteration

The residual vector of is r _ω , let ${x_q^{\&prime;}=\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_0}^{\&prime;}{x}_q,$ ${\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{\&prime;}={\left({\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{*}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}\right)}^{-1}{\mathrm{D.}}_{\mathrm{lq}-\mathrm{the\; best}-{\tilde{I}}_{\mathrm{\&omega;}}}^{*},$ In the current iteration, the column number index of the K sequence atoms corresponding to the maximum value of the element in x′ _p is I _ω , combined with the formula

{the column number index of the K column atom with the greatest correlation with rω _-1 in _Dlq-best } and

Update r _ω , and I _ω , r _ω-1 represents the projection

The residual vector at the ω-1th iteration,

Represents the matrix composed of atoms whose column number is I _ω in D _lp-best ,

Indicates that the column number in D _lp-best is

A matrix composed of atoms; ⑤-4 Compare r _ω with r _ω-1 , if r _ω || ₂ >r _ω-1 || ₂ , then the iteration ends, otherwise, add 1 to ω as the current number of iterations and return to step ⑤-3 for iteration renew; ⑤-5 After the iteration is completed, use the least squares method to obtain the optimal sparse representation coefficient α _q of x _q in the dictionary D _lq-best ; ⑥ Select the high-resolution target sub-dictionary D _{hq- best corresponding to D lq-best} , and use the sparse representation coefficient α _q and D _hq-best to solve the high-resolution image reconstruction block of x _q corresponding to D _hq-best y _q =D _hq-best α _q ; ⑦According to the method of steps ③～⑥, process all the small image blocks of the low-resolution feature image X' according to the "zigzag" shape, and obtain the initial high-resolution reconstructed image ⑧ Reconstruction of high-resolution images Deblocking is performed to obtain the final high-resolution reconstructed image Y. 2. the image super-resolution method of a kind of adaptive online dictionary learning according to claim 1, is characterized in that described step 3. the concrete steps that initial sub-dictionary set is optimized are as follows: ③-1 Optimize the low-resolution sub-dictionary D _l1 , the specific steps are: a. Assuming that the total number of optimization iterations is T, in the tth iteration, input any feature image block f _l in the feature image of the low-resolution sample, 1≤t≤T; b. Fix the target sub-dictionary obtained by the t-1th iteration

Solving using Subspace Pursuit

λ represents the regularization coefficient and 0<λ<1, where argmin represents the minimum value of the solution functional, |||| represents the solution norm, and the feature image block f _l in the target subdictionary in the t-th iteration

The sparse decomposition coefficient Λ _t under ; c. Fixed sparse decomposition coefficient Λ _t , assuming that the low-resolution sub-dictionary D _l1 has p atoms, so that $A_t=[a_1,a_2,\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,a_j,\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,a_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{\mathrm{\&Lambda;}}_i{\mathrm{\&Lambda;}}_i^T,$ $B_t=[b_1,b_2,\&CenterDot;\&Center\; Dot;\&Center\; Dot;,b_j,\&CenterDot;\&CenterDot;\&CenterDot;,b_p]=\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}{f}_l{\mathrm{\&Lambda;}}_i^T,$ i=1,2,...t, a _j represents the jth column in A _t , b _j represents the jth column in B _t , a _j and b _j are p×1 matrix, 1≤j≤p; d. Let the j-th atom of the target sub-dictionary corresponding to the low-resolution sub-dictionary D _l1 be d _j , and optimize d _j , then $d_j=\frac{1}{\max ({\left|\right|u_j\left|\right|}_2,1)}{u}_j,$ $u_j=\frac{1}{A_{\mathrm{jj}}}(b_j-{\mathrm{D.}}_{{l1}_{t-j}}^{\&prime;}{a}_j)+d_j,$

Represents the target sub-dictionary being updated of the low-resolution sub-dictionary D _l1 , A _jj represents the value of the jth row in a _j , and max represents the maximum value of the solution function; e. Update the p atoms of the low-resolution sub-dictionary D _l1 according to step d, complete the t-th iteration, and obtain the target sub-dictionary of the low-resolution sub-dictionary D _l1 after the t-th iteration

f. Will

Into the formula ${\mathrm{D.}}_{{l1}_t}=\arg \min \frac{1}{t}\underset{i=1}{\overset{t}{\mathrm{\&Sigma;}}}(\frac{1}{2}{\left|\right|f_l-{\mathrm{D.}}_{{l1}_t}^{\&prime;}{\mathrm{\&Lambda;}}_i\left|\right|}_2^2+\mathrm{\&lambda;}{\left|\right|{\mathrm{\&Lambda;}}_i\left|\right|}_1),$ 0<λ<1, get the target sub-dictionary updated in the t-th cycle

g. By analogy, when the input image is the feature image block f _l , T iterations of the low-resolution sub-dictionary D _l1 are completed to obtain the optimized target sub-dictionary corresponding to the feature image block f _l

h. Input all feature image blocks except f _l in the low-resolution sample feature image in sequence, and perform training according to the same principles as steps a~g, complete the training of the initial low-resolution sub-dictionary D _l1 , and convert the last round The updated target sub-dictionary is used as the optimal target sub-dictionary D _l1-best of the initial low-resolution sub-dictionary D _l1 . ③-2 Use the same principle of optimizing the low-resolution sub-dictionary D _l1 to respectively optimize D _l2 , D _l3 , ..., D _lk , D _h1 , D _h2 , D _h3 ,…, D _hk are optimized to obtain the optimal target sub-dictionary corresponding to D _l2 , D _l3 ,…,D _lk , D _h1 , D _h2 , D _h3 ,…,D _hk , and finally Get the optimal target sub-dictionary set {(D _l1-best ,D _h1-best ),(D _l2-best ,D _h2-best ),(D _l3-best ,D _h3-best ),…,(D _lk-best ,D _hk-best ) }. 3. the image super-resolution method of a kind of self-adaptive online dictionary learning according to claim 1, it is characterized in that in described step 8., deblocking processing method is to get mean value to the pixel of overlapping part after each iteration.

Images