CN101763633A

CN101763633A - Visible light image registration method based on salient region

Info

Publication number: CN101763633A
Application number: CN200910088975A
Authority: CN
Inventors: 田捷; 郑健; 杨鑫; 邓可欣; 徐敏
Original assignee: Institute of Automation of Chinese Academy of Science
Current assignee: Institute of Automation of Chinese Academy of Science
Priority date: 2009-07-15
Filing date: 2009-07-15
Publication date: 2010-06-30
Anticipated expiration: 2029-07-15
Also published as: CN101763633B

Abstract

The invention relates to the field of image registration, in particular to a visible light image registration method based on a salient region, including: (1) loading an image. (2) Extract the salient region of the image. (3) For the extracted salient regions, calculate the regional feature descriptors, and match the salient regions according to the similarity of the regional feature descriptors. (4) Perform local rigid registration on the salient regions on the preliminary matching in step (3). (5) The center of the salient region after local rigid registration is used as the control point to perform global quadratic polynomial transformation registration. The method of the invention is a fast, accurate and robust automatic image registration method, and has great application value in image registration.

Description

Visible light image registration method based on salient regions

技术领域technical field

本发明涉及图像处理，模式识别技术，特别涉及一种基于显著性区域的自动图像配准技术。The invention relates to image processing and pattern recognition technology, in particular to an automatic image registration technology based on salient regions.

背景技术Background technique

目前主流的自动图像配准方法，主要有基于特征点的配准，基于图像灰度值的配准，基于互信息的配准方法等。这些方法都还存在一些不足之处，基于特征点的配准，对一些质量比较差的可见光图像，特征点难以准确提取，基于图像灰度值的配准要求两幅图像的灰度值必须要一致，对受光照等环境影响的图像配准精度不高，基于互信息的配准，配准需要的时间比较长，而且可能会陷入局部极值，无法得到精确配准结果。因此，对于低质量的图像很多还是采用手动配准，手动配准的成功率和精度都比较高，但是它加大了操作者的负担，配准的速度比较慢。At present, the mainstream automatic image registration methods mainly include registration based on feature points, registration based on image gray value, and registration methods based on mutual information. These methods still have some shortcomings. Based on the registration of feature points, for some visible light images with poor quality, it is difficult to accurately extract the feature points. The registration based on image gray value requires that the gray value of the two images must be equal. Consistent, the registration accuracy of images affected by lighting and other environments is not high. Registration based on mutual information takes a long time for registration, and may fall into local extremum, and accurate registration results cannot be obtained. Therefore, manual registration is still used for many low-quality images. The success rate and accuracy of manual registration are relatively high, but it increases the burden on the operator and the registration speed is relatively slow.

发明内容Contents of the invention

针对现有技术的缺陷，本发明的目的是提供一种快速、精确、鲁棒、基于显著性区域的可见光图像配准方法。Aiming at the defects of the prior art, the purpose of the present invention is to provide a fast, accurate and robust visible light image registration method based on salient regions.

为达成所述目的，本发明提供一种基于显著性区域的自动图像配准方法，该方法的步骤如下：To achieve the stated purpose, the present invention provides a method for automatic image registration based on salient regions, the steps of which are as follows:

步骤1：在计算机上加载两幅待配准图像，选择一幅作为参考图像，另一幅作为浮动图像；Step 1: Load two images to be registered on the computer, select one as a reference image and the other as a floating image;

步骤2：将参考图像和浮动图像分成M×N个矩形区域，计算每一个区域R的局部显著性函数Ls(R)；对局部显著性函数Ls(R)进行高斯拟合，计算拟合后的局部显著性函数值Fls(R)；选择Fls(R)的局部极值区域的中心作为显著性区域中心；对每一个显著性区域中心，根据邻域内的Fls(R)分布计算区域的半径，提取参考图像和浮动图像的显著性区域R；Step 2: Divide the reference image and the floating image into M×N rectangular areas, and calculate the local saliency function Ls(R) of each area R; perform Gaussian fitting on the local saliency function Ls(R), and calculate the fitted The local significance function value Fls(R) of Fls(R); select the center of the local extremum region of Fls(R) as the center of the significant region; for each significant region center, calculate the radius of the region according to the distribution of Fls(R) in the neighborhood , to extract the salient region R of the reference image and the floating image;

步骤3：对提取出的显著性区域R构建一个72维的尺度无关特征描述子Lfd(R)；定义一个距离度量函数为Dist(Lfd(R₁)，Lfd(R₂))，衡量两个特征描述子Lfd(R₁)，Lfd(R₂)间的相似性；对两幅图像的任一显著性区域对C(i，j)，计算C(i，j)的区域匹配相似度，采用由粗到精的匹配策略，进行显著性区域的匹配；Step 3: Construct a 72-dimensional scale-independent feature descriptor Lfd(R) for the extracted salient region R; define a distance metric function as Dist(Lfd(R ₁ ), Lfd(R ₂ )), measure two Similarity between feature descriptors Lfd(R ₁ ), Lfd(R ₂ ); for any significant region pair C(i, j) of two images, calculate the region matching similarity of C(i, j), Use a coarse-to-fine matching strategy to match the salient regions;

步骤4：对初步匹配上的显著性区域对Cmp(i，j)，采用基于归一化相关系数的相似性度量进行局部刚体配准；Step 4: For the salient region pair Cmp(i, j) on the preliminary matching, use the similarity measure based on the normalized correlation coefficient to perform local rigid body registration;

步骤5：对局部刚体配准后的区域，采用聚类分析的方法，提取精确匹配成功的区域中心点作为控制点进行全局二次多项式变换配准，实现两幅图像的精确配准。Step 5: For the area after the local rigid body registration, cluster analysis method is used to extract the center point of the area with successful accurate matching as the control point for global quadratic polynomial transformation registration to achieve accurate registration of the two images.

其中，所述局部显著性函数Ls(R)是：Ls(R)＝Av(R)·Lge(R)，式中：Av(R)是区域R的归一化区域差异函数表示为Av(R)＝σ/μ，σ是区域R的标准差，μ是区域R的均值；Lge(R)是区域R的梯度场熵表示如下：Wherein, the local saliency function Ls(R) is: Ls(R)=Av(R) Lge(R), where: Av(R) is the normalized regional difference function of the region R, expressed as Av( R)=σ/μ, σ is the standard deviation of the region R, μ is the mean value of the region R; Lge(R) is the gradient field entropy of the region R expressed as follows:

$Lge Lge ((R R)) = = - - {Σ Σ}_{i i = = 11}^{3636} {p p}_{i i} ((R R)) {log log}_{22} {p p}_{i i} ((R R)),,$

p_i(R)是梯度方向位于第i个扇形区域的像素点集在区域R中占的梯度幅值比例表示如下：p _i (R) is the gradient amplitude ratio of the pixel point set in the i-th fan-shaped area where the gradient direction occupies in the area R, expressed as follows:

R_i是由所有梯度方向位于第i个扇形区域内的像素点构成的点集，

是像素点X梯度幅值，X_i是点集R_i中的像素点。

R _i is a point set composed of all the pixels whose gradient direction is located in the i-th fan-shaped area,

is the gradient magnitude of the pixel point X, and _Xi is the pixel point in the point set R _i .

其中，所述拟合后的局部显著性函数值Fls(R)的计算为：

式中：用σ＝1.5的高斯核函数进行拟合，M是图像在X方向上划分的矩形区域数，N是图像在Y方向上划分的矩形区域数，(a，i)∈{1，2，...，M}，是区域R在M×N矩形区域阵列中X方向上的坐标，(b，j)∈{1，2，...，N}是区域R在M×N矩形区域阵列中Y方向上的坐标，矩形区域为R_ab。Wherein, the calculation of the local significance function value Fls(R) after the fitting is:

In the formula: the Gaussian kernel function of σ=1.5 is used for fitting, M is the number of rectangular areas divided by the image in the X direction, N is the number of rectangular areas divided by the image in the Y direction, (a, i)∈{1, 2,...,M}, is the coordinates of the region R in the X direction in the M×N rectangular region array, (b,j)∈{1,2,...,N} is the region R in the M×N Coordinates in the Y direction in the rectangular area array, the rectangular area is R _ab .

其中，所述每一个显著性区域中心的区域半径的计算包括：以显著性区域中心所在矩形区域R_ab为中心，构建一个半径最大的正方形矩形区域集合Ω，Ω必须满足以下条件：Wherein, the calculation of the area radius of each salient area center includes: taking the rectangular area R _ab where the salient area center is located as the center, constructing a square rectangular area set Ω with the largest radius, Ω must meet the following conditions:

Fls(R_ij)≥λ·Fls(R_ab)， $&ForAll; R_{ij} &Element; Ω,$ Fls(R _ij )≥λ·Fls(R _ab ), $&ForAll; R_{ij} &Element; Ω,$

式中：Fls(R_ij)是区域R_ij拟合后的局部显著性函数值，λ是矩形区域半径控制参数，经验取值为0.75；选择Ω的长度和宽度中的较小值作为显著性区域半径。In the formula: Fls(R _ij ) is the local saliency function value after fitting of the region R _ij , λ is the control parameter of the radius of the rectangular region, and the empirical value is 0.75; the smaller value of the length and width of Ω is selected as the saliency area radius.

其中，所述构建一个72维的尺度无关特征描述子为：Lfd(R)＝(p₁(R)，...p₃₆(R)，da₁(R)，...da₃₆(R))，式中p_i(R)是梯度方向位于第i个扇形区域的像素点集在区域R中占的梯度幅值比例，da_i(R)∈[0，2π)是梯度方向位于第i个扇形区域的像素点集的几何中心到显著性区域中心的方向角；然后定义一个距离度量函数Dist(Lfd(R₁)，Lfd(R₂))，衡量两个特征描述子Lfd(R₁)，Lfd(R₂)间的相似性为：Wherein, the construction of a 72-dimensional scale-independent feature descriptor is: Lfd(R)=(p ₁ (R),...p ₃₆ (R), da ₁ (R),...da ₃₆ (R )), where p _i (R) is the gradient amplitude ratio of the pixel point set whose gradient direction is located in the i-th fan-shaped area in the area R, da _i (R)∈[0, 2π) is the gradient direction located in the i-th sector The direction angle from the geometric center of the pixel point set in the i fan-shaped area to the center of the salient area; then define a distance metric function Dist(Lfd(R ₁ ), Lfd(R ₂ )), measure the two feature descriptors Lfd(R ₁ ), the similarity between Lfd(R ₂ ) is:

$Dist Dist ((Lfd Lfd (({R R}_{11})),, Lfd Lfd (({R R}_{22})))) = = {Σ Σ}_{i i = = 11}^{3636} {{Eud Eud {(({da da}_{i i} (({R R}_{11})),, {da da}_{i i} (({R R}_{22}))))}^{22} \cdot \cdot Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))))}}$

$\cdot &Center Dot; {Σ Σ}_{i i = = 11}^{3636} {{log log \frac{Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))}{Min Min (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))} \cdot \cdot Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))))}},,$

式中Eud(da_i(R₁)，da_i(R₂))是R₁和R₂中对应第i个方向角间的夹角。In the formula, Eud(da _i (R ₁ ), da _i (R ₂ )) is the angle between R ₁ and R ₂ corresponding to the i-th direction angle.

其中，所述显著性区域的匹配包括：1)遍历两幅图像的每一个可能的显著性区域匹配C(i，j)，式中i表示参考图像的第i个显著性区域R_i，j表示浮动图像的第j个显著性区域R_j；满足以下条件的C(i，j)则认为是粗匹配上的显著性区域对Cmp(i，j)：Wherein, the matching of the salient region includes: 1) traversing every possible salient region matching C(i, j) of the two images, where i represents the ith salient region R _i , j of the reference image Represents the jth salient region R _j of the floating image; C(i, j) satisfying the following conditions is considered to be a salient region pair Cmp(i, j) on rough matching:

$\frac{Min Min ((Av Av (({R R}_{i i})),, Av Av (({R R}_{j j}))))}{Max Max ((Av Av (({R R}_{i i})),, Av Av (({R R}_{j j}))))} \cdot \cdot \frac{Min Min ((Lge Lge (({R R}_{i i})),, Lge Lge (({R R}_{j j}))))}{Max Max ((Lge Lge (({R R}_{i i})),, Lge Lge (({R R}_{j j}))))} > > T T,,$

式中Min(·)函数是求最小值，Max(·)函数是求最大值，T是粗匹配控制参数，经验取值为0.6；In the formula, the Min( ) function is to seek the minimum value, the Max( ) function is to seek the maximum value, T is the rough matching control parameter, and the empirical value is 0.6;

2)对每一个粗匹配区域对Cmp(i，j)，按如下方法计算R_i和R_j间的相似性S(i，j)以及旋转角度θ_ij： 2) For each rough matching area pair Cmp(i, j), calculate the similarity S(i, j) and the rotation angle θ _ij between R _i and R _j as follows:

S(i，j)＝Dist(Lfd(R_i)，Lfd(R_jk))，S(i, j) = Dist(Lfd(R _i ), Lfd(R _j k)),

k＝arg_k Min(Dist(Lfd(R_i)，Lfd(R_jk)))，k∈{0，1，...35}，k = arg _k Min(Dist(Lfd(R _i ), Lfd(R _j k))), k ∈ {0, 1, ... 35},

式中，R_jk是将R_j逆时针旋转10·k度得到的新区域；每一个粗匹配区域对Cmp(i，j)确定三个全局刚性变换参数为：二维平移(由R_i和R_j的区域中心决定)以及旋转参数θ_ij；In the formula, R _j k is the new area obtained by rotating R _j counterclockwise by 10 k degrees; each rough matching area pair Cmp(i, j) determines three global rigid transformation parameters: two-dimensional translation (by R _i and the area center of R _j ) and the rotation parameter θ _ij ;

3)按S(i，j)升序排列所有的Cmp(i，j)，选取前2000(如果提取的粗匹配区域对数目不足则以实际的数目为准)个粗匹配区域对Cmp(i，j)作为输入样本集合，设置合适的类内距离阈值，对Cmp(i，j)进行全局刚性变换参数空间上的聚类，选取聚类内数目最多的类作为初步匹配上的显著性区域对F(i，j)；按S(i，j)的大小来剔除重复的区域，保证初步匹配上的显著性区域对F(i，j)中不包含重复的区域，减小后续的计算量。3) Arrange all Cmp(i, j) in ascending order of S(i, j), and select the first 2000 (if the number of extracted rough matching region pairs is insufficient, the actual number shall prevail) rough matching region pairs Cmp(i, j) As the input sample set, set an appropriate intra-class distance threshold, cluster Cmp(i, j) in the global rigid transformation parameter space, and select the class with the largest number in the cluster as the significant region pair for preliminary matching F(i, j); eliminate repeated regions according to the size of S(i, j), to ensure that the significant region pair F(i, j) on the preliminary matching does not contain repeated regions, reducing the amount of subsequent calculations .

其中，所述局部刚体配准是对初步匹配上的显著性区域对F(i，j)，以显著性区域R_i，R_j的中心位置和旋转角度θ_ij作为初始配准参数，进行局部刚体配准。Wherein, the local rigid body registration is for the salient region pair F(i, j) on the preliminary matching, and the center position and rotation angle θ _ij of the salient regions R _i and R _j are used as initial registration parameters to perform local Rigid body registration.

其中，所述实现两幅图像的精确配准包括：对局部刚体配准后的区域，设置更精细的类内距离阈值，进行全局刚性变换参数空间上的聚类分析，提取精确匹配上的区域对；并以精确匹配上的区域中心点作为控制点进行全局二次多项式变换配准，实现两幅图像的精确配准。Wherein, the precise registration of the two images includes: setting a finer intra-class distance threshold for the region after the local rigid body registration, performing cluster analysis on the global rigid transformation parameter space, and extracting the region on the exact match Right; and the center point of the area on the precise matching is used as the control point to perform global quadratic polynomial transformation registration to realize the precise registration of the two images.

本发明的有益效果：本发明利用提取图像中的显著性区域，计算区域特征描述子，然后采用由粗到精的显著性区域匹配策略，将初步匹配上的显著性区域对进行局部刚体配准，最后提取局部刚体配准成功的区域中心点作为控制点进行全局二次多项式变换配准，实现两幅图像的精确配准。由于我们很好的定义了区域显著性函数和特征描述子，采用由粗到精的显著性区域匹配策略以及图像配准策略，整个算法的计算量大大减少，同时对低质量的图像也能很好的完成精确配准，算法具有非常高的鲁棒性。实验结果表明，本方法可以在4s左右完成一般图像的配准，低质量图像配准时间会长一些，在10s左右也可完成精确配准，对1548×1260的图像配准精度可以达到2个像素。因此，具有重大的应用价值。Beneficial effects of the present invention: the present invention utilizes the extraction of salient regions in the image, calculates the regional feature descriptor, and then adopts a coarse-to-fine salient region matching strategy to perform local rigid-body registration on the salient region pairs on the preliminary match , and finally extract the center point of the area where the local rigid body registration is successful as the control point for global quadratic polynomial transformation registration to achieve accurate registration of the two images. Since we have well defined regional saliency functions and feature descriptors, and adopted a coarse-to-fine salient region matching strategy and image registration strategy, the calculation of the entire algorithm is greatly reduced, and it can also be used for low-quality images. Accurate registration is completed well, and the algorithm has very high robustness. The experimental results show that this method can complete the registration of general images in about 4s, the registration time of low-quality images will be longer, and the precise registration can be completed in about 10s, and the registration accuracy of 1548×1260 images can reach 2 pixels. Therefore, it has great application value.

附图说明Description of drawings

图1是本发明方法执行流程示意图；Fig. 1 is a schematic diagram of the execution flow of the method of the present invention;

图2(a)浮动图像；Figure 2(a) floating image;

图2(b)参考图像；Figure 2(b) reference image;

图2(c)是提取的浮动图像显著性区域；Figure 2(c) is the extracted salient region of the floating image;

图2(d)是提取的参考图像显著性区域：Figure 2(d) is the extracted salient region of the reference image:

图2(e)是初步匹配上的显著性区域示意图；Figure 2(e) is a schematic diagram of the salient regions on the preliminary matching;

图3是梯度方向划分示意图；Fig. 3 is a schematic diagram of gradient direction division;

图4(a)是显著性区域R的梯度分布示意图；Figure 4(a) is a schematic diagram of the gradient distribution of the salient region R;

图4(b)是显著性区域特征描述子前36维特征向量示意图；Figure 4(b) is a schematic diagram of the first 36-dimensional feature vector of the feature descriptor of the salient region;

图4(c)是显著性区域特征描述子后36维特征向量示意图；Figure 4(c) is a schematic diagram of the 36-dimensional feature vector after the feature descriptor of the salient region;

图5是配准后的浮动图像示意图；Fig. 5 is a schematic diagram of a floating image after registration;

图6是融合后的图像示意图；Figure 6 is a schematic diagram of an image after fusion;

具体实施方式Detailed ways

为使本发明的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本发明进一步详细说明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

如图1示出本发明方法执行流程示意图；在计算机101上加载两幅待配准图像，选择一幅作为参考图像，另一幅作为浮动图像；利用计算机101实现以下四个顺序处理单元对可见光图像配准：显著性区域提取单元102，用来提取可见光图像中的显著性区域；区域特征描述子计算及显著性区域匹配单元103，用来计算显著性区域的特征描述子，以及基于特征描述子间的相似性进行显著性区域匹配；局部刚体配准单元104，对初步匹配上的显著性区域进行局部刚体配准；全局二次多项式变换配准单元105，提取局部刚体配准后的区域中心点作为控制点进行全局二次多项式变换，实现图像的精确配准。Figure 1 shows a schematic diagram of the execution flow of the method of the present invention; load two images to be registered on the computer 101, select one as a reference image, and the other as a floating image; use the computer 101 to realize the following four sequential processing units for visible light Image registration: the salient area extraction unit 102 is used to extract the salient area in the visible light image; the area feature descriptor calculation and salient area matching unit 103 is used to calculate the feature descriptor of the salient area, and based on the feature description The similarity between sub-groups is used to perform salient region matching; the local rigid body registration unit 104 performs local rigid body registration on the salient region on the preliminary matching; the global quadratic polynomial transformation registration unit 105 extracts the region after local rigid body registration The center point is used as a control point for global quadratic polynomial transformation to achieve precise image registration.

所述显著性区域提取单元102，具体实现是应用计算机101，并可以利用编程语言C++编写程序，根据我们定义的拟合后的局部显著性函数值Fls(R)，提取两幅图像中的显著性区域。The salient area extracting unit 102 is specifically realized by using a computer 101, and can utilize the programming language C++ to write a program, and extract the salient features in the two images according to the fitted local saliency function value Fls(R) defined by us. sex area.

区域特征描述子计算及显著性区域匹配单元103，具体实现是应用计算机101，并可以利用编程语言C++编写程序，实现如下功能：对提取出的显著性区域，计算它的72维的尺度无关特征描述子Lfd(R)，定义距离度量函数Dist(Lfd(R₁)，Lfd(R₂))，衡量两个特征描述子Lfd(R₁)，Lfd(R₂)间的相似性，对两幅图像的任一显著性区域对，计算它们的区域匹配相似度，采用由粗到精的匹配策略，进行显著性区域的匹配。The regional feature descriptor calculation and salient region matching unit 103 is specifically realized by using the computer 101, and can use the programming language C++ to write a program to realize the following functions: for the extracted salient region, calculate its 72-dimensional scale-independent feature Descriptor Lfd(R), define the distance measurement function Dist(Lfd(R ₁ ), Lfd(R ₂ )), measure the similarity between two feature descriptors Lfd(R ₁ ), Lfd(R ₂ ), for two For any pair of salient regions in an image, calculate their region matching similarity, and use a coarse-to-fine matching strategy to match the salient regions.

所述局部刚体配准单元104，具体实现是应用计算机101，并可以利用编程语言C++编写程序，实现如下功能：对初步匹配上的显著性区域对，采用基于归一化相关系数的相似性度量进行局部刚体配准。The local rigid body registration unit 104 is specifically implemented by using the computer 101, and can use the programming language C++ to write a program to realize the following functions: for the salient region pairs on the preliminary matching, the similarity measure based on the normalized correlation coefficient is adopted Perform local rigid body registration.

所述全局二次多项式变换配准单元105，具体实现是应用计算机101，并可以利用编程语言C++编写程序，实现如下功能：对局部刚体配准后的区域，设置更精细的类内距离阈值，进行全局刚性变换参数空间上的聚类分析，提取精确匹配成功的区域中心点作为控制点进行全局二次多项式变换配准，实现两幅图像的精确配准。The global quadratic polynomial transformation registration unit 105 is specifically implemented by using the computer 101, and can use the programming language C++ to write a program to realize the following functions: for the region after the local rigid body registration, a finer intra-class distance threshold is set, Carry out cluster analysis on the parameter space of the global rigid transformation, extract the center point of the area with successful accurate matching as the control point for global quadratic polynomial transformation registration, and realize the precise registration of the two images.

本发明的配准方法主要包括以下步骤：The registration method of the present invention mainly includes the following steps:

步骤1：加载两幅待配准图像，应用计算机读取图像，将图像转化为二维数组，存储在计算机中，以方便后续单元进行处理，选择一幅作为参考图像，另一幅作为浮动图像，如图2(a)，图2(b)所示。Step 1: Load two images to be registered, use a computer to read the images, convert the images into a two-dimensional array, and store them in the computer to facilitate subsequent unit processing. Select one as a reference image and the other as a floating image , as shown in Figure 2(a) and Figure 2(b).

步骤2：运行显著性区域提取单元102，提取两幅图像的显著性区域。Step 2: Run the salient region extraction unit 102 to extract the salient regions of the two images.

显著性区域的提取主要由以下几步完成：The extraction of salient regions is mainly completed by the following steps:

1、将参考图像和浮动图像分成M×N个矩形区域，M，N的取值跟图像大小相关，在我们的方法中，将1548×1260的图像分成100×100个矩形区域，计算每一个矩形区域R的局部显著性函数Ls(R)：1. Divide the reference image and the floating image into M×N rectangular areas. The values of M and N are related to the size of the image. In our method, the 1548×1260 image is divided into 100×100 rectangular areas, and each The local saliency function Ls(R) of the rectangular area R:

Ls(R)＝Av(R)·Lge(R)，Ls(R)=Av(R)·Lge(R),

其中：Av(R)是区域R的归一化区域差异函数，表示为：Av(R)＝σ/μ，式中σ是区域R的标准差，μ是区域R的均值；Among them: Av(R) is the normalized regional difference function of the region R, expressed as: Av(R)=σ/μ, where σ is the standard deviation of the region R, and μ is the mean value of the region R;

Lge(R)是区域R的梯度场熵，表示如下：Lge(R) is the gradient field entropy of the region R, expressed as follows:

式中：p_i(R)是梯度方向(像素点X的梯度方向由该点的梯度向量g(X)＝[G_x(X)，G_y(X)]确定，如图3所示，我们将整个G_xG_y平面分成36等分)位于第i个扇形区域的像素点集在区域R中占的梯度幅值比例表示如下：

In the formula: p _i (R) is the gradient direction (the gradient direction of the pixel point X is determined by the gradient vector g (X)=[G _x (X), G _y (X)] of the point, as shown in Figure 3, We divide the entire G _x G _y plane into 36 equal parts) The proportion of the gradient amplitude value of the pixel point set in the i-th fan-shaped area in the area R is expressed as follows:

${p p}_{i i} ((R R)) = = \frac{{&Integral; &Integral;}_{{R R}_{i i}} | | g g (({X x}_{i i})) | | d d {X x}_{i i}}{{&Integral; &Integral;}_{R R} | | g g ((X x)) | | dX wxya},,$

式中R_i是由所有梯度方向位于第i个扇形区域内的像素点构成的点集，

是像素点X梯度幅值，X_i是点集R_i中的像素点。如果区域的归一化区域差异函数Av(R)的值比较小，那么区域的像素值分布比较一致，区域同质性比较强，区域显著性不明显，如果Av(R)的值比较大，区域像素值分布比较复杂，区域异质性比较强，区域显著性比较明显。另一方面，如果区域是同质的，那么局部梯度场分布应该是规则的，梯度场熵Lge(R)的值比较小，区域显著性低，如果区域是异质的，那么局部梯度场的分布就比较复杂，梯度场熵Lge(R)的值比较大，区域显著性增强。结合Av(R)和Lge(R)的区域显著性度量，Ls(R)能更好的度量区域的显著性水平。In the formula, R _i is a point set composed of all the pixels whose gradient directions are located in the i-th fan-shaped area,

is the gradient magnitude of the pixel point X, and _Xi is the pixel point in the point set R _i . If the value of the normalized regional difference function Av(R) of the region is relatively small, then the pixel value distribution of the region is relatively consistent, the regional homogeneity is relatively strong, and the regional significance is not obvious. If the value of Av(R) is relatively large, The distribution of regional pixel values is relatively complex, the regional heterogeneity is relatively strong, and the regional salience is relatively obvious. On the other hand, if the region is homogeneous, then the distribution of the local gradient field should be regular, the value of the gradient field entropy Lge(R) is relatively small, and the significance of the region is low. If the region is heterogeneous, then the local gradient field The distribution is more complicated, the value of the gradient field entropy Lge(R) is relatively large, and the regional significance is enhanced. Combining the regional saliency measures of Av(R) and Lge(R), Ls(R) can better measure the saliency level of the region.

2、在完成局部显著性函数Ls(R)的计算之后，我们采用σ＝1.5的高斯核函数进行拟合，计算拟合后的局部显著性函数值Fls(R)：2. After completing the calculation of the local significance function Ls(R), we use the Gaussian kernel function of σ=1.5 for fitting, and calculate the fitted local significance function value Fls(R):

$Fls Fls (({R R}_{ab ab})) = = {Σ Σ}_{i i = = 11}^{M m} {Σ Σ}_{j j = = 11}^{N N} \frac{Ls ls (({R R}_{ij ij}))}{22 π π {σ σ}^{22}} ex ex {p p}^{- - (({((a a - - i i))}^{22} + + {((b b - - j j))}^{22})) / / ((22 {σ σ}^{22}))},,$

M是图像在X方向上划分的矩形区域数，N是图像在Y方向上划分的矩形区域数，(a，i)∈{1，2，...，M}，是区域R在M×N矩形区域阵列中X方向上的坐标，(b，j)∈{1，2，...，N}是区域R在M×N矩形区域阵列中Y方向上的坐标，通过对Ls(R)进行高斯拟合，Fls(R)可以反映区域R的邻域的显著性水平，从而能更准确的度量R的区域显著性，所以我们选择Fls(R)的局部极值区域的中心点作为显著性区域中心。M is the number of rectangular areas divided by the image in the X direction, N is the number of rectangular areas divided by the image in the Y direction, (a, i) ∈ {1, 2, ..., M}, is the area R in M× The coordinates in the X direction of the N rectangular area array, (b, j) ∈ {1, 2, ..., N} are the coordinates of the area R in the Y direction in the M×N rectangular area array, through the pair Ls(R ) for Gaussian fitting, Fls(R) can reflect the significance level of the neighborhood of the region R, so that the regional significance of R can be measured more accurately, so we choose the center point of the local extremum region of Fls(R) as The center of the salience region.

3、在完成显著性区域中心提取后，根据邻域内的Fls(R)分布计算每一个显著性区域中心的区域半径，包括：3. After completing the extraction of the center of the salient area, calculate the area radius of each salient area center according to the Fls(R) distribution in the neighborhood, including:

以显著性区域中心所在矩形区域R_ab为中心，构建一个半径最大的正方形矩形区域集合Ω，Ω必须满足以下条件：Taking the rectangular area R _ab where the center of the salient area is located as the center, construct a square rectangular area set Ω with the largest radius, Ω must meet the following conditions:

其中：Fls(R_ij)是区域R_ij拟合后的局部显著性函数值，λ是区域半径控制参数，经验取值为0.75。选择Ω的长度(X方向包含的像素数)和宽度(Y方向包含的像素数)中的较小值作为显著性区域半径，完成图像的显著性区域提取，如图2(c)，图2(d)所示。Among them: Fls(R _ij ) is the local saliency function value after fitting the region R _ij , λ is the region radius control parameter, and the empirical value is 0.75. Select the smaller value of the length (the number of pixels included in the X direction) and the width (the number of pixels included in the Y direction) of Ω as the radius of the salient area to complete the extraction of the salient area of the image, as shown in Figure 2(c), Figure 2 (d) shown.

步骤3：运行区域特征描述子计算及显著性区域匹配单元103，完成两幅图像的显著性区域匹配。Step 3: Run the region feature descriptor calculation and salient region matching unit 103 to complete the salient region matching of the two images.

显著性区域匹配主要由以下几步完成：Salient region matching is mainly completed by the following steps:

1、对每一个显著性区域R，我们遍历区域R的每一个像素，计算其梯度幅值和梯度方向。然后统计区域R的梯度向量分布和每个梯度方向上包含的像素点集合，构建一个72维的尺度无关特征描述子Lfd(R)：1. For each salient region R, we traverse each pixel of the region R and calculate its gradient magnitude and gradient direction. Then count the gradient vector distribution of the region R and the set of pixels contained in each gradient direction to construct a 72-dimensional scale-independent feature descriptor Lfd(R):

Lfd(R)＝(p₁(R)，...p₃₆(R)，da₁(R)，...da₃₆(R))，Lfd(R)=(p ₁ (R), . . . p ₃₆ (R), da ₁ (R), . . . da ₃₆ (R)),

其中p_i(R)是梯度方向位于第i个扇形区域的像素点集在区域R中占的梯度幅值比例，da_i(R)∈[0，2π)是梯度方向位于第i个扇形区域的像素点集的几何中心到显著性区域中心C的方向角。Lfd(R)的前36维特征描述了区域R内36个梯度方向上的像素点梯度幅值分布(示意图见图4(b)，限于图像大小并未画齐所有36个方向上的特征分布)，后36维特征描述了36个梯度方向的像素点几何中心相对于区域中心C的方位特征(示意图见图4(c)，限于图像大小并未画齐所有36个方向上的特征分布)。一个示意性的尺度无关特征描述子Lfd(R)的构造如图4(a)，图4(b)，图4(c)所示，其中，图4(a)中m示意性表示第三个梯度方向上的像素点集的(梯度方向以虚线表示)几何中心，C是区域中心，da₃(R)表示从C到m的方向角。Where p _i (R) is the gradient amplitude ratio of the pixel point set whose gradient direction is located in the i-th fan-shaped area in the area R, da _i (R)∈[0, 2π) is the gradient direction located in the i-th fan-shaped area The direction angle from the geometric center of the pixel point set to the center C of the salient region. The first 36-dimensional features of Lfd(R) describe the gradient amplitude distribution of pixels in 36 gradient directions in the region R (see Figure 4(b) for the schematic diagram, and the feature distribution in all 36 directions is not drawn due to the image size ), the last 36-dimensional features describe the orientation characteristics of the geometric center of the pixel point in the 36 gradient directions relative to the center C of the area (see Figure 4(c) for the schematic diagram, and the feature distribution in all 36 directions is not drawn due to the image size) . The construction of a schematic scale-independent feature descriptor Lfd(R) is shown in Figure 4(a), Figure 4(b), and Figure 4(c), where m in Figure 4(a) schematically represents the third The geometric center of the pixel point set in the gradient direction (the gradient direction is indicated by a dotted line), C is the center of the region, and da ₃ (R) represents the direction angle from C to m.

2、定义一个距离度量函数Dist(Lfd(R₁)，Lfd(R₂))，衡量两个特征描述子Lfd(R₁)，Lfd(R₂)间的相似性为：2. Define a distance metric function Dist(Lfd(R ₁ ), Lfd(R ₂ )) to measure the similarity between two feature descriptors Lfd(R ₁ ), Lfd(R ₂ ) as:

$Dist Dist ((Lfd Lfd (({R R}_{11})),, Lfd Lfd (({R R}_{22})))) = = {Σ Σ}_{i i = = 11}^{3636} {{Eud Eud {(({da da}_{i i} (({R R}_{11})),, {da da}_{i i} (({R R}_{22}))))}^{22} \cdot &Center Dot; Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))))}}$

$\cdot &Center Dot; {Σ Σ}_{i i = = 11}^{3636} {{log log \frac{Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))}{Min Min (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))} \cdot &Center Dot; Max Max (({p p}_{i i} (({R R}_{11})),, {p p}_{i i} (({R R}_{22}))))}},,$

其中Eud(da_i(R₁)，da_i(R₂))是R₁和R₂中对应第i个方向角间的夹角。Dist(Lfd(R₁)，Lfd(R₂))的第二项类似于K-L散度的定义，但是与K-L散度相比，我们的距离度量函数定义具有对称性的优点。Wherein Eud(da _i (R ₁ ), da _i (R ₂ )) is the angle between R ₁ and R ₂ corresponding to the i-th direction angle. The second term of Dist(Lfd(R ₁ ), Lfd(R ₂ )) is similar to the definition of KL divergence, but compared with KL divergence, our distance metric function definition has the advantage of symmetry.

3、遍历两幅图像的每一个可能的显著性区域匹配C(i，j)，其中i表示参考图像的第i个显著性区域R_i，j表示浮动图像的第j个显著性区域R_j；满足以下条件的C(i，j)则认为是粗匹配上的显著性区域对Cmp(i，j)：3. Traverse every possible salient region matching C(i, j) of the two images, where i represents the i-th salient region R _i of the reference image, and j represents the j-th salient region R _j of the floating image ; C(i, j) that satisfies the following conditions is considered to be a significant region pair Cmp(i, j) on rough matching:

$\frac{Min Min ((Av Av (({R R}_{i i})),, Av Av (({R R}_{j j}))))}{Max Max ((Av Av (({R R}_{i i})),, Av Av (({R R}_{j j}))))} \cdot &Center Dot; \frac{Min Min ((Lge Lge (({R R}_{i i})),, Lge Lge (({R R}_{j j}))))}{Max Max ((Lge Lge (({R R}_{i i})),, Lge Lge (({R R}_{j j}))))} > > T T,,$

其中Min(·)函数是求最小值，Max(·)函数是求最大值，T是粗匹配控制参数，T值设定越高，则粗匹配条件要求越严格，粗匹配上的显著性区域对Cmp(i，j)也越少，经验取值为0.6。Among them, the Min( ) function is to find the minimum value, the Max( ) function is to find the maximum value, and T is the rough matching control parameter. The higher the T value is set, the stricter the rough matching conditions are, and the significant area Cmp(i, j) is also less, and the empirical value is 0.6.

4、对每一个粗匹配区域对Cmp(i，j)，按如下方法计算R_i和R_j间的相似性S(i，j)以及旋转角度θ_ij：4. For each rough matching area pair Cmp(i, j), calculate the similarity S(i, j) and the rotation angle θ _ij between R _i and R _j as follows:

${θ θ}_{ij ij} = = \frac{22 kπ kπ}{3636},,$

其中，R_ik是将R_j逆时针旋转10·k角度得到的新区域。Among them, R _i k is the new region obtained by rotating R _j counterclockwise by an angle of 10·k.

每一个粗匹配区域对Cmp(i，j)都可以确定三个全局刚性变换参数为：浮动图像中心相对参考图像中心的平移t_x，t_y(t_x＝O_jx-O_ix，t_y＝O_jy-O_iy)以及绕浮动图像中心的旋转角度θ_ij。其中O_ix，O_iy是显著性区域R_i中心O_i的X和Y方向上的坐标，O_jx，O_jy是显著性区域R_j中心O_j的X和Y方向上的坐标。Each coarse matching area pair Cmp(i, j) can determine three global rigid transformation parameters as: the translation t _x of the center of the floating image relative to the center of the reference image, _ty (t _x = O _j xO _i x, _ty = O _j yO _i y) and the rotation angle θ _ij around the center of the floating image. where O _i x, O _i y are the coordinates in the X and Y directions of the center O _i of the salient region R _i , and O _j x, O _j y are the coordinates in the X and Y directions of the center O _j of the salient region R _j .

5、按S(i，j)升序排列所有的Cmp(i，j)，选取前2000(如果提取的粗匹配区域对数目不足则以实际的数目为准)个粗匹配区域对Cmp(i，j)作为输入样本集合，设置合适的类内距离阈值，对Cmp(i，j)进行全局刚性变换参数空间上的聚类，选取聚类内数目最多的类作为初步匹配上的显著性区域对F(i，j)。具体的聚类方法如下：固定浮动图像的中心Cm，通过每一个粗匹配对Cmp(i，j)确定的三个刚体变换变换参数，将Cm映射到参考图像上的Cf_ij点，具体表达式如下：5. Arrange all Cmp(i, j) in ascending order according to S(i, j), and select the first 2000 (if the extracted rough matching region pairs are insufficient, the actual number shall prevail) coarse matching region pairs Cmp(i, j) As the input sample set, set an appropriate intra-class distance threshold, cluster Cmp(i, j) in the global rigid transformation parameter space, and select the class with the largest number in the cluster as the significant region pair for preliminary matching F(i,j). The specific clustering method is as follows: fix the center Cm of the floating image, map Cm to the Cf _ij point on the reference image through the three rigid body transformation parameters determined by each rough matching pair Cmp(i, j), the specific expression as follows:

$[\begin{matrix} {Cf Cf}_{ij ij} x x \\ {Cf Cf}_{ij ij} y the y \end{matrix}] = = [\begin{matrix} cos cos {θ θ}_{ij ij} & - - sin sin {θ θ}_{ij ij} \\ sin sin {θ θ}_{ij ij} & cos cos {θ θ}_{ij ij} \end{matrix}] \times \times [\begin{matrix} Cmx Cmx - - {O o}_{j j} x x \\ Cmy Cmy - - {O o}_{j j} y the y \end{matrix}] - - [\begin{matrix} {t t}_{x x} \\ {t t}_{y the y} \end{matrix}] + + [\begin{matrix} {O o}_{j j} x x \\ {O o}_{j j} y the y \end{matrix}]$

其中Cmx，Cmy，Cf_ijx，Cf_ijy分别是Cm和Cf_ijX和Y方向上的坐标。我们对映射后的Cf_ij点在二维欧式空间上进行聚类，通过设置合适的类内阈值(跟图像的尺寸大小相关，在我们的实验中选取的类内阈值t为50)，选取聚类内数目最多的类作为初步匹配上的显著性区域对F(i，j)。二维欧式空间上的聚类步骤如下：Where Cmx, Cmy, Cf _ij x, Cf _ij y are the coordinates of Cm and Cf _ij in the X and Y directions, respectively. We cluster the mapped Cf _ij points in the two-dimensional Euclidean space. By setting an appropriate intra-class threshold (related to the size of the image, the intra-class threshold t selected in our experiment is 50), select the cluster The class with the largest number in the class is used as the significant region pair F(i, j) on the preliminary matching. The clustering steps on the two-dimensional Euclidean space are as follows:

1.初始化分类数目N为0。1. Initialize the number of categories N to be 0.

2.按顺序遍历映射后的Cf_ij点，如果N＝0，将当前Cf_ij作为第一类的中心，N＝N+1；反之，依次计算Cf_ij与第k(k＝1，...，N)类中心的距离d_k，选取最小值d_kmin，如果d_kmin＜t，则将Cf_ij归入第kmin类并更新第kmin类的中心以及类内元素数，如果d_kmin≥t，则将Cf_ij作为第N+1类的中心，N＝N+1。2. Traverse the mapped Cf _ij points in order, if N=0, take the current Cf _ij as the center of the first category, N=N+1; otherwise, calculate Cf _ij and the kth (k=1, .. ., N) The distance d _k of the center of the class, select the minimum value d _kmin , if d _kmin <t, then classify Cf _ij into the kminth class and update the center of the kminth class and the number of elements in the class, if d _kmin ≥ t , then take Cf _ij as the center of the N+1th category, N=N+1.

3.选取类内数目最多的类对应的粗匹配Cmp(i，j)对作为初步匹配上的显著性区域对F(i，j)，然后，我们按S(i，j)的大小来剔除重复的区域，如果F(3，5)和F(3，7)都是F(i，j)中的元素，我们比较S(3，5)和S(3，7)的大小，剔除值比较大的匹配。保证初步匹配上的显著性区域对F(i，j)中不包含重复的区域，减小后续的计算量。初步匹配上的显著性区域对如图2(e)所示。3. Select the rough matching Cmp(i, j) pair corresponding to the class with the largest number in the class as the salient region pair F(i, j) on the preliminary matching, and then we eliminate it according to the size of S(i, j) Repeated areas, if both F(3, 5) and F(3, 7) are elements in F(i, j), we compare the size of S(3, 5) and S(3, 7), and remove the value relatively large match. It is guaranteed that the significant region pair F(i, j) on the preliminary matching does not contain repeated regions, reducing the amount of subsequent calculations. The salient region pairs on the preliminary matching are shown in Fig. 2(e).

步骤4：运行局部刚体配准单元104。对步骤3中初步匹配上的显著性区域对F(i，j)，以区域R_i，R_j的中心确定初始平移参数t_x，t_y，以旋转角度θ_ij作为初始旋转参数，以归一化相关系数作为区域相似性度量，进行局部刚体配准。归一化相关系数的定义如下：Step 4: Run the local rigid body registration unit 104 . For the significant region pair F(i, j) on the initial matching in step 3, the initial translation parameters t _x , t _y are determined with the center of the region R _i , R _j , the rotation angle θ _ij is used as the initial rotation parameter, and the normalized The normalized correlation coefficient is used as a measure of regional similarity for local rigid body registration. The normalized correlation coefficient is defined as follows:

$((F f,, M m)) = = - - \frac{{Σ Σ}_{i i = = 11}^{N N} (({F f}_{i i} ((x x)) - - \overset{&OverBar; &OverBar;}{F f ((x x))})) \cdot &Center Dot; (({M m}_{i i} ((x x)) - - \overset{&OverBar; &OverBar;}{M m ((x x))}))}{\sqrt{{Σ Σ}_{i i = = 11}^{N N} {(({F f}_{i i} ((x x)) - - \overset{&OverBar; &OverBar;}{F f ((x x))}))}^{22}} \cdot \cdot \sqrt{{Σ Σ}_{i i = = 11}^{N N} {(({M m}_{i i} ((x x)) - - \overset{&OverBar; &OverBar;}{M m ((x x))}))}^{22}}},,$

其中N是参考图像和浮动图像重叠区域内包含的像素数，F_i(x)和M_i(x)分别是参考图像和浮动图像第i个像素值，

和分别是参考图像和浮动图像区域内像素的均值。Where N is the number of pixels contained in the overlapping area of the reference image and the floating image, F _i (x) and M _i (x) are the i-th pixel value of the reference image and the floating image, respectively,

and are the mean values of the pixels in the reference image and floating image regions, respectively.

步骤5：运行全局二次多项式变换配准单元105。在步骤4中完成局部刚体配准后，可能存在错误的区域配准结果，在进行全局二次多项式变换前需要剔除错误的区域配准结果，在这里我们对局部刚体配准后的区域及对应的全局刚性配准参数，采用步骤3中第5步所描述的聚类分析方法，设置更精细的类内距离阈值(和图像大小相关，在我们的实验中取为20)，进行全局刚性变换参数空间上的聚类分析，选择聚类内数目最多的类作为精确匹配上的区域对。并以精确匹配上的区域中心点作为控制点进行全局二次多项式变换配准，实现两幅图像的精确配准。全局二次多项式变换配准的数学模型如下：Step 5: Run the global quadratic polynomial transformation registration unit 105 . After the local rigid body registration is completed in step 4, there may be wrong regional registration results. Before the global quadratic polynomial transformation, the wrong regional registration results need to be eliminated. The global rigid registration parameters of , using the cluster analysis method described in step 5 in step 3, set a finer intra-class distance threshold (related to the image size, set as 20 in our experiment), and perform global rigid transformation Cluster analysis on the parameter space, select the cluster with the largest number of clusters as the region pair on the exact match. And the center point of the area on the precise matching is used as the control point to carry out the global quadratic polynomial transformation registration to realize the precise registration of the two images. The mathematical model of global quadratic polynomial transformation registration is as follows:

X_C＝A·B，X _C =A·B,

$A A = = [\begin{matrix} {a a}_{0000} & {a a}_{1010} & {a a}_{0101} & {a a}_{1111} & {a a}_{2020} & {a a}_{0202} \\ {b b}_{0000} & {b b}_{1010} & {b b}_{0101} & {b b}_{1111} & {b b}_{2020} & {b b}_{0202} \end{matrix}],,$

$B B = = {[\begin{matrix} 11 & {x x}_{D D.} & {y the y}_{D D.} & {x x}_{D D.} {y the y}_{D D.} & {x x}_{D D.}^{22} & {y the y}_{D D.}^{22} \end{matrix}]}^{T T},,$

X_D＝[x_D，y_D]^T是浮动图像中的点坐标，X_C＝[x_C，y_C]^T是参考图像中的点坐标，A是二次多项式变换矩阵，A可以通过以下方法进行求解，其中K是控制点个数：X _D =[x _D , y _D ] ^T is the point coordinates in the floating image, X _C =[x _C , y _C ] ^T is the point coordinates in the reference image, A is the quadratic polynomial transformation matrix, and A can pass the following method to solve, where K is the number of control points:

$R R = = [\begin{matrix} {x x}_{C C 11} & {x x}_{C C 22} & \cdot \cdot \cdot \cdot \cdot \cdot & {x x}_{CK CK} \\ {y the y}_{C C 11} & {y the y}_{C C 22} & \cdot \cdot \cdot \cdot \cdot \cdot & {y the y}_{CK CK} \end{matrix}],,$

$D D. = = [\begin{matrix} 11 & 11 & \cdot \cdot \cdot \cdot \cdot \cdot & 11 \\ {x x}_{D D. 11} & {x x}_{D D. 22} & \cdot \cdot \cdot \cdot \cdot \cdot & {x x}_{DK DK} \\ {y the y}_{D D. 11} & {y the y}_{D D. 22} & \cdot \cdot \cdot \cdot \cdot &Center Dot; & {y the y}_{DK DK} \\ {x x}_{D D. 11} {y the y}_{D D. 11} & {x x}_{D D. 22} {y the y}_{D D. 22} & \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; & {x x}_{DK DK} {y the y}_{DK DK} \\ {x x}_{D D. 11}^{22} & {x x}_{D D. 22}^{22} & \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; & {x x}_{DK DK}^{22} \\ {y the y}_{D D. 11}^{22} & {y the y}_{D D. 22}^{22} & \cdot \cdot \cdot &Center Dot; \cdot \cdot & {y the y}_{DK DK}^{22} \end{matrix}],,$

A＝RD^T(DD^T)^-1。A = RD ^T (DD ^T ) ^-1 .

运行结果：operation result:

为了验证本发明方法，我们选取了20对图像作为实验样本，其中包含5对低质量的图像，实验结果表明，本算法可以在4s左右完成一般图像的配准，低质量的图像配准时间会长一些，在10s左右也可完成精确配准，对1548×1260的图像配准精度可以达到2个像素。具体的配准结果如图5，图6所示。实验表明，我们的方法是快速，精确，鲁棒的，具有巨大的应用价值。In order to verify the method of the present invention, we selected 20 pairs of images as experimental samples, including 5 pairs of low-quality images. The experimental results show that this algorithm can complete the registration of general images in about 4s, and the registration time of low-quality images will be slower. If it is longer, accurate registration can be completed in about 10s, and the registration accuracy of 1548×1260 images can reach 2 pixels. The specific registration results are shown in Figure 5 and Figure 6. Experiments show that our method is fast, accurate, robust and has great application value.

以上所述，仅为本发明中的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉该技术的人在本发明所揭露的技术范围内，可理解想到的变换或替换，都应涵盖在本发明的包含范围之内，因此，本发明的保护范围应该以权利要求书的保护范围为准。The above is only a specific implementation mode in the present invention, but the scope of protection of the present invention is not limited thereto. Anyone familiar with the technology can understand the conceivable transformation or replacement within the technical scope disclosed in the present invention. All should be covered within the scope of the present invention, therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. based on the visible light image registration method of salient region, it is characterized in that, may further comprise the steps:

Step 1: load two images subject to registration on computers, select a width of cloth as the reference image, another width of cloth is as floating image;

Step 2: reference picture and floating image are divided into M * N rectangular area, calculate the local significance function Ls (R) of each region R; Local significance function Ls (R) is carried out Gauss curve fitting, the local significance function value Fls (R) after the calculating match; The center in the local extremum zone of selection Fls (R) is as the salient region center; To each salient region center,, extract the salient region R of reference picture and floating image according to the radius in the Fls in the neighborhood (R) Distribution calculation zone;

Step 3: the yardstick extraneous features descriptor Lfd (R) that the salient region R that extracts is made up one 72 dimension; Defining a distance metric function is Dist (Lfd (R ₁), Lfd (R ₂)), weigh two sub-Lfd (R of feature description ₁), Lfd (R ₂) between similarity; To arbitrary salient region of two width of cloth images to C (i, j), (i, regional matching similarity j) adopt by thick matching strategy to essence, carry out the coupling of salient region to calculate C;

Step 4: (i j), adopts the similarity measurement based on normalized correlation coefficient to carry out local rigid body registration to Cmp to the salient region on the preliminary coupling;

Step 5: to the zone behind the local rigid body registration, adopt the method for cluster analysis, extract the regional center point that accurately the match is successful and carry out overall quadratic polynomial conversion registration, realize the accurate registration of two width of cloth images as the reference mark.

2. method according to claim 1 is characterized in that, described local significance function Ls (R) is: Ls (R)=Av (R) Lge (R), wherein:

Av (R) is that the normalization area differentiation function representation of region R is Av (R)=σ/μ, and σ is the standard deviation of region R in the formula, and μ is the average of region R;

Lge (R) is that the gradient fields entropy of region R is expressed as follows:

Lge (R) = - Σ_{i = 1}^{36} p_{i} (R) \log_{2} p_{i} (R),

In the formula: p _i(R) be that the pixel point set that gradient direction is arranged in i sector region is expressed as follows in the gradient magnitude ratio that region R accounts for:

p_{i} (R) = \frac{{&Integral;}_{R_{i}} | g (X_{i}) | {dX}_{i}}{{&Integral;}_{R} | g (X) | dX},

R in the formula _iBe to be positioned at the point set that the pixel of i sector region constitutes by all gradient directions, Be pixel X gradient magnitude, X _iBe point set R _iIn pixel.

3. method according to claim 1 is characterized in that, local significance function value Fls (R) after the described match is calculated as:

Fls (R_{ab}) = Σ_{i = 1}^{M} Σ_{j = 1}^{N} \frac{Ls (R_{ij})}{{2 πσ}^{2}} \exp^{- ({(a - i)}^{2} + {(b - j)}^{2}) / ({2 σ}^{2})},

In the formula: the gaussian kernel function with σ=1.5 carries out match, and M is the rectangular area number that image is divided on directions X, and N is the rectangular area number that image is divided on the Y direction, (a, i) ∈ { 1,2, ..., M} is region R coordinate on the directions X in M * N rectangular area array, (b, j) ∈ { 1,2 ..., N} is region R coordinate on the Y direction in M * N rectangular area array.

4. method according to claim 1 is characterized in that, the calculating of the zone radius at described each salient region center comprises:

With rectangular area, place, salient region center R _AbBe the center, make up the square rectangular area set omega of a radius maximum, Ω must meet the following conditions:

Fls(R _ij)≥λ·Fls(R _ab)，

&ForAll; R_{ij} &Element; Ω,

In the formula: Fls (R _Ij) be region R _IjLocal significance function value after the match, λ is a rectangular area radius controlled variable, the experience value is 0.75; The length of selection Ω and the smaller value in the width are as the salient region radius.

5. method according to claim 1 is characterized in that, the yardstick extraneous features descriptor of one 72 dimension of described structure is:

Lfd (R)=(p ₁(R) ... p ₃₆(R), da ₁(R) ... da ₃₆(R)), p in the formula _i(R) be that gradient direction is arranged in the gradient magnitude ratio that the pixel point set of i sector region accounts in region R, da _i(R) ∈ [0,2 π) be the deflection of the geometric center of the gradient direction pixel point set that is positioned at i sector region to the salient region center; Define a distance metric function Dist (Lfd (R then ₁), Lfd (R ₂)), weigh two sub-Lfd (R of feature description ₁), Lfd (R ₂) between similarity be:

Dist (Lfd (R_{1}), Lfd (R_{2})) = Σ_{i = 1}^{36} {Eud {({da}_{i} (R_{1}), {da}_{i} (R_{2}))}^{2} \cdot Max (p_{i} (R_{1}), p_{i} (R_{2}))}

\cdot Σ_{i = 1}^{36} {\log \frac{Max (p_{i} (R_{1}), p_{i} (R_{2})}{Min (p_{i} (R_{1}), p_{i} (R_{2})} \cdot Max (p_{i} (R_{1}), p_{i} (R_{2}))},

Eud (da in the formula _i(R ₁), da _i(R ₂)) be R ₁And R ₂In angle between corresponding i deflection.

6. method according to claim 1 is characterized in that, the coupling of described salient region comprises:

1) (i, j), wherein i represents i salient region R of reference picture to each possible salient region coupling C of traversal two width of cloth images _i, j represents j salient region R of floating image _jThe C that meets the following conditions (i, j) then think on the thick coupling salient region to Cmp (i, j):

\frac{Min (Av (R_{i}), Av (R_{j}))}{Max (Av (R_{i}), Av (R_{j}))} \cdot \frac{Min (Lge (R_{i}), Lge (R_{j}))}{Max (Lge (R_{i}), Lge (R_{j}))} > T,

Min () function is to minimize in the formula, and Max () function is a maximizing, and T slightly mates controlled variable, and the experience value is 0.6;

2) (i j), calculates R as follows to Cmp to each thick matching area _iAnd R _jBetween similarity S (i, j) and anglec of rotation θ _Ij:

θ_{ij} = \frac{2 kπ}{36},

S(i，j)＝Dist(Lfd(R _i)，Lfd(R _jk))，

k＝arg _kMin(Dist(Lfd(R _i)，Lfd(R _jk)))，k∈{0，1，...35}，

In the formula, R _jK is with R _jBe rotated counterclockwise the new region that the 10k degree obtains;

(i j) determines that three global rigid transformation parameters are: two-dimension translational and rotation parameter θ to each thick matching area to Cmp _Ij, two-dimension translational is by R _iAnd R _jRegional center decision;

3) press S (i, j) ascending order is arranged all Cmp (i, j), (i is j) as the input sample set to Cmp to choose preceding 2000 thick matching areas, if the thick matching area that extracts is to number deficiency then be as the criterion with the number of reality, distance threshold in the suitable class is set, and (i j) carries out cluster on the global rigid transformation parameter space to Cmp, choose number is maximum in the cluster class as the salient region on the preliminary coupling to F (i, j); (i, size j) is rejected the zone of repetition, guarantees that (i does not comprise the zone of repetition in j), reduces follow-up calculated amount to F for salient region on the preliminary coupling to press S.

7. method according to claim 1 is characterized in that, described local rigid body registration be to the salient region on the preliminary coupling to F (i, j), with salient region R _i, R _jCenter and anglec of rotation θ _IjAs the initial registration parameter, carry out local rigid body registration.

8. method according to claim 1, it is characterized in that the accurate registration of described realization two width of cloth images comprises:, distance threshold in the meticulousr class is set to the zone behind the local rigid body registration, carry out the cluster analysis on the global rigid transformation parameter space, extract accurately mate regional right; And carry out overall quadratic polynomial conversion registration as the reference mark with the regional center point on the accurate coupling, realize the accurate registration of two width of cloth images.