Proceedings of Spie: An Improved Non-Rigid Registration Algorithm For SAR Image Based On Thin Plate Spline
Proceedings of Spie: An Improved Non-Rigid Registration Algorithm For SAR Image Based On Thin Plate Spline
Proceedings of Spie: An Improved Non-Rigid Registration Algorithm For SAR Image Based On Thin Plate Spline
SPIEDigitalLibrary.org/conference-proceedings-of-spie
Jiqing Zhao, Yun Gong, "An improved non-rigid registration algorithm for SAR
image based on thin plate spline," Proc. SPIE 11373, Eleventh International
Conference on Graphics and Image Processing (ICGIP 2019), 1137316 (3
January 2020); doi: 10.1117/12.2557330
ABSTRACT
SAR image registration is a key step in SAR data processing, and it plays an important role in change detection,
image mosaic fusion and target recognition. The traditional rigid registration model can only realize the rotation and
translation transformation of a whole image. However, for SAR images with coherent speckle noise and complex
deformation, the local transformation result can’t meet the accuracy requirement of the later data processing. In this
paper, the existing SAR image registration method is improved. After finishing SAR image feature matching by means
of SURF and RANSAC algorithms, we construct an image conversion model with thin plate spline, which can
decompose the space transformation of SAR image into global affine transformation and local non affine transformation.
In this way, the whole and local mapping transformation of images can be realized simultaneously without the local
distortion. The experimental results show that the improved method can achieve better transformation and obtain higher
registration accuracy in the whole and local processing of SAR images.
Keywords: SURF, RANSAC, SAR Image, Non-rigid Registration, Thin Plate Spline.
1. INTRODUCTION
Synthetic Aperture Radar (SAR) is a kind of high resolution microwave remote sensing imaging radar, which can't
be affected by weather and climate. SAR imaging can make up for the defects of infrared imaging and visible light
imaging, so SAR images have a wide application prospect. Image fusion is one of the most effective ways to make full
use of the advantages of SAR images. Image registration is the basis of image fusion, the key step of processing SAR
data, and the basis of SAR image change detection and extraction of interested targets, which has important research
significance [1].
At present, scholars at home and abroad have done a lot of research on SAR image registration methods, and put
forward a number of successful methods. However, these methods can only achieve better registration results under
certain specific conditions, and have certain local limitations. SAR image registration can be divided into gray-based
registration method, transform domain-based registration method and feature-based registration method [2]. In gray-
level-based registration method, Lixia Shu et al. firstly de-noised SAR image, and then used mutual information to
achieve SAR image registration [3]. Loeckx et al. proposed a registration method based on conditional mutual
information [4]. This algorithm introduces the spatial relation of adjacent pixels into the joint histogram, constructs a
new similarity matching criterion, and obtains the best registration result. Xiao et al proposed a method based on
Reyleign mutual information to realize SAR image registration with the same origin [5]. However, this method is very
sensitive to noise, and there is coherent speckle noise in SAR images, so it has some limitations in SAR image
registration. In the transformation domain-based registration method, Yuesong Lin et al. proposed a phase correlation
registration algorithm based on scale estimation for SAR image [6]; Dengsong Xin et al. proposed an improved phase-
dependent registration method, applied to SAR image registration with scaling, rotation and translation [7]. But, this
method is difficult to ensure the registration results of SAR images with rich texture and complex gray scale distribution.
However, the feature-based registration method can't be affected by the change of gray level. It is more suitable for the
registration of SAR images by obtaining the stable feature structure in the image. Chio et al. proposed an image
registration method based on maintaining local geometric consistency [8]. Xiufang Liu et al. applied SIFT algorithm to
feature extraction and automatic matching of multitemporal SAR images [9]. Jianxun Zhang et al. used PCA-SIFT
algorithm and Mahalanobis distance to complete the feature matching, then they used RANSAC to eliminate the
mismatching to complete the automatic registration of SAR images [10].The above research results verify the reliability
and robustness of the feature registration method.
Eleventh International Conference on Graphics and Image Processing (ICGIP 2019), edited by
Zhigeng Pan, Xun Wang, Proc. of SPIE Vol. 11373, 1137316 · © 2020 SPIE
CCC code: 0277-786X/20/$21 · doi: 10.1117/12.2557330
2. RESEARCH METHOD
2.1 SURF Feature Extraction Algorithm
The SURF algorithm is a stable feature extraction algorithm proposed by Bay et al. based on the SIFT algorithm,
which not only inherits the superiority of the SIFT algorithm, but also the calculation speed is faster than the former.
(1) Detection of extreme points and construction of scale space
The SURF algorithm performs the detection of extreme points by constructing a Hessian matrix. Since the feature
points in the image need to be scale-independent, the Gaussian filtering of the image is required before the Hessian
matrix is constructed. The Hessian matrix constructed by Gaussian filtering is expressed as formula (1). In formula
(1), 𝐿𝑥𝑥 (𝑥, 𝜎) is the second derivative of image 𝑔(𝜎) in the x direction after Gaussian filtering, and 𝐿𝑦𝑦 (𝑥, 𝜎), 𝐿𝑥𝑦 (𝑥, 𝜎)
is also the second derivative of 𝑔(𝜎). In order to calculate the three unknown matrix elements of H matrix, it is
necessary to use the convolution between specific kernels to calculate the second partial derivative, and use the second
order standard Gaussian function as the filter to calculate [14].
Bay et al. used the approximate values of 𝐿 in 𝑥, 𝑦 and 𝑥𝑦 directions to improve the operation speed of Gaussian
convolution, and recorded the approximate values in the three directions as 𝐷𝑥𝑥 , 𝐷𝑦𝑦 and 𝐷𝑥𝑦 . Then, the discriminant can
be expressed as formula (2).
Where 𝑙 is the weighting factor and the value is 0.9 to balance the error caused by the approximation. Whether the data is
an extreme point can be determined by the 𝐻 matrix discriminant calculation result.
SURF algorithm simplified the second-order Gaussian differential template by converting the Gaussian convolution
operation between the second-order Gaussian differential template and the image into a box filtering operation [15]. The
simplified template is shown in Fig. 1. Gaussian filtering is realized by the addition and subtraction of integral image in
the processed image. As long as all the pixels in the original image are scanned once, the integral image of the image can
be obtained. When building the image pyramid, the size of the box filter template is repeatedly enlarged to realize the
construction of linear scale space, and the speed of the establishment of scale space is also accelerated.
Building feature descriptors is the process of building feature vectors. Take the feature point as the center, rotate the
coordinate axis to the main direction, and divide the 20s*20s rectangular region into four sub-regions along the main
direction, and calculate its response value in each sub-region. The 4-dimensional strength structure description vector of
each sub-region can be obtained which expressed as formula (3).
The feature descriptors with good discrimination can be generated by the description vector of the four-dimensional
structure.
2.2 RANSAC Algorithm
The method uses the RANSAC algorithm to reject the wrong matching point pairs. First, the RANSAC algorithm
randomly selects several points as the inner group points in the matching point set, and calculates the appropriate
transformation matrix H, as shown in the following equation.
The matrix in the above equation is the model parameter sought, 𝑆 is the scale factor, (𝑥, 𝑦) and (𝑥1 , 𝑦1 ) are
corresponding points to two images. The entire data set is then filtered using the model parameters sought. Repeat the
above steps to get the model with the largest number of inner group points, that is, the corresponding value in formula (5)
is the smallest, then the model is optimal.
RANSAC calculates the error of the matching point after the positive and negative transformation of the
transformation matrix while eliminating the mismatched point, and uses the set threshold to eliminate the point with
larger error, and obtain the further refined registration point [17].
𝑓𝑥 (𝑥, 𝑦) is mapping function in 𝑥 direction, 𝑓𝑦 (𝑥, 𝑦) is a mapping function in 𝑦 direction. Every mapping function
includes a global transformation and a local transformation [14]. As shown in formula (7), take the mapping function in
the x direction as an example.
Where A(x, y) = 𝑃𝑚 (𝑥, 𝑦), 𝑅(𝑥, 𝑦) = ∑𝑛𝑖=1 𝐴𝑖 𝑔(𝑟𝑖 ).The function 𝐴(𝑥, 𝑦) is a global linear transformation and 𝑃𝑚 (𝑥, 𝑦) is
a linear polynomial. The function 𝑅(𝑥, 𝑦) is the weighted sum of the nonlinear function 𝑔(𝑟𝑖 ), where 𝑔(𝑟𝑖 ) takes the thin
plate spline transformation function, as shown in formula (8) .
Similarly, a mapping function in the 𝑦 direction can be obtained, so that the thin image spline function can be used to
transform from the image to be registered to the reference image. Thin plate spline registration can ensure the accurate
alignment of the registered image at the control point, and the offset of other regions can make the whole image have the
minimum bending energy [19, 20], so the registered image appears smoother.
3. EXPERIMENT
In this paper, the simulation experiment data is obtained from the relevant websites. The experiment data are divided
into two groups, one is the SAR image of a certain field, and the other is a lake data of a certain place. Considering the
limited experimental equipment, the image size is selected as 1000*1000. The number, position and angle of ground
objects contained in the main image and the auxiliary image of experimental data are different. As shown in Fig. 4.
Because SAR images have speckle noise, it can affect feature extraction. Therefore, the image is edge-detected
before registration, as shown in Fig.5 (a-d). On the basis of edge detection, SURF algorithm is used to extract feature
points quickly, Euclidean distance is used as feature similarity measure to complete feature matching, and then
RANSAC algorithm is used to eliminate wrong matching points on the whole. The tie-points after eliminating are
displayed on two groups of SAR images, as shown in Fig.5 (e-f). The exact matching point pairs are obtained, and then
the global and local conversion of SAR image is realized based on thin plate spline registration model. The results are
shown in Fig.5 (g-h).
(e) (f)
4. RESULT ANALYSIS
In this paper, the accuracy detection of the registered image is carried out. The Fig.6 is a new image from the local
region fusion of the two SAR images after registration. It can be seen from the natural degree and smooth degree of the
ground object that the registration accuracy is higher.
(a) (b)
Figure 6. Superimposed result of reference image and registered image
In order to further verify the registration accuracy, the feature points on the registered SAR image are quantitatively
examined. Feature points are evenly selected on the SAR images after registration, and 20 feature points are selected for
each group of data, as shown in the Table 1 and Table 2. According to the pixel coordinates of these feature points, the
pixel coordinate difference and registration accuracy of corresponding points can be obtained. The registration accuracy
of the two groups of images is 0.523 and 0.564, respectively. The results show that the registration accuracy of the two
sets of data reaches the sub-pixel level, and the image registration effect is better.
Table 1. Experimental data (1) checkpoint pixel coordinates (unit: pixel)
REFERENCES
[1] Ying X.: Survey of SAR Image Registration Methods. Geospatial Information, 63–66 (2013).
[2] Suqian L. Research on the Feature-based SAR Image Automatic Registration of SAR Im-ages. Doctor, National
University of Defense Technology (2017).
[3] Lixai Shu, Tieniu Tan.: SAR and SPOT Image Registration Based on Mutual Information with Contrast Measure. In
IEEE International Conference on image processing, pp. 429-432 (2007).
[4] LOECKK D, SLAGMOLEN P,MAES F, et al Nonrigid image registration using conditional mutual information.
IEEE Transactions on Medical Imaging,725-737(2007).
[5] Xiao X.: Bivariate Rayleigh Distribution and its Mutual Information for SAR Image Reg-istration. Chinese Journal
of Engineering Mathematics 28(2),157-164 (2011).
[6] Yuesong L.: A FFT-based SAR Image Registration Algorithm Via Priority Estimation of Scale Distortion. Modern
Radar 32(07), 39-44 (2010).
[7] Dengsong X.: A Modified Phase Correlation Registration Method for SAR Image. Com-puter & Digital Engineering
39(08), 126-128(2011).