CN100525452C - Frequency domain fast sub picture element global motion estimating method for image stability - Google Patents

Abstract

The invention discloses a frequency-domain fast sub-pixel global motion estimation method for image stabilization. In this method, first calculate the cross-power spectrum function between two adjacent frames of images in the video sequence, perform inverse Fourier transform on the cross-power spectrum function, generate a motion distribution function, search for its maximum peak point to obtain the global motion vector of the whole pixel; judge and Recording several peaks adjacent to the coordinates of the maximum peak point in the row and column directions of the motion distribution function, and calculating the sub-pixel global motion vector according to the correspondence between the sub-pixel global motion and the distribution of the primary and secondary peaks in the motion distribution function; Approximate processing is performed on the calculated sub-pixel global motion vector to obtain a sub-pixel global motion vector with a specified precision. The invention avoids the image interpolation process of the traditional method, not only greatly reduces the calculation burden, but also improves the precision of global motion estimation, and can effectively support accurate and real-time image stabilization processing.

Description

Frequency-Domain Fast Sub-Pixel Global Motion Estimation Method for Image Stabilization

technical field

The present invention relates to a frequency-domain fast sub-pixel global motion estimation method for image stabilization, in particular to a method capable of reducing calculation and storage costs of sub-pixel global motion estimation under frequency domain conditions and improving sub-pixel global motion estimation A method for improving accuracy and image stability belongs to the technical field of image processing.

Background technique

Global motion estimation technology can be used to obtain the overall motion between two adjacent frames in a video sequence. It is the most critical technical link in the process of image stabilization. Global motion estimation affects the performance of image stabilization systems from the following two aspects. First, the accuracy and precision of global motion estimation directly determine the image stabilization quality of the system. Secondly, since the calculation amount of the global motion estimation part generally accounts for more than 90% of the calculation amount of the system, the speed of the global motion estimation determines the real-time processing capability of the image stabilization system.

Referring to FIG. 1 , the global motion estimation consists of two parts: integer-pixel global motion estimation and sub-pixel global motion estimation. Integer-pixel global motion estimation is based on directly sampling images, and estimates the integer-pixel-level global motion vectors between images; sub-pixel global motion estimation first estimates the non-direct sampling points (sub-pixel points) in the image through interpolation. Based on this value, the sub-pixel global motion vector is further estimated on the basis of the known integer-pixel global motion vector. Relevant experiments show that sub-pixel global motion estimation can more accurately describe the global motion between images and effectively improve the effect of motion compensation. Therefore, sub-pixel global motion estimation has become a research hotspot in many application fields.

Sub-pixel global motion estimation can be divided into two implementations in time domain and frequency domain. In the Chinese invention patent application "A fast sub-pixel motion estimation method" with the application number 200410073837.5 and the filing date of September 3, 2004, a fast sub-pixel motion estimation method based on time domain is disclosed. Compared with the time-domain implementation method, the global motion estimation method based on the frequency domain has the advantages of wide application range and strong anti-interference ability, so it is more suitable for image stabilization applications. However, in actual implementation, the global motion estimation method in the frequency domain needs to obtain the global motion between images in the spatial domain by comparing the phase relationship between the frequency spectra of two adjacent frames of images in the video sequence based on the translation characteristics of Fourier transform. In this process, there is a Fourier transform process between the spatial domain and the frequency domain, and the calculation amount increases rapidly with the change of the image size. For example, when performing sub-pixel global motion estimation with 1/2 ^k pixel precision, the size of the interpolated image becomes 4 ^k times that of the original image, and the corresponding storage cost will increase by 4 ^k -1 times, while the calculation cost when using discrete Fourier transform It is close to 16 ^k times of the whole pixel case, even if the fast Fourier transform algorithm is used, the calculation amount will reach 8 ^k times of the original calculation amount. For the frequency-domain global motion estimation method, this sub-pixel expansion method not only increases the computational complexity, but also increases the storage cost exponentially with the increase of sub-pixel precision, so it is difficult to meet the applications with limited storage and computing resources. actual needs of the environment. On the other hand, when implementing the global motion estimation method based on the frequency domain, the interference of the image edge content change on the global motion estimation in the frequency domain is obvious, so it must be overcome in a targeted manner.

Contents of the invention

The primary purpose of the present invention is to provide a frequency-domain fast sub-pixel global motion estimation method for image stabilization. This method can use the frequency-domain phase change information generated by the image sub-pixel global motion, and obtain the image sub-pixel global motion vector through direct calculation without image interpolation.

Another object of the present invention is to provide a method capable of effectively suppressing interference caused by image edge content changes to frequency-domain global motion estimation. The method can improve the accuracy of frequency-domain global motion estimation, especially the accuracy of frequency-domain sub-pixel global motion estimation.

In order to achieve the above object, the present invention adopts the following technical solutions.

A frequency-domain fast sub-pixel global motion estimation method for image stabilization, characterized in that it comprises the following steps:

(1) Calculate the original cross-power spectrum of two adjacent frames of images in the video sequence;

(2) Obtain the maximum interference coefficient corresponding to the row and column direction of each frequency point under the specified accuracy range, and then multiply the amplitude of each frequency point in the image spectrum of the previous frame by the maximum interference coefficient corresponding to the row and column direction respectively, Get the size of the maximum interference component that each frequency point can overcome under the specified accuracy range, compare the minimum value of the maximum interference component at each frequency point with the empirical threshold, and judge whether the frequency point is a frequency point that is susceptible to interference;

(3) Suppress the frequency points that are judged to be susceptible to interference, and obtain the cross-power spectrum after interference suppression;

(4) carry out inverse Fourier transform to the cross power spectrum after described interference suppression, generate motion distribution function, search the maximum peak value in the motion distribution function, obtain the whole pixel global motion vector according to the coordinate of maximum peak point;

(5) Judging and recording several peaks adjacent to the coordinates of the maximum peak point in the row and column directions in the motion distribution function, and calculating the sub-pixel global motion based on the corresponding relationship between the sub-pixel global motion and the distribution of the main and secondary peaks in the motion distribution function Pixel global motion vector;

(6) Perform approximate processing on the calculated sub-pixel global motion vector to obtain a sub-pixel global motion vector with a specified precision.

Wherein, the size of the empirical threshold is positively correlated with the richness of texture details of the image, and is between 200 and 400.

Described step (5) is divided into two steps:

First, according to the maximum peak value searched in step (4), record the function values of the three coordinate points closest to the maximum peak point in the horizontal direction and the vertical direction respectively;

Secondly, according to the obtained maximum peak value and the recorded function values of the three coordinate points closest to the maximum peak point in the horizontal and vertical directions, respectively calculate the corresponding sub-pixel global motion vector with arbitrary precision.

The frequency-domain fast sub-pixel global motion estimation method for image stabilization provided by the present invention can effectively reduce the calculation and storage overhead of traditional frequency-domain sub-pixel global motion estimation, and has better interference suppression for image edge content changes As a result, the accuracy of sub-pixel global motion estimation in the frequency domain is improved. Relevant test results show that this method can achieve good results for sub-pixel global motion estimation of various scene images in image stabilization.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

FIG. 1 is a schematic diagram of pixel distribution of two-dimensional sub-pixel global motion;

Fig. 2 is the flow chart of the frequency-domain fast sub-pixel global motion estimation method for image stabilization provided by the present invention;

FIG. 3 is a schematic diagram of the peak value distribution of the motion distribution function during one-dimensional sub-pixel global motion.

Detailed ways

In the frequency domain, a good sub-pixel global motion estimation method must simultaneously consider the impact of motion estimation accuracy, accuracy, operating efficiency and storage overhead, and at the same time be able to effectively overcome the adverse effects of image edge content changes. Therefore, the solution idea of the present invention is: according to the corresponding relationship between the sub-pixel global motion in the spatial domain image and the frequency spectrum change in the frequency domain, on the basis of the global motion estimation of the whole pixel in the frequency domain, use the intermediate results to obtain directly by calculation The sub-pixel global motion vector realizes fast and accurate sub-pixel global motion estimation in the frequency domain.

The specific implementation of the present invention will be described below in conjunction with the accompanying drawings. Fig. 2 is a flow chart of the frequency-domain fast sub-pixel global motion estimation method for image stabilization provided by the present invention. The method includes the following steps: First, calculate the cross-power spectrum function S ₀ between two adjacent frames of images in the video sequence, and suppress the interference of the cross-power spectrum function according to the form of interference caused by the change of the edge content of the image, and obtain Cross-power spectral function S ₁ containing more accurate global motion information. Perform inverse Fourier transform on the newly obtained cross-power spectrum function _S1 to generate motion distribution function I, search for its maximum peak point to obtain the global motion vector of the whole pixel; secondly, judge and record the relationship between the row (column) direction and For several secondary peaks (preferably three times) adjacent to the coordinates of the maximum peak point, the sub-pixel global motion vector can be calculated according to the correspondence between the sub-pixel global motion and the distribution of the primary and secondary peaks in the motion distribution function; finally, the calculated The sub-pixel global motion vector is approximated, and the sub-pixel global motion vector under the specified precision is obtained.

The above-mentioned step of obtaining the global motion vector of an entire pixel specifically includes the following three sub-steps: first, calculate the original cross-power spectrum of two adjacent frames of images in the video sequence according to the definition of the cross-power spectrum function; The information of different frequency points in the cross power spectrum is screened and suppressed to obtain a cross power spectrum with higher reliability; finally, the inverse Fourier transform is performed on the new cross power spectrum obtained after interference suppression to generate a motion distribution function and search for motion The maximum peak in the distribution function, according to the coordinates of the maximum peak point to get the whole pixel global motion vector.

The specific calculation steps are as follows:

(1) Calculation of the original cross power spectrum

The cross power spectrum characterizes the mutual relationship between the frequency spectrum of two frames of images. In theory, each frequency point in the cross-power spectrum contains the global motion information between two frames of images. When calculating the cross-power spectrum, first use formula (1) and formula (2) to calculate the spectrum F ₁ (u, v) and F ₂ (u, v) of the two images to be estimated, and then calculate according to formula (3) Determine the original cross-power spectral function S ₀ (u,v).

F ₁ (u, v) = F(f ₁ (x, y)) (1)

F ₂ (u, v) = F(f ₂ (x, y)) (2)

In formulas (1) and (2), f ₁ (x, y) and f ₂ (x, y) represent two frames of images to be estimated respectively, while F ₁ (u, v) and F ₂ (u, v) represent two The frequency spectrum corresponding to the frame image to be estimated, and the symbol F represents Fourier transform. In order to improve the speed of global motion estimation in practical applications, fast Fourier transform is usually used instead of ordinary discrete Fourier transform, and the image size is taken as an integer power of 2.

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

In formula (3), S ₀ (u, v) is the original cross-power spectrum between two frames of images to be estimated.

(2) Interference suppression of cross power spectrum

Ideally, the global motion information contained in each frequency point in the cross-power spectrum should be accurate, but in reality, due to the influence of factors such as image edge content changes, the information in the cross-power spectrum will be distorted. The reliability of the information in the cross-power spectrum determines the accuracy of the global motion estimation in the frequency domain. Therefore, suppressing the interference caused by the above factors on the cross-power spectrum and improving the reliability of the information in the cross-power spectrum is the key to achieving accurate global motion estimation in the frequency domain. key.

Different frequency points are affected to different extents by changes in image edge content. In order to improve the accuracy of global motion estimation, the present invention evaluates the accuracy of the information of each frequency point in the cross-power spectrum, and suppresses the information that is susceptible to interference according to the specified accuracy range. A new cross-power spectrum containing more accurate global motion information is obtained.

When evaluating the information of each frequency point in the cross-power spectrum, the present invention first uses formulas (4) and (5) to respectively obtain the maximum interference coefficient R ₁ (u) corresponding to the row and column directions of each frequency point under the specified accuracy range , R ₂ (v), this coefficient describes the ratio between the modulus value of the image spectrum and the interference component when the interference effect caused by the change of the image content is the most significant; then use formula (6), (7) to The amplitude of each frequency point in the image spectrum of the previous frame is multiplied by the maximum interference coefficient corresponding to the row and column direction, and the maximum interference component size V ₁ (u), V ₂ (v); Finally, use formula (8) to compare the minimum value of V ₁ (u) and V ₂ (v) at each frequency point with an empirical threshold to determine whether the frequency point is susceptible to interference. Frequency points judged to be susceptible to interference need to be suppressed.

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; sin</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;u</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; sin</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;v</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

表示指定精度范围(如1/2像素，1/4像素，1/8像素)。In formulas (4) and (5), m and n represent the number of rows and columns of the image, u and v represent the frequency values in the row and column direction respectively,

Indicates the specified precision range (such as 1/2 pixel, 1/4 pixel, 1/8 pixel).

V ₁ (u)=|F ₁ (u, v)|·R ₁ (u) (6)

V ₂ (v)=|F ₁ (u, v)|·R ₂ (v) (7)

In formulas (6) and (7), |F ₁ (u, v)| represents the modulus value of the spectrum of the previous image at the frequency point (u, v).

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; if</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="Threshold"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">Threshold</figure-callout></span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; if</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; Threshold</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

The constant Threshold in formula (8) is an empirical threshold, which indicates the size of the interference component caused by the content change of the edge of the image. Through the interference suppression experiment under the condition of a large number of different image content changes, the interference suppression effect is better when the threshold value Threshold is between 200 and 400. For images with rich texture details, a slightly larger threshold can be selected, while for images with less texture details, a smaller threshold is better.

Compared with the existing frequency-domain integer-pixel global motion estimation technology, the above-mentioned technical solution can effectively overcome the influence of the change of image edge content on the accuracy of global motion estimation, especially the adverse effect on the accuracy of subsequent sub-pixel global motion estimation. Influence.

(3) Integer pixel global motion vector acquisition

A new cross power spectrum is obtained after interference suppression is performed on the original cross power spectrum function above, and then an inverse Fourier transform is performed on the new cross power spectrum to obtain a motion distribution function I. The maximum peak point P ₀ in the motion distribution function is searched, and the whole pixel global motion vector is obtained according to its coordinate position. Equation (9) is the calculation formula of the integer pixel global motion vector (m ₁ , m ₂ ).

(m ₁ , m ₂ )=arg max(I(x,y)) (9)

The sub-steps of calculating the sub-pixel global motion vector at arbitrary precision are further described below. Different from the traditional sub-pixel global motion estimation technology in the frequency domain, in the present invention, in order to solve the problem of a surge in calculation and storage burden caused by the traditional method, the sub-pixel global motion vector calculation method under arbitrary precision is divided into two steps: first , according to the maximum peak point P ₀ searched in the previous step, respectively record the function values of the three coordinate points adjacent to the coordinate position of the peak point in the row and column directions; secondly, according to the obtained maximum peak value and the recorded For several peaks adjacent to the maximum peak horizontally and vertically, their corresponding sub-pixel global motion vectors with arbitrary precision are calculated respectively.

The specific implementation steps are as follows:

(1) Record the secondary peak value

Referring to FIG. 3 , let the coordinates of the maximum peak point P ₀ in the motion distribution function I be (x ₀ , y ₀ ). Record the function values of the three nearest coordinate points on the left and right sides of the maximum peak point P ₀ , denoted as P ₁ , P ₂ , and P ₃ respectively; record the three most adjacent coordinate points on the upper and lower sides of the maximum peak point P ₀ The function values of the coordinate points are recorded as P ₄ , P ₅ , and P ₆ respectively. Equations (10), (11), (12), and (13) are the selection formulas for the secondary peak values in the row and column directions above.

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="P"\; filenames="C200710099576E00121.png,C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="3"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">y</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}\right.& \mathrm{<span\; class="notranslate">\; if\; i</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">y</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="P"\; filenames="C200710099576E00121.png,C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">y</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">y</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="3"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}\right.& \mathrm{<span\; class="notranslate">\; if\; i</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">y</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="5"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\end{array}\right.& \mathrm{<span\; class="notranslate">\; if\; i</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="5"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\end{array}\right.& \mathrm{<span\; class="notranslate">\; if\; i</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

(2) Sub-pixel global motion vector calculation

According to the obtained maximum peak P ₀ and the recorded secondary peaks P ₁ , P ₂ , P ₃ and P ₄ , P ₅ , P ₆ adjacent to the maximum peak horizontally and vertically, using equations (14), (15 ) Calculate sub-pixel global motion vectors m ₁ and m ₂ in row and column directions.

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="m"\; filenames="C200710099576E00121.png,C200710099576E00131.png"\; state="\{\{state\}\}">m</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="3"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="P"\; filenames="C200710099576E00121.png,C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 14</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="m"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">m</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}{{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="5"\; label="P"\; filenames="C200710099576E00131.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 6</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 15</span>}<span\; class="notranslate">\; )</span>$

It should be noted that in the above formula for calculating the sub-pixel global motion vector, the sub-peaks used are three adjacent sub-peaks. This is derived under the condition of the Catmull-Rom third-order interpolation method by comparing the corresponding relationship between the sub-pixel global motion of the image and the spectral change. Catmull-Rom third-order interpolation is a more commonly used sub-pixel image interpolation method, which can better express the change of image texture. However, other well-established interpolation methods may also be used. On the premise of using other interpolation methods, the selected secondary peak may also be other secondary peaks.

The above-mentioned sub-pixel global motion vector calculation method can directly obtain the sub-pixel global motion vector between images by using the intermediate results on the basis of frequency-domain integer-pixel global motion estimation, avoiding the calculation and calculation caused by image interpolation. Increased storage burden.

The following describes in detail how to obtain the sub-pixel global motion vector at a specified precision.

The previous calculation steps have calculated the sub-pixel global motion vector with arbitrary precision, but in practical applications, the sub-pixel precision is not as high as possible, and factors such as application effect and calculation cost need to be considered comprehensively. Generally, the sub-pixel accuracy commonly used is 1/2, 1/4 or 1/8 pixel. Therefore, in order to meet the requirements of practical applications, it is necessary to perform approximate processing on the calculated sub-pixel global motion vector with arbitrary precision. When the specified sub-pixel precision is 1/2 ^K , the sub-pixel global motion vector with the required precision can be calculated according to equations (16) and (17).

m ₁ ′=round(m ₁ ×2 ^K )/2 ^K (16)

m ₂ ′=round(m ₂ ×2 ^K )/2 ^K (17)

Where round means rounding up. From this, a sub-pixel global motion vector at a specified precision is obtained.

The above disclosures are only specific examples of the present invention. According to the idea provided by the present invention, changes conceivable by those skilled in the art should fall within the protection scope of the present invention.

Claims (3)

1. a frequency domain fast sub picture element global motion estimating method that is used for image stabilization is characterized in that comprising the steps:

(1) the original crosspower spectrum of adjacent two two field pictures in the calculating video sequence;

(2) the maximum interference coefficient of acquisition each Frequency point row, column direction correspondence under the designated precision scope, then that the amplitude of each Frequency point in the former frame image spectrum is corresponding with the row, column direction respectively maximum interference multiplication, obtain the size of the maximum interference component that each Frequency point can overcome under the designated precision scope, minimum value and empirical value in the maximum interference component of each Frequency point are compared, judge whether this Frequency point is the Frequency point that is subject to disturb;

(3) suppress for being judged as the Frequency point that is subject to disturb, obtain to disturb the crosspower spectrum after suppressing;

(4) crosspower spectrum after described interference is suppressed carries out inverse-Fourier transform, generates the distribution of movement function, and the peak-peak in the searching moving distribution function obtains whole picture element global motion vector according to the coordinate of maximal peak point;

(5) judge and note in the described distribution of movement function several times peak value adjacent on the row, column direction, calculate the sub picture element global motion vector according to the corresponding relation of primary and secondary peak Distribution in sub picture element global motion and the distribution of movement function with the peak-peak point coordinates;

(6) the sub picture element global motion vector that calculates is carried out approximate processing, obtain the sub picture element global motion vector under the designated precision.

2. the frequency domain fast sub picture element global motion estimating method that is used for image stabilization as claimed in claim 1 is characterized in that:

It is relevant that the size of described empirical value and the grain details of image are enriched the degree forward, between 200～400.

3. the frequency domain fast sub picture element global motion estimating method that is used for image stabilization as claimed in claim 1 is characterized in that:

Described step (5) is divided into two steps:

At first, according to the peak-peak that search in the step (4) obtains, note respectively in the horizontal direction with vertical direction on the functional value of the most contiguous three coordinate points of maximal peak point;

Secondly, according to the peak-peak that obtains and note in the horizontal direction with vertical direction on the functional value of the most contiguous three coordinate points of maximal peak point, calculate the sub picture element global motion vector under the corresponding arbitrary accuracy respectively.

Images