CN1766928A - A kind of motion object center of gravity track extraction method based on the dynamic background sport video - Google Patents

Abstract

The invention discloses a method for extracting the center-of-gravity trajectory of a moving object based on a dynamic background moving video, which is used for extracting the center-of-gravity trajectory of a moving object in a background moving video; the method includes the following steps: performing global motion estimation on video frames , to find the global motion parameters; according to the global motion parameters, separate the foreground motion area of the moving object, and reconstruct the background of the moving image; realize background elimination, quickly segment out the moving object area; record motion history information, and obtain a complete moving object area; extract The outline of the moving object; find the trajectory of the center of gravity of the moving object. The invention has the advantages that it can be used for various types of moving objects and has good versatility; the two-dimensional center of gravity of the moving object can be obtained through simple and efficient two-dimensional calculation, and the real-time effect can be achieved.

Description

A method for extracting the trajectory of the center of gravity of moving objects based on dynamic background motion video

technical field

The invention relates to a method for extracting a track of a center of gravity of a moving object in a moving video, in particular to a method for extracting a track of a center of gravity of a moving object based on a dynamic background moving video.

Background technique

Using video to analyze the movement of moving objects is a hot issue in the fields of pattern recognition, virtual reality, and intelligent human-machine interface, and has great application value. For example, analyzing the movement of athletes in sports videos has important guiding significance for guiding the training of coaches and athletes and improving the level of training and competition.

Among them, the two-dimensional center of gravity trajectory of the moving object when performing a specific action is an extremely important parameter in motion analysis. If the two-dimensional center of gravity trajectory of the moving object during the movement can be obtained from the video, it can be used for judging the motion of the moving object. The quality of the movement provides a strong basis. For example, if it is applied to sports videos, it can provide an effective means to guide athletes' training, and it is of great value in improving the quality of athletes' movement, training level and competitive level.

The so-called dynamic background motion video refers to a video that includes camera movement in a video image sequence, therefore, the scene background of the dynamic background motion video changes. For example, in diving, gymnastics and other sports videos, the camera position will move with the movement of the moving object, which is the so-called dynamic background motion video.

Corresponding to the dynamic background motion video is the static background motion video, which refers to a video in which the scene background does not change in the video image sequence. For example, in a trampoline video, the camera is stationary, and the scene background is also static, so the trampoline video belongs to the static background video.

If the trajectory of the center of gravity of the moving object in the video can be extracted during the movement, it is helpful to analyze the movement of the moving object. Compared with the method of extracting the trajectory of the center of gravity of moving objects based on static background moving video, the method of extracting the trajectory of the center of gravity of moving objects based on dynamic background video undoubtedly has a wider range of applications and has greater use value and practical significance.

However, in the existing video processing software at home and abroad, there is no such function of automatically extracting the trajectory of the center of gravity of the moving object in the dynamic background video. When searching for relevant patents, no relevant patent information was retrieved.

Contents of the invention

The object of the present invention is to provide a method for extracting the trajectory of the center of gravity of a moving object based on a dynamic background moving video, so as to realize the extraction of the trajectory of the center of gravity of a moving object in the video of the background movement.

In order to achieve the above object, the present invention provides a method for extracting the center of gravity track of a moving object based on a dynamic background motion video, the main steps of the method are as follows:

1) Do global motion estimation between frames of video frames, and find global motion parameters;

2) According to the global motion parameters, the foreground motion area of the moving object is separated, and the background of the motion image is reconstructed;

3) Realize background elimination and quickly segment the moving object area;

4) Record motion history information to obtain a complete motion object area;

5) Extract the outline of the moving object;

6) Find the trajectory of the center of gravity of the moving object.

In the above-mentioned technical scheme, in step 4) record motion history information is realized like this:

Calculate the binarization template of the current frame and the previous frame;

Find the foreground binarization template of the previous frame;

Find the intersection of the current frame and the binarization template of the previous frame and the foreground binarization template of the previous frame to obtain the temporarily static moving object area;

Merge the temporarily static moving object area with the foreground area obtained by background elimination to obtain a complete moving object area.

In the above technical solution, the extraction of the contour of the moving object in step 5) is realized by adopting the Snake method.

In the above-mentioned technical solution, the two-dimensional center of gravity of the moving object in a single frame is required before the track of the center of gravity of the moving object in step 6), and then each two-dimensional center of gravity is transformed to a designated frame to form a center of gravity track using global motion parameters.

The two-dimensional center of gravity of the moving object is obtained by calculating the geometric center of the outline of the two-dimensional moving object.

The advantages of the present invention are:

1. The method of the present invention realizes the extraction of the trajectory of the center of gravity of the moving object in the dynamic background motion video, and has a wide range of applications and great use value.

2. The method of the present invention can be used for various types of moving objects and has good versatility.

3. The present invention can obtain the two-dimensional center of gravity of the moving object through simple and efficient two-dimensional calculation, which can achieve a real-time effect.

Description of drawings

Figure 1 is a flow chart of extracting the trajectory of the center of gravity of a moving object in a dynamic background video.

Detailed ways

The method of the present invention will be further described below in conjunction with the accompanying drawings.

As shown in FIG. 1 , it is a flow chart of the method of this embodiment. In the flow chart, dotted-line boxes indicate operations, and solid-line boxes indicate results obtained from related operations.

A kind of center of gravity track extraction method of moving object based on dynamic background motion video of the present invention is mainly divided into the following steps:

Step 10. Extracting a sequence of video frames from the video where the moving object is located, performing inter-frame global motion estimation on each frame in the video, and calculating the global motion parameters between each frame. Global motion refers to the motion of pixels that account for a relatively large proportion in video sequence frames, and is mostly caused by camera motion in motion video. Global motion estimation refers to estimating the law of global motion between images based on two frames of images, and the law can be represented by global motion parameters. In motion video, a 6-parameter affine transformation model is usually used to represent the global motion, for example: let p=( _xi , y _i ) be the coordinates of the current frame I _t , p′=(xi _′ , y _i ′) is the coordinates on the previous frame I _t-1 , and the corresponding relationship between them can be expressed as p=θ _{t, t-1} p′, where, θ _{t, t-1} = (a, b, c, d, e, f) are camera motion parameters, which specifically represent the global motion law between the i-th frame and its previous frame i-1 frame, where the components e, f are related to the translational motion of the camera lens, and the components a, b, c , d is related to the zoom and rotation of the camera lens. There are many ways to obtain the global motion parameters, for example: the initial solution of θ _{t, t-1} can be obtained by selecting several feature point pairs between adjacent frames, and using the minimum mean square error rule, and then gradually improve the accuracy through iteration . The above method is the prior art, and those skilled in the art should be able to easily solve the global motion parameters.

Step 20. Using the global motion parameters obtained in step 10, separate the foreground motion area of the moving object, and construct the background of the motion image.

a1. Through the global motion parameters obtained in the previous step, the pixel points of the image in the previous frame of the current frame are transformed point by point. For example, the pixel point (x, y) of the previous frame corresponds to the position (X, Y) in the current frame after being transformed by the global motion parameter, and the pixel brightness of these two points is the same.

b1. After transformation, the difference between the two adjacent frames can be compared, and the area with difference between the two frames can be recorded. This area includes the foreground motion area of the moving object in the two frames. After obtaining the foreground moving area of the moving object in the two frames, the foreground area of the moving object can be separated to eliminate the influence of the foreground area on background synthesis. The above process of comparing the difference between two frames is actually similar to the logical XOR concept, that is, 0 when the two are the same, and 1 when the two are different.

c1. Due to the existence of global motion, the background area of the adjacent frame of the current frame is not exactly the same as the background area of the current frame, so the XOR operation of the pixels of the two frames after the global transformation cannot obtain all the pixels of the current frame background area. Since the background of the current frame is missing, part of its background must be obtained from adjacent frames. After the inverse transformation of the global motion transformation, it can be known which part of the pixels of the adjacent frame the missing background pixel of the current frame corresponds to. The background of the current frame can be completed by filling the corresponding missing pixels of the current frame with those pixels of the adjacent frame.

Step 30. Realize the elimination of the background, and quickly segment out the moving object area. After the background of the current frame is reconstructed, the moving object area can be quickly segmented by background elimination. The specific method of background elimination can be expressed by the following formula in this embodiment: Let D _k be the frame difference after background elimination, then D _k (p)=|W*I _k (p)-B _k (p) |, where W is a smoothing operator, which is a known factor, I _k (p) represents the current frame, and B _k (p) represents the background of the current frame. Then perform black-and-white binarization on D _k to obtain the foreground template M _k ^B . Due to the influence of noise such as accumulated errors in background alignment and interpolation calculation errors, there will be small area noise areas caused by these factors in the foreground template M _k ^B , so it is necessary to use connected component analysis and morphological opening operations to eliminate Small-area noise regions ⁱⁿ _MkB and preserve moving-object regions with larger areas.

Step 40. Record motion history information to realize segmentation of moving objects in video frames. If a certain part of the body of the moving object remains still for a period of time, this part of the body may remain in the constructed background, and this part of the body will not appear in the region of the moving object obtained after the background is eliminated. Body regions can be called static foreground regions. In order to solve the above problems, logical operations are performed on the binarized templates D _{k, k-1} of the current frame and the previous frame and the moving object area template M _k-1 of the previous frame to detect the static foreground area M _k ^H . D _{k, k-1} refers to the image template obtained by binarizing the frame difference between the current frame and the previous frame image. The solution of M _k-1 is similar to the solution method of M _k below. The two are iterative relationship. After obtaining the static foreground area M _k ^H , merge it with the moving object area M _k ^B obtained by background elimination to obtain a complete moving object area M _k , namely:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}& {\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; comp</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; land</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; else</span>}\end{array}\right.$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; B</span>}}{<span\; class="notranslate">\; \&cup;</span>\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}$

Among them, M _k-1 ^comp is the template of the moving object region in the previous frame that is spatially aligned with the current frame after global motion compensation.

Obtaining the complete moving object region M _k also means the completion of the moving object segmentation in the video frame.

Step 50. After obtaining the moving object area, extract the moving object contour. In this embodiment, the method for extracting the contour of the moving object adopts the active contour model Snake. The active contour model Snake is an effective tool for edge detection or contour extraction of objects in current image processing and computer vision. Specifically, snakes are deformable curves defined in the image domain, which produce continuously smooth contours by minimizing their energy functions and deforming and adjusting the snake's natural shape to match specific objects. That is to say, when the snake's energy is minimum, the snake coincides with the object. The shape of the active contour model (snake) is controlled by the internal force of the curve itself and the external force of the image data, and the force acting on the snake will vary locally in space depending on the position and shape of the contour. The internal and external forces act differently: the internal force acts as a smooth constraint, while the external force guides the snake towards image features. The details of the Snake method can be found in Reference 1 "DP Correction Algorithm Based on Active Contour Model", Journal of Shanghai University, Volume 5, Issue 5, 1999. The advantage of using the Snake method is that the contours of moving objects can be extracted accurately.

The specific steps of using the Snake method are:

a2. Set the initial position of Snake on the edge of the moving object area, and the external energy of Snake can be given by the space-time gradient;

The minimization of the total energy of b2 and Snake can be obtained through a fast solution based on the greedy algorithm;

c2. When the total energy of Snake reaches the minimum, the outline of the moving object can be extracted.

The contour of the moving object in the video is two-dimensional, so when the contour of the moving object is obtained, the two-dimensional contour information in the video is also obtained.

Step 60: Obtain the two-dimensional center of gravity of the moving object from the contour of the moving object. The two-dimensional center of gravity of the moving object can be obtained through simple two-dimensional operations. The simple two-dimensional operation refers to calculating the geometric center of the outline of the two-dimensional moving object, for example:

Let G _L be the contour area of the moving object in the Lth frame image in the video sequence; and (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n ) are the contour area G of the moving object All the pixels of _L , then (X, Y) are the coordinates of the two-dimensional center of gravity track point of the frame image, where:

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; .</span>$

Step 70: After the obtained center of gravity of the moving object in a single frame undergoes global motion compensation, the coordinates of the center of gravity are converted to a specified frame, such as the initial frame, and then the same operation is performed on the center of gravity of the moving object in each frame, and finally Obtain the trajectory of the continuous center of gravity of the moving object in the specified frame.

Claims (5)

1, a kind of motion object center of gravity track extraction method based on the dynamic background sport video is used for the barycenter trajectory at the video extraction motion object of background motion; This method may further comprise the steps:

1) interframe of frame of video is done overall motion estimation, innings kinematic parameter of demanding perfection;

2) according to the global motion parameter, the foreground moving zone of disengaging movement object, the background of reconstitution movement image;

3) realize the background elimination, be partitioned into the motion subject area fast;

4) record motion history information obtains complete motion subject area;

5) extract motion object outline;

6) ask motion object barycenter trajectory.

2, the motion object center of gravity track extraction method based on the dynamic background sport video according to claim 1 is characterized in that, the record motion history information in the step 4) is achieved in that

Calculate the binaryzation template of present frame and previous frame;

Ask the prospect binaryzation template of previous frame;

Ask the common factor of the prospect binaryzation template of the binaryzation template of present frame and previous frame and previous frame, obtain temporary transient static motion subject area;

Temporary transient static motion subject area is eliminated the foreground area merging that obtains with background obtain complete motion subject area.

3, the motion object center of gravity track extraction method based on the dynamic background sport video according to claim 1 is characterized in that, the extraction of motion object outline is to adopt the Snake method to realize in the step 5).

4, the motion object center of gravity track extraction method based on the dynamic background sport video according to claim 1, it is characterized in that, ask motion object barycenter trajectory to require the two-dimentional center of gravity of the motion object in the single frame before in the step 6), utilize the global motion parameter that each two-dimentional gravity-center-change is formed barycenter trajectory to designated frame then.

5, the motion object center of gravity track extraction method based on the dynamic background sport video according to claim 4 is characterized in that the two-dimentional center of gravity of described motion object is to try to achieve by the geometric center of asking the two dimensional motion object outline.

Images