JP4625949B2 - Object tracking method, object tracking apparatus, and program - Google Patents

Description

  The present invention relates to an object tracking method, an object tracking apparatus, and a program for sequentially detecting position coordinates of tracking target objects in a moving image.

  Object tracking is a technique for sequentially detecting the position coordinates of a tracking target object in a moving image. A feature amount of an image (tracking target image) of a partial region or the entire region of the tracking target object is obtained, and the feature amount is An area that matches or approximates the feature amount of the tracking target image is searched from each frame, and the position coordinates of the area are determined as the position coordinates of the tracking target object. Object tracking includes those that perform template matching, and those that compare the color histogram of the object region with the color histogram in the input frame and set the position of the region having a high similarity of the color histogram as the position of the target object.

  Object tracking by template matching is to create a template image from which the geometric features of the tracking target image are extracted, and detect an area where the template image matches the geometric features from the input frame. Since the shape feature of the image changes due to movement, rotation, deformation, or illumination variation of the tracking target object, there is a problem that tracking accuracy and stability are low. Therefore, each time a tracking target object is successfully detected in one frame, the template image is updated to prepare for the input of the next frame, or the similarity between the input image and the template image is below a predetermined threshold For example, a method of updating a template image is used (Patent Document 1).

  Since the color histogram does not change due to movement or rotation of the tracking target object, it is suitable for object tracking. For example, in Patent Document 2, a reference area histogram and a reference extended area histogram are created from a reference image, a candidate area histogram and a candidate extended area histogram are created from a processed image, and histogram matching is performed using the reference area histogram and the candidate area histogram. Histogram intersection is calculated, histogram matching is performed using the reference extended region histogram and the candidate extended region histogram, and the extended histogram intersection is calculated, and the integrated histogram intersection is obtained by integrating the calculated histogram intersection and the extended histogram intersection. An image processing method is disclosed in which a candidate region having the maximum is detected as a target moving object position in a processed image.

  Further, in order to make the object tracking by the color histogram more robust, every time the tracking target object is successfully detected, the color histogram is updated to prepare for the input of the next frame. Such an object tracking method, that is, a reference area histogram is created from the image of the tracking target object in the first frame, a candidate area histogram is created from the input frame, and a histogram intersection between the reference area histogram and the candidate area histogram is set. The position coordinates of the candidate area where the histogram intersection is maximized are determined as the position coordinates of the tracking target object in the input image, and the color histogram of the candidate area is set as a new reference area histogram, and the next input frame is calculated. The object tracking method that repeats the operation of waiting for is called “conventional method”.

JP 2004-118485 A JP 2004-348273 A

  The conventional method aims to achieve robust object tracking that is not affected by disturbances such as fluctuations in lighting conditions. However, it detects only one candidate region with the maximum histogram intersection from the input frame. The detected color histogram of the candidate area is used for detection of the next frame, so that each time the candidate area is detected, detection errors accumulate and the detection position gradually deviates from the true position of the tracking target object. A phenomenon called drifting occurred, and there was a risk of being unable to track.

  The present invention has been made to solve such problems, and is a highly accurate and highly stable object tracking that is robust against movement, rotation, deformation, variation of illumination conditions, and the like of an object to be tracked. An object is to provide a method, an apparatus, and a program.

  The first configuration of the object tracking method of the present invention is the position of a partial image corresponding to an image of a partial area or the entire area of a tracking target object (hereinafter referred to as “tracking target image”) in one frame of a moving image. In the object tracking method for tracking the tracking target object by sequentially repeating the corresponding position determining step for determining coordinates for each frame of the moving image, the corresponding position determining step belongs to the one frame. A similarity measure calculating step for calculating a similarity measure between a feature amount of the partial image and a feature amount of the tracking target image with respect to the partial images at a plurality of positions; distribution of the similarity measure in the one frame A region extracting step of extracting a region in which a partial image similar to the tracking target image in the one frame exists (hereinafter referred to as “similar region”); Characterized by having a; and calculates the center coordinates of the similar region, the centroid coordinate position calculating step of determining the position coordinates of the target object in the frame.

  According to this configuration, since the barycentric coordinates of a plurality of partial images similar to the tracking target image are used as the position coordinates of the tracking target object, the error is canceled out and the object tracking can be performed with high accuracy. Occurrence is suppressed and stable object tracking becomes possible.

  Note that the feature amount of an image is a variable that distinguishes the image from other images, such as the hue, saturation, brightness, luminance, or histogram of these images. Further, a variable indicating the shape feature of the image may be used as the feature amount. Further, the feature amount used for object tracking is not limited to one. A plurality of feature amounts may be used in combination.

  According to a second configuration of the object tracking method of the present invention, in addition to the first configuration, the corresponding position determining step sets a threshold value for separating a hill portion and a skirt portion of the distribution shape of the similarity scale having a bell shape. A threshold calculating step for calculating, wherein the region extracting step classifies the partial image belonging to the one frame into a partial image similar to the tracking target image and another partial image based on the threshold. .

  According to this configuration, since the frame is classified into the region similar to the tracking target image and the background region based on the distribution shape of the similarity measure, it is possible to perform tracking robust against noise. For calculating the threshold value, a method based on the binarization criterion of Otsu, the criterion of Kittler, or a method in which the weighted average of the maximum value and the minimum value of the similarity measure is used as the threshold value can be selected.

  Here, the high part of the distribution shape of the similarity scale means a part with high similarity, and the skirt part means a part with low similarity surrounding the outside of the high part. When a measure such as a histogram intersection indicating a high degree of similarity is used as the similarity measure, the value of the similarity measure at the height is large. On the other hand, when the distance is used for the similarity scale, the magnitude of the distance indicates low similarity, so that the value of the similarity scale for the hills is small.

  According to a third configuration of the object tracking method of the present invention, in addition to the first configuration, the region extraction step is configured such that the similarity measure is k (k is a natural number of 2 or more) from a partial image having the maximum similarity measure. ) The k partial images up to the largest partial image are extracted as the similar regions.

  According to this configuration, since the partial image is extracted only by comparing the similarity scales, the calculation is simple and easy, and high-speed processing is possible.

  According to a fourth configuration of the object tracking method of the present invention, in any one of the first to third configurations, the feature amount is a color histogram.

  According to this configuration, since the color histogram is used as the feature amount, it is robust against the rotation, movement, and deformation of the target object.

  According to a fifth configuration of the object tracking method of the present invention, in the fourth configuration, the color histogram is color space converted by color space conversion for converting the color space of the image into an rg color space defined by the following equation. It is a two-dimensional histogram created by using the hue component thus made.

  Since the rg color space accurately represents human skin color, the object tracking method according to this configuration is effective for tracking a human face and the like.

  According to a sixth configuration of the object tracking method of the present invention, in any one of the first to fifth configurations, the similarity measure is a histogram intersection.

  Since the histogram intersection can be calculated by a simple operation compared to other similarity measures, this configuration enables high-speed processing.

  According to a seventh configuration of the object tracking method of the present invention, in any one of the first to fifth configurations, the similarity measure is a distance.

Here, the distance is a measure of similarity between vectors such as Euclidean distance, Mahalanobis distance, Manhattan distance, and similarity. In this configuration, when the feature amount can be expressed by a combination of multiple variables, that is, when the feature amount can be handled as a vector, the similarity of the partial images can be accurately evaluated, so that the accuracy and stability of object tracking are improved.

  The first configuration of the object tracking apparatus of the present invention is the position of a partial image corresponding to an image of a partial area or the entire area of a tracking target object (hereinafter referred to as “tracking target image”) in one frame of a moving image. In the object tracking device that tracks the tracking target object by sequentially repeating the operation for determining the coordinates for each frame of the moving image, the partial image at a plurality of positions belonging to the one frame A similarity measure calculating means for calculating a similarity measure between the feature amount of the partial image and the feature amount of the tracked image; and based on the distribution of the similarity measure in the one frame, A region extracting unit that extracts a region where a partial image similar to the tracking target image exists (hereinafter referred to as a “similar region”), and a centroid coordinate of the similar region are calculated, and the centroid coordinate is determined. Characterized in that it comprises a position calculating means for determining the position coordinates of the target object in the frame.

  In addition to the first configuration, the second configuration of the object tracking device of the present invention includes threshold calculation means for calculating a threshold for dividing the height portion and the skirt portion of the distribution shape of the similarity scale having a bell shape. At the same time, the region extracting means classifies the partial image belonging to the one frame into a partial image similar to the tracking target image and another partial image according to the threshold value.

  According to a third configuration of the object tracking apparatus of the present invention, the region extraction unit is configured to perform k from a partial image having a maximum similarity measure to a partial image having a similarity measure of k (k is a natural number of 2 or more). The partial images are extracted as the similar regions.

  According to a fourth configuration of the object tracking apparatus of the present invention, in addition to any of the first to third configurations, the feature amount is a color histogram.

  According to a fifth configuration of the object tracking apparatus of the present invention, in the fourth configuration, the color histogram is color space converted by color space conversion that converts the color space of the image into an rg color space defined by the following equation. It is a two-dimensional histogram created by using the hue component thus made.

  According to a sixth configuration of the object tracking apparatus of the present invention, in any one of the first to fifth configurations, the similarity measure is a histogram intersection.

  According to a seventh configuration of the object tracking apparatus of the present invention, in any one of the first to fifth configurations, the similarity measure is a distance.

  The configuration of the program of the present invention is characterized by causing a computer to execute the object tracking method of any one of the first to seventh configurations.

  As described above, the object tracking method and the object tracking device of the present invention are robust against changes in the imaging environment such as illumination conditions in addition to the movement, rotation, and deformation of the tracking target object. There is a great contribution to improving the capabilities of robot vision sensors.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a tracking target area set in the frame of the tracking start time, and shows an example in which a human face is a tracking target. As shown in the figure, the worker manually sets the tracking target area W (x) in the portion where the face of the tracking target person on the frame is shown. The tracking target area W (x) is a rectangular window area having a size of a × b centered on coordinates x = (x, y) on the image plane, and coordinates x = (x, y) are set to the area W ( The position x).

  After setting the tracking target area W (x), an appropriate tracking radius D is selected and input. The tracking radius D is a parameter that limits the range in which the tracking target object is tracked in the frame at the next time, and the tracking target object is tracked within the tracking range R (x) of the radius D centered on the tracking target region W (x). Is done.

  The tracking radius D is set to a necessary and sufficient value based on the predicted moving speed of the tracking target object, the moving speed of the imaging means, and the sampling rate (number of imaging frames / time) of the imaging means. If the tracking range cannot be limited due to the moving speed of the tracking target object, etc., the entire range of the frame can be set as the tracking target by setting the tracking radius D to a length larger than the diagonal length of the frame. .

  Further, when all of the region W (x) is inside the object region, the position of the region W (x), that is, the coordinate x = (x, y) is called the position of the tracking target object, and the tracking target detected at the time of tracking If the position of the object satisfies this condition, it is assumed that tracking is successful.

  The feature amount is a parameter that distinguishes the region W (x) from other regions, and is a parameter that displays the shape, area, color, brightness, and the like of an image included in the region W (x). In this embodiment, a color histogram is used as the feature amount.

  The color histogram is a frequency distribution of each color component appearing in a certain area of the image, and is expressed by the number of pixels of each color component when the pixels belonging to the area are divided into color components. Since the color histogram has a smaller variation with respect to the movement of the imaging means and the movement of the tracking object than other characteristics, it is convenient for searching for the tracking object from the moving image.

  A space that expresses a specific color is called a color space. When a color space is defined, the color is expressed by coordinates indicating a position in the color space. The color space includes an RGB space that expresses colors with three gradations of red (R), green (G), and blue (B), cyan (C), magenta (M), yellow (Y), black ( There are various types such as a CYMK space expressed by four gradations of K), a YUV space expressed by luminance (Y) and red color difference (U), and blue color difference (V). In object tracking, it is necessary to select a color space suitable for dividing an image into an area in which an object to be tracked is shown and a background area.

  The color histogram used for object tracking has (1) little variation with respect to illumination changes. (2) The production cost is required to be low. Here, lowering the creation cost is not only a matter of cost, but also means reducing the computation time for creation and enabling high-speed processing and real-time processing. In order to meet such a requirement, in this embodiment, the color space of the region W (x) is converted to the rg color space defined by the following equations (1) and (2). A two-dimensional color histogram created using the color space-converted hue component is used. R, G, and B are the luminances of the primary color components of red, green, and blue.

ｒｇ色空間は、画素中の赤色および緑色成分の比率を表現する２次元空間であり、肌色の表現に適している。しかしながら、ｒｇ色空間は無彩色（つまり、Ｒ＝Ｇ＝Ｂとなる色）および無彩色に近い色を適切に表現できない性質があるので、無彩色の画素および無彩色に近い色をヒストグラムに組み入れるとノイズとして働くことが問題になる。そこで、（ｒ，ｇ）の組又は下記の式（３）および式（４）で定義される（ｒ’，ｇ’）が（１／３，１／３）から半径０．０２以内にある画素をヒストグラムの要素から除外することにした。なお、（ｒ’、ｇ’）は黒色（Ｒ＝Ｇ＝Ｂ＝０）近傍の無彩色を除外するため設定したパラメータである。また、半径０．０２は、人物の黒色の頭髪領域を１００枚の画像から抽出し、これらの画素の９０％を無彩色として判定する値を選択した。

The rg color space is a two-dimensional space that represents the ratio of red and green components in a pixel, and is suitable for the expression of skin color. However, since the rg color space has a property that an achromatic color (that is, a color satisfying R = G = B) and a color close to the achromatic color cannot be appropriately expressed, an achromatic color pixel and a color close to the achromatic color are incorporated in the histogram. And working as noise becomes a problem. Therefore, the set of (r, g) or (r ′, g ′) defined by the following equations (3) and (4) is within a radius of 0.02 from (1/3, 1/3). We decided to exclude the pixels from the histogram elements. Note that (r ′, g ′) are parameters set to exclude achromatic colors near black (R = G = B = 0). For the radius 0.02, a black hair region of a person was extracted from 100 images, and a value for determining 90% of these pixels as an achromatic color was selected.

  FIG. 2 is a diagram showing a color histogram in which achromatic colors and colors close to achromatic colors are excluded, and shows that pixels in the portions indicated by white circles in the figure are excluded. In this way, a color histogram is created using a color space obtained by uniformly quantizing an rg color space excluding achromatic colors and colors close to an achromatic color with an appropriate color depth. In the present embodiment, a normalized color histogram H (r, g) such as the following formulas (5) and (6) is used.

In this embodiment, the scale is defined by the following equation (7) as a measure for evaluating similarity for searching a region having a color histogram closest to the color histogram of the tracking target region W (x) from the frame at the next time. Histogram intersection Sc was used. In this histogram intersection Sc, the number of pixels existing in the overlapping portion of the two color histograms H ₁ (r, g) and H ₂ (r, g) is used as a measure of similarity, compared to other measures. Because calculation is easy, it is suitable for high-speed processing.

Since the color histograms H ₁ (r, g) and H ₂ (r, g) are normalized, the value of the histogram intersection Sc ranges from 0 to 1 and is set to 1 when the two histograms are equal. Become.

Further, instead of the histogram intersection Sc, the color histograms H ₁ (r, g) and H ₂ (r, g) are treated as vectors, and the distances between them (Euclidean distance, Mahalanobis distance, Manhattan distance, similarity, etc.) And these distances may be used as a measure of the similarity between the two.

[Similarity scale distribution method]
FIG. 3 is a conceptual block diagram of the image tracking apparatus according to the present embodiment, and FIG. 4 is a flowchart showing the image tracking method. In FIG. 3, reference numeral 1 denotes an image tracking device. The image tracking device 1 includes a similarity scale distribution calculating unit 2, a region extracting unit 3, a position calculating unit 4, and a threshold calculating unit 5, and includes an imaging unit 6 or a recording / reproducing unit. 7 is a device that estimates the position of the tracking object in the moving image input from 7 and outputs it to the output means 8.

  The similarity scale distribution calculating means 2 includes a display device and an input device (such as a keyboard and a mouse) (not shown), displays an image being tracked, and allows manual input by an operator. The imaging unit 6 is a unit that captures a moving image and outputs it in the form of an electrical signal, such as a video camera. The electrical signal output from the imaging means 6 is temporarily recorded in a recording / reproducing means 7 such as a VTR device or a DVD device, and then reproduced and input to the image tracking device 1. You may input into the apparatus 1. FIG. The output means 8 is, for example, a display device that displays a mark indicating the position of the tracking target calculated by the image tracking device 1 in a superimposed manner on a moving image, but instead of the display device, a VTR device, a DVD device or data You may output to a recorder etc.

  The image tracking apparatus 1 can be realized by a combination of a computer and a computer-readable medium describing a program for an image tracking method described in detail below. Also, since a general image analysis method, that is, a technique for reading and processing the information of pixels existing in a specific area of a specific frame of a moving image and performing statistical processing is already known, a description thereof will be omitted.

  Next, details of the image tracking method will be described with reference to the step numbers given in FIG.

[S1]
First, the first frame (time t = 0) of the moving image is read from the recording / reproducing means 7 to the similarity scale distribution calculating means 2.

[S2]
The operator observes the first (time t = 0) frame displayed on the display device (not shown) by the similarity scale distribution calculating means 2 while viewing the tracking target region W (x ₀ ) and then the tracking radius D is entered to set the tracking range R (x ₀ ).

[S3]
When the tracking target area W (x ₀₎ is determined, the similarity measure distribution calculating means 2 reads the color information of the pixels existing in the tracking target area W (x _0), the tracking target area W of the (x ₀₎ A color histogram H ₀ (x ₀ ) is calculated.

[S4]
If the recording / reproducing means 7 does not have a frame at the next time, the process is terminated.

[S5]
The frame at the next time (t = 1) is read from the recording / reproducing means 7 to the similarity scale distribution calculating means 2.

[S6]
The similarity scale distribution calculation means 2 divides the frame at the next time (t = 1) into rectangular window areas W (x) having a size a × b with the coordinate x as the center, and each window area W ( calculating a color histogram _H 1 (x) of x), between the color histogram _H 0 color histograms _H 1 (x) and the tracking object region W of each window region _{W (x) (x 0)} (x 0) histogram intersection _{Sc (H 1 (x),} H 0 (x 0)) to calculate the next time (t = 1) histogram intersection _Sc on the frame of the _{(H 1 (x), H} 0 (x 0 )) Distribution.

FIG. 5 is a diagram showing an example of distribution of histogram intersections. As shown in FIG. 5, the schematic shape of the distribution of the histogram intersection Sc (H ₁ (x), H ₀ (x ₀ )) has a bell shape and does not change depending on environmental changes such as lighting.

[S7]
Next, the similarity scale distribution calculating means 2 performs the histogram intersection S of the window region W (x) existing in the tracking range R (x ₀ ) as defined by the following equations (8) and (9). A set V of ₁ (x | x ₀ ) is obtained and transmitted to the threshold value calculation means 5.

The threshold calculation means 5 calculates a threshold T ₁ that divides the set V into a target object class Vobj and a background class Vbg. Vobj corresponds to the high part of the distribution shape of the histogram intersection Sc (H ₁ (x), H ₀ (x ₀ )) having a bell shape, and Vbg corresponds to the base part. T ₁ corresponds to a height that demarcates the boundary between the hill and the base. T ₁ is calculated by determining the threshold T ₁ according to a standard (Otsu's standard) that minimizes the ratio between the intra-class variance and the inter-class variance of the object class Vobj and the background class Vbg. or a method of determining the thresholds T ₁ according to the criteria of the misidentification rate when approximated to a minimum (reference for Kittler) used. The Otsu's standard and Kittler's standard are described in detail in the following documents, and will not be described here.

  Otsu, "Automatic threshold selection method based on discriminant and least-squares criteria" IEICE Transactions, Vol. J63-D, No. 4, pp. 41-47, 1980

  J. Kittler and J. Illingworth, "Minimum Error Thresholding," Pattern Recognition, Vol. 19, No. 1, pp 41-47, 1986

Further, the maximum value S _{_max} and minimum value S _{_min} set V, may be a weighted average of the two calculated depending on the following equation (10) to the threshold T _1. In this case, α is approximately 0.5. This method has an advantage that the calculation speed is overwhelmingly faster than the method based on the Otsu standard and the Kittler standard.

The threshold value T ₁ determined by any one of the above methods is transmitted to the region extracting unit 3, and the region extracting unit 3 sets the set U ₁ shown in FIG. 6 and the following equation (11), that is, the tracking range R (x ₀ ), and a set of coordinates of the center of the window region W (x) such that the histogram intersection S ₁ (x | x ₀ ) is equal to or greater than the threshold T ₁ is obtained. To the position calculation means 4.

[S8]
The position calculation means 4 calculates an arithmetic mean x ₁ of coordinates included in the set U ₁ as shown in the following equation (12), and uses this as the position of the target object at time t = 1, and the output means 8 And transmitted to the similarity scale distribution calculating means 2.

Needless to say, _{x 1} is the histogram intersection Sc _(x / x ₀₎ is the center of the region such that the threshold value above _{T 1,} that is, the center of gravity coordinates. In this way, the coordinates of the center of gravity of the area where the similarity measure for the tracking target area W (x ₀ ) exceeds a predetermined level is determined as the position coordinates of the tracking target object.

[S9]
The similarity scale distribution calculating means 2 sets the tracking target area W (x ₁ ) centered on x ₁ to the frame at the next time (t = 1), and the color histogram H _{1 of the} tracking target area W (x ₁ ). (X ₁ ) is calculated and prepared for the input of the frame at the next time (t = 2).

  As described above, the steps from S4 to S9 are repeated to determine the position of the target object in the frame at each time (t = 3, 4,...). The steps from S4 to S9 are collectively referred to as a corresponding position determination step.

  The object tracking method described above is referred to as “similarity scale distribution method” in this specification.

[K-maximum similarity measure]
The similarity measure distribution method, the thresholds T ₁ for dividing an image similar area and other areas, histogram intersection S ₁ | was calculated from the distribution shape of the _(x x _0), calculation of the thresholds T ₁ Therefore, if priority is given to speeding up the processing time, the tracking range R (x ₀ ) is extracted in step S 7 in order to extract a region having a large histogram intersection S ₁ (x | x ₀ ). The existing window region W (x) is extracted k (k is a natural number of 2 or more) window regions W (x) from the larger histogram intersection S ₁ (x | x ₀ ). The center of gravity of the k window regions W (x) may be set as the position of the tracking target object. This method will be referred to herein as “k-maximum similarity measure”.

Note that the basic configuration of the object tracking apparatus for performing the k-maximum similarity measure method is the same as that of the object tracking apparatus 1 described above. However, since not calculated threshold T _1, the threshold calculating unit 5 is unnecessary.

[Performance comparison]
In order to compare the performance of the conventional method and the method of the present invention, the results of object tracking for five types of images are shown below.

  FIG. 7 is a sample of five types of images used in the experiment.

  Images (a) and (b) are images captured by a fixed camera, and there is no change in illumination conditions. The tracking target of the image (a) is a doll that is placed on a swivel and turns, and the image (b) is the face of a person facing the front.

  Images (c) and (d) are images obtained by capturing the face of a moving person with a moving camera, image (c) is an example in which the variation in illumination conditions is small, and image (d) is a variation in illumination conditions. Is a large example.

  The image (e) is an image obtained by capturing a state of waving a flag in front of a fixed camera, and is an example of tracking a non-rigid body, that is, an object whose shape changes with time.

The similarity measure distribution method, the method of equation (10) that is chose a method of determining the thresholds T ₁ by a weighted average of the maximum value S _{_max} and minimum value S _{_min} set V. However, α = 0.5.

  For the k-maximum similarity measure, k = 10 was used.

  All moving images were input to a computer with QVGA (320 × 240) resolution, and the width and height of the tracking target area W (x) were 20 to 30 pixels.

  Table 1 shows the results of object tracking under the above conditions. Note that the denominator of the numbers in the table is the total number of frames of the image, and the numerator is the number of the last frame that has successfully tracked the target object. In other words, the denominator and the numerator are equal when tracking is successful until the end, and the numerator becomes smaller if it fails halfway. Those that have been successfully traced to the end are marked with *.

  As shown in Table 1, the conventional method failed to track all the images at an extremely early stage, but the similarity scale distribution method succeeded to track all the images to the end. In addition, the k-maximum similarity measure has been successfully tracked to the end for the images (b) and (c), but has failed midway for the other images.

  It can be seen that the similarity measure distribution method shows very high performance, and the k-maximum similarity measure method shows slightly higher performance than the conventional method, although it is somewhat inferior to the similarity measure distribution method.

  Since the k-maximum similarity scale method is somewhat inferior to the similarity scale distribution method, the image to be applied is limited to an image captured in a stable environment with little change in lighting or the like. Since the processing time is shorter than that of the sex scale distribution method, it is suitable for applications requiring high-speed processing.

It is a figure which shows the tracking object area | region set to the flame | frame at the time of tracking start. It is a figure which shows the histogram which excluded the achromatic color and the color close | similar to an achromatic color. 1 is a conceptual configuration diagram of an image tracking apparatus showing an embodiment of the present invention. It is a flowchart of the image tracking method which shows the Example of this invention. It is a figure which shows the example of distribution of a histogram intersection. It is a figure explaining the area | region classification by a threshold value. It is the sample of the image used for experiment.

Explanation of symbols

1 Image tracking device
2 Similarity scale distribution calculation means
3 area extraction means 4 position calculation means 5 threshold calculation means 6 imaging means 7 recording / reproducing means 8 output means

Claims (5)

A corresponding position determining step for determining position coordinates of a partial image corresponding to a tracking target image that is an image of a partial area or a whole area of the tracking target object in one frame of the moving image is performed for each frame of the moving image. In the object tracking method for tracking the tracking target object by repeating sequentially,
The corresponding position determining step includes:
A similarity scale distribution calculating step for calculating a similarity scale between the feature amount of the partial image and the feature amount of the tracking target image for the partial images at a plurality of positions belonging to the one frame ;
For each partial image in the one frame , a partial image whose similarity measure is equal to or greater than a predetermined threshold is extracted, or a similarity measure is k (k is 2 or more) from a partial image having the maximum similarity measure A region extraction step of extracting a similar region which is a region where a partial image similar to the tracking target image exists in the one frame by extracting k partial images up to the largest partial image. When,
Calculating a barycentric coordinate that is an arithmetic mean of the coordinates of each partial image in the similar region , and determining the barycentric coordinate as a position coordinate of the target object in the frame , and
The feature amount is a color histogram which is a frequency distribution of each color component appearing in each pixel in a certain area of the image,
The corresponding position determining step includes a step of calculating the color histogram H ₀ of the tracking target image ;
The similarity measure calculating step includes:
On the partial image of the plurality of positions belonging to the one frame, and calculating the color histogram H ₁ of the partial images,
Calculating a histogram intersection that is the similarity measure between the color histogram H ₁ of the partial image and the color histogram H ₀ of the tracking target image according to the following equation (1):
The color histogram uses a hue component that has been color space converted by color space conversion for converting the color space of the image into an rg color space defined by the following equation (2), and is defined by the following equation (3): An object tracking method, wherein (r ′, g ′) is a two-dimensional histogram created by excluding pixels having a radius within 0.02 from (1/3, 1/3) .

The corresponding position determining step includes a threshold value calculating step for calculating a threshold value for separating a hill part and a hill part of the distribution shape of the similarity scale having a bell shape,
2. The object tracking according to claim 1, wherein in the region extraction step , the partial image belonging to the one frame is classified into a partial image similar to the tracking target image and another partial image based on the threshold value. Method. By sequentially repeating the operation for determining the position coordinates of the partial image corresponding to the tracking target image , which is an image of a partial area or the entire area of the tracking target object in one frame of the moving image, for each frame of the moving image In the object tracking device for tracking the tracking target object,
Similarity scale distribution calculating means for calculating a similarity scale between the feature amount of the partial image and the feature amount of the tracking target image with respect to the partial images at a plurality of positions belonging to the one frame;
For each partial image in the one frame , a partial image whose similarity measure is equal to or greater than a predetermined threshold is extracted, or a similarity measure is k (k is 2 or more) from a partial image having the maximum similarity measure Region extracting means for extracting a similar region , which is a region where a partial image similar to the tracking target image exists in the one frame, by extracting k partial images up to the largest partial image. When,
Position calculation means for calculating a center-of-gravity coordinate that is an arithmetic average of the coordinates of each partial image in the similar region , and determining the center-of-gravity coordinate as a position coordinate of the target object in the frame ,
The feature amount is a color histogram which is a frequency distribution of each color component appearing in each pixel in a certain area of the image,
The similarity scale distribution calculating means includes:
Means for calculating the color histogram H ₀ of the tracking target image ;
On the partial image of the plurality of positions belonging to the one frame, and means for calculating the color histogram H ₁ of the partial images,
And means for calculating said by the following equation (1) the histogram intersection is the similarity measure between the color histogram H ₀ of the color histogram H ₁ and the tracking object image of the partial image,
The color histogram uses a hue component that has been color space converted by color space conversion for converting the color space of the image into an rg color space defined by the following equation (2), and is defined by the following equation (3): (r ', g') is (1 / 3,1 / 3) object tracking apparatus according to claim 2 dimensional histogram der Rukoto created by excluding pixels that are within a radius from 0.02.

Having a threshold value calculating means for calculating a threshold value that separates a hill part and a skirt part of the distribution shape of the similarity scale having a bell shape;
The object tracking device according to claim 3 , wherein the region extracting unit classifies the partial image belonging to the one frame into the partial image similar to the tracking target image and the other partial image based on the threshold value. . By executing read into a computer program, characterized in that to execute the object tracking method according to claim 1 or 2 in a computer.

Images