JPH0520366A - Animated image collating method - Google Patents

Abstract

PURPOSE:To easily execute the retrieval by using a timewise feature amount especially, in the case the expression of a scene by a speech is difficult, in the collating method for an animated image for extracting a desired scene from an assembly of animated images. CONSTITUTION:The extraction of information (position, area) of an object body in one frame is executed S1-S4 with regard to all frames, and subsequently, a calculation is executed S5 with regard to timewise feature amounts (speed, locus, area variation), and based on this calculation result, a distance calculation is executed S6 to an index animated image in an animated image data base, several images are presented S7 from the index animated images of a small distance, obtained by this calculation, and a scene intended by a retriever is detected.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image search method which is required when searching materials from a moving image database or collecting materials when editing a multimedia document. The present invention relates to the moving image matching method when searching for.

[0002]

2. Description of the Related Art In the conventional moving image retrieval method, it is necessary to remove the human presence of pushing the fast-forward or rewind switch and releasing the switch when it is judged by the human eyes that the desired scene is reached. There was only a premised method.

On the other hand, there is also a method of assigning a keyword to a target screen and managing it as a database.

[0004]

However, in these conventional techniques, the former method which presupposes the existence of a human being searches for a desired scene in a moving image by visual observation, and therefore the switch operation is complicated and labor-intensive. In short, it lacks immediacy.

In the latter method of assigning a keyword to a screen and managing it as a database, the interpretation of an image is
It varies depending on the time and environment, and it was impossible to express and manage this with limited words. Further, this method is a database management method for still images, and even in the case of moving images, only the title is a keyword.

[0006] When the contents of the moving image, in particular, the feature amount is temporally added as a keyword, the expression as a word is delicate, the labor of the indexer (the person who gives the keyword) is great, and the searcher ( (A person who searches using a keyword) must also be strongly aware of the difference in interpretation with the indexer, and there is a problem in that much labor is required.

In any case, the prior art does not have a method for searching using a temporal feature amount. In other words, there is no way to automatically detect an object of "same speed" or a moving object that performs "similar motion", and there is no choice but to search with human eyes.

The present invention solves the above-mentioned conventional problems, and in the moving image collating method required when a desired scene is extracted from a set of moving images, the above-mentioned verbal scene It is an object of the present invention to provide a method for a case where it is difficult to express, especially when a search is performed using a temporal feature amount.

[0009]

According to the present invention, in a moving image collation method for collating a plurality of moving images, a target object is cut out for each frame in the moving image, and position information of the target object in a series of frames is extracted. Based on the moving speed information, the key moving image is collated, or the moving locus information consisting of a chain of position information is used to collate the key moving image. It is characterized in that matching is performed with the key moving image using the temporal shape information of the moving object calculated based on the original.

[0010]

According to the present invention, a searcher first presents a scene close to a desired scene, and a temporal feature quantity automatically calculated from this is used to extract a temporal feature quantity in advance. From the image, the moving speed information, the moving locus information, or the temporal shape information is used to collate with the key operation image to search for a similar scene.

By such search collation, the searcher does not need to devise or select a keyword for which an accurate expression is expected. The searcher only has to present a scene that seems to be close to the desired scene based on his interpretation, and the know-how cultivated by experience is unnecessary.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a schematic diagram of a moving image used in a search according to an embodiment of the present invention, and FIG. 2 is a flow chart showing a search procedure of the present invention.

FIG. 1A shows a moving image (hereinafter referred to as a key moving image) which is input by the searcher to the moving image search system as an image intended for output by the searcher. (2), (3) and
(4) is each moving image (hereinafter referred to as an index moving image) stored in a normal moving image database.

Here, it is assumed that the searcher requests a moving image similar to the key moving image (1) and inputs the search image.
(S ₁ ). In FIG. 1, one frame in the moving image is schematically shown by one square. Three squares in (1) (1)
Since -1, (1) -2 and (1) -3 are shown, (1/3
It becomes a moving image of 0) × 3 = 1/10 seconds. Here, for convenience of explanation, it is assumed that a short moving image is given. In the moving image retrieval system that is actually provided, a certain long moving image is used as a key moving image.
It goes without saying that setting as (1) is easier to use.

From the key moving image, the existing image processing technology is used.
The target object is designated and extracted for each frame using (for example, extraction of a moving area using a method of inter-frame difference) (S ₂ ).

The target object is designated by inputting an image showing the target object.

Then, extraction is performed for all frames (S ₃ ), and (S ₄ ) Relative speed with respect to the frame is calculated from the extracted position information of the target object for each frame.
In FIG. 1, 1/10 second, 1/3 frame in the x direction,
1/3 frame in the y direction,

[0018]

[Outer 1]

On the other hand, as another characteristic amount, the locus y of the object is y =
f (x) is calculated. In the case of (1) in FIG. 1, y = x is given.

Further, the time change a of the shape of the target object is calculated from the extracted area information of the target object for each frame. In the case of (1) in FIG. 1, from frame (1) -1 to frame
(1) -2 area change per 1/30 second (2/3)-
1. Also, the area change from frame (1) -2 to frame (1) -3 is (1/3)-(2/3) per 1/30 second, that is,
a is on average

[0021]

[Equation 1]

Is the area change rate per time (S ₅ ).

That is, for each index image ((2), (3), (4) in FIG. 1) stored in the moving image database, the feature amounts of speed, locus, and area change are set in advance (S ₅ ).

In the feature amount calculation in the step (S ₅ ), the moving distance of the target object related to the locus is calculated by using the moving distance of the center of gravity of the target object. Even if the difference is small, it is possible to calculate the distance.

The area is calculated by a well-known means such as measuring the number of pixels in a divided area.

Next, after calculating the characteristic amount (speed, locus, area change), the index moving image in the moving image database (see FIG.
(2), (3), and (4)) are calculated (S ₆ ).

That is, in the case of (2) in FIG. 1, the total time is 4
[Frame] x (1/30) [second / frame] = (2/15)
It becomes [second].

The moving speed is

[0029]

[Equation 2]

It becomes

The movement locus is y = x. The area change is (2/3) -1, (1/3)-(2/3), (1
/ 6)-(1/3), so averaging this,

[0032]

[Equation 3]

It becomes

Similarly, in (3) of FIG.
(1/10) [second], speed (10/3) [frame frame / second], locus x = 0, area change −10 [unit area / second]. Figure 1
In (4), the total time (1/10) [sec], speed 5 [frame / sec], locus y = 0, area change 0 [unit area /
Seconds].

At this time, consider an appropriate distance function D,
The one with the smaller value is presented to the searcher as a candidate image.

As an example of the above distance function D,

[0037]

[Equation 4]

The following are conceivable. Of course, other distance functions can be applied.

At (S ₆ ) in FIG. 2 showing the processing procedure for retrieval using this distance function, some index moving images with small distances obtained by calculation are presented at (S ₇ ). .

[0040]

As described above, the moving image collation method of the present invention enables a moving image search without a human being interpreting the image and guiding a keyword. Needless to say, this method can be used not only for searching but also for editing and analyzing moving images.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a moving image used in a search according to the present invention.

FIG. 2 is a flowchart showing a search procedure of the present invention.

Claims (3)

[Claims] 1. A moving image collation method for collating a plurality of moving images, wherein a target object is cut out for each frame in a moving image, and movement is calculated based on position information of the target object in a series of frames. A moving image matching method characterized by matching with a key moving image using speed information. 2. A moving image collation method for collating a plurality of moving images, wherein a target object is cut out for each frame in a moving image, and movement locus information consisting of a chain of position information of the target object in a series of frames is obtained. A moving image matching method characterized by using the key to match a moving image. 3. A moving image collating method for collating a plurality of moving images, wherein a target object is cut out for each frame in a moving image and calculated based on image area information of the target object in a series of frames. A moving image matching method characterized by matching with a key moving image using temporal shape information of a moving object.