JPH1079957A - Special cut point detecting device for moving image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a fast and highly accurate cut point detecting device which can detect a cut point using a special effect like dissolving, etc., in addition to a normal cut point and improves an entire detection rate. SOLUTION: A moving-average operating part 4 finds moving average by using an intra-screen activity acquired by an intra-screen activity operating part 2. A moving-average finite difference operating part 6 finds the finite difference from the moving average and sends it to a special cut point detection processing part 8. On the other hand, a color difference correlation operating part 9 finds color difference correlation from a color difference histogram acquired in a color difference histogram operating part 3. The color difference correlation is sent to the part 8. The part 8 detects a special cut point like dissolving, etc., from the moving-average finite difference and color difference correlation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for detecting a cut point of a moving image, and more particularly, to a high-speed switching of moving image information comprising unencoded original moving image information or encoded moving image information. The present invention relates to a moving image cut point image detecting device that can be detected with high accuracy.

[0002]

2. Description of the Related Art Indexing of moving images (indexing)
Is one of the important technologies for efficient searching and editing in digital libraries and video databases. A cut point indicates a boundary point of a scene, which is a basic unit of video, and as a basic indexing tool, research on its detection method has been actively conducted in recent years.

In particular, recently, a detection method has been reported for not only a normal cut point where a scene changes instantaneously but also a special cut screen such as a dissolve screen where a scene changes using an editing effect.

[0004] A dissolve image is basically an image of a process of transitioning from one scene to another scene while two different scenes are spatially synthesized. For example, a "video image" by Murata, Nakamura, Ota et al. Automatic Detection of Cut Change in IPSJ, 50th National Convention of IPSJ, 6D-
7 (1995), a method of detecting a dissolve by modeling a change in luminance with respect to the dissolve has been proposed. Other conventional techniques include "Dissolve Detection Focusing on Monotonicity of Luminance Change" by Nagasaka, Miyatake, Taniguchi et al., IEICE Spring Conference, D-615 (199)
As described in 6), a method of performing dissolve detection by focusing on a scene synthesis process in dissolve has also been proposed.

[0005]

However, in the former case, the temporal change of the ratio of the pixel having a positive ratio of the luminance component inter-frame difference to the entire screen is obtained, and the maximum point is determined as the dissolve screen. Even if the scene moves greatly from a dark place to a bright place, the scene may be erroneously detected as a dissolve screen.

In the latter case, focusing on the monotonous increase / decrease of the luminance change of the pixel in the dissolve section, a difference value between each pixel in the frame and the immediately preceding frame is obtained. Although the image is determined to be a screen, for example, even when the subject has a subtle movement, the variance may be reduced, and the image may be erroneously detected as a dissolve screen.

Therefore, the present invention uses an in-screen activity having different luminance change characteristics between a moving image and a dissolve image.
Utilizing the characteristic that in-screen activity is reduced in the scene synthesis part, it performs strong detection for movement, and in addition to normal cut points, enables detection of cut points using special effects such as dissolve, and overall detection It is an object of the present invention to provide a high-speed and high-accuracy cut point detecting device with improved efficiency.

[0008]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an in-screen activity calculating means for obtaining an activity of a luminance component of each frame from input screen data of a moving image; Moving average calculating means for calculating a moving average of the in-screen activity of each frame from the in-screen activity of each frame extracted by the calculating means; and moving average of the moving average data of each frame extracted by the moving average calculating means. Moving average difference calculating means for calculating an inter-frame difference value, and a target screen from the screen data and a histogram of color difference signals temporally a plurality of frames earlier from the target screen, thereby obtaining a color difference signal histogram correlation value between frames. Color difference histogram correlation calculating means, the moving average difference data and the color It is characterized in that and a special cut detection processing means for performing special cut detection process using a histogram correlation.

According to the present invention, the special cut point detection processing is performed using the inter-frame difference value of the moving average of the in-screen activity and the color difference histogram correlation, so that a special effect such as dissolve is used. The cut point can be accurately detected.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of a cut point detecting device according to the present invention. This embodiment is based on MPEG1 which is an international standard for moving picture coding.
This is to detect a special cut point frame from encoded video data compressed by (ISO / IEC 11172), but the present invention is not limited to this.

As shown in FIG. 1, coded data a of a moving image which has been compression-coded is input to a variable length decoding unit 1. The variable length decoding unit 1 includes a variable length decoding unit 3
1 and an average component extraction unit 32. The variable length decoding unit 31 decodes the quantized two-dimensional DCT coefficients of each block from the encoded data a of the moving image, and
The T coefficient b is input to the average value component extraction unit 32.

The average value component extraction method of the average value component extraction unit 32 includes, for example, "Detection of cut points from compressed moving image data by simple decoding" by Umihara and Nakajima et al. , 6S-9 (1995),
The method disclosed in Japanese Patent Application No. Hei 7-263681 can be used.

[0013] Briefly, the average value component extraction unit 3
2, for example, as shown in FIG. 2A, the average value shown in FIG. 2B is obtained from the quantized two-dimensional DCT coefficients of each block of 8 pixels × 8 lines of the frame t. The (0,0) component representing the component data is extracted, and the average value component data YDC [i, k] of the luminance signal is sent to the in-screen activity calculation unit 2 to output the average value component data Cb [i, k] of the color difference signal, Cr [i,
k] are input to the color difference histogram calculator 3.

On the other hand, when the input moving image is not the moving image data that has been compression-encoded, the data of each frame is directly input to the average component extracting unit 32. in this case,
The average component extraction unit 32 calculates the average component data by integrating the pixels in the small area of each frame.

In the in-screen activity calculation section 2,
Using the average value component data YDC [i, k] of the input luminance signal, an in-screen activity (in-screen deviation) Y-VAR ² [t] of the luminance signal of frame t is calculated according to the following equation. The in-screen activity is input to the moving average calculation unit 4 and the first memory 5. Y-VAR ² [t] = <YDC ² [i, k]> − <YDC [i, k]> ² (1) where t is a frame number (No.), and YDC [i, k] is 8x
The DC components of eight blocks [i, k], <> indicate the average value in frame t.

The in-screen activity Y-VAR ² [t] of the luminance component in the dissolve section thus obtained.
Has a downwardly convex U-curve shape, but even in a place other than the dissolve section, the shape of the activity may be downwardly convex due to local fluctuation due to motion. Therefore, the following processing is performed in the moving average calculation unit 4.

In the moving average calculating section 4, for example, as shown in FIG. 3, the in-screen activity of the frame of interest t inputted from the in-screen activity calculating section 2 is stored in the first memory 5. Then, Fv frame moving averages are calculated in accordance with the following equation by summing the in-screen activities from the frame t to the Fv-1 frame before in time.

[0018]

(Equation 1) By performing this processing, it becomes possible to perform filtering that absorbs movement and local fluctuation, and it is possible to prevent erroneous detection of an image that is not a cut point.

However, the shape of the moving average of the in-screen activity of the luminance component is a flat U-curve when the dissolve section length is long, and a sharp U-curve when the dissolve section length is short.
That is, the shape of the moving average of the in-screen activity of the luminance component changes between a flat U curve and a steep U curve depending on the dissolve section length. Therefore, for example, a relatively flat U curve may not be detected by simple peak detection such as minimum value detection.

In the present embodiment, the moving average data is input to the moving average difference calculation section 6 and the second memory 7, and the moving average data for one frame before, which is stored in the second memory 7 in advance, is also moved. The moving average difference DMV [t] is calculated according to the following equation, and the calculation result is input to the special cut point detection processing unit 8 to calculate the special cut point image from the moving average difference value. Is determined.

DMV [t] = MV [t] -MV [t-1] (3) Another feature of the dissolve section is that even if the correlation between the color difference signals between consecutive frames is small, the frame interval is increased to some extent. The obtained inter-frame correlation tends to increase.

Therefore, the color difference histogram calculator 3 calculates a color difference histogram of the frame t from the average value component data of the input color difference signals, and inputs the calculation result to the color difference correlation calculator 9 and the third memory 10. I do. The color difference correlation calculator 9 calculates the color difference histogram correlation based on the color difference histogram of the input frame t and the color difference histogram of the frame (t−α) (where α is 1 or more) previously stored in the third memory 10. ρ _{t, t-} α Is calculated and input to the special cut point detection processing unit 8.

As a method of calculating the color difference histogram and the color difference correlation, for example, Nakajima et al., “Detection of Cut from Compressed Moving Image Data by Inter-frame Luminance Difference and Color Difference Correlation”, IEICE Autumn Meeting D-501 ( 19
94), Japanese Patent Application No. 5-216895 or Japanese Patent Application No. 6
No. 46561 can be used.

The special cut point detection processing section 8 detects a special cut point using the moving average difference of the input luminance signal and the color difference histogram correlation. The output signal c from the special cut point detection processing section 8 is sent to the first determination section 11.

FIG. 4 is a graph showing the time change of the moving average difference DMV [t] of the luminance signal, for example, as shown in FIG. The dissolve section can be detected as a portion that changes from a downwardly convex peak to an upwardly convex peak as shown on the left side of the figure. However, as shown in the right side b of the figure, depending on the fade section, the dissolve while moving, or the shape of the screen, the peak may be either positive or negative, so the absolute value of the DMV is When the threshold value β is exceeded, the frame when the threshold value β is exceeded is determined as a dissolve candidate screen.

| DMV [t] |> β Next, the color difference histogram correlation ρ _{t, t-} α is set for the frame determined as the dissolve candidate screen by the moving average difference.
Is used to perform final dissolve screen detection. Color difference histogram correlation ρ _{t, t-} α FIG. 5 shows, for example, the time change of the graph in FIG. 5. Therefore, the final dissolve screen is determined according to the following equation.

Ρ _{t, t-} α <Γ> a and b in FIG. 5 correspond to a and b in FIG. In addition,
The dissolve section is empirically several frames (for example, 50 to
Since it is known that the dissolve screen is continued (about 70 frames), when the dissolve screen is detected, the search for the next dissolve screen is restarted by skipping the several frames. As a result,
It is possible to prevent a plurality of dissolve screens from being detected from one dissolve section.

The first determination unit 11 sends the special cut point frame d detected by the special cut point detection processing unit 8 to the cut point image holding unit 12. On the other hand, the first determination unit 11
In the special cut point detection processing unit 8, for the frame e not determined to be a special cut point, the normal cut point detection processing unit 13 further determines whether or not the frame e is a normal cut point. Send to

The normal cut point detection processing unit 13 detects a normal cut point, and the frame f determined to be a normal cut point is input to the cut point image holding unit 12 in the same manner as the special cut point. You. On the other hand, the frame g determined to be a non-cut screen is discarded. As a method of normal cut point detection, for example, the above-mentioned "Cut point detection from compressed moving image data by simple decoding" by Ushihara and Nakajima et al., 51st National Convention of IPSJ, 6S-9
(1995), and the method disclosed in Japanese Patent Application No. Hei 7-263681 can be used.

The cut point image h of the cut point image holding unit 12
By transferring the data to the cut point image display unit 14, a list of cut point images can be displayed as shown in FIG. 6, for example.

Next, the operation of the control unit 15 of FIG. 1 will be described with reference to the flowcharts of FIGS. In step S1, k and i representing the block number of the frame are set to 0.
And put. In step S2, the coded data a of the block [i, k] of the moving image input to the variable length decoding processing unit 1 is variable length decoded. In step S3, the average component extraction unit 32 extracts the average component data of the luminance signal and the color difference signal. In step S4, it is determined whether or not the block [i, k] is the last block in the frame t. If the determination is negative, the process proceeds to step S5, where 1 is added to k or i, and the process proceeds to the next block. Then, the processing of steps S2 and S3 is performed again. The above processing is repeated, and if the determination in step S4 becomes affirmative, the process proceeds to step S6, where the average value component data of the luminance and color difference signals are transferred to the in-screen activity calculation unit 2 and the color difference histogram calculation unit 3.

In step S7, the in-screen activity Y-VAR ² [t] and the chrominance histogram data Cb [i, k] and Cr [i, k obtained by the in-screen activity calculation unit 2 and the chrominance histogram calculation unit 3 are described. ] To the moving average calculation unit 4, the first memory 5, the color difference correlation calculation unit 9, and the third memory 10, respectively. In step S8,
The moving average data obtained by the moving average calculator 4 is transferred to the moving average difference calculator 6. In step S9, the moving average difference and color difference correlation data obtained by the moving average difference calculation unit 6 and the color difference correlation calculation unit 9 are transferred to the special cut point detection processing unit 8. In step S10, it is determined whether or not the frame t is a special cut point, that is, a dissolve screen. If this determination is affirmative, the process proceeds to step S13 in FIG. On the other hand, if not,
Proceed to step S11.

In step S13, the screen data of the frame t is transferred to the cut point image holding unit 12. Then
Proceeding to step S14, it is determined whether the frame t is the final image. If this determination is negative, the process proceeds to step S17 in FIG. 7, and the frame t is updated, for example, by 60 frames. The 60 frames are used to prevent one or more dissolve screens from being detected from one dissolve section, since one dissolve section is about 60 frames. Then, returning to step S2, the above-described processing is repeated.

On the other hand, when the process proceeds to step S11, a normal cut point detection process is performed on the frame t in step S11. In step S12, it is determined whether or not the frame t is a normal cut point. When the determination is affirmative, the process proceeds to step S15 in FIG. In step S15, the screen data of the frame t is transferred to the cut point image holding unit 12. Next, step S16
To determine whether the frame t is the final image. If this determination is negative, the process proceeds to step S18 in FIG. 7, and the frame t is updated by one frame. Then, returning to step S2, the above-described processing is repeated. If the determination in step S12 is negative, the process proceeds to step S16.

If the determination in step S14 or S16 is affirmative, the flow advances to step S19 to determine whether or not to display the cut image on the cut point image display unit 14. If this determination is affirmative, step S20
To the cut point image display unit 1
Transfer to 4. Then, in step S21, a determination is made as to whether or not to end the processing, and if this determination is affirmative,
The above-described series of processing ends.

[0036]

As described above, according to the present invention,
Special cut point detection is performed using the average value component of the luminance signal and color difference signal that can be extracted only by variable-length decoding, so in addition to normal cut point detection, cuts using special effects such as dissolves Point detection can be performed while suppressing an increase in the processing amount to a very small one. In addition, it is possible to improve the overall cut point detection rate.

Incidentally, when the present invention was actually operated,
The following results were obtained. That is, when cut detection is performed on a bit stream coded by the MPEG1 system standardized by ISO for a material of about one hour including a news, a cooking program, a variety program, an advertising program (CM program), etc. Comparing the results of detection of the ratio of the detected cut points to the cut points (detection rate) with the above-mentioned method of Ushihara et al. And the method of the present invention, the former detection rate is 92.8%. The detection rate of the latter is 9
It became 4.5%. This is because the detection rate is improved because it is possible to detect a cut point using a special effect such as dissolve, the detection device according to the present invention, compared with the conventional device, comprehensively, The cut point screen could be detected with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving image special cut screen detection device according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of an operation of an average value component extraction unit in FIG. 1;

FIG. 3 is an explanatory diagram of an example of an operation of a moving average calculation unit in FIG. 1;

FIG. 4 is an explanatory diagram showing a moving average difference of a luminance signal.

FIG. 5 is an explanatory diagram showing a color difference histogram correlation.

FIG. 6 is an explanatory diagram illustrating a display example of a cut image display unit in FIG. 1;

FIG. 7 is a flowchart illustrating the operation of the control device of FIG. 1;

FIG. 8 is a flowchart continued from FIG. 7;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Variable length decoding processing part, 2 ... In-screen activity calculation part, 3 ... Color difference histogram calculation part, 4 ... Moving average calculation part, 5 ... 1st memory, 6 ... Moving average difference calculation part, 7 ...
Second memory, 8: special cut point detection processing unit, 9: color difference calculation unit, 10: third memory, 11: first determination unit, 1
2 cut point image holding unit, 13 normal cut point detection processing unit, 14 cut point image display unit, 15 control unit, 31
Variable length decoding unit, 32... Average value component extraction unit.

Claims (7)

[Claims] 1. An apparatus for detecting a special cut point of a moving image, comprising: an in-screen activity calculating unit for obtaining an activity of a luminance component of each frame from input screen data of the moving image; A moving average calculating means for calculating a moving average of the in-screen activity of each frame from the in-screen activity of each frame; and a moving average inter-frame difference value from the moving average data of each frame extracted by the moving average calculating means. A moving average difference calculating means for calculating; and a color difference histogram correlation calculating means for obtaining a target screen from the screen data and a histogram of color difference signals a plurality of frames earlier from the target screen in time, and obtaining a color difference signal histogram correlation value between frames. The moving average difference data and the color difference histogram Special cut detection device of the moving image, characterized by comprising a special cut detection processing means for performing special cut detection process using uncorrelated. 2. The moving image special cut point detecting device according to claim 1, wherein the in-screen activity calculating means extracts an in-screen activity from average value component data in a small area of each frame of the input moving image. A special cut point detecting apparatus for a moving image. 3. The apparatus for detecting a special cut point of a moving image according to claim 1, wherein the in-screen activity calculating means decodes screen data when the input moving image is a compressed moving image. A moving image special cut point detecting device for extracting an in-screen activity from small-region average value component data obtained by using a simple decoding process for extracting a part of the decoded data. 4. The apparatus for detecting a special cut point of a moving image according to claim 1, wherein said in-screen activity calculating means includes a luminance component average value component data among the extracted small region average value component data. A special cut point detecting device for a moving image, wherein an in-screen deviation is obtained by using the method. 5. The moving image special cut point detecting apparatus according to claim 1, wherein the color difference histogram correlation calculating means uses the average value component data of the color difference component among the average value component data of the small area of the frame. An apparatus for detecting a special cut point of a moving image, wherein a histogram is obtained. 6. The special cut point detecting apparatus for a moving image according to claim 1, wherein the special cut point detection processing means performs a special cut based on a temporal change of the moving average difference data and a temporal change of the color difference histogram correlation. An apparatus for detecting a special cut point of a moving image, characterized by determining a screen. 7. The moving image special cut point detecting apparatus according to claim 6, wherein a temporal change of the moving average difference data and a temporal change of the color difference histogram correlation are provided for each of the threshold values. A special cut point detection unit for a moving image, wherein a special cut screen is determined when the number exceeds a predetermined value.