JP2010114529A - Video processor, and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately determine whether or not telecine video images include closed captions. <P>SOLUTION: A combing degree detection part 15 calculates the combing degree c1 of a frame generated from an attention field C and a succeeding field N. A combing degree detection part 16 calculates the combing degree c2 of a frame generated from the attention field C by intra-field interpolation, A combing degree detection part 17 calculates the combing degree c3 of a frame generated from the attention field C and a previous field P. An optimum frame detection part 18 calculates an evaluation value indicating the field to generate the frame corresponding to the attention field on the basis of the combing degree. A state machine part 19 determines whether or not the attention field C includes the closed caption on the basis of the evaluation value. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to video processing for converting an interlaced scanning video signal into a progressive scanning video signal.

  For example, an interlaced scanned image of 60 fields per second is used as a format for transmitting and storing moving image data such as a television. For storing digital moving images by digital broadcasting or DVD, interlaced scanned images are compressed by MPEG-2 moving image encoding. Further, even with video camera photography, photography by interlaced scanning at 60 fields per second is common.

  On the other hand, images of 24 frames per second are used in film movie shooting and screening. Therefore, when a movie is broadcast on television or recorded on a DVD, it is necessary to convert a progressive scan image of 24 frames per second into an interlaced scan image of 60 fields per second, and this conversion is generally 2-3 pulldown. is called.

  Also, there are scenes where the dialogue is entered in subtitles in the movie. This subtitle is created at 30 frames per second. When a movie is broadcast on television or recorded on a DVD, a conversion called 30-frame subtitle image (progressive scan image) is converted to an interlaced scan image of 60 fields per second, generally called 2-2 pulldown. .

  FIG. 1 is a diagram for explaining telecine for converting a film movie into a television signal.

  In FIG. 1, (24p) indicates a frame of a movie (24 frames per second), and (60i) indicates a field of interlace scanning (60 fields per second). 2-3 pull-down shows four frames of a movie that are continuous with A, B, C, D, and two fields alternately like Ao, Ae, Bo, Be, Bo, Ce, Co, De, Do, De. Convert to 3-field interlaced scanned image. That is, 4 frames of a movie are converted into a television signal of 10 fields (5 frames).

  FIG. 2 is a diagram for explaining telecine for converting a caption image into a television signal.

  In FIG. 2, (30p) indicates a subtitle frame (30 frames per second), and (60i) indicates an interlaced scanning field (60 fields per second). 2-2 pulldown is a 5-field subtitle sequence consisting of A, B, C, D, E, 2 fields like Ao, Ae, Bo, Be, Co, Ce, Do, De, Eo, Ee. Convert to interlaced scanned image. That is, 5 frames of subtitles are converted into a television signal of 10 fields (5 frames).

  A method using a state machine (state transition device) is known to detect patterns that repeat 2 fields and 3 fields alternately and patterns that repeat 2 fields as shown in (60i) of FIG. 1 and FIG. 2 ( (See Patent Document 1). The details of the state machine, which is described in FIG. 4 of Patent Document 1, is a device that changes a predetermined state in a predetermined order according to a predetermined condition.

  In the past, devices that displayed television signals used CRTs to display interlaced scanned images as they were. However, in recent years, progressive scanning display devices such as LCD and plasma have become widespread, and a method of converting an interlaced scanned image into a progressive scanned image for display has become widespread.

  Weave processing is known as a method of converting a 2-3 pulldown or 2-2 pulldown interlaced scan image into a progressive scan image (hereinafter referred to as IP conversion: interlace to progressive conversion). The Weave process is a process for generating a progressive image by combining two fields before and after (see, for example, Patent Document 2).

  (60p) in Fig. 1 shows a progressive scan image obtained by Weave processing on a 2-3 pulldown interlaced scan image, and (60p) in Fig. 2 shows a progressive image obtained by Weave processing on an interlaced scan image pulled down 2-2. Is shown.

  The reference field (hereinafter referred to as the interpolation partner) when interpolating each frame of the progressive scanned image that is IP-converted from the 2-3 pull-down interlaced scanned image is the subsequent field (hereinafter referred to as the following field) in the case of the A1 frame. ) Is an Ae field. In the case of an A2 frame, the Ao field is a preceding field (hereinafter referred to as the previous field). When the relationship between the interpolation partners of each frame is extracted, the pattern “back, front, back, front, front” is repeated.

  On the other hand, when the interpolation partner is extracted from each frame of the progressive scan image that is IP-converted from the interlaced scan image that has been pulled down 2-2, “after, before” is repeated. However, considering the correspondence with the progressive scan image that is IP-converted from the 2-3 pull-down interlaced scan image, as shown in FIG. 2, “back, front, back, front, back” and “front, back, The pattern of “before, after, before” is repeated.

  Therefore, when there are subtitles in the video, these two repeated patterns are mixed, and at the moment when the subtitles enter or disappear, comb noise in the scanning line direction may occur in the subtitle portion. In addition, although the subtitles are normally displayed, there may be a phenomenon that the video is disturbed in a comb shape in the scanning line direction. Such noise and disturbance are called combing noise.

  For example, Patent Document 3 proposes a method of detecting subtitles from high-frequency components between peripheral pixels and performing separate IP conversion on the subtitle portion and other portions. However, such a method may detect a portion that is not a subtitle as a subtitle. If subtitle IP conversion is performed on a portion that is not a subtitle, image quality degradation of the movie occurs.

  In this way, in IP conversion, it is possible to obtain a high-quality progressive scan image (video) by inputting a 2-3 pull-down interlaced scan image, but when subtitles are included in the video, It is not necessarily obtained.

Japanese Patent Laid-Open No. 10-065964 Japanese Patent Laid-Open No. 11-088845 JP2007-074439

  An object of the present invention is to determine with high accuracy whether or not a telecine video includes subtitles.

  The present invention has the following configuration as one means for achieving the above object.

  The present invention provides a combing degree of a frame generated from a field of interest and a subsequent field, and a combing degree of a frame generated from the field of interest by intra-field interpolation when an interlaced scanning video signal is converted into a video signal of progressive scanning. In addition, a combing degree of a frame generated from the attention field and the previous field is calculated, and an evaluation value indicating a field in which a frame corresponding to the attention field is to be generated is calculated based on the combing degree. Based on this, it is determined whether or not the field of interest includes a caption.

  ADVANTAGE OF THE INVENTION According to this invention, it can be determined with high precision whether a telecine image | video contains a caption.

  Hereinafter, video signal processing according to an embodiment of the present invention will be described in detail with reference to the drawings.

  Movie subtitle positions vary from the lower and right edges of the screen. However, the caption position in a single movie is almost unchanged. In the embodiment, the subtitle portion is detected with high accuracy using the characteristic of the subtitle position.

[Device configuration]
FIG. 3 is a block diagram illustrating a configuration example of the video signal processing apparatus according to the embodiment. In order to simplify the description, an example in which the processing target is a luminance signal of a video signal will be described, but the present invention is not limited to this.

  An interlaced scanning video signal input to the input terminal 11 (in order to simplify the explanation, a luminance signal having no color component) is used as a field memory 12, a combing degree detection unit 15, and a pixel correction unit 22. To be supplied.

  The field memories 12 and 13 have a memory capacity for storing one field of interlaced scanning video signals. The field memory 12 delays the video signal by one field time, and outputs the delayed video signal to the field memory 13, the intra-field interpolation unit 14, the combing degree detection units 15 and 17, and the caption area detection unit 20. The field memory 13 further delays the video signal by one field time, and outputs the video signal delayed by two fields (one frame) time to the combing degree detection unit 17 and the pixel correction unit 22.

  In the following, a field delayed by two field times is referred to as a previous field P, a field delayed by one field time is referred to as a target field C, and a field input to the input terminal 11 is referred to as a subsequent field N.

  The intra-field interpolation unit 14 interpolates a line in which the pixel of the target field C does not exist (target line) by using data of two lines (adjacent lines) adjacent to the target line, for example, by a bicubic method. An image Fc is generated. The frame image Fc is output to the combing degree detection unit 16 and the pixel correction unit 22.

  Although described in detail later, each of the combing degree detection units 15, 16, and 17 detects an index (hereinafter referred to as a “combing degree”) indicating the amount of combing noise of an input frame. The combing degree detection unit 15 detects the combing degree c1 of the frame image Fcn created by combining the target field C and the subsequent field N. The combing degree detection unit 16 detects the combing degree c2 of the frame image Fc interpolated in the field. The combing degree detection unit 17 detects the combing degree c3 of the frame image Fpc created by combining the previous field P and the target field C. The detection result of each combing degree detection unit is input to the optimum frame detection unit 18.

  Although the details will be described later, the optimal frame detection unit 18 temporarily determines a combination of fields when generating a frame based on the combing degree. In other words, whether to generate a frame by combining the field of interest C and any of the fields before and after, or whether it is better to generate a frame from only the field of interest C without combining with any field. Make a provisional decision.

  Although the details will be described later, the state machine unit 19 determines whether there is a caption in the field based on the frame generation information provisionally determined by the optimum frame detection unit 18, and whether the field is generated by telecine. Judgment is made.

  The subtitle area detection unit 20 roughly detects an area where the subtitle of the field of interest C exists (hereinafter, a subtitle area) based on the field information of the field of interest C and the information output by the state machine unit 19. The caption identifying unit 21 identifies the caption position from the caption area. The pixel correction unit 22 corrects the pixel based on the information output from the state machine unit 19 and the specification result of the caption specifying unit 21, and outputs a video signal of the correction result to the output terminal 23.

● Combing degree detector 15
FIG. 4 is a diagram illustrating generation of the frame image Fcn by the combing degree detection unit 15.

  As shown in FIG. 4, the combing degree detection unit 15 generates a frame image Fcn by synthesizing the field of interest C and the post field N, calculates the combing degree in units of blocks (hereinafter referred to as combing degree). Block). Then, the number of combing blocks in the entire frame image Fcn is counted, and the combing degree c1 of the frame image Fcn is calculated.

  FIG. 5 is a diagram for explaining pixels constituting a block for calculating the combing degree. For example, as shown in FIG. 5, 3 × 2 pixels Cij (i = 0, 1, j = 0 to 2) in the field of interest C and 3 × 3 pixels Nij (i, j = 0 to 2) in the subsequent field N The 3 × 5 pixels of the frame image Fcn consisting of

As shown in FIG. 6, the combing degree detection unit 15 forms three frame structures (each referred to as a sub-block) using the pixel of the field of interest C and the pixel of the subsequent field N constituting the block. Then, the following calculation is performed in sub-block A, sub-block B, and sub-block C shown in FIG.
if (Subblock A) {
A1 = min (| N00-C00 |, | N01-C01 |, | N02-C02 |);
A2 = min (| C00-N10 |, | C01-N11 |, | C02-N12 |);
A3 = min (| N00-N10 |, | N01-N11 |, | N02-N12 |);
SA = A1 + A2-A3;
}
if (Subblock B) {
B1 = min (| C00-N10 |, | C01-N11 |, | C02-N12 |);
B2 = min (| N10-C10 |, | N11-C11 |, | N12-C12 |);
B3 = min (| C00-C10 |, | C01-C11 |, | C02-C12 |);
SB = B1 + B2-B3;
}
if (Subblock C) {
C1 = min (| N10-C10 |, | N11-C11 |, | N12-C12 |);
C2 = min (| C10-N20 |, | C11-N21 |, | C12-N22 |);
C3 = min (| N10-N20 |, | N11-N21 |, | N12-N22 |);
SC = C1 + C2-C3;
}… (1)
Where min () is a function that outputs the minimum value.

Further, based on the calculation result of each sub-block, the block score S is calculated according to the following equation.
M1 = min (SA, SB);
M2 = min (SB, SC);
S = max (M1, M2);… (2)
Here, max () is a function that outputs the maximum value.

  Then, when the score S is compared with a predetermined threshold Cth and S> Cth, the block is determined as a combing block.

The combing degree detection unit 15 performs the above processing over the entire frame image Fcn, and counts the number C_count of combing blocks in the frame image Fcn. Then, the combing degree c1 is calculated by the following equation.
c1 = C_count / C_count_max1 (3)

● Combing degree detector 17
FIG. 7 is a diagram showing the generation of the frame image Fpc by the combing degree detection unit 17.

  As shown in FIG. 7, the combing degree detection unit 15 combines the previous field P and the field of interest C to generate a frame image Fpc, calculates the combing degree for each block, and determines whether or not it is a combing block. Then, the number of combing blocks in the entire frame image Fpc is counted to calculate the combing degree c3 of the frame image Fpc.

  FIG. 8 is a diagram for explaining pixels constituting a block for calculating the combing degree. For example, as shown in FIG. 8, 3 × 3 pixels Pij (i, j = 0 to 2) of the previous field P and 3 × 2 pixels Cij (i = 0, 1, j = 0 to 2) of the field of interest C The 3 × 5 pixels of the frame image Fpc consisting of

As shown in FIG. 9, the combing degree detection unit 17 forms a sub-block using the pixels in the previous field P and the pixel in the field of interest C that form the block. Then, the following calculation is performed in sub-block D, sub-block E, and sub-block F shown in FIG.
if (Subblock D) {
D1 = min (| P00-C00 |, | P01-C01 |, | P02-C02 |);
D2 = min (| C00-P10 |, | C01-P11 |, | C02-P12 |);
D3 = min (| P00-P10 |, | P01-P11 |, | P02-P12 |);
SD = D1 + D2-D3;
}
if (Subblock E) {
E1 = min (| C00-P10 |, | C01-P11 |, | C02-P12 |);
E2 = min (| P10-C10 |, | P11-C11 |, | P12-C12 |);
E3 = min (| C00-C10 |, | C01-C11 |, | C02-C12 |);
SE = E1 + E2-E3;
}
if (Subblock F) {
F1 = min (| P10-C10 |, | P11-C11 |, | P12-C12 |);
F2 = min (| C10-P20 |, | C11-P21 |, | C12-P22 |);
F3 = min (| P10-P20 |, | P11-P21 |, | P12-P22 |);
SF = F1 + F2-F3;
} …(Four)

Further, based on the calculation result of each sub-block, the block score S is calculated according to the following equation.
M3 = min (SD, SE);
M4 = min (SE, SF);
S = max (M3, M4);… (5)

  Then, when the score S is compared with a predetermined threshold Cth and S> Cth, the block is determined as a combing block.

The combing degree detection unit 17 performs the above processing over the entire frame image Fpc, and counts the number C_count of combing blocks in the frame image Fpc. Then, the combing degree c3 is calculated by the following equation.
c3 = C_count / C_count_max3… (6)

● Combing degree detector 16
FIG. 10 is a diagram illustrating the frame image Fc input by the combing degree detection unit 16.

  As shown in FIG. 10, the combing degree detection unit 16 inputs the frame image Fc interpolated from the field of interest C, calculates the combing degree for each block, and determines whether or not it is a combing block. Then, the number of combing blocks in the entire frame image Fc is counted, and the combing degree c2 of the frame image Fc is calculated.

  FIG. 11 is a diagram for explaining pixels constituting a block for calculating the combing degree. For example, as shown in FIG. 11, 3 × 2 pixels Cij (i = 0, 1, j = 0 to 2) in the field of interest C and 3 × 3 pixels Iij (i, j = interpolated from the field of interest C) A 3 × 5 pixel of the frame image Fc consisting of 0 to 2) is made into a block.

As shown in FIG. 12, the combing degree detection unit 16 configures a sub-block using a pixel in the field of interest C constituting the block and a pixel interpolated from the field of interest C. Then, the following calculation is performed in sub-block G, sub-block H, and sub-block J shown in FIG.
if (Subblock G) {
G1 = min (| I00-C00 |, | I01-C01 |, | I02-C02 |);
G2 = min (| C00-I10 |, | C01-I11 |, | C02-I12 |);
G3 = min (| I00-I10 |, | I01-I11 |, | I02-I12 |);
SG = G1 + G2-G3;
}
if (Subblock H) {
H1 = min (| C00-I10 |, | C01-I11 |, | C02-I12 |);
H2 = min (| I10-C10 |, | I11-C11 |, | I12-C12 |);
H3 = min (| C00-C10 |, | C01-C11 |, | C02-C12 |);
SH = H1 + H2-H3;
}
if (subblock J) {
J1 = min (| P10-C10 |, | P11-C11 |, | P12-C12 |);
J2 = min (| C10-P20 |, | C11-P21 |, | C12-P22 |);
J3 = min (| P10-P20 |, | P11-P21 |, | P12-P22 |);
SJ = J1 + J2-J3;
}… (7)

Further, based on the calculation result of each sub-block, the block score S is calculated according to the following equation.
M5 = min (SG, SH);
M6 = min (SH, SJ);
S = max (M5, M6);… (8)

  Then, when the score S is compared with a predetermined threshold Cth and S> Cth, the block is determined as a combing block.

The combing degree detection unit 16 performs the above processing over the entire frame image Fc, and counts the number C_count of combing blocks in the frame image Fc. Then, the combing degree c2 is calculated by the following equation.
c2 = C_count / C_count_max2 (9)

  As described above, the combing degree detection units 15 to 17 create a frame structure having a predetermined size, and calculate the strength of the artifact generated in the frame structure as the combing degree.

Optimal Frame Detection Unit The optimal frame detection unit 18 outputs a signal fp (evaluation value) indicating a combination of fields for generating frame images based on the combing degrees c1 to c3.
if (fp == p1) Combining field of interest C and back field N is optimal;
if (fp == p2) Optimal combination of previous field P and field of interest C;
if (fp == p3) Neither combination is optimal;
if (fp == p4) Any combination is not optimal, but the combination of field of interest C and any field is appropriate according to past pattern information;

  When the signal fp is p4, the optimum frame detection unit 18 finally outputs one of p1, p2, and p3 as the signal fp using the information of the state machine unit 19.

  FIG. 13 is a flowchart for explaining the process of the optimum frame detection unit 18.

  The optimum frame detector 18 and the counter n are initialized to 0 (S101).

  Next, the optimal frame detection unit 18 calculates the combing degree c1 and the absolute difference value | Δc1 | of min (c1, c2, c3) (S102), and compares the absolute difference value | Δc1 | with the threshold mb_th ( S103). If | Δc1 | ≧ mb_th, the counter n is incremented (S104).

  Next, the optimal frame detection unit 18 calculates the combing degree c3 and the absolute difference value | Δc3 | of min (c1, c2, c3) (S105), and compares the absolute difference value | Δc3 | with the threshold mb_th ( S106). If | Δc3 | ≧ mb_th, the counter n is incremented (S107).

  Next, the optimum frame detection unit 18 branches the process according to the value of the counter n (S108).

  Counter n = 1 means that c1 or c3 has a larger value than other combing degrees. For example, it indicates that combing has occurred in the frame image obtained by combining the attention field C and the previous field P, and no combing has occurred in the frame image obtained by combining the attention field C and the subsequent field N (of course, the case where the front and back are reversed) is there). In other words, it can be said that it is optimal to generate a frame image by combining the field of interest C and a field that does not cause combing. Therefore, when n = 1, the optimum frame detection unit 18 compares c1 and c3 (S109), and outputs signal fp = p1 if c1 <c3 (S110), and signal fp = if c1> c3. p2 is output (S111).

  The counter n = 2 means that the combing degrees c1 and c2 have a large value. That is, it indicates that combing has occurred in the frame image obtained by combining the target field C and the previous or subsequent field, and combining with any field is not suitable. Therefore, when n = 2, the optimum frame detection unit 18 outputs the signal fp = p3 (S112).

  The counter n = 0 is a situation where there is little motion in the moving image, or a central field (such as the B2 field in FIG. 1) of the 3 fields pulled down 2-3 (such as the B1 to B3 fields in FIG. 1). . Therefore, the optimum field to be combined with the field of interest C cannot be determined. Therefore, when n = 0, the optimal frame detection unit 18 outputs the signal fp = p4 (S113). As described above, when the signal fp = p4, the optimum frame detection unit 18 finally determines the optimum field to be combined with the field of interest C based on the accumulation pattern detected by the state machine unit 19. To do.

State Machine Unit FIG. 14 is a diagram for explaining the process of producing subtitles in telecine, and shows a time chart when a telecine of one program (movie) is broadcasted on television.

  First of all, commercials etc. enter before the main part of the movie starts. CMs are expected to contain various pull-down patterns, but basically there is no problem considering that it is not a normal 60i video, in other words, a telecine.

  The main part of the movie starts when the commercial ends. Since no subtitles are entered at the start of the movie, only the 2-3 pull-down video is broadcast. Subsequently, when subtitles appear, the video that has been pulled down 2-2 is superimposed on the video that has been pulled down 2-3. Later, if a commercial enters the middle of the movie, it will become a normal 60i video again.

  As described above, the subtitle position varies depending on whether it is the lower end or the right end of the screen, but it can be said that “the subtitle position is almost unchanged in one movie”. That is, in the main part of the movie, the subtitle position can be specified to some extent by examining the subtitle area when the subtitle appears for the first time.

  Therefore, the state machine unit 19 switches the state machine 902 for 24p + 30p detection and the state machine 903 for 24p detection appropriately by the selector 901, and detects the field in which the caption appears first.

  The state machine 902 for detecting 24p + 30p detects a 2-3 pull-down video pattern and a 2-2 pull-down video pattern. Broadly speaking, there are three states: a state in which 2-3 pull-down is detected, a state in which 2-2 pull-down is detected, and a state in which nothing is detected. Three states cannot be mixed in one field.

  The state machine 902 selected by the selector 901 receives the signal fp output from the optimum frame detection unit 18. The state machine 902 transitions according to the transition rule when the signal fp = p1 or p2 is input, and returns to the initial state when the signal fp = p3 is input.

  For example, when the signal fp is input in the order of p1 → p2 → p1 → p2 from the initial state, the state of the state machine 902 changes from 1 → 2 → 1 → 2 (hereinafter referred to as 1212). There are five transition states 112121, 121121, 121211, 2111212, and 212112 in which 2-3 pulldown is detected, and two transition states 212121 and 121212 in which 2-2 pulldown is detected. Transition states other than these are states in which nothing is detected.

  The state machine 902 outputs a signal to the pixel correction unit 22 when 2-3 pulldown is detected, and outputs a signal to the selector 901 and the caption area detection unit 10 when 2-2 pulldown is detected.

  On the other hand, the state machine 903 for detecting 24p detects only the 2-3 pull-down video pattern. Broadly speaking, there are two states, a state in which 2-3 pull-down is detected and a state in which nothing is detected, and two states cannot be mixed in one field.

  The state machine 903 selected by the selector 901 inputs the signal fp output from the optimum frame detection unit 18. The state machine 903 transitions according to the transition rule when the signal fp = p1 or p2 is input, and returns to the initial state when the signal fp = p3 is input.

  For example, when the signal fp is input in the order of p1 → p2 → p1 → p2 from the initial state, the state of the state machine 903 transitions to 1212. There are five transition states 11212, 12112, 12121, 21121, and 21211 in which 2-3 pull-down is detected, and other transition states are states in which nothing is detected.

  However, the transition of 12121 is made only from four transition states other than 12121 where 2-3 pull-down is detected, and signal fp = p1 is input following the transition of 1212 when 2-3 pull-down is not detected. Even if it is done, the transition to 12121 is prevented. This is because 12121 is a pattern that also exists in 2-2 pulldown, so that 2-2 pulldown is not erroneously detected as 2-3 pulldown.

  The state machine 903 outputs a signal to the selector 901 and the pixel correction unit 22 when 2-3 pulldown is detected.

  In FIG. 14, SM1 represents a state machine 902 for detecting 24p + 30p, and SM2 represents a state machine 903 for detecting 24p. SM1 detects 2-3 pulldown and 2-2 pulldown, but cannot detect both simultaneously. Furthermore, when a caption appears, the vertical spatial frequency near the caption is higher than that of surrounding pixels, and 2-2 pulldown tends to be detected with a fairly high probability. Moreover, even if subtitles appear, SM2 can detect 2-3 pulldown without being affected by it.

  As shown in FIG. 14, first, SM1 is selected by the selector 901. During CM, SM1 (SM2) is basically not detecting anything. When the movie starts, SM1 detects 2-3 pulldown. Subsequently, when subtitles appear, SM1 detects 2-2 pulldown. When subtitle appearance is detected (2-2 pull-down is detected), SM2 is selected by the selector 901, and SM2 detects 2-3 pull-down. When it becomes CM again, since SM2 is in a state where nothing is detected, SM1 is selected by the selector 901. Thereafter, the above operation is repeated.

  As a result, the state machine unit 19 outputs a signal indicating that 2-3 pulldown is detected to the pixel correction unit 22, and outputs a signal indicating the moment when the appearance of the caption is detected to the caption area detection unit 20. Will do.

Subtitle Area Detection Unit The subtitle area detection unit 20 inputs a signal indicating the moment when a subtitle appears from the state machine unit 19. When the signal is input, the caption area detection unit 20 searches for a pixel (subtitle pixel candidate) having a very large luminance difference from surrounding pixels in the field of interest C. If the subtitle pixel candidates are concentrated in a certain area, the area is determined as a subtitle area.

  In order to determine the caption area, for example, the screen is divided into several blocks in consideration of the area where the caption is entered, and the number of caption pixel candidates is counted in each block. Then, a block whose count value is very large compared to other blocks is determined as a caption area. When the caption area detection unit 20 determines the caption area, the caption area detection unit 20 outputs a signal indicating the detection of the caption area to the selector 901 of the state machine unit 19 until the state machine 902 for 24p + 30p detection is selected again. maintain.

  Also, the subtitle area detection unit 20 outputs a signal indicating that the subtitle area is not detected to the state machine unit 19 without determining the subtitle area when there is no block in which subtitle pixel candidates are concentrated. In response to this signal, the selector 901 again selects the state machine 902 for detecting 24p + 30p.

  In this way, the subtitle area can be specified, and when the 2-2 pull-down pattern is detected in a place other than the subtitle, the subtitle area is not determined, and thus erroneous detection can be prevented.

● Subtitle Identification Unit The subtitle identification unit 21 inputs the image of the subtitle area from the subtitle area detection unit 20, searches the subtitle area for a pixel (subtitle pixel) that has a very large luminance difference from surrounding pixels, Outputs pixel position information.

● Pixel Correction Unit When the signal indicating 2-3 pulldown detection is not input from the state machine unit 19, the pixel correction unit 22 performs IP, for example, by a method such as motion adaptive progressive scan conversion or motion compensated sequential scan conversion. Perform conversion.

  When a signal indicating 2-3 pulldown detection is input, the pixel correction unit 22 divides the frame of interest into a 2-3 pulldown region and a caption region based on the position information of the caption pixel input from the caption specifying unit 21. To do. Then, in the 2-3 pull-down area (other than the caption area), interpolation by Weave processing is performed according to the signal fp.

  In the caption area, an average value of two pixels (adjacent pixels and pixels existing in the field of interest) positioned above and below the interpolation pixel of the interpolation line of the frame of interest is calculated. Then, an absolute difference value between the average value and the values of the pixels (corresponding pixels) in the preceding and following fields corresponding to the interpolation pixel is calculated. Then, the pixel value having the smaller difference absolute value (the value of the pixel in the previous or subsequent field) is set as the value of the interpolation pixel.

  In this way, by switching between the two state machines, subtitles are detected accurately, and different types of IP conversion are performed in the subtitle area and other areas, so that a high-quality progressive image can be obtained even if subtitles are entered. Can do.

  The video signal processing according to the second embodiment of the present invention will be described below. Note that the same reference numerals in the second embodiment denote the same parts as in the first embodiment, and a detailed description thereof will be omitted.

  The video signal processing apparatus of the first embodiment can be realized by supplying a computer program to a general-purpose information processing apparatus such as a personal computer. Example 2 describes such an implementation.

[Device configuration]
FIG. 15 is a block diagram illustrating a configuration example of the information processing apparatus according to the second embodiment.

  The CPU 101 executes an operating system (OS) and various programs stored in the ROM of the main memory 102 and the hard disk drive (HDD) 105 using the RAM of the main memory 102 as a work memory. Each component is controlled via a system bus 114 such as a PCI (peripheral component interconnect) bus. Further, various programs including a video signal processing application described later are executed.

  The CPU 101 accesses the HDD 105 via the system bus 114 and the serial ATA interface (SATA I / F) 103. Further, a network 113 such as a local area network (LAN) is accessed via the communication I / F 104. Note that the communication I / F 104 is not necessarily connected to the network 113 alone, but can be connected to a public line or a wireless line, or can receive broadcast radio waves.

  In the following description, the program and various data are described as being read from the HDD 105, but can be read from a server device on the network 113.

  Further, the CPU 101 displays a user interface (UI) and a processing result on the monitor 107 via the graphic accelerator 106, and inputs a user instruction via the keyboard 111 and the mouse 112 connected to the keyboard / mouse controller 110. The monitor 107 is a progressive scanning display device.

  Further, the CPU 101 inputs a video signal in field units from, for example, a tuner 109 connected to a USB (Universal Serial Bus) controller 108 via a serial bus. Note that a field unit video signal may be input from a video player instead of the tuner.

[Video signal processing]
The user operates the UI, inputs a video signal from the tuner 109, and instructs the monitor 107 to display a moving image represented by the shooting signal. In response to a user instruction, the CPU 101 loads a video signal processing application program from the HDD 105 into the main memory 102 and executes video signal processing.

  FIG. 16 is a diagram illustrating a memory map example of the main memory 102 when the information processing apparatus performs video signal processing.

  An OS exists at the beginning of the RAM area of the main memory 102. Thereafter, there are video signal processing software for performing video signal processing including IP conversion processing, image input software, image display software for displaying a moving image on the monitor 107, and the like. The image input software controls the USB controller 108 to input (capture) video signals field by field. Furthermore, there is an image area for storing moving image data and a working area for storing various calculation parameters.

  FIG. 17 is a flowchart for explaining video signal processing executed by the CPU 101.

  The CPU 101 executes the video signal processing software, and initializes the image area and various calculation parameters (S11). Then, it is determined whether or not to end the video signal processing (S12). When the user operates the UI to instruct the end of the video signal processing, the CPU 101 ends the video signal processing.

  When the video signal processing is continued, the CPU 101 inputs the video signal from the tuner 109 and stores it in the image area for each field (S13). Then, a caption is detected from the video signal stored in the image area (S14), interpolation processing is performed according to the detection of the caption, and the video signal resulting from the IP conversion process is stored in the image area (S15). Then, the process returns to step S12.

  Further, the CPU 101 executes image display software and displays the video signal of the IP conversion processing result stored in the image area on the monitor 107.

  FIG. 18 is a flowchart showing details of subtitle detection (S14).

  The CPU 101 creates a frame Fpc by combining the previous field P and the target field C, a frame Fc by interpolation within the target field C, and a frame Fcn by combining the target field C and the subsequent field N from the video signal stored in the image area. To do. Then, the combing degrees c1 to c3 of each frame are calculated (S400).

  Next, the CPU 101 calculates an evaluation value (fp) based on the three combing degrees c1 to c3 (S401), and determines the current state by updating and switching the two states based on the evaluation value ( State machine processing (S402). Then, it is determined whether or not a caption has appeared based on the state information (S403). When a caption appears, a caption area is detected (S404), and the position of the caption pixel is specified from the caption area (S405).

[Modification]
In the above, for the sake of simplicity, the luminance signal of the video signal has been described as an example. However, the signal may be an RGB signal or a chromaticity signal.

  In addition, although an example of an interlaced scanned image of 1: 2 has been described, the present invention is not limited to this, and it is possible to deal with interlaced scanned images of other ratios by adjusting each calculation unit.

[Other embodiments]
Note that the present invention can be applied to a system constituted by a plurality of devices (for example, a computer, an interface device, a reader, a printer, etc.), but an apparatus (for example, a copier, a facsimile machine, a control device) composed of a single device. Etc.).

  Another object of the present invention is to supply a recording medium or a recording medium recording a computer program for realizing the functions of the above embodiments to a system or apparatus. This can also be achieved by the computer (CPU or MPU) of the system or apparatus executing the computer program. In this case, the software read from the recording medium itself realizes the functions of the above embodiments, and the computer program and the computer-readable recording medium storing the computer program constitute the present invention. .

  Further, the above functions are not only realized by the execution of the computer program. That is, according to the instruction of the computer program, the operating system (OS) and / or the first, second, third,... This includes the case where the above function is realized.

  The computer program may be written in a memory of a device such as a function expansion card or unit connected to the computer. That is, it includes the case where the CPU of the first, second, third,... Device performs part or all of the actual processing according to the instructions of the computer program, thereby realizing the above functions.

  When the present invention is applied to the recording medium, the recording medium stores a computer program corresponding to or related to the flowchart described above.

The figure explaining the telecine which converts the film movie into the television signal, The figure explaining the telecine which converts a subtitle picture into a television signal, Block diagram showing a configuration example of a video signal processing apparatus of an embodiment, The figure which shows the production | generation of the frame image Fcn by the combing degree detection part 15; The figure explaining the pixel which comprises the block which calculates a combing degree, The figure explaining calculation of the combing degree, The figure which shows the production | generation of the frame image Fpc by the combing degree detection part 17, The figure explaining the pixel which comprises the block which calculates a combing degree, The figure explaining calculation of the combing degree, The figure which shows the frame image Fc which the combing degree detection part 16 inputs, The figure explaining the pixel which comprises the block which calculates a combing degree, The figure explaining calculation of the combing degree, A flowchart for explaining processing of the optimum frame detection unit; Diagram explaining the process of subtitles in telecine Block diagram showing a configuration example of the information processing apparatus of Example 2, The figure which shows the memory map example of the main memory at the time of an information processing apparatus performing video signal processing, A flowchart for explaining video signal processing executed by the CPU; It is a flowchart which shows the detail of the detection of a caption.

Claims (11)

A video processing device that converts an interlaced scanning video signal into a progressive scanning video signal,
The combing degree of the frame generated from the attention field and the subsequent field, the combing degree of the frame generated from the attention field and the preceding field, and the combing degree of the frame generated from the attention field and the previous field by intra-field interpolation. A calculation means;
An evaluation value calculating means for calculating an evaluation value indicating a field in which a frame corresponding to the field of interest is to be generated based on the combing degree;
A video processing apparatus comprising: determination means for determining whether or not the field of interest includes a caption based on the evaluation value.   2. The video processing apparatus according to claim 1, further comprising detection means for detecting a position of a subtitle included in the field of interest.   3. The video processing apparatus according to claim 2, further comprising generating means for generating a frame corresponding to the field of interest based on the position information of the caption and the evaluation value.   The generating unit extracts a caption area including a caption pixel based on position information of the caption, and sets an interpolation pixel of the caption area as a value of an adjacent pixel of the field of interest and the previous field corresponding to the interpolation pixel or the 4. The video processing apparatus according to claim 3, wherein the frame is generated by interpolating from a value of a pixel in a subsequent field.   5. The video processing apparatus according to claim 4, wherein the generation unit generates the frame by synthesizing a region other than the caption area from the field of interest and a field based on the evaluation value.   2. The video processing apparatus according to claim 1, wherein the combing degree calculation unit generates a frame structure of a predetermined size from the frame, and calculates the strength of an artifact generated in the frame structure as a combing degree. .   The determination means detects a 2-3 pull-down video pattern based on the evaluation value, and further determines that the field of interest includes the subtitle when a 2-2 pull-down video pattern is detected. 2. The video processing apparatus according to claim 1, wherein   8. The determination unit according to claim 7, wherein after determining that the subtitle is included, the determination unit determines that the target field does not include the subtitle when the 2-3 pull-down video pattern is not detected. The described video processing device. A video processing method for converting an interlaced scanning video signal into a progressive scanning video signal,
The combing degree of the frame generated from the attention field and the previous field is calculated by the combing degree of the frame generated from the attention field and the subsequent field, and the intra-field interpolation, the combing degree of the frame generated from the attention field and the previous field,
Based on the combing degree, an evaluation value indicating a field in which a frame corresponding to the field of interest is to be generated is calculated,
A video processing method characterized by determining whether or not the field of interest includes a caption based on the evaluation value.   9. A non-transitory computer-readable storage medium storing a program for controlling a computer device to function as each unit of the video processing device according to claim 1.   11. A computer-readable recording medium on which the program according to claim 10 is recorded.

Images