JPH0235515B2 - - Google Patents

Description

Detailed Description of the Invention [Field of Industrial Application] The present invention relates to a television format conversion device for converting the number of frames and the number of scanning lines of a television image signal, and in particular, for converting the number of frames and scanning lines of a television image signal. This is designed to reduce image quality deterioration such as moth-eaten malfunctions that occur in moving image parts due to frame number conversion.

(Prior art) Conventionally, when performing method conversion, linear interpolation methods such as linear interpolation, position interpolation methods using motion vectors, and methods that combine these have been proposed (Japanese Patent Application No. 60-712). , when selecting the optimal motion vector-compensated interpolated image signal from among multiple motion-compensated interpolated image signals based on multiple detected motion vectors, malfunctions may occur depending on the picture, and conversion There was a moth-eaten image quality deterioration in the image.

FIG. 2 shows a block diagram outlining the structure of an embodiment of the system converter described in the specification of Japanese Patent Application No. 1983-712 "System Conversion Apparatus". In order to convert an M-line/2N field (interlaced scanning) television signal to a K-line/2L field (interlaced scanning) television signal, first convert the M-line/2N field (interlaced scanning) television signal into a K-line/2L field (interlaced scanning) television signal, as shown in Figure 3. The /2N field signal is subjected to K run/2N frame scanning line number conversion and progressive scanning conversion, and then frame number conversion to K lines/2L frames and interlaced scanning conversion.

Both the above patent application and the present invention are mainly concerned with the frame number conversion part. For example, when converting an input television signal of N=30 frames/second to an output television signal of L=25 frames/second. As shown in FIG.
The conversion cycle is to convert the frame into five frames a to e of the output signal.

In such frame number conversion, the simplest method is to form the output image signal of a new frame located between two consecutive frames of input image signals by weighting the input image signals of the previous and following frames. There is a linear interpolation method for this purpose, but it has the drawback that when there is movement in the image, blurring and jitteriness occur at the edges of the moving parts of the output image, resulting in a significant deterioration in image quality.

As a method for eliminating the above-mentioned drawbacks, in Japanese Patent Application No. 59-244638 "Television System Conversion System" by the inventors of the present application, the motion and motion vector (representative motion and motion vector) of the image are We proposed a method that detects the image position and synthesizes it with a linear interpolation signal controlled by a motion vector, which greatly improves the above-mentioned drawbacks of ordinary images. It is true that many of the images contain simple movements such as panning and tilting to follow a specific subject, and although this method was quite effective, it is relatively easy to notice different movements in the same image. Some images include a plurality of image parts, and in such cases, a sufficient effect of improving image quality cannot be obtained.

The above-mentioned drawbacks were further overcome and improved in the Japanese Patent Application No. 60-712 entitled "Format Conversion Apparatus" by the inventors of the present application, and the outline of this apparatus will be explained below with reference to FIG. 2.

In the apparatus shown in FIG. 2, when converting the number of frames of the television system, a linearly interpolated image signal of a new frame is formed by linear interpolation from the original image signal of consecutive N and N+1 frames. , 43, 44, 45, Divide the original image into multiple regions, detect motion vectors according to the direction and distance of image movement in each region, 49, and calculate N and N+1 according to these vectors.
forming a plurality of N and N+1 frames of motion compensated image signals by respectively shifting the entire original image signal of the frame 46, 47; forming a plurality of new frames of motion compensated interpolated image signals 48, N and N+1; From the absolute values 50 of the inter-frame differences between the original image signal of a frame and the motion compensated image signals of a plurality of N and N+1 frames, 51 linearly interpolated image signals 45 corresponding to the minimum of these absolute values, or a plurality of compensation interpolation image signal 48
52 to select one of them.

(Problems to be Solved by the Invention) According to the device equipped with the means described above, even if there are a plurality of image parts that are relatively easy to see and have different movements, the problems caused by frame number conversion can be solved. It was possible to considerably reduce image quality deterioration such as blurring and jitteriness.

However, even with the above-mentioned apparatus, malfunctions have occurred when selecting the minimum value from among the absolute values of the plurality of inter-frame differences for certain types of images.

Figure 5 shows a situation in which a black point a moves on a white background, but for this moving point, the interpolated frame point based on the motion vector V ₁ becomes a black point, so it is correct, and the vector V ₂ This is an error because it becomes a white dot. Here, vector V ₁ and vector V ₂ are motion vectors detected from the image area where the black point will move and from other image areas, respectively.

Here, the motion compensated interpolated image signal at point i in the interpolated frame is selected as follows. In other words, the motion compensated image signals of N and N+1 frames based on the motion vector V ₁ are the a i signal directly above point i, which is obtained by moving the black point a signal on frame N to the right, and the a _i signal directly above point i, which is obtained by moving the black point a signal on frame N+1 to the _left . This is the a _1i signal directly below point i, which has moved in the opposite direction. Similarly, the motion compensated image signals of the N and N+1 frames based on the vector V ₂ are the b _i signal and the b _1i signal (not shown). Here, in order to select the interpolated image signal with the correct motion vector, the absolute value of the difference between signal a _i and signal a _i is compared with the absolute value of the difference between signal bi and signal b _1i ,
That is, by comparing the absolute value of the difference between signal a and signal _a1 and the absolute value of the difference between signal b and signal _b1 , in this case both become zero, and the optimum value for selection cannot be specified. This occurred because we assumed a situation in which the sunspot a moved on a white background, but such a situation is quite possible in reality.

In this way, the minimum value label detection circuit 51 is unable to make an accurate determination, and it is not certain which of the motion compensated image signals based on the vector V ₁ and the vector V ₂ to select. In this example, we have taken the case of two vectors, but since the same state will occur even if there are many motion vectors, a white point and a black point will be randomly selected as the motion compensation interpolation image signal of a black moving object.
In this way, it becomes a worm-eaten state where black and white alternate.

(Means for Solving the Problems) An object of the present invention is to eliminate the above-mentioned device malfunction,
When converting the frame number of a television image signal, try to achieve high-quality frame number conversion by removing blurring, jitteriness, and moth-eaten situations in multiple moving image parts even in special images such as a black point moving on a white background. That is.

That is, the format conversion device of the present invention is a format conversion device that converts the number of television frames.
Means for detecting motion vectors according to moving directions and distances of a plurality of area images on the same screen, and a plurality of N and N+1 frames in which the entire original image signals of consecutive N and N+1 frames are respectively moved in accordance with these vectors. means for forming motion-compensated interpolated image signals of a plurality of frames by forming motion-compensated interpolated image signals of a plurality of new interpolated frames; and pattern matching of the plurality of motion-compensated image signals with the plurality of detected motion vectors. means for calculating the sum of absolute values of a plurality of interframe differences related to each motion vector corresponding to the pattern matching for each pixel of the motion compensation interpolated image signal, and and means for selecting a motion compensated interpolation image signal whose position is corrected in accordance with the detected motion vector as a frame interpolation signal. .

(Embodiment) A new algorithm and the configuration of the device for implementing the system conversion device of the present invention will be explained in detail below with reference to the drawings.

FIGS. 6 and 7 are vector diagrams for explaining the new algorithm, and FIG. 1 is a block diagram of only the main parts of the device configuration according to the present invention.

In the examples of FIGS. 6 and 7, the number of motion vectors detected by the motion vector detection circuit 49 of FIG. 2 is four, but it goes without saying that this number can be expanded to any number. In this case, assume that the image is a scene in which a black point a moves on a white background, as in FIG. 5, and consider obtaining a motion-compensated interpolated image signal for the image. In FIG. 6, the interpolation point i in the interpolation frame is a black point that can be correctly obtained by interpolating motion compensated image signals obtained from the signals a ₀ and a ₁ using the vector V ₁ . Since vectors V ₂ to _{V 4} are vectors detected from other moving image regions, they are error vectors for interpolation point i. The reason is that for the interpolation point i, the signals b ₀ , c ₀ , d ₀ on the corresponding N frames by these vectors
Since this is a white background, when motion compensation interpolation is performed using vectors V ₂ to V ₄ , the interpolation point i becomes a white point, which causes moth damage.

Now, from the interpolation point i, the interpolation candidates for i are a, b, c, d4 corresponding to vectors V ₁ to _{V 4}
The point signals a ₀ , b ₀ , c ₀ , d ₀ are, but as mentioned above, the motion vector label is simply detected as the minimum value of the absolute value of the frame difference of the motion compensated image signals of N and N+1 frames. This will cause an error. Among the interpolation candidates for interpolation point i, vector V ₁ must be selected as a label in this example. As shown in Fig. 7 _, the i point of the interpolated frame to be subjected to frame conversion is _{the 2} vector ( _vector k ₂ V ₁ , vector
k ₁ V ₁ ), two motion-compensated image signals whose pixel positions have been corrected by motion compensation are synthesized and obtained.

Therefore, the present invention selects the optimal motion compensation interpolation image signal using the procedure described below.

First, as shown in FIG _. 6, for interpolation point _i , pixels a ₀ ,
Set b ₀ , c ₀ , d ₀ . Next, regarding a ₀ point,
Signals a ₀ and a ₁ due to V ₁ , signal a ₀ due to vector V ₂
and a ₂ , the signals a ₀ and a ₃ by the vector V ₃ , the vector
A ₀ and a ₄ by V ₄ are extracted as motion-compensated image signals, and the frame difference between N and N+1 frames for each vector is calculated (vector verification unit 20 in FIG. 1). The frame difference is zero for signals a ₀ and a ₁ , and is not zero for the others. In these cases, the signal a ₀ is the black level, whereas the signal a ₂ ,
This is because a ₃ and a ₄ are the white level. The same thing is done for other candidate signals b ₀ , c ₀ , and d ₀ (vector testing units 21, 22, and 23 in FIG. 1). 6th
In the figure, vectors corresponding to signals b ₀ , c ₀ , d ₀
b ₂ , c _{3 ,} of N+1 frame signal by V ₂ , V _{3 ,} V ₄ ,
Items other than d ₄ have been omitted to avoid complication of the figure.

The above operation is nothing but an operation of performing pattern matching using four vectors on each interpolation candidate signal. In this example, there are 16 combinations in total, and if the absolute value of the frame difference related to the signals a ₀ and a ₁ is written as θ _1-1 , then the absolute value of the frame difference related to the movement of point a is: θ _1-1 =0, θ _1-2 ≠0, θ _1-3 ≠0, θ _1-4 ≠0 Similarly, the absolute value of the frame difference related to the movement of points b, c, and d is θ _2-1 = θ _2-2 = θ _2-3 = θ _2-4 = 0 θ _3-1 = θ _3-2 = θ _3-3 = θ _3-4 = 0 θ _4-1 = θ _4-2 = θ _{4 -3} = θ _4-4 = 0. When the above combinations are rearranged in terms of vectors, the set related to vector V ₁ is θ _1-1 =
θ _2-1 = θ _3-1 = θ _4-1 = 0 The pairs related to similar vectors V ₂ , V ₃ , and V ₄ are θ _1-2 ≠0, θ _2-2 = θ _3-2 = θ, respectively. _4-2 = 0 θ _1-3 ≠0, θ _2-3 = θ _3-3 = θ _4-3 = 0 θ _1-4 ≠0, θ _2-4 = θ _3-4 = θ _4-4 = It becomes 0.

Here, the sum of the absolute value of the frame difference related to the vector V ₁ and the absolute value of the frame difference related to the vectors V ₂ , V ₃ , and V ₄ is calculated (using the adder 2 in Figure 1).
4, 25, 26, 27), respectively i ₁ ,
If i ₂ , i ₃ , and i ₄ , the minimum value is i ₁ , and the vector giving i ₁ is the correct movement vector of the moving object (here, V ₁ ) (minimum value detection unit 2 in Fig. 1).
8).

Similarly, if V ₂ is correct, the output i ₂ of adder 25
is the minimum, and accurate vector labels are detected using the same algorithm for V ₃ and V ₄ as well. A motion compensated interpolation image signal whose position has been correctly corrected by the control signal C corresponding to the accurate vector determined by the minimum value vector label detection unit 28 is selected.

(Effects of the Invention) By using the device of the present invention, in a motion-compensated frame number conversion device, accurate motion vector label detection and corresponding selection can be achieved in selecting a motion-compensated interpolated image signal for each frame. Therefore, there is an advantage that good format conversion can be performed without causing a moth-eaten image quality deterioration in the output image after frame number conversion.

Furthermore, since the device of the present invention can be applied to converting the number of frames of any television signal, it can also be applied to a system conversion device between current standard television systems, and between a high-definition television system and a standard television system.

[Brief explanation of drawings]

FIG. 1 is a block diagram for explaining the main parts of the configuration of the device of the present invention, FIG. 2 is a block diagram of the configuration of the conventional device, FIG. 3 is a diagram showing the flow of the system conversion device, and FIG. 4 is a diagram showing the flow of the system conversion device. A diagram showing an example of the conversion cycle of frame number conversion, Figures 5 and 6 are diagrams for explaining conventional vector selection and vector selection by the new algorithm of the present invention, respectively, and Figure 7 is an interpolation frame FIG. 3 is a diagram showing the relationship between and vector calculation. 1...Frame memory (F), 2, 3, 6, 7,
8... Buffer memory (BF), 4, 5... Coefficient multiplier (k ₁ , k ₂ ), 9, 10, 11, 24, 25,
26, 27... Adder (+), 12, 13, 14,
15...Subtractor (-), 16, 17, 18, 19
...Absolute value circuit (ABS), 20, 21, 22, 2
3...First, second, third, and fourth block configuration circuits, respectively, 28...Minimum value vector label detection section, 41...Frame order circuit, 42...Frame memory, 43, 44...Coefficient multiplier (k ₁ , k ₂ ),
45, 48... Adder (+), 46, 47... Buffer memory, 49... Motion vector detection circuit,
50... Absolute value circuit (ABS), 51... Minimum value label detection circuit, 52... Signal selection circuit, 53...
Scanning line number/progressive scanning conversion, 54... Frame number conversion.

Claims (1)

[Claims] 1. In a format conversion device that converts the number of television frames, there is a means for detecting motion vectors according to the moving direction and distance of multiple area images on the same screen, and continuous N
and means for forming motion compensated interpolated image signals of a plurality of new interpolated frames by forming a plurality of motion compensated image signals of N and N+1 frames by moving the entire original image signal of N+1 frames, respectively;
Pattern matching is performed on the plurality of motion-compensated image signals using the plurality of detected motion vectors, and the sum of the absolute values of the plurality of inter-frame differences related to each of the motion vectors corresponding to the pattern matching is calculated as the motion compensation. A means for obtaining each pixel of an interpolated image signal, a means for detecting a motion vector that minimizes the sum of the obtained interpolated image signals, and a means for detecting a motion compensation interpolated image signal whose position is corrected corresponding to the detected motion vector into a frame. 1. A system conversion device comprising: means for selecting an interpolation signal.