JP3480015B2 - Apparatus and method for generating image data - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data generating apparatus and an image data generating apparatus which can be applied to a scanning line interpolating device for interpolating a horizontal scanning line missing in an interlaced scan .
And the generation method .

[0002]

2. Description of the Related Art Current standard television signals (for example, NTSC system) have problems such as line flicker due to interlaced scanning. In order to solve this problem, there has already been proposed a scanning line interpolating device which interpolates a missing scanning line on the receiver side and converts it into a progressive scanning television signal. FIG. 5 is an example of a conventional motion adaptive scanning line interpolation device.

The input terminal indicated by 21 has, for example, 13.5.
An interlaced-scan digital video signal is supplied, which is sampled at a frequency of MHz and 1 sample (1 pixel) is quantized to 8 bits. This input signal is supplied to the intra-field interpolation circuit 22, the inter-field interpolation circuit 23, and the motion detection circuit 24, respectively. The intra-field interpolation circuit 22 performs interpolation with signals in the same field. For example, the average value of two samples located on the upper and lower scanning lines is used as the interpolation value. Inter-field interpolation circuit 23
Is a circuit that interpolates at the pixel at the same position as the interpolation pixel on the scanning line at the same position as the interpolation scanning line of the previous field in terms of time.

The motion detection circuit 24 detects the amount of motion for each pixel. The motion detection circuit 24 uses the motion coefficients k and 1-k.
To occur. The motion coefficient k is (k =
1), and the coefficient becomes smaller as the amount of movement increases.

The output signal of the intra-field interpolation circuit 22 is supplied to the multiplication circuit 25, and the output signal of the inter-field interpolation circuit 23 is supplied to the multiplication circuit 26. The multiplication circuit 25 multiplies the coefficient by 1-k, and the multiplication circuit 26 multiplies k.
Is multiplied by the coefficient of. These multiplication circuits 25 and 26
Are supplied to the adder circuit 27, and the adder circuit 27 generates an interpolated output signal in which the output signals of the interpolators 22 and 23 are mixed.

[0006]

The conventional interpolating circuit uses an intra-field interpolated signal and an inter-field interpolated signal by weighting according to the motion amount which is empirically determined according to the characteristics of the image signal assumed from the motion amount. It mixes with the signal. However, when the assumed image signal characteristic and the actual image signal characteristic are different, there is a problem that not only good interpolation cannot be performed but also an output signal is deteriorated such as an afterimage. Furthermore, when the intra-field interpolation and the inter-field interpolation are used, the ability of the interpolator is not sufficient, and the degree of improvement in image quality due to the conversion from interlaced scanning to progressive scanning is insufficient.

Therefore, an object of the present invention is to provide an image data generating apparatus and method capable of obtaining a good interpolated image quality without causing deterioration of an output signal.

[0008]

Means for Solving the Problems of claims 1 invention, the apparatus for generating image data to generate pixel data that does not exist in the input image signal, based on an input image signal, generating
Motion amount detecting means for detecting a motion amount at a target pixel position of interest , detected motion amount, and pixel position of interest
Multiple reference images spatially and / or spatially nearby
Generate class code according to the pattern of elementary level distribution
A class code generating means for, in a target pixel position during training
Of the target pixel position and the target
Multiple references spatially and / or temporally close to a pixel
Create a class according to the pattern of pixel level distribution,
The prediction coefficient obtained in advance by learning for each class ,
Memorize each class and classify to the class corresponding to the class code.
And engaging count storage means you outputs paid prediction coefficients, the target pixel position and the spatial and prediction coefficients from the coefficient storage unit and / or
Alternatively, the image data generating device is characterized by having a pixel data generating unit for generating pixel data at a pixel position of interest by calculation with a plurality of peripheral pixel data that are temporally close to each other.

[0009]

The prediction coefficient for generating the optimum interpolated value is determined for each motion amount in advance by learning, and this is stored in the memory. The motion amount of the actual input image data is detected, the motion amount corresponding to this motion amount is read from the memory, and the linear linear combination of the read coefficient and the spatially and temporally neighboring pixel data of the interpolation target pixel is performed. , Interpolation values are generated.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a scanning line interpolation device will be described below. In FIG. 1, 1
Is an input terminal for interlaced scanning digital video signals. Specifically, a transmission signal by broadcasting or the like, a reproduction signal from a VTR or the like is supplied to the input terminal 1. Reference numeral 2 is a time series conversion circuit for converting an input signal into a block structure signal.

The output signal of the time series conversion circuit 2 is supplied to the interpolation calculation circuit 3 and the motion detection circuit 5. The interpolation calculation circuit 3 is connected to memories 4a and 4b in which prediction coefficients acquired by learning in advance are stored as described later. The prediction coefficient is a coefficient for accurately estimating the interpolated value on the skipped horizontal scanning line.

As will be described later, the motion detection circuit 5 detects a motion amount from the sum of absolute values of inter-frame differences for each block, for example. The inter-frame difference is obtained by subtracting pixel data at the same position between the current frame and the temporally previous frame. This interframe difference is converted into an absolute value, and the sum totaled for one block is compared with a predetermined threshold value to determine whether the pixel of interest in the current frame is a moving pixel or a still pixel.

The output signal of the motion detection circuit 5 is supplied to the class code generation circuits 6a and 6b. Is the target of interpolation,
The pattern of the level distribution of a plurality of pixels near the target pixel, that is, the class is determined by the class code generation circuits 6a and 6b. A class code is created that points to this class.

The class code is supplied to the memories 4a and 4b as addresses, and the prediction coefficient corresponding to the class is read from the memories 4a and 4b, respectively. The memories 4a and 4b are not limited to separate memories, but a memory area of one memory may be divided and a memory area may be selected according to the result of motion / still determination. The prediction coefficient is supplied to the interpolation calculation circuit 3, and the interpolation value of the pixel of interest is formed. The interpolation calculation circuit 3 outputs the interpolated value of the pixel on the scanning line skipped to the output terminal 7. The existing scanning line data (input data) and interpolation scanning line data can be combined by a frame memory (not shown).

FIG. 2 is for explaining the class classification and the interpolation calculation, and shows a part of each of the current field (nth field) and the previous field ((n-1) field). When generating the pixel value y on the non-existing scan line Ln + 1 between the scan lines Ln and Ln + 2 of the current field, the pixel values shown are used to correspond to the level distribution. Class classification and interpolation operations are performed.

As an example, the values x ₂ and x _{5 of} two pixels located on the interpolation scanning line in the current field and located above and below the interpolation pixel, and the values x ₁ and x of pixels on both sides of the pixel x _{2. 3} and the pixel values x ₄ and x on both sides of the pixel x _5.
6 , the pixel value x ₈ located on the scanning line Ln + 1 in the previous field and located at the same position as the pixel of interest, and the pixel values x ₇ and x _{9 on} both sides thereof are used. The interpolation value y of the pixel of interest is generated by the interpolation calculation circuit 3 by the following linear linear combination when the prediction coefficients are represented by w ₁ to w ₉ .

Y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ + w ₄
x ₄ + w ₅ x ₅ + w ₆ x ₆ + w ₇ x ₇ + w ₈ x ₈ + w ₉
x ₉ (1)

Such interpolation processing is performed for each of the moving pixel and the still pixel in response to the result determined by the motion detecting circuit 5. In the class code generation circuits 6a and 6b, the level distribution pattern is detected by referring to, for example, the same nine pixels used in the interpolation calculation. Here, the pixel referred to for class classification and the pixel used for the interpolation calculation are the same, but they are not necessarily required. If the value of each pixel is 8 bits, the number of classes will be very large, and the hardware scale will increase due to an increase in memory capacity.
The number of bits in these pixels is compressed.

As an example, ADRC (Adaptive Dynamic
The value of the pixel referred to by the range coding is compressed. ADRC is a technique for adaptively removing redundancy in the level direction by utilizing local correlation of images. That is, within the dynamic range of 0 to 255 possessed by the 8-bit original data, the intra-block dynamic range required for requantization for each block composed of a plurality of spatially and temporally neighboring pixels is significantly small. It is noted that the number of quantization bits of each pixel is significantly reduced from the original 8 bits.

More specifically, 1-bit ADRC can be used. That is, the maximum value and the minimum value of the block including the reference pixels x _{1 to} x ₉ described above are detected, the dynamic range that is the difference between the maximum value and the minimum value is detected, and the value of the reference pixel is divided by the dynamic range. , Its quotient is compared with 0.5, and those with 0.5 or more are coded as `1 ', and those with less than 0.5 are coded as` 0'. Therefore, a 9-bit class code is generated. ADRC that generates an output of a bit number other than 1 bit may be adopted.

Not only ADRC but also DPCM (Differentia
l pulse code modulation), BTC (Block Trancation
Coding), VQ (vector quantization) and other compression encoding encoders can be used as the class code generating circuits 6a and 6b. Further, although the reference pixels are converted into data having the same number of bits, the number of allocated bits may be changed in consideration of the distance between the pixel of interest and the reference pixel. That is, the number of allocated bits of the reference pixel closer to the pixel of interest is made larger than the number of bits of the farther pixel.

An example of motion detection will be described with reference to FIG. The nth frame is composed of the nth field and the (n-1) th field of FIG. The motion detection circuit 5, 9 pixels at the same position in the one block formed from the above nine pixels in the current frame (n th frame) (x ₁ ~x _9), the previous frame ((n-1) frame) The absolute value sum ΣΔF of the frame differences between one block consisting of (x ₁ ′ to x ₉ ′) is obtained. That is, ΣΔF = | x ₁ −x ₁ ′ | + | x ₂ −x ₂ ′ | + ...
_{··· + | x 9 -x 9 '} | (2)

This sum of absolute values ΣΔF is compared with a predetermined threshold value. When it is larger than the threshold value, the central pixel (x ₈ ) in the block is judged to be a moving pixel. It is determined as a still pixel. Then, as described above, the class code generation circuit 6a forms a class code corresponding to the level distribution pattern for the moving pixel, and the class code generation circuit 6b similarly corresponds to the level distribution pattern for the still pixel. To form.

[0024] It should be noted that, as the sum of the absolute values of frame difference,
Not only the previous frame but also the sum of absolute values of the frame differences of the subsequent frame may be used together. Further, not limited to the method of detecting the motion amount from the absolute value of the frame difference, various known motion amount detection methods such as the block matching method can be used.

Prediction coefficients acquired by learning in advance are stored in the memories 4a and 4b. FIG. 4 is a flowchart showing the operation when learning is performed by software processing. The control of the learning process is started from step 11, and in the learning data formation of step 12, learning data corresponding to a known image is formed. Specifically, as described above, a plurality of pixels in the arrangements of FIGS. 2 and 3 can be used. At the end of the data in step 13, if the processing of all the input data, for example, one frame of data has been completed, the process proceeds to step 16 to determine the prediction coefficient, and if not, the process proceeds to step 14 for motion detection and class determination. Shifts.

The motion detection and class determination in step 14 are
As described above, the motion / stillness of the pixel of interest is determined, and
This is a step of detecting a pattern of level distribution centered on the pixel of interest. In the normal equation addition of the next step 15, a normal equation described later is created.

After the processing of all the data is completed from the end of the data in step 13, the control is transferred to step 16,
In the determination of the prediction coefficient of No. 6, the coefficient is determined by solving the equation (10) described later using the matrix solution method. The predictive coefficient store in step 17 stores the predictive coefficient in the memory, and step 1
At 8, the control of the learning process ends.

Step 15 in FIG. 4 (normal equation generation)
The process of step 16 (determination of prediction coefficient) will be described in more detail. Let the true value of the pixel of interest be y and its predicted value be y
′ And the values of the surrounding pixels are x ₁ to x _n ,
Setting the coefficient for each class w ₁ linear combination of n taps by _{~w n y'= w 1 x 1} + w 2 x 2 + ‥‥ + w n x n (3). Before learning, w _i is an undetermined coefficient.

As mentioned above, learning is done for each class,
When the number of data is m, according to the equation (3), y _j ′ = w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (4) (where j = 1, 2, ...

When m> n, w _{1 to} w _n are not uniquely determined, so the elements of the error vector E are e _j = y _j − (w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (5) (where j = 1, 2, ..., M), and the coefficient that minimizes the following equation (6) is obtained.

[0031]

[Equation 1]

This is a so-called least squares method. Here, the partial differential coefficient by w _i of the equation (6) is obtained.

[0033]

[Equation 2]

Since each w _i may be determined so that the equation (7) is set to 0,

[0035]

[Equation 3]

Using a matrix as

[0037]

[Equation 4]

[0038] This equation is generally called a normal equation. If this equation is solved for w _i using a general matrix solution method such as a sweeping method, the prediction coefficient w _i is obtained, and the prediction result w _i is stored in memory using the motion / still determination result and class code as an address. Store it.

Although FIG. 4 shows a software configuration for learning, learning can also be performed by a hardware configuration. Further, the interpolation value is not limited to the linear linear combination using the prediction coefficient, but the data value itself may be created in advance by learning, and this value may be used as the interpolation value. Further, the present invention can be applied not only to the scanning line interpolation but also to the case of interpolating the values of non-existing pixels such as interpolating the values of pixels thinned out by subsampling.

[0040]

According to the present invention, the optimum prediction coefficient for each motion amount is previously obtained by learning, the prediction coefficient is stored in the memory, and this coefficient is used during interpolation, so that the output signal is deteriorated. It is possible to obtain a good interpolation image quality without causing

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment in which the present invention is applied to a scanning line interpolation device.

FIG. 2 is a schematic diagram showing an array of pixels used for class classification and interpolation in the present invention.

FIG. 3 is a schematic diagram for explaining an operation of detecting a motion amount in the present invention.

FIG. 4 is a flowchart when learning for obtaining a prediction coefficient is performed by software processing.

FIG. 5 is a block diagram of a conventional scanning line interpolation device.

[Explanation of symbols]

3 Interpolation calculation circuit 4a, 4b Memory storing prediction coefficients 5 Motion detection circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-5-167992 (JP, A) JP-A-1-108886 (JP, A) JP-A-63-48088 (JP, A) JP-A-1- 236881 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N ^7/ 00-7/088

Claims (4)

(57) [Claims] 1. An image data generation device for generating pixel data that does not exist in an input image signal, wherein a motion amount for detecting a motion amount at a target pixel position to be generated based on the input image signal. Detecting means, the detected motion amount, and the pixel position and space of interest
Of multiple reference pixels that are temporally and / or temporally adjacent
Class that generates a class code according to the distribution pattern
The code amount generating means calculates the motion amount at the target pixel position at the time of learning, and calculates the motion amount at the target pixel position and the sky of the target pixel.
A plurality of reference pixels that are adjacent to each other temporally and / or temporally
Prediction coefficient obtained in advance by creating a class according to the Bell distribution pattern and learning for each class
The stores for each of the classes, and engaging number storing means you output pre <br/> measuring coefficients stored in the class corresponding to the class code, the prediction coefficient and the target pixel from the coefficient storage means location and
A device for generating image data, comprising: a pixel data generation means for generating pixel data at the pixel position of interest by calculation with a plurality of neighboring pixel data spatially and / or temporally nearby. 2. The image data generating device according to claim 1, wherein the motion detecting means detects the amount of motion based on a difference between frames. 3. The image data generating device according to claim 1, wherein the input image signal is an interlaced scanning signal, and pixel data at an interlaced horizontal scanning line position in the field is generated by the pixel data generating means. An image data generation device characterized by being generated. 4. Pixel data not present in an input image signal
In the method of generating image data for generating , the target pixel to be generated is generated based on the input image signal.
A motion amount detecting step for detecting a motion amount at a position, the detected motion amount, and the pixel position of interest and the space
Of multiple reference pixels that are temporally and / or temporally adjacent
Class that generates a class code according to the distribution pattern
In the code generation step , the motion amount at the target pixel position is obtained during learning, and the motion amount at the target pixel position and the sky of the target pixel are calculated.
A plurality of reference pixels that are adjacent to each other temporally and / or temporally
Prediction coefficient obtained in advance by creating a class according to the Bell distribution pattern and learning for each class
Is stored for each of the above classes and stored in the above class corresponding to the above class code.
Coefficient storage step for outputting the measured coefficient, the prediction coefficient from the coefficient storage step, and the target pixel position
Multiple peripheral images spatially and / or spatially nearby
By calculating with the raw data, the pixel
A pixel data generation step for generating the data
A method of generating image data, characterized by: