JP2000152233A - Image information converter and conversion method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the image quality in a transmission-reception system in compliance with the moving picture image coding experts group phase 2 MPEG 2 standards. SOLUTION: An image signal decoded by a receiver side is fed to area cut- out circuits 100, 120 and a field/frame processing identification circuit 11 as an input image signal. The area cut-out circuits 100, 120 extract a class tap and a prediction tap on the basis of the input image signal and the field/frame processing identification circuit 11 identifies which of field DCT processing/frame DCT processing is conducted at compensation coding. A characteristic quantity extract circuit 130 generates an ADRC code based on the output of the area cut-out circuit 100 and generates a code corresponding to the output of the field/frame processing identification circuit 11. A class code generating circuit 14 generates a class code based on the output of the characteristic quantity extract circuit 130. Proper processing is conducted corresponding to the changeover of the DCT processing mode by using the class code to conduct classification adaptive processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to MPEG2 (Mo
The present invention relates to an image information conversion device and a conversion method having a function of improving image quality of a restored image signal generated by inter-image compression encoding and decompression decoding according to the specification of the ving Picture Expert Group phase 2).

[0002]

2. Description of the Related Art MPE is used as a compression encoding method for image signals.
An encoding system based on the G2 standard has been conventionally used.
In a transmission / reception system based on the MPEG2 standard, an image signal to be transmitted is subjected to a compression coding process according to the MPEG2 standard, and is transmitted.
The original image signal is restored by performing decompression decoding corresponding to the compression encoding process according to the PEG2 standard.

[0003] Here, the compression encoding process according to the MPEG2 standard includes DCT (Discrete Cosine Transform) processing. As DCT processing, field DCT processing,
Any one of the frame DCT processes is performed by switching on a macroblock basis.

In such a transmission / reception system based on the MPEG2 standard, an image signal restored on the receiving side sometimes has partial problems such as blurring of a high frequency portion and block distortion.

[0005]

In order to cope with the above-mentioned problems, the image quality is improved by performing image information conversion using the classification adaptive processing on the image signal restored on the receiving side. It is possible. In this case, the switching between the field DCT processing and the frame DCT processing may result in insufficient improvement of blurring, or may enhance block distortion.

Accordingly, an object of the present invention is to provide a transmission / reception system using an encoding system conforming to the MPEG2 standard.
An object of the present invention is to provide an image information conversion device and a conversion method capable of obtaining better image quality.

[0007]

According to the first aspect of the present invention, an MP is provided.
In an image information conversion apparatus for performing image information conversion on an input image signal generated by expanding and decoding an image signal compressed and encoded according to the EG2 standard, a predetermined pixel region is extracted from the expanded and decoded input image signal. A first area cutout means for performing, in the input image signal, whether the intra-field DCT processing or the intra-frame DCT processing has been performed at the time of compression encoding in units of macroblocks, and a field for outputting a code indicating the identification result. / Frame processing identification means, a feature quantity extraction means for extracting a feature quantity based on pixel data supplied from the first area cutout means, and an output of the field / frame processing identification means, and an output of the feature quantity extraction means And a class code generating means for generating a class code corresponding to, and storing a predetermined prediction coefficient. Coefficient storage means for outputting a prediction coefficient corresponding to the output of the class code generation means from among the prediction coefficients, a second area cutout means for extracting a predetermined pixel area from an input image signal, and a second area cutout An image information conversion apparatus comprising: an arithmetic processing unit that performs weighted addition based on an output of a unit and an output of a coefficient storage unit to generate a pixel value.

According to a sixth aspect of the present invention, there is provided an image information conversion method for performing image information conversion on an input image signal generated by expanding and decoding an image signal compressed and encoded according to the MPEG2 standard. A first area extracting step of extracting a predetermined pixel area from the input image signal, and identifying whether the input image signal has been subjected to intra-field DCT processing or intra-frame DCT processing at the time of compression encoding in macroblock units. A DCT processing identifying step of outputting a code indicating the identification result, a feature amount extracting step of extracting a feature amount based on the pixel data supplied from the first area cutout step, and a result of the processing identifying step; A class code generation step of generating a class code corresponding to the result of the quantity extraction step; A coefficient storing step of storing a predicted coefficient corresponding to the result of the class code generating step from among the stored predicted coefficients, and a second area for extracting a predetermined pixel area from the input image signal An image information conversion method comprising: an extraction step; a calculation processing step of performing a weighted addition based on a result of the second area extraction step and a result of the coefficient storage step to generate a pixel value. is there.

According to the above-described invention, it is possible to perform the classification adaptive processing appropriately according to the type (mode) of the DCT processing performed at the time of compression encoding.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Prior to description of an embodiment of the present invention, a classification adaptive process will be described for easy understanding. First, a configuration related to generation of a predicted image signal will be described with reference to FIG. The input image signal is supplied to the region extracting circuits 10 and 12. Here, the input image signal is an image signal restored on the receiving side in the transmission / reception system based on the MPEG2 standard. The region cutout circuit 10 extracts a predetermined pixel region from the input image signal, and supplies pixel data relating to the extracted pixel region to the feature amount extraction circuit 13.

The feature quantity extraction circuit 13 performs a process such as 1-bit ADRC on the supplied pixel data, thereby
A DRC code is generated, and the generated ADRC code is supplied to the class code generation circuit. The class code generation circuit 14 generates a class code representing the result of the classification based on the supplied ADRC code, and
To supply. The ROM 15 supplies coefficient data corresponding to the supplied class code to the estimation operation circuit 16. Here, the prediction coefficient is determined in advance as described later, and is stored in the ROM 15 for each class, more specifically, in such a form that a class code is used as an address. Note that a pixel area used by the area extraction circuit 10 and used for class classification is referred to as a class tap.

On the other hand, the area cutout circuit 12 extracts a predetermined pixel area from the input image signal, and supplies pixel data of the extracted pixel area to the estimation calculation circuit 16. The estimation operation circuit 16 generates a predicted image signal by performing a product-sum operation according to the following equation (1) based on the pixel data supplied from the region cutout circuit 12 and the prediction coefficient supplied from the ROM 15. Here, the region extraction circuit 12
The pixel area used for the product-sum operation that is extracted is referred to as a prediction tap.

Y = w ₁ × x ₁ + w ₂ × x ₂ + ‥‥ + w _n × x _n (1) where x ₁ , ‥‥, x _n are each prediction tap,
w ₁ , ‥‥, and w _n are respective prediction coefficients.

Next, learning, that is, a process of obtaining a prediction coefficient for each class will be described. Generally, an image signal having the same signal format as an image signal to be predicted by the classification adaptive processing (hereinafter, referred to as a teacher signal) is used for converting the teacher signal into image information conversion intended for the classification adaptive processing. A prediction coefficient is determined by a predetermined calculation process based on an image signal obtained by performing a related process.
In the class classification adaptive processing performed to improve the image quality by improving blurring of a high frequency portion, block distortion, and the like caused by compression encoding / decompression decoding of an image signal according to the MPEG2 standard or the like, learning is performed by, for example, FIG. This is performed by the configuration shown in FIG.

A predetermined image signal is an MPEG signal as a teacher signal.
2 encoder 20 and normal equation addition circuit 26. The MPEG2 encoder 20 outputs the MP
The compression coding according to the EG2 standard is performed, and the compression coding signal is supplied to the MPEG2 decoder 21. The MPEG2 decoder 21 expands and decodes the supplied compressed and coded signal to generate a restored image signal, and supplies the generated restored image signal to the region cutout circuit 22 and the feature amount extraction circuit 23. Here, the output of the MPEG2 decoder 21 includes blurring of a high-frequency portion, block distortion, and the like.

The feature amount extracting circuit 23 generates an ADRC code by performing processing such as 1-bit ADRC on the output of the MPEG2 decoder 21, and supplies the generated ADRC code to the class code generating circuit 25. The class code generation circuit 25 receives the supplied A
Based on the DRC code, a class code corresponding to the class to be classified is generated and supplied to the normal equation adding circuit 26.

On the other hand, the area extracting circuit 22
A predetermined pixel area is extracted from the output of the two decoder 21 as a prediction tap, and pixel data of the prediction tap is supplied to the normal equation adding circuit 26. The normal equation adding circuit 26 performs normal arithmetic processing based on the output of the area cutout circuit 22 and the teacher signal, and obtains data of a normal equation having a prediction coefficient corresponding to a class code supplied from the class code generating circuit 25 as a solution. Generate Normal equation addition circuit 26
Is supplied to the prediction coefficient calculation circuit 27. The prediction coefficient calculation circuit 27 performs an arithmetic process for solving a normal equation based on the supplied data. The prediction coefficients calculated by this calculation process are supplied to the memory 28 and stored. Prior to performing the processing relating to the prediction estimation, the storage content of the memory 28 is loaded into the ROM 15 in FIG.

The normal equation will be described below. In the above equation (1), before learning, the prediction coefficients w ₁ , ‥‥, w
_n is an undetermined coefficient. Learning is performed by inputting a plurality of teacher signals for each class. The number of types of teacher signals is m
In this case, the following equation (2) is set from the equation (1).

Y _k = w ₁ × x _k1 + w ₂ × x _k2 + ‥‥ + w _n × x _kn (2) (k = 1,2, ‥‥, m) When m> n, the prediction coefficients w ₁ , Since ‥‥ and w _n are not uniquely determined, the element e _k of the error vector e is defined by the following equation (3), and the prediction coefficient is set so as to minimize the error vector e defined by the equation (4). To be determined. That is, the prediction coefficient is uniquely determined by the so-called least square method.

E _k = y _k − {w ₁ × x _k1 + w ₂ × x _k2 + Δ + w _n × x _kn } (3) (k = 1, 2, Δm)

[0021]

(Equation 1)

As a practical calculation method for obtaining the prediction coefficient minimizing e ² in Equation (4), partial differentiation of e ² with the prediction coefficient w _i (i = 1, 2 ‥‥) is performed (Equation 4). (5)) Each prediction coefficient w _i may be determined so that the partial differential value becomes 0 for each value of _i .

[0023]

(Equation 2)

A specific procedure for determining each prediction coefficient w _i from equation (5) will be described. If X _ji and Y _i are defined as in equations (6) and (7), equation (5) can be written in the form of a determinant of equation (8).

[0025]

(Equation 3)

[0026]

(Equation 4)

[0027]

(Equation 5)

Equation (8) is generally called a normal equation. The prediction coefficient calculation circuit 27 calculates a prediction coefficient w _i by performing a calculation process for solving the normal equation (8) according to a general matrix solution such as a sweeping method.

The above-described processing has the following problems. That is, in the encoding method based on the MPEG2 standard, the frame DCT processing / field DCT processing is switched on a macroblock basis according to the degree of motion of an image. If the classification adaptation process is applied without considering this point, the blur may not be sufficiently improved,
Alternatively, block distortion may be emphasized. Therefore, the present invention provides a method for compressing and encoding a frame DC.
It identifies which of the T processing and the field DCT processing has been performed, and reflects the identification result in the result of the classification.

FIG. 3 shows an example of a configuration relating to the generation of a predicted image signal in one embodiment of the present invention. As described above, the input image signal is an image signal restored on the receiving side in the transmission / reception system based on the MPEG2 standard.
Also, the same components as those in FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. The field / frame processing identification circuit 11 identifies which of the field DCT processing and the frame DCT processing has been performed at the time of the compression encoding based on the input image signal, and extracts the identification result as the area cutout circuits 100 and 120 and the feature amount extraction. Supply to the circuit 130. Each of the region cutout circuits 100 and 120 performs an operation of extracting a class tap and a prediction tap according to the supplied identification result.

The feature amount extraction circuit 130 adds 1 to the prediction tap data supplied from the region extraction circuit 100.
By performing processing such as bit ADRC, an ADRC code is generated, and a code corresponding to the identification result supplied from the field / frame processing identification circuit 11 is generated. The class code generation circuit 14 generates a class code based on the output of the feature amount extraction circuit 130. This class code is used for the field / frame processing identification circuit 11.
Represents a class reflecting the identification result supplied from.

Next, FIG. 4 shows an example of the configuration relating to the determination of the prediction coefficient in one embodiment of the present invention. here,
The same components as those in FIG. 2 are denoted by the same reference numerals, and redundant description will be omitted. The field / frame processing identification circuit 24 identifies which of the field DCT processing / frame DCT processing is being performed for each macro block based on the output of the MPEG2 decoder 21.
A feature amount extraction circuit 230 and an area extraction circuit 1
And 2.

The feature amount extraction circuit 230 generates an ADRC code by subjecting the output of the MPEG decoder 21 to processing such as 1-bit ADRC.
A code corresponding to the identification result supplied from the frame processing identification circuit 24 is generated, and an ADRC code and a code corresponding to the identification result are supplied to the class code generation circuit 25. The class code generation circuit 25 includes a feature amount extraction circuit 23
A class code is generated based on the output of 0. This class code represents a class reflecting the identification result supplied from the field / frame processing identification circuit 24.

Next, a method of identifying which of the field DCT processing and the frame DCT processing is being performed will be described. To facilitate understanding, first, D
The CT processing will be described with reference to FIGS. 5A and 5B. In the compression coding according to the MPEG standard, image data in each macroblock is composed of 8 × 8 (pixels).
It is divided into DCT blocks, and DCT processing is performed in units of DCT blocks. Each coefficient generated by the DCT processing is quantized in a predetermined quantization step, thereby generating a data stream. On the other hand, in decompression encoding of data, inverse DCT processing is performed for each DCT block.

As described above, in the MPEG2 standard, it is possible to switch between field DCT processing and frame DCT processing in units of macroblocks. In the field DCT processing, when the macroblock luminance signal is decomposed into four blocks, each DCT block is decomposed so as to be constituted by fields as shown in FIG. 5A. That is, each DCT block is composed of only odd-numbered lines or only even-numbered lines in a macroblock. On the other hand, the frame DCT
In the process, as shown in FIG. 5B, each DCT block is decomposed so as to be constituted by a frame. Generally, field DCT processing achieves high compression efficiency for moving images,
On the other hand, the frame DCT process achieves high compression efficiency for a still image (an image whose motion amount is small to some extent).

Although the field DCT processing or the frame DCT processing is not standardized in the MPEG2 standard, it is determined as follows based on, for example, a vertical line-to-line correlation in a field and a frame. Can be used. In order to evaluate the correlation between lines, for example, the square of the difference value between pixels adjacent in the vertical direction can be used as the evaluation value.

More specifically, the square var of the difference value between adjacent pixels in the field and the frame in the vertical direction is used.
1 and var2 are calculated according to the following equations (9) and (10), respectively. In Equations (9) and (10), as shown in FIG. 5, u and v are variables representing horizontal and vertical pixel positions in the macroblock, respectively. Accordingly, the right side of the equation (9) indicates the sum of the square sum of the even-numbered pixel difference and the square sum of the odd-numbered pixel difference, and the smaller the value is, the more vertically the field becomes in the field. The correlation between adjacent pixels is large. The right side of the equation (10) indicates the sum of the squares of the differences between the even-numbered pixels and the odd-numbered pixels. The smaller the value is, the more the correlation between the vertically adjacent pixels in the frame becomes. It will be big.

[0038]

(Equation 6)

[0039]

(Equation 7)

Then, based on the calculation results according to the equations (9) and (10), the DCT processing is selected as follows.

(A) var1 ≧ var2 → intra-frame DC
T processing (b) var1 <var2 → in-field DCT processing In one embodiment of the present invention, DC processing as described above is performed.
A DCT process in the input image signal is identified by performing a calculation process similar to the calculation process related to the selection of the T process based on the input image signal (that is, an image signal restored on the receiving side in the transmission / reception system according to the MPEG2 standard). . More specifically, calculation is performed according to Expressions (9) and (10), and a class value blk_class representing the identification result is generated based on the calculation result. The generated class value
An example of blk_class is shown below.

(A) 'var1 ≧ var2 → D in frame
CT and blk_class = 1 (b) 'var1 <var2 → DCT in the field, class value related to the classification result generated as blk_class = 0 (a)', (b) ' By combining the blk_class with the class value related to the waveform of the input image signal generated by the commonly used 1-bit ADRC to generate a class code and performing the class classification adaptive processing, the field D
It is possible to accurately cope with the switching for each macroblock of the CT processing / frame DCT processing. For example, the following is used as the class code class.

Class = adrc_class + 32 × blk_class (11) Next, an example of a tap structure of a class tap and a prediction tap according to an embodiment of the present invention will be described with reference to FIGS. 6A and 6B.
Are shown below. 6A and 6B, circles indicate pixels extracted as class taps and prediction taps. Further, a double circle indicates a target pixel. In this example, the pixel of interest is also extracted as a class tap and a prediction tap. The squares indicate pixels near the class tap and the prediction tap. Note that an arrangement in which the pixel of interest is located at the center of the macroblock and all of the class taps fall within the macroblock is ideal from the viewpoint of processing accuracy and the like, but it is not necessarily required that all of the class taps be within the macroblock. It does not have to fit in.

In one embodiment of the present invention described above, the MP
In the transmission / reception system based on the EG2 standard, an input image signal, which is an image signal restored on the receiving side, is evaluated for inter-line correlation, and based on the evaluation result, which of field DCT processing and frame DCT processing is performed is determined. The identification is performed, and the identification result is reflected in the classification result. On the other hand, the MP that generates the input image signal
1-bit information (field information of a macroblock layer in the data stream) indicating whether field DCT processing or frame DCT processing has been performed from the data stream generated by the compression encoding based on the MPEG2 standard input to the EG2 decoder. (Included) for class classification.

In this case, more specifically, the 1-bit information is extracted from the received data stream by a predetermined circuit and used for class classification, or the received data stream is subjected to decompression decoding. A configuration or the like can be used in which the 1-bit information extracted from the data stream in the MPEG2 decoder is used for class classification.

The present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications can be made without departing from the gist of the present invention.

[0047]

As described above, the present invention provides an MPEG
An image signal restored on the receiving side in a transmission / reception system conforming to the two standards is supplied as an input image signal, and when applying the classification adaptive processing to the input image signal, DCT processing in a field / DCT processing in a frame is performed during compression encoding. The identification is performed in units of macroblocks, and the identification result is reflected in the class classification together with the waveform classification by ADRC processing or the like.

For this reason, DCT performed at the time of compression encoding is performed.
It is possible to perform the class classification adaptive processing appropriately corresponding to the processing type (mode).

Therefore, when the image signal restored on the receiving side in the transmission / reception system based on the MPEG2 standard is improved in image quality by using the classification adaptive processing, it is necessary to switch between DCT processing in a field and DCT processing in a frame. It is possible to prevent the degree of improvement from becoming insufficient due to this.

Therefore, the image quality of the image signal restored on the receiving side in the transmission / reception system based on the MPEG2 standard can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of a configuration related to prediction estimation in a general classification adaptive process.

FIG. 2 is a block diagram illustrating an example of a configuration related to learning when performing a class classification adaptive process for a transmission / reception system using an encoding scheme based on the MPEG2 standard.

FIG. 3 is a block diagram illustrating an example of a configuration related to prediction estimation according to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating an example of a configuration related to learning according to an embodiment of the present invention.

FIG. 5 is a schematic diagram for explaining a DCT mode in MPEG2.

FIG. 6 is a schematic diagram illustrating an example of a tap structure of a class tap and a prediction tap according to an embodiment of the present invention.

[Explanation of symbols]

11,... Field / frame processing identification circuit, 24
... Field / frame processing identification circuit, 130 ...
.Characteristic amount extraction circuit, 230 ... Characteristic amount extraction circuit

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Masaaki Hattori 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yasushi Node 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. Sony Corporation F term (reference) 5C059 KK03 MA00 MA23 PP04 TA13 TB07 TC02 TD05 TD06 UA05 UA34 UA38 5C063 AA01 AB03 AB07 AC01 CA01 CA05 CA36 CA38

Claims (6)

[Claims] An image information conversion apparatus for performing image information conversion on an input image signal generated by expanding and decoding an image signal compressed and encoded in accordance with the MPEG2 standard. A first area cutout unit for extracting a pixel area of the macroblock; and identifying whether the input image signal has been subjected to intra-field DCT processing or intra-frame DCT processing at the time of compression encoding, using a macroblock as a unit. Field / frame processing identifying means for outputting a code shown in the table, a feature quantity extracting means for extracting a feature quantity based on pixel data supplied from the first area cutout means and an output of the field / frame processing identifying means. Class code generating means for generating a class code corresponding to the output of the feature amount extracting means; Coefficient coefficient storing means for storing the determined prediction coefficient and outputting a prediction coefficient corresponding to the output of the class code generating means from the stored prediction coefficients; and extracting a predetermined pixel area from the input image signal. A second area cutout unit that performs weighted addition based on an output of the second area cutout unit and an output of the coefficient storage unit to generate a pixel value. Image information conversion device. 2. An image information conversion apparatus according to claim 1, wherein said field / frame processing identification means obtains an identification result by comparing vertical line-to-line correlations within a field and within a frame. . 3. The method according to claim 2, wherein the field / frame processing identification means calculates a sum of squares of a difference value between pixels adjacent in a field and a frame in a vertical direction, and calculates a calculated value of a correlation between lines. An image information conversion device characterized by using as an evaluation value. 4. The method according to claim 1, wherein the feature amount extracting unit generates an ADRC code by performing an ADRC process on the pixel data supplied from the first region extracting unit. An image information conversion apparatus characterized in that the feature amount is generated by combining an output of a frame processing identification unit. 5. The image processing apparatus according to claim 1, wherein the prediction coefficient is an image signal restoring unit that performs compression encoding and decompression decoding on a predetermined image signal according to the MPEG2 standard. Learning field / frame processing identifying means for identifying which of the DCT processing / intra-frame DCT processing has been performed on a macroblock basis and outputting a code indicating the identification result; and a predetermined pixel from the output of the image signal restoring means. A feature extracting unit for learning for extracting a feature based on pixel data relating to the extracted pixel region and an output of the learning field / frame processing identifying unit; and an output of the feature extracting unit Learning class code generating means for generating a class code corresponding to the image signal; Learning region extracting means for extracting a region, the predetermined image signal, the output of the learning class code generating device, and the data for solving the normal equation based on the output of the learning region extracting device. The normal coefficient adding means to be generated, and the predetermined arithmetic processing performed based on the output of the normal equation adding means, whereby the prediction coefficient determining means comprising the prediction coefficient calculating means for calculating the prediction coefficient, Characteristic image information conversion device. 6. An image information conversion method for performing image information conversion on an input image signal generated by expanding and decoding an image signal compressed and encoded in accordance with the MPEG2 standard, comprising the steps of: A first area cutout step of extracting a pixel area of the input image signal; identifying whether the input image signal has been subjected to intra-field DCT processing or intra-frame DCT processing at the time of compression encoding, on a macroblock basis; A DCT processing identifying step of outputting a code indicating the above, a feature quantity extracting step of extracting a feature quantity based on the pixel data supplied from the first area cutout step, and a result of the processing identifying step; A class code generating step of generating a class code corresponding to the result of the extracting step; Storing the predicted coefficients, and outputting a predicted coefficient corresponding to the result of the class code generating step from among the stored predicted coefficients; and extracting a predetermined pixel area from the input image signal. 2 and a calculation processing step of performing a weighted addition to generate a pixel value based on the result of the second area cutting step and the result of the coefficient storing step. Image information conversion method.