JP2001346209A - Data processing unit and data processing method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processing unit that efficiently applies decoding or the like to data subjected to JPEG coding. SOLUTION: JPEG coded data are subjected to entropy decoding and converted into quantized DCT coefficients, which are fed to a class tap extract circuit 42. The class tap extract circuit 42 extracts the quantized DCT coefficient on the basis of a block of DCT coefficients corresponding to a target pixel and its surrounding blocks to configure a class tap. A classification circuit 43 classifies blocks on the basis of the class tap to output a class code. Then a tap coefficient corresponding to the class code is read from a coefficient table storage section 44, and a cross product arithmetic circuit 45 applies a linear prediction calcutation to the tap coefficient and the quantized DCT coefficient to obtain decoded image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing apparatus, a data processing method, and a recording medium, and more particularly, to a data processing apparatus and a data processing suitable for use in decoding lossy-compressed images and the like. The present invention relates to a method and a recording medium.

[0002]

2. Description of the Related Art For example, digital image data has a large data amount.
A large-capacity recording medium and transmission medium are required. Therefore, in general, recording and transmission are performed after compressing and encoding image data to reduce the data amount.

As a method of compressing and encoding an image, for example, a JPEG (Joint Photo) which is a compression encoding method of a still image is used.
There is an MPEG (Moving Picture Experts Group) method, which is a moving image compression encoding method, and the like.

For example, encoding / decoding of image data according to the JPEG system is performed as shown in FIG.

[0005] That is, FIG. 1A shows a configuration of an example of a conventional JPEG encoding device.

[0006] The image data to be encoded is input to a blocking circuit 1, which divides the input image data into blocks of 64 8x8 pixels. Each block obtained by the blocking circuit 1 is supplied to a DCT (Discrete Cosine Transform) circuit 2. The DCT circuit 2 performs a DCT (Discrete Cosine Transform) process on the block from the blocking circuit 1, and generates one DC (Direct Current) component (DC component),
Of 63 frequency components (AC (Alternating Current) components) (AC components) in the horizontal and vertical directions,
This is converted into a total of 64 DCT coefficients. The 64 DCT coefficients for each block are converted from the DCT circuit 2 to the quantization circuit 3
Supplied to

The quantization circuit 3 quantizes the DCT coefficient from the DCT circuit 2 according to a predetermined quantization table, and uses the quantization result (hereinafter, appropriately referred to as a quantized DCT coefficient) for quantization. The information is supplied to the entropy encoding circuit 4 together with the quantization table.

FIG. 1B shows an example of a quantization table used in the quantization circuit 3. Generally, quantization tables take into account human visual characteristics,
The low-frequency DCT coefficients of high importance are finely quantized,
A quantization step for coarsely quantizing the DCT coefficient of a low-frequency high frequency is set, and thereby the image quality of the image is suppressed from being degraded, and the compression is performed efficiently.

The entropy coding circuit 4 performs an entropy coding process such as Huffman coding on the quantized DCT coefficient from the quantization circuit 3 and adds a quantization table from the quantization circuit 3. Then, the resulting encoded data is output as a JPEG encoded result.

Next, FIG. 1C shows the JPE of FIG.
1 shows a configuration of an example of a conventional JPEG decoding device that decodes encoded data output from a G encoding device.

The encoded data is transmitted to an entropy decoding circuit 1
1, the entropy decoding circuit 11 separates the encoded data into entropy-encoded quantized DCT coefficients and a quantization table. Further, the entropy decoding circuit 11 performs the entropy-encoded quantization D
The CT coefficients are entropy-decoded, and the resulting quantized DCT coefficients are supplied to an inverse quantization circuit 12 together with a quantization table. The inverse quantization circuit 12 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 11 in accordance with the quantization table from the entropy decoding circuit 11, and converts the resulting DCT coefficient into the inverse DCT coefficient.
It is supplied to the circuit 13. The inverse DCT circuit 13 performs an inverse DCT process on the DCT coefficient from the inverse quantization circuit 12, and supplies the resulting (decoded) block of 8 × 8 pixels to the block decomposition circuit 14. The block decomposition circuit 14 obtains and outputs a decoded image by deblocking the block from the inverse DCT circuit 13.

[0012]

SUMMARY OF THE INVENTION The JPEG shown in FIG.
In the encoding device, the quantization circuit 3 can reduce the data amount of the encoded data by increasing the quantization step of the quantization table used for quantizing the block. That is, high compression can be realized.

However, when the quantization step is increased, the so-called quantization error is also increased.
(C) The image quality of the decoded image obtained by the JPEG decoding device is degraded. That is, blur, block distortion, mosquito noise, and the like appear conspicuously in the decoded image.

Therefore, in order to prevent the image quality of the decoded image from deteriorating while reducing the data amount of the encoded data, or to improve the image quality of the decoded image while maintaining the data amount of the encoded data. It is necessary to perform some kind of processing for improving image quality after JPEG decoding.

However, performing processing for improving the image quality after JPEG decoding complicates the processing.
The time until a decoded image is finally obtained also increases.

The present invention has been made in view of such a situation, and is intended to efficiently obtain a high-quality decoded image from a JPEG-coded image or the like. .

[0017]

A first data processing apparatus according to the present invention comprises: an obtaining means for obtaining a tap coefficient obtained by performing learning; and a processing data of interest, which is processing data of interest. The conversion data used to classify into any of several classes is at least
Class tap extracting means for extracting from a block other than the block corresponding to the processing data of interest and outputting it as a class tap; class classification means for performing class classification for obtaining a class of the processing data of interest based on the class tap; And calculating means for calculating a predicted value of the processing data of interest by performing a predetermined prediction operation using the tap coefficients of the class and the conversion data.

In the first data processing device, the operation means can perform a linear primary prediction operation using the tap coefficients and the converted data.

The first data processing device may further include a storage unit for storing the tap coefficients. In this case, the obtaining unit may cause the obtaining unit to obtain the tap coefficients from the storage unit.

In the first data processing device, the converted data may be at least discrete cosine transformed data.

In the first data processing apparatus, the class classifying means includes, based on the DC component or AC component power of the discrete cosine-transformed data, which is the conversion data that is the class tap, the target processing data of the target processing data. You can ask for a class.

In the first data processing apparatus, the class tap extracting means can extract conversion data to be used as class taps from blocks around the block corresponding to the processing data of interest.

In the first data processing apparatus, the class tap extracting means can extract the conversion data to be used as the class tap from the block corresponding to the processing data of interest.

In the first data processing device, the prediction error of the prediction value of the processing data obtained by performing a predetermined prediction operation using the tap coefficient and the conversion data is statistically minimized. Thus, it can be obtained by learning.

In the first data processing device, the data may be moving image or still image data.

According to a first data processing method of the present invention, an acquisition step of acquiring a tap coefficient obtained by performing learning, and a process data of interest, which is a process data of interest, among several classes. A class tap extraction step of extracting, at least, the conversion data used for classifying into any one of the blocks other than the block corresponding to the processing data of interest, and outputting as a class tap,
Based on the class tap, a class classification step of performing a class classification for obtaining a class of the target processing data, and a predetermined prediction operation using the tap coefficient of the class of the target processing data and the conversion data, thereby obtaining the target processing data. And a calculation step of calculating a predicted value of

According to the first recording medium of the present invention, an acquisition step of acquiring a tap coefficient obtained by performing learning, and a process data of interest, which is a process data of interest,
A class tap extraction step of extracting at least conversion data used to classify the class into any of several classes from blocks other than the block corresponding to the processing data of interest and outputting the class tap as a class tap; Based on a class classification step of performing a class classification for obtaining a class of the processing data of interest based on a tap coefficient of the class of the processing data of interest and the conversion data, a prediction operation of the processing data of interest is performed. And a calculation step for calculating

According to a second data processing apparatus of the present invention, a generating means for generating converted data in block units as student data to be a student at the time of learning, and processing data as teacher data to be a teacher at the time of learning The student data used to classify the teacher data of interest, which is the teacher data of interest, into one of several classes is extracted from at least the blocks other than the block corresponding to the teacher data of interest. Class tap extracting means for outputting as a tap, class classifying means for performing class classification for obtaining a class of the teacher data of interest based on the class tap,
Learning is performed so that the prediction error of the prediction value of the teacher data obtained by performing the prediction operation using the tap coefficient for each class and the student data is statistically minimized, and the tap coefficient is obtained for each class. Learning means.

In the second data processing apparatus, the learning means statistically minimizes the prediction error of the prediction value of the teacher data obtained by performing the linear primary prediction operation using the tap coefficients and the student data. Learning can be performed as follows.

In the second data processing device, the generating means can generate the student data by at least performing discrete cosine transform on the data.

In the second data processing apparatus, the class classifying means includes, based on the DC component or AC component power of the discrete cosine transformed data, which is the student data serving as the class tap, for the teacher data of interest. You can ask for a class.

In the second data processing apparatus, the class tap extracting means can extract student data to be used as class taps from blocks around the block corresponding to the teacher data of interest.

In the second data processing device, the class tap extracting means can extract the student data to be used as the class tap from the block corresponding to the teacher data of interest.

In the second data processing device, the data may be moving image or still image data.

According to a second data processing method of the present invention, a generating step of generating converted data in block units as student data to be a student at the time of learning, and processing data as teacher data to be a teacher at the time of learning The student data used to classify the teacher data of interest, which is the teacher data of interest, into one of several classes is extracted from at least the blocks other than the block corresponding to the teacher data of interest. A class tap extraction step of outputting as a tap, a class classification step of performing a class classification for obtaining a class of the teacher data of interest based on the class tap, and a prediction operation using a tap coefficient for each class and student data. Learning is performed so that the prediction error of the prediction value of the obtained teacher data is statistically minimized, and tap The number, characterized in that it comprises a learning step of determining for each class.

In the second recording medium of the present invention, a generation step of generating converted data in block units as student data to be a student at the time of learning, and processing data as teacher data to be a teacher at the time of learning. The student data used to classify the target teacher data, which is the target teacher data, into one of several classes is extracted from at least blocks other than the block corresponding to the target teacher data, and the class tap is performed. A class tap extracting step of outputting as a class tap; a class classification step of performing a class classification for obtaining a class of the teacher data of interest based on the class tap; and a prediction operation using the tap coefficient and the student data for each class. The training is performed so that the prediction error of the predicted value of the teacher data is statistically minimized, and the tap coefficient is Program and a learning step of determining for each class is characterized in that it is recorded.

In the first data processing apparatus, the first data processing method, and the recording medium according to the present invention, at least conversion data used to classify the target processing data into one of several classes includes: It is extracted from blocks other than the block corresponding to the processing data of interest and output as class taps. Then, based on the class tap, a class classification for obtaining a class of the processing data of interest is performed, and a tap coefficient of the class of the processing data of interest,
By performing a predetermined prediction operation using the conversion data and the conversion data, a prediction value of the processing data of interest is obtained.

In the second data processing apparatus, data processing method, and recording medium of the present invention, at least student data used to classify the noted teacher data into any of several classes includes at least: It is extracted from blocks other than the block corresponding to the teacher data of interest and output as class taps. Further, a class classification for obtaining a class of the teacher data of interest is performed based on the class tap. Learning is performed so that the prediction error of the predicted value of the teacher data obtained by performing the prediction operation using the tap coefficient for each class and the student data is statistically minimized. Required for each.

[0039]

FIG. 2 shows a configuration example of an embodiment of an image transmission system to which the present invention is applied.

The image data to be transmitted is transmitted to the encoder 21
The encoder 21 encodes the image data supplied thereto, for example, by JPEG encoding to obtain encoded data. That is, the encoder 21 is configured, for example, in the same manner as the above-described JPEG encoding apparatus shown in FIG. 1A, and JPEG-encodes image data. Encoded data obtained by the encoder 21 performing JPEG encoding includes, for example, a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape,
It is recorded on a recording medium 23 such as a phase change disk or the like.
(Cable Television) is transmitted via a transmission medium 24 such as a network, the Internet, or a public line.

The decoder 22 receives the encoded data provided via the recording medium 23 or the transmission medium 24,
Decode to original image data. The decoded image data is supplied to, for example, a monitor (not shown) and displayed.

FIG. 3 shows an example of the configuration of the decoder 22 shown in FIG.

The encoded data is supplied to the entropy decoding circuit 3
1, the entropy decoding circuit 31 performs entropy decoding of the encoded data, and obtains a quantized DCT coefficient Q for each block obtained as a result.
It is supplied to the coefficient conversion circuit 32. Note that the coded data includes the entropy-coded quantized DCT as in the case described in the entropy decoding circuit 11 in FIG.
In addition to the coefficients, a quantization table is also included. The quantization table can be used for decoding the quantized DCT coefficients as needed, as described later.

The coefficient conversion circuit 32 performs a predetermined prediction operation using the quantized DCT coefficient Q from the entropy decoding circuit 31 and a tap coefficient obtained by performing learning, which will be described later. DCT
The coefficients are decoded into the original block of 8 × 8 pixels.

The block decomposition circuit 33 includes a coefficient conversion circuit 3
By deblocking the decoded block (decoded block) obtained in step 2, a decoded image is obtained and output.

Next, referring to the flowchart of FIG.
The processing of the decoder 22 in FIG. 3 will be described.

The encoded data is supplied to the entropy decoding circuit 3
1, the entropy decoding circuit 31 entropy-decodes the encoded data in step S1, and supplies the quantized DCT coefficient Q for each block to the coefficient transforming circuit 32. The coefficient conversion circuit 32 determines in step S2
In, the quantized DCT coefficient Q for each block from the entropy decoding circuit 31 is decoded into a pixel value for each block by performing a prediction operation using a tap coefficient,
This is supplied to the block decomposition circuit 33. Block decomposition circuit 3
In step S3, block decomposition is performed to unblock the pixel value block (decoded block) from the coefficient conversion circuit 32, and a decoded image obtained as a result is output, and the process ends.

Next, the coefficient conversion circuit 32 shown in FIG. 3 can decode the quantized DCT coefficients into pixel values by using, for example, a class classification adaptive process.

The class classification adaptation process includes a class classification process and an adaptation process. The class classification process classifies data into classes based on the nature of the data, and performs an adaptation process for each class. Is based on the following method.

That is, in the adaptive processing, for example, the quantization DC
The quantized DCT coefficient is decoded into the original pixel value by obtaining the predicted value of the original pixel by a linear combination of the T coefficient and a predetermined tap coefficient.

More specifically, for example, a certain image is now used as teacher data, and the image is subjected to DCT processing in block units and further quantized to obtain a quantized DCT.
Using the coefficients as student data, a prediction value E [y] of a pixel value y of a pixel as teacher data is converted to some quantized DCT coefficients x
_1, x _2, a set of ..., predetermined tap coefficients w _1, w _2,
.. Are considered by a linear first-order combination model defined by the linear combination. In this case, the predicted value E [y]
Can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (1)

To generalize equation (1), a matrix W consisting of a set of tap coefficients w _j , a matrix X consisting of a set of student data x _ij , and a matrix Y consisting of a set of predicted values E [y _j ] '
To

(Equation 1) Defines the following observation equation.

XW = Y ′ (2) Here, the component x _ij of the matrix X is a set of i-th student data (a set of student data used for predicting the _i-th teacher data y _i ). Means the j-th student data in the matrix W, and the component w _{j of the} matrix W represents a tap coefficient by which a product with the j-th student data in the set of the student data is calculated. Also, y
_i represents the i-th teacher data, and therefore, E [y _i ]
Represents the predicted value of the i-th teacher data. Note that y on the left side of the equation (1) is obtained by omitting the suffix i of the component y _i of the matrix Y. Further, x ₁ , x ₂ ,. The suffix i of the component x _ij is omitted.

Then, it is considered that a least square method is applied to this observation equation to obtain a predicted value E [y] close to the original pixel value y. In this case, a true pixel value y serving as teacher data
And a predicted value E for a pixel value y
A matrix E consisting of a set of residuals e of [y] is

(Equation 2) From equation (2), the following residual equation is established.

XW = Y + E (3)

In this case, the predicted value E close to the original pixel value y
The tap coefficient w _j for obtaining [y] is a square error

(Equation 3) Can be obtained by minimizing.

Therefore, the above square error is calculated by tap coefficient w _j
Is zero, that is, a tap coefficient w _j that satisfies the following equation is an optimum value for obtaining a predicted value E [y] close to the original pixel value y.

[0059]

(Equation 4) ... (4)

Therefore, first, the equation (3) is changed to the tap coefficient w
By differentiating with _j , the following equation is established.

[0061]

(Equation 5) ... (5)

From equations (4) and (5), equation (6) is obtained.

[0063]

(Equation 6) ... (6)

Further, considering the relationship among the student data x _ij , the tap coefficient w _j , the teacher data y _i , and the residual e _{i in} the residual equation of the equation (3), the following equation is obtained from the equation (6). A normal equation can be obtained.

[0065]

(Equation 7) ... (7)

The normal equation shown in equation (7) is obtained by converting a matrix (covariance matrix) A and a vector v into

(Equation 8) If the vector W is defined as shown in Expression 1, it can be expressed by the following expression: AW = v (8)

Each normal equation in the equation (7) is prepared by preparing a certain number of sets of the student data x _ij and the teacher data y _i , and forming the same number as the number J of the tap coefficients w _{j to} be obtained. And therefore equation (8)
Is solved for the vector W (however, in order to solve the equation (8), the matrix A in the equation (8) needs to be regular) to obtain an optimal tap coefficient (here, the square error is minimized). Tap coefficient) w _j can be obtained. In addition,
In solving equation (8), for example, the sweeping method (Ga
uss-Jordan elimination method) can be used.

As described above, the optimum tap coefficient w _j
The adaptive processing is to obtain a predicted value E [y] close to the original pixel value y by using the tap coefficient _wj and using the equation (1).

For example, as teacher data, JPE
While using an image of the same image quality as the image to be G-coded,
When DCT and quantized DCT coefficients obtained by quantizing the teacher data are used as student data, a prediction error is used as a tap coefficient when decoding JPEG-coded image data into original image data. However, a statistically minimum one is obtained.

Therefore, even if the compression ratio at the time of performing JPEG encoding is increased, that is, even if the quantization step used for quantization is made coarse, according to the adaptive processing, the prediction error is statistically minimized. Will be performed, and in effect,
The decoding process of the JPEG encoded image and the process of improving the image quality are performed at the same time.
As a result, even if the compression ratio is increased, the image quality of the decoded image can be maintained.

For example, as teacher data, JPE
An image having a higher image quality than the image to be G-encoded is used, and the image quality of the teacher data is used as student data according to JP.
The image quality is degraded to the same image quality as the image to be EG-coded.
When a quantized DCT coefficient obtained by CT and quantization is used, as a tap coefficient, when decoding JPEG-coded image data into high-quality image data, the prediction error is statistically minimized. Will be obtained.

Therefore, in this case, according to the adaptive processing, J
The decoding process of the PEG-encoded image and the process of further improving the image quality are performed at the same time. As described above, by changing the image quality of the image serving as the teacher data or the student data, it is possible to obtain a tap coefficient for setting the image quality of the decoded image to an arbitrary level.

In the above case, the image data is used as the teacher data, and the quantized DCT coefficients are used as the student data. In addition, for example, the DCT coefficients are used as the teacher data, and the student data is used as the student data. DCT
It is also possible to use quantized DCT coefficients obtained by quantizing the coefficients. In this case, according to the adaptive processing, a DC in which the quantization error is reduced (suppressed) from the quantized DCT coefficient.
A tap coefficient for predicting the T coefficient is obtained.

FIG. 5 shows a first configuration example of the coefficient conversion circuit 32 shown in FIG. 3 for decoding quantized DCT coefficients into pixel values by the above-described class classification adaptive processing.

The quantized DCT coefficients for each block output from the entropy decoding circuit 31 (FIG. 3) are supplied to the prediction tap extraction circuit 41 and the class tap extraction circuit 42.

The prediction tap extraction circuit 41 supplies a block of quantized DCT coefficients (hereinafter referred to as DCT
A block having a pixel value corresponding to the block (hereinafter, referred to as a block) (hereinafter, referred to as a pixel block, as appropriate)
The target pixel block is sequentially set as a target pixel block, and each pixel constituting the target pixel block is set as a target pixel sequentially in a so-called raster scan order, for example. Further, the prediction tap extracting circuit 41 extracts a quantized DCT coefficient used for predicting the pixel value of the pixel of interest, and sets it as a prediction tap.

That is, for example, as shown in FIG. 6A, the prediction tap extracting circuit 41 outputs all the quantized DCs of the DCT block corresponding to the pixel block to which the pixel of interest belongs.
T coefficients, ie, 64 8 × 8 quantized DCT coefficients,
Extract as a prediction tap. Therefore, in the present embodiment, the same prediction tap is formed for all pixels in a certain pixel block. However, the prediction tap can be configured with different quantized DCT coefficients for each target pixel.

The quantized DCT constituting the prediction tap
The coefficients are not limited to those of the pattern described above.

The prediction taps for each pixel constituting the pixel block, that is, 64 sets of prediction taps for each of the 64 pixels, obtained in the prediction tap extraction circuit 41, are supplied to the product-sum operation circuit 45. However, in the present embodiment, as described above, since the same prediction tap is configured for all pixels of the pixel block, one set of prediction taps is actually set for one pixel block. It may be supplied to the product-sum operation circuit 45.

The class tap extracting circuit 42 extracts a quantized DCT coefficient used for class classification for classifying the target pixel into one of several classes, and sets it as a class tap.

In JPEG encoding, an image is encoded (DCT processing and quantization) for each pixel block. Therefore, all pixels belonging to a certain pixel block are classified into the same class, for example. And
Therefore, the class tap extracting circuit 42 forms the same class tap for each pixel of a certain pixel block.

That is, in the present embodiment, the class tap extracting circuit 42 includes, for example, as shown in FIG. 6B, a DCT block corresponding to a pixel block to which a pixel of interest belongs, and four DCT blocks adjacent vertically and horizontally. Of DCT blocks
320 of 5 DCT blocks in total (= 8 × 8 × 5)
The quantized DCT coefficients are extracted as class taps.

Here, each pixel belonging to the pixel block is represented by
Classifying all pixel classes into the same class is equivalent to classifying the pixel block. Therefore, the class tap extracting circuit 42 configures one set of class taps for classifying the target pixel block, instead of 64 sets of class taps for classifying each of the 64 pixels constituting the target pixel block. Therefore, the class tap extraction circuit 42
Extracts, for each pixel block, a DCT block corresponding to the pixel block and quantized DCT coefficients of four DCT blocks adjacent above, below, left, and right to classify the pixel block into classes. And so on.

Note that the quantized DC constituting the class tap is
The T coefficient is not limited to the pattern described above.

That is, in JPEG encoding, a DCT block composed of 8 × 8 quantized DCT coefficients is formed by performing DCT and quantization in units of 8 × 8 pixel blocks. The pixels of the block
When decoding by the classification adaptive processing, it is conceivable that only the quantized DCT coefficient of the DCT block corresponding to the pixel block is used as a class tap.

However, in an image, when attention is paid to a certain pixel block, there is generally not less than a correlation between pixels of the pixel block and pixels of peripheral pixel blocks. Therefore, as described above, not only the DCT block corresponding to a certain pixel block, but also the other DCT blocks, the quantized DCT block.
By extracting coefficients and using them as class taps, the target pixel can be more appropriately classified, and as a result, only the quantized DCT coefficients of the DCT block corresponding to the pixel block are used as class taps. As compared with the case, the image quality of the decoded image can be improved.

Here, in the above case, the DCT block corresponding to a certain pixel block and the quantized DCT coefficients of the four DCT blocks adjacent to the upper, lower, left and right sides are set as class taps. Quantized DCT coefficients to be used as taps are other than DCT coefficients corresponding to a certain pixel block.
DCT blocks obliquely adjacent to CT blocks,
It may be extracted from a DCT block or the like that is not adjacent but is adjacent. That is, the range of DCT blocks from which quantized DCT coefficients to be used as class taps are extracted is not particularly limited.

The class tap of the target pixel block obtained by the class tap extracting circuit 42 is supplied to the class classifying circuit 43. The class classifying circuit 43 converts the class tap from the class tap extracting circuit 42 into the class tap. Based on this, the target pixel block is classified into classes, and a class code corresponding to the resulting class is output.

Here, as a method of classifying,
For example, ADRC (Adaptive Dynamic Range Coding) or the like can be adopted.

In the method using ADRC, the quantized DCT coefficients constituting the class tap are subjected to ADRC processing, and the class of the target pixel block is determined according to the ADRC code obtained as a result.

In the K-bit ADRC, for example,
The maximum value MAX of the quantized DCT coefficient constituting the class tap
And the minimum value MIN is detected, and DR = MAX-MIN is set as the local dynamic range of the set.
, The quantized DCT coefficients making up the class tap are re-quantized to K bits. That is, from among the quantized DCT coefficients forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided (quantized) by DR / 2 ^K. Then, a bit string obtained by arranging the K-bit quantized DCT coefficients constituting the class tap in a predetermined order, which is obtained as described above, is output as an ADRC code. Therefore,
When the class tap is subjected to, for example, 1-bit ADRC processing, the quantized DCT coefficients constituting the class tap are obtained by subtracting the minimum value MIN from the maximum value MAX and the minimum value MI.
Divided by the average value with N, so that each quantized DCT
The coefficient is one bit (binarized). Then, a bit string in which the 1-bit quantized DCT coefficients are arranged in a predetermined order is output as an ADRC code.

The class classification circuit 43 includes, for example,
It is also possible to output the pattern of the level distribution of the quantized DCT coefficients constituting the class taps as it is as the class code. In this case, however, the class tap is constituted by N quantized DCT coefficients, and Assuming that K bits are assigned to the DCT coefficient, the number of class codes output from the classifying circuit 43 is (2
^N) becomes ^K Street, an enormous number exponentially proportional to the number of bits K of quantized DCT coefficients.

Therefore, in the classification circuit 43,
It is preferable to perform the class classification after compressing the information amount of the class tap by the above-described ADRC processing or vector quantization.

By the way, in the present embodiment, the class tap is composed of 320 quantized DCT coefficients as described above. Therefore, for example, if the class tap is 1
By bit ADRC processing, as to perform the classification, the number of cases of the class code is a huge value of 2 ³²⁰ ways.

Therefore, in the present embodiment, the classifying circuit 43 uses the quantized DCT constituting the class tap.
A feature amount having high importance is extracted from the coefficient, and the class is classified based on the feature amount, thereby reducing the number of classes.

That is, FIG. 7 shows the classification circuit 43 of FIG.
Is shown.

The class taps are supplied to the power calculation circuit 51. The power calculation circuit 51 divides the quantized DCT coefficients constituting the class taps into several spatial frequency bands, and Calculate the power in the frequency band.

That is, in the present embodiment, the class tap is composed of quantized DCT coefficients of five DCT blocks as shown in FIG.
1 is 8 × of each DCT block constituting the class tap.
The eight quantized DCT coefficients are divided into, for example, four spatial frequency bands S ₀ , S ₁ , S ₂ , and S _{3 as shown} in FIG.

Here, each of the 8 × 8 quantized DCT coefficients of one DCT block is represented by an alphabet x,
As shown in FIG. 6A, a sequential integer from 0 is added to the raster scan order.
The spatial frequency band S ₀ has four quantized DCT coefficients x ₀ , x
_1, is composed of x _8, x _9, the spatial frequency band S ₁ is 12
Quantized DCT coefficients x ₂ , x ₃ , x ₄ , x ₅ , x ₆ , x ₇ ,
_{x 10, x 11, x 12} , x 13, composed of x _14, x _15.
Further, the spatial frequency band S ₂ is 12 quantized DCT coefficients _{x 16, x 17, x 24} , x 25, x 32, x 33, x 40, x 41,
consists _{x 48, x 49, x 56} , x 57, the spatial frequency band S ₃ is 36 quantized DCT coefficients _{x 1 8, x 19, x} 20,
_{x 21, x 22, x 23} , x 26, x 27, x 28, x 29, x 30, x
_{31, x 3 4, x 35} , x 36, x 37, x 38, x 39, x 42,
_{x 43, x 44, x 45} , x 46, x 47, x 5 0, x 51, x 52, x
_{53, x 54, x 55,} x 58, x 59, x 60, x 61, x 62, x 63
Consists of

Further, the power operation circuit 51 performs the following for each of the five DCT blocks constituting the class tap.
For each of the spatial frequency bands S ₀ , S ₁ , S ₂ , S ₃ , the powers P ₀ , P ₁ , AC of the AC components of the quantized DCT coefficients
P ₂ and P ₃ are calculated and output to the class code generation circuit 52.

That is, the power operation circuit 51 operates in the spatial frequency band.
Area S₀For the above four quantized DCT coefficients x₀,
x₁, X₈, X₉AC component x of₁, X₈, X₉Sum of squares
x₁ ^Two+ X₈ ^Two+ X₉ ^TwoAnd calculate the power P₀As
Output to the Las code generation circuit 52. Also, the power calculation time
The road 51 corresponds to the above-mentioned 12 for the spatial frequency band S1.
AC components of the quantized DCT coefficients, ie, all 12
Is calculated, and the sum of squares of the quantized DCT coefficients of₁
To the class code generation circuit 52. Further
In addition, the power operation circuit 51 has a spatial frequency band S_TwoAnd S_ThreeNitsu
The spatial frequency band S₁As in
And each power P_TwoAnd P_ThreeAnd generate class code
Output to the circuit 52.

The class code generation circuit 52 calculates the powers P ₀ , P ₁ , P ₂ , and P ₃ from the power operation circuit 51 for each of the five DCT blocks constituting the class tap,
The corresponding threshold value T stored in the threshold value table storage unit 53
H ₀ , TH ₁ , TH ₂ , and TH ₃ are compared with each other, and a class code is output based on the magnitude relation. That is, the class code generation circuit 52 determines the power P ₀ and the threshold TH ₀
And a 1-bit code representing the magnitude relationship is obtained. Similarly, the class code generation circuit 52 determines the power P ₁ and the threshold TH ₁ , the power P ₂ and the threshold TH ₂ , and the power P ₃ and the threshold TH ₃
By comparing
Get 1-bit code.

[0103] The class code generation circuit 52 performs the following operations on each of the five DCT blocks constituting the class tap.
As described above, four 1-bit codes, that is, a total of 20-bit codes are obtained. Then, the class code generation circuit 52 outputs the 20-bit code as a class code representing the class of the target pixel block. In this case, the target pixel block will be classification into one of 2 ²⁰ classes.

The threshold value table storage unit 53 stores threshold values TH _{0 to} TH ₃ to be compared with the electric powers P _{0 to} P ₃ of the spatial frequency bands S _{0 to} S ₃ , respectively.

In the above case, the DC component x ₀ of the quantized DCT coefficient is not used in the class classification process, but the class classification process can be performed using the DC component x _0. .

Returning to FIG. 5, the class code output from the classifying circuit 43 as described above is stored in the coefficient table storage unit 4.
4 is given as an address.

The coefficient table storage section 44 stores a coefficient table in which tap coefficients obtained by performing a learning process described later are registered, and stores an address corresponding to the class code output from the class classification circuit 43. Is output to the product-sum operation circuit 45.

Here, in the present embodiment, the pixel blocks are classified into classes, so that the target pixel block is
One class code is obtained. On the other hand, in the present embodiment, since the pixel block is composed of 64 pixels of 8 × 8 pixels, 64 sets of tap coefficients for decoding each of the 64 pixels constituting the target pixel block are required. is there. Therefore, the coefficient table storage unit 44 stores, for an address corresponding to one class code,
64 sets of tap coefficients are stored.

The product-sum operation circuit 45 includes a prediction tap output from the prediction tap extraction circuit 41 and a coefficient table storage unit 44.
Is obtained, and the linear prediction operation (product-sum operation) shown in Expression (1) is performed using the prediction taps and the tap coefficients. The pixel value of the pixel is output to the block decomposition circuit 33 (FIG. 3) as a decoding result of the corresponding DCT block.

Here, in the prediction tap extracting circuit 41, as described above, each pixel of the target pixel block is sequentially set as the target pixel. Operation mode corresponding to the position of the pixel (hereinafter, appropriately referred to as pixel position mode)
And perform the processing.

That is, for example, among the pixels of the pixel block of interest, the i-th pixel in the raster scan order is represented as p _i, and when the pixel p _i is the pixel of interest, the product-sum operation circuit 45 , The processing of the pixel position mode #i is performed.

[0112] More specifically, as described above, the coefficient table storage unit 44 is to output the tap coefficients of the 64 sets for decoding the respective 64 pixels constituting the pixel block of interest, the pixel p _i of which If the set of tap coefficients for decoding denoted W _i, sum-of-products operation circuit 45, the operation mode is, when the pixel position mode #i includes a prediction tap, and a set W _i of the tap coefficients of the 64 sets Is used, and the result of the product-sum operation is used as the decoding result of the pixel p _i .

Next, referring to the flowchart of FIG.
The processing of the coefficient conversion circuit 32 in FIG. 5 will be described.

The quantized DCT coefficient for each block output from the entropy decoding circuit 31
1 and the class tap extraction circuit 42 sequentially receive the prediction block, and the prediction tap extraction circuit 41 sequentially sets the pixel blocks corresponding to the blocks of the quantized DCT coefficients (DCT blocks) supplied thereto as target pixel blocks.

In step S11, the class tap extracting circuit 42 uses the quantized DCT coefficients received there to classify the target pixel block, that is, in this embodiment, the class tap extraction circuit 42 uses the target pixel block. , And a total of five quantized DCT coefficients of the DCT block corresponding to, and four DCT blocks adjacent to the upper, lower, left, and right sides thereof are extracted to form a class tap, which is supplied to the class classification circuit 43. I do.

In step S12, the class classification circuit 43 classifies the pixel block of interest using the class tap from the class tap extraction circuit 42, and outputs the resulting class code to the coefficient table storage unit 44. .

That is, in step S12, as shown in the flowchart of FIG.
, The power calculation circuit 51 of the class classification circuit 43 (FIG. 7) calculates the power P _{0 of} each of the four spatial frequency bands S _{0 to} S ₃ shown in FIG. 8 for each of the five DCT blocks constituting the class tap. or it calculates the P _3. The powers P _{0 to} P ₃ are output from the power calculation circuit 51 to the class code generation circuit 52.

The class code generation circuit 52 determines in step S
At 22, the threshold value TH _{0 is} stored in the threshold value table storage unit 53.
To read the TH _3, compared from power calculating circuit 51, the five DCT blocks forming the class tap and the power P ₀ to P _3, respectively, and a threshold value TH0 to TH3 respectively, based on the respective magnitude relationships Generate class code and return.

Returning to FIG. 9, the class code obtained as described above in step S12 is given from the class classification circuit 43 to the coefficient table storage unit 44 as an address.

When the coefficient table storage unit 44 receives the class code as the address from the class classification circuit 43, it reads out the 64 sets of tap coefficients stored at the address in step S13, and sends it to the product-sum operation circuit 45. Output.

Then, the process proceeds to step S14, in which the prediction tap extracting circuit 41 sets, as a target pixel, a pixel which has not been set as the target pixel in the raster scan order among the pixels of the target pixel block, and sets the target pixel as a target pixel. A quantized DCT coefficient used for predicting a value is extracted and configured as a prediction tap. This prediction tap is a prediction tap extraction circuit 4
1 is supplied to the product-sum operation circuit 45.

Here, in this embodiment, for each pixel block, all the pixels in the pixel block are
Since the same prediction tap is configured, in practice, if the processing of step S14 is performed only on the pixel that is initially set as the target pixel for the target pixel block, the remaining 63
There is no need to do this for pixels.

In step S15, the product-sum operation circuit 45 acquires a set of tap coefficients corresponding to the pixel position mode for the target pixel from among the 64 sets of tap coefficients output from the coefficient table storage unit 44 in step S13. Using the set of tap coefficients and the prediction tap supplied from the prediction tap extraction circuit 41 in step S14, the product-sum operation shown in Expression (1) is performed to obtain a decoded value of the pixel value of the target pixel.

Then, proceeding to step S16, the prediction tap extracting circuit 41 determines whether or not all pixels of the target pixel block have been processed as the target pixel.
If it is determined in step S16 that all the pixels of the target pixel block have not been processed yet as the target pixel, the process returns to step S14, and the prediction tap extraction circuit 41 returns the raster tap among the pixels of the target pixel block. In the scanning order, a pixel that has not been set as a target pixel is set as a new target pixel, and the same processing is repeated.

If it is determined in step S16 that all pixels of the target pixel block have been processed as the target pixel, that is, if the decoded values of all the pixels of the target pixel block have been obtained, The arithmetic circuit 45
Pixel block composed of the decoded value (decoded block)
Is output to the block decomposition circuit 33 (FIG. 3), and the process is terminated.

The process according to the flowchart of FIG. 9 is repeated each time the prediction tap extracting circuit 41 sets a new pixel block of interest.

Next, in the above case, the classifying circuit 4
In 3, for each of five DCT blocks forming the class taps, the power P ₀ to P ₃ of the same spatial frequency band S ₀ to S ₃ calculated, based on the power,
The class classification is performed. In addition, the class classification is performed based on the calculated power of different spatial frequency bands for some of the five DCT blocks forming the class tap, for example. It is also possible.

That is, for example, as shown by hatching in FIG. 11, of the five DCT blocks constituting the class tap, the DCT block corresponding to the target pixel block (hereinafter referred to as the target DCT block as appropriate) is the power P _v and the power P _h in the horizontal direction of the high-frequency band in the vertical direction of the high frequency band, for the DCT block adjacent to the upper side of the target DCT block, the power P _u in the vertical direction of the high frequency band, for DCT block adjacent to the bottom of the target DCT block, the power P _d in the vertical direction of the high-frequency band, the DCT block adjacent to the left of the target DCT block, the power P _l horizontal high frequency band
The, for the DCT block to the right of the target DCT block, the power P _r of the horizontal high frequency band, respectively calculated, their power _{P v, P h, P u} , P d, P l, P r , It is possible to perform the class classification in the same manner as in the case described with reference to FIGS.

In this case, the classification circuit 4 shown in FIG.
In 3, the process as shown in FIG. 12 is performed in step S12 in FIG.

That is, first, in step S31, the power calculation circuit 51 of the class classification circuit 43 (FIG. 7)
But the five DCT blocks forming the class tap, and arithmetic power P _v of each frequency band as described in FIG. _{11, P h, P u,} P d, P l, and P _r, the class code generation Output to the circuit 52.

The class code generation circuit 52 executes the processing in step S
At 32, the threshold is read from the threshold table storage 53.
put out. Here, the threshold table storage unit 53 stores
Force P _v, P_h, P_u, P_d, P_l, P_rCompare with each
Threshold TH_v, TH_h, TH _u, TH_d, TH_l, TH_rBut
It shall be stored.

The class code generation circuit 52 stores the threshold values TH _v , TH _h , TH _u , TH _d , T
When H _l and TH _r are read, each is read by the power operation circuit 5.
Power P _v from _{1, P h, P u,} P d, compared to P _l, P _r respectively, based on the respective magnitude relation, obtaining six 1 bits. Then, the class code generation circuit 52 generates a 6-bit code including the six 1-bit codes,
Output as class code and return. Therefore, in this case, the target pixel (target pixel block) is 64 (= 2
⁶ ) Classification into one of the classes.

Next, in the above description, the AC component of the quantized DCT coefficient as a class tap is used for the class classification. However, the class classification can also be performed using the DC component of the quantized DCT coefficient. It is possible.

That is, the class classification is, for example, as shown in FIG.
As shown, the DC component C of the DCT block of interest₀,Moreover
DC component C of each DCT block adjacent to the lower left and right
_u, C _d, C_l, C_rIt is possible to perform using.

In this case, the class classification circuit 43 of FIG.
For example, it is configured as shown in FIG.

The class taps are supplied to a difference calculation circuit 151. The difference calculation circuit 151 selects one of the five DCT blocks constituting the class tap, which is adjacent to the DCT block above, below, left and right. DC component of block _{C u, C d, C l} , and C _r, respectively, attention DCT
Absolute value D _u of the difference between the DC component C ₀ of the block, D _d,
Dl and _Dr are calculated and supplied to the class code generation circuit 152. That is, the difference calculation circuit 151 calculates the expression _Du = | _Cu- C
₀ |, D _d = | C _d −C ₀ |, D _l = | C ₁ −C ₀ |, _Dr = |
C _r −C ₀ | is calculated, and the calculation result is supplied to the class code generation circuit 152.

The class code generation circuit 152 calculates the operation results (absolute difference values) D _u , D _d and D from the difference operation circuit 151.
_l, the D _r, stored in the threshold table storage unit 153, compares corresponding threshold TH _u, TH _d, TH _l, and TH _r respectively, based on the respective magnitude relationship, and outputs the class code. That is, the class code generating circuit 152 compares the difference absolute value D _u and the threshold TH _u, representing the magnitude relation 1
Get a bit of code. Similarly, the class code generation circuit 152 calculates the difference absolute value D _d , the threshold value TH _d , and the difference absolute value D _l
The threshold TH _l, the difference absolute value D _r and the threshold TH _r, by comparing each for each, to obtain a 1-bit code.

Then, the class code generation circuit 152
The four 1-bit codes obtained as described above are
For example, a 4-bit code (accordingly, any value from 0 to 15) obtained by arranging in a predetermined order is output as a class code representing the class of the pixel block of interest. Therefore, in this case, the target pixel block is classified into any one of 2 ⁴ (= 16) classes.

The threshold value table storage unit 153 stores threshold values TH _u and T _u to be compared with the absolute difference values D _u , D _d , D _l and _Dr respectively.
H _d ,, TH _l, stores TH _r.

When the classifying circuit 43 is configured as shown in FIG. 14, in step S12 in FIG.
The processing shown in FIG.

That is, in this case, first, step S
In 41, the difference calculation circuit 151, five DCT blocks each DC component constituting the class tap _{C 0, C u, C d} , C l, using a C _r, the above-described difference absolute value D _u, D _d , _Dl , and _Dr are calculated, and the class code generation circuit 1
52.

In step S42, the class code generation circuit 152 compares the threshold values TH _u , TH _d , TH _l , and TH _r stored in the threshold value table storage section 153 with the absolute difference values D _u , D _d, compared to D _l, D _r respectively, to obtain the four 1-bit code representing the magnitude relationship. Then, the class code generation circuit 152
The 4-bit code consisting of the four 1-bit codes is output as a class code, and the process returns.

The class classification is performed using only the AC component or only the DC component of the quantized DCT coefficient.
It is also possible to carry out using both the C component and the DC component. That is, the method of class classification is not limited to the method described above.

Next, FIG. 16 shows an example of the configuration of an embodiment of the learning apparatus for performing the learning process of the tap coefficients stored in the coefficient table storage section 44 of FIG.

One or more pieces of learning image data are supplied to the blocking circuit 61. The blocking circuit 61 converts the learning image into the same data as in the case of JPEG encoding. , 8 × 8 pixels.

The DCT circuit 62 sequentially reads out the pixel blocks formed by the blocking circuit 61 as a pixel block of interest, and subjects the pixel block of interest to a DCT process to obtain a block of DCT coefficients. This DC
The block of the T coefficient is supplied to the quantization circuit 63.

The quantization circuit 63 quantizes the block of DCT coefficients from the DCT circuit 62 in accordance with the same quantization table used for JPEG encoding, and obtains a block of quantized DCT coefficients (DCT coefficient) obtained as a result. Block) are sequentially supplied to the prediction tap extraction circuit 64 and the class tap extraction circuit 65.

The prediction tap extraction circuit 64 selects, from the pixels of the pixel block of interest, pixels which have not been set as the pixel of interest in the raster scan order as the pixel of interest, and sets the prediction tap extraction circuit of FIG. From the output of the quantization circuit 63, the same prediction tap
It is configured by extracting necessary quantized DCT coefficients. This prediction tap is sent from the prediction tap extraction circuit 64 to the normal equation addition circuit 6 as student data to be a student during learning.
7 is supplied.

The class tap extracting circuit 65 extracts, from the output of the quantizing circuit 63, the same quantized DCT coefficients as the class taps formed by the class tap extracting circuit 42 of FIG. It is constituted by doing. This class tap is a class tap extraction circuit 65.
Is supplied to the class classification circuit 66.

The class classification circuit 66 performs the same processing as the class classification circuit 43 of FIG. 5 using the class taps from the class tap extraction circuit 65, thereby classifying the pixel block of interest and obtaining the result. Class code
It is supplied to the normal equation adding circuit 67.

The normal equation adding circuit 67 reads out the pixel of interest (the pixel value thereof) from the blocking circuit 61 as teacher data, and the quantized DCT coefficient constituting the prediction tap as the student data from the prediction tap forming circuit 64. ) And addition for the pixel of interest.

That is, the normal equation addition circuit 67 uses the prediction taps (student data) for each class corresponding to the class code supplied from the class classification circuit 66 to generate each component in the matrix A of the equation (8). Multiplication (x _in x _im ) between the student data, and an operation corresponding to summation (Σ).

Further, the normal equation adding circuit 67 also uses the prediction tap (student data) and the pixel of interest (teacher data) for each class corresponding to the class code supplied from the class classification circuit 66 to obtain the equation (8). )), Multiplication (x _in y _i ) of student data and teacher data, which are components in the vector v, and an operation corresponding to summation (Σ) are performed.

The normal equation adding circuit 67
The above-described addition is performed for each class in each pixel position mode for the target pixel.

The normal equation adding circuit 67 performs the above addition using all the pixels constituting the learning image supplied to the blocking circuit 61 as a pixel of interest. Then, the normal equation shown in equation (8) is established.

The tap coefficient determination circuit 68 solves the normal equation generated for each class (and for each pixel position mode) in the normal equation addition circuit 67, thereby obtaining 64 sets of tap coefficients for each class. The coefficients are supplied to addresses corresponding to each class in the coefficient table storage unit 69.

Depending on the number of images prepared as learning images, the content of the images, and the like, the normal equation adding circuit 67 may not be able to obtain the number of normal equations necessary for obtaining the tap coefficients. In such a case, the tap coefficient determination circuit 68 outputs, for example, a default tap coefficient for such a class.

The coefficient table storage unit 69 stores 64 sets of tap coefficients for each class supplied from the tap coefficient determination circuit 68.

Next, the processing (learning processing) of the learning apparatus of FIG. 16 will be described with reference to the flowchart of FIG.

The image data for learning is supplied to the blocking circuit 61, and the blocking circuit 61 proceeds to step S5.
In step 1, the learning image data is divided into 8 × 8 pixel blocks as in the case of JPEG encoding, and the process proceeds to step S52. In step S52, the DCT circuit 62 sequentially reads out the pixel blocks that have been blocked by the blocking circuit 61, performs DCT processing on the pixel blocks to obtain DCT coefficient blocks, and proceeds to step S53. In step S53, the quantization circuit 63 outputs the DCT obtained by the DCT circuit 62.
Coefficient blocks are sequentially read out and quantized in accordance with the same quantization table used for JPEG encoding to obtain blocks (DCT blocks) composed of quantized DCT coefficients.

Then, the process proceeds to step S54, where the class tap extracting circuit 65 sets a pixel block which has not been set as a target pixel block among the pixel blocks divided by the blocking circuit 61 as a target pixel block. further,
The class tap extracting circuit 65 extracts a quantized DCT coefficient used for classifying the pixel block of interest from the DCT block obtained by the quantizing circuit 63, forms a class tap, and supplies the class tap to the class classifying circuit 66. I do. In step S55, the class classification circuit 66 classifies the pixel block of interest using the class tap from the class tap extraction circuit 65, as in the case described with reference to the flowchart of FIG. Is supplied to the normal equation adding circuit 67, and the process proceeds to step S5.
Proceed to 6.

In step S56, the prediction tap extracting circuit 64 sets a pixel which has not been set as the target pixel in the raster scan order among the pixels of the target pixel block as the target pixel, and sets the target pixel in FIG. The same prediction tap as that formed by the prediction tap extraction circuit 41 is configured by extracting necessary quantized DCT coefficients from the output of the quantization circuit 63. Then, the prediction tap extraction circuit 64
The prediction tap for the pixel of interest is used as student data.
This is supplied to the normal equation addition circuit 67, and the process proceeds to step S57.

In step S57, the normal equation adding circuit 67 reads out the pixel of interest from the blocking circuit 61 as teacher data, predicts taps as student data (quantized DCT coefficients forming the same), and pays attention as teacher data. The above-described addition of the matrix A and the vector v in Expression (8) is performed on the pixels. This addition is performed for each class corresponding to the class code from the class classification circuit 66 and for each pixel position mode for the target pixel.

Then, proceeding to step S58, the prediction tap extracting circuit 64 determines whether or not all pixels of the target pixel block have been added as the target pixel. If it is determined in step S58 that all the pixels of the target pixel block have not been added as the target pixels, the process returns to step S56, and the prediction tap extracting circuit 64 determines, from among the pixels of the target pixel block, In the raster scan order, a pixel that has not been set as a target pixel is newly set as a target pixel, and the same processing is repeated thereafter.

If it is determined in step S58 that all the pixels of the target pixel block have been added as the target pixel, the process proceeds to step S59, where the blocking circuit 61 obtains the image from the learning image. It is determined whether or not all the processed pixel blocks have been processed as target pixel blocks. In step S59, if it is determined that all the pixel blocks obtained from the learning image have not been processed yet as the pixel block of interest, the process returns to step S54, and the pixels blocked by the blocking circuit 61 are returned. Of the blocks, those that have not yet been set as the target pixel block are newly set as the target pixel block, and the same processing is repeated thereafter.

On the other hand, if it is determined in step S59 that all the pixel blocks obtained from the learning image have been processed as the target pixel block, that is, the normal equation When the normal equation for each position mode is obtained, the process proceeds to step S60, where the tap coefficient determination circuit 68 solves the normal equation generated for each class of the pixel position mode, and for each class, 64 sets of tap coefficients corresponding to each of the 64 pixel position modes are obtained, supplied to and stored in the coefficient table storage unit 69 at addresses corresponding to each class, and the process is terminated.

As described above, the coefficient table storage unit 6
The tap coefficients for each class stored in 9 are stored in the coefficient table storage unit 44 in FIG.

Therefore, the tap coefficients stored in the coefficient table storage unit 44 are such that the prediction error (here, the square error) of the prediction value of the original pixel value obtained by performing the linear prediction operation is statistically minimized. The coefficient conversion circuit 32 shown in FIG. 5 decodes a JPEG-coded image into an image that is as close as possible to the original image. Can be.

As described above, the decoding process of the JPEG-coded image and the process of improving the image quality are performed at the same time. Thus, a decoded image with good image quality can be obtained.

Next, FIG. 18 shows the coefficient conversion circuit 32 of FIG.
2 shows a second configuration example. In the figure, portions corresponding to those in FIG. 5 are denoted by the same reference numerals, and a description thereof will be omitted as appropriate below. That is, the coefficient conversion circuit 32 in FIG. 18 has basically the same configuration as that in FIG. 5 except that an inverse quantization circuit 71 is newly provided.

In the embodiment of FIG. 18, the inverse quantization circuit 71 is supplied with quantized DCT coefficients for each block obtained by entropy decoding the encoded data in the entropy decoding circuit 31 (FIG. 3). .

As described above, the entropy decoding circuit 31 converts the coded data from the quantized DCT
In addition to the coefficients, a quantization table is obtained. In the embodiment shown in FIG. 18, this quantization table is also supplied from the entropy decoding circuit 31 to the inverse quantization circuit 71.

The inverse quantization circuit 71 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 31 in accordance with the quantization table from the entropy decoding circuit 31, and outputs the resulting DCT coefficient to the prediction tap extraction circuit 41. And to the class tap extraction circuit 42.

Therefore, in the prediction tap extraction circuit 41 and the class tap extraction circuit, not the quantized DCT coefficient but
A prediction tap and a class tap are respectively configured for DCT coefficients, and thereafter, for DCT coefficients,
The same processing as in FIG. 5 is performed.

As described above, in the embodiment of FIG. 18, the processing is performed not on the quantized DCT coefficient but on the DCT coefficient (the inversely quantized DCT coefficient). Need to be different from that in FIG.

FIG. 19 shows an example of the configuration of an embodiment of a learning apparatus for performing a learning process of tap coefficients stored in the coefficient table storage section 44 of FIG. In the figure, portions corresponding to the case in FIG. 16 are denoted by the same reference numerals, and the description thereof will be appropriately omitted below. That is, the learning device in FIG. 19 is configured basically in the same manner as in FIG. 16 except that an inverse quantization circuit 81 is newly provided at a subsequent stage of the quantization circuit 63.

In the embodiment of FIG. 19, the inverse quantization circuit 81 inversely quantizes the quantized DCT coefficient output from the inverse quantization circuit 63 in the same manner as the inverse quantization circuit 71 of FIG.
The DCT coefficients obtained as a result are extracted by the prediction tap extraction circuit 6.
4 and the class tap extraction circuit 65.

Therefore, the prediction tap extraction circuit 64 and the class tap extraction circuit 65 do not use the quantized DCT coefficient but
A prediction tap and a class tap are respectively configured for DCT coefficients, and thereafter, for DCT coefficients,
The same processing as in FIG. 16 is performed.

As a result, a tap coefficient that reduces the effect of a quantization error caused by the DCT coefficient being quantized and further dequantized is obtained.

Next, FIG. 20 shows the coefficient conversion circuit 32 of FIG.
3 shows a third configuration example. In the figure, portions corresponding to those in FIG. 5 are denoted by the same reference numerals, and a description thereof will be omitted as appropriate below. That is, the coefficient transforming circuit 32 of FIG. 20 is different from the product-sum calculating circuit 45 in that an inverse DCT circuit 101 is newly provided at the subsequent stage.
Basically, the configuration is the same as in FIG.

The inverse DCT circuit 101 includes a product-sum operation circuit 45
Is subjected to an inverse DCT process, thereby decoding and outputting the image. Therefore, in the embodiment of FIG. 20, the product-sum operation circuit 45 uses the quantized DCT coefficients constituting the prediction taps output by the prediction tap extraction circuit 41 and the tap coefficients stored in the coefficient table storage unit 44. The DCT coefficient is output by performing the sum-of-products operation.

As described above, in the embodiment of FIG. 20, the quantized DCT coefficients are not decoded into pixel values by the product-sum operation with the tap coefficients, but are converted into DCT coefficients.
Further, the DCT coefficient is converted into an inverse D
By being subjected to CT, it is decoded into a pixel value. Therefore,
The tap coefficients stored in the coefficient table storage section 44 need to be different from those in FIG.

FIG. 21 shows an example of the configuration of an embodiment of the learning apparatus for performing the learning process of the tap coefficients stored in the coefficient table storage section 44 of FIG. In the figure, portions corresponding to the case in FIG. 16 are denoted by the same reference numerals, and the description thereof will be appropriately omitted below. That is, the learning apparatus in FIG. 21 gives the normal equation addition circuit 67, as teacher data, not the pixel value of the learning image but the DCT coefficient output from the DCT circuit 62 and obtained by performing the DCT processing on the learning image. Other than that, the configuration is the same as that in the case of FIG.

Therefore, in the embodiment of FIG. 21, the normal equation adding circuit 67 outputs the DCT signal output from the DCT circuit 62.
The above addition is performed using the coefficients as teacher data and the quantized DCT coefficients constituting the prediction taps output from the prediction tap configuration circuit 64 as the student data. And
The tap coefficient determination circuit 68 obtains a tap coefficient by solving a normal equation obtained by such addition. As a result, in the learning device of FIG.
A tap coefficient for converting the coefficient into a DCT coefficient in which a quantization error due to the quantization in the quantization circuit 63 is reduced (suppressed) is obtained.

In the coefficient conversion circuit 32 shown in FIG. 20, since the product-sum operation circuit 45 performs the product-sum operation using the tap coefficients as described above, the output is the quantized DCT output from the prediction tap extraction circuit 41. The coefficient is converted into a DCT coefficient whose quantization error is reduced. Therefore, such a DCT coefficient is subjected to inverse DCT by the inverse DCT circuit 101, so that a decoded image with reduced image quality deterioration due to the influence of the quantization error can be obtained.

Next, FIG. 22 shows the coefficient conversion circuit 32 of FIG.
4 shows a fourth configuration example. In FIG. 5, FIG.
8, or portions corresponding to those in FIG. 20 are denoted by the same reference numerals, and the description thereof will be made below.
Omitted as appropriate. That is, in the coefficient conversion circuit 32 of FIG. 22, an inverse quantization circuit 71 is newly provided, as in the case of FIG. 18, and, as in the case of FIG.
The configuration is the same as that in FIG. 5 except that a T circuit 101 is newly provided.

Accordingly, in the embodiment shown in FIG. 22, the prediction tap extraction circuit 41 and the class tap extraction circuit 42 form prediction taps and class taps not for quantized DCT coefficients but for DCT coefficients. Further, in the embodiment of FIG. 22, the product-sum operation circuit 45 uses the DCT coefficient constituting the prediction tap output from the prediction tap extraction circuit 41 and the tap coefficient stored in the coefficient table storage unit 44. By performing a sum operation, a DCT coefficient with a reduced quantization error is obtained, and the inverse DCT circuit 101
Output to

Next, FIG. 23 shows an example of the configuration of an embodiment of the learning apparatus for performing the learning process of the tap coefficients stored in the coefficient table storage section 44 of FIG. In the drawings, the same reference numerals are given to portions corresponding to the case in FIG. 16, FIG. 19, or FIG. 21, and the description thereof will be appropriately omitted below. That is, the learning device of FIG.
21. Further, as in the case of FIG. 21, the learning image output from the DCT circuit 62 is output to the normal equation adding circuit 67 as teacher data instead of the pixel value of the learning image. The configuration is the same as that in FIG. 16 except that DCT coefficients subjected to DCT processing are given.

Therefore, in the embodiment of FIG. 23, the normal equation adding circuit 67
Coefficients (DCT coefficients without quantization errors) are used as teacher data, and DCT coefficients (DCT coefficients that have been quantized and dequantized) constituting prediction taps output from the prediction tap configuration circuit 64 are used as student data. The above addition is performed. Then, the tap coefficient determination circuit 68 solves the normal equation obtained by such addition,
Find the tap coefficient. As a result, in the learning device of FIG. 23, the quantized and further dequantized DCT coefficients are
A tap coefficient to be converted to a DCT coefficient in which a quantization error due to the quantization and the inverse quantization is reduced is obtained.

Next, the above-described series of processing can be performed by hardware or software. When a series of processing is performed by software, a program constituting the software is
Installed on a general-purpose computer.

FIG. 24 shows an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processing is installed.

The program is stored in a hard disk 205 or a ROM 2 as a recording medium built in the computer.
03 can be recorded in advance.

Alternatively, the program may be a floppy (registered trademark) disk, a CD-ROM (Compact Disc Read Onl
y Memory), MO (Magneto optical) disc, DVD (Digita
l Versatile Disc), a magnetic disk, a semiconductor memory, etc., can be temporarily or permanently stored (recorded) in a removable recording medium 211. Such a removable recording medium 211 can be provided as so-called package software.

The program can be installed in the computer from the removable recording medium 211 as described above, or transmitted from a download site to the computer wirelessly via a satellite for digital satellite broadcasting, or transmitted to a LAN (Local Area). Network) or the Internet, and the program can be transferred to the computer by wire, and the computer can receive the transferred program by the communication unit 208 and install the program on the built-in hard disk 205.

The computer has a CPU (Central Processing).
Unit 202. The CPU 202 has a bus 2
01, the input / output interface 210 is connected. The CPU 202 operates the input unit 207 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 210. When a command is input, the ROM (Read O
(Nly Memory) 203 is executed. Alternatively, the CPU 202 may execute a program stored in the hard disk 205, a program transferred from a satellite or a network, received by the communication unit 208 and installed on the hard disk 205, or a removable recording medium 211 mounted on the drive 209. The program read and installed on the hard disk 205 is stored in a RAM (Random Access Memory).
y) Load to 204 and execute. This allows the CPU 20
2 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 202 transmits the processing result as necessary, for example, via the input / output interface 210.
An output is made from an output unit 206 including an LCD (Liquid CryStal Display), a speaker, or the like, or transmitted from a communication unit 208, and further recorded on the hard disk 205.

Here, in this specification, processing steps for writing a program for causing a computer to perform various processes do not necessarily have to be processed in chronological order in the order described in the flowchart, and may be performed in parallel. Alternatively, it also includes processing executed individually (for example, parallel processing or processing by an object).

The program may be processed by one computer, or may be processed by a plurality of computers in a distributed manner. Further, the program may be transferred to a remote computer and executed.

Although the present embodiment is directed to image data, the present invention is also applicable to, for example, audio data.

Further, in the present embodiment, a JPEG-encoded image for compressing and encoding a still image is targeted.
The present invention can be applied to an image in which a moving image is compression-encoded, for example, an MPEG-encoded image.

In the present embodiment, at least D
Although the decoding of the JPEG-encoded data for performing the CT process is performed, the present invention provides decoding and decoding of data converted in block units (a predetermined unit) by other orthogonal transformation or frequency transformation. Applicable for conversion. That is, for example, the present invention decodes subband encoded data, Fourier transformed data, and the like,
The present invention is also applicable to a case where the data is converted into data in which the quantization error or the like is reduced.

Further, in the present embodiment, the decoder 22
In the above, the tap coefficients are stored in advance, but the tap coefficients can be included in the encoded data and provided to the decoder 22.

Further, in the present embodiment, decoding and conversion are performed by a linear primary prediction operation using tap coefficients. However, decoding and conversion are also performed by a second-order or higher-order prediction operation. It is also possible to do.

Further, in the present embodiment, the class tap is set to the quantized DCT of the DCT block corresponding to the pixel of interest.
Although the configuration is made using not only the coefficients but also the quantized DCT coefficients of the other DCT blocks, the prediction taps can be similarly configured.

[0204]

According to the first data processing apparatus, the data processing method, and the recording medium of the present invention, the conversion data used to classify the processing data of interest into one of several classes can be obtained. , Are extracted from at least blocks other than the block corresponding to the processing data of interest and output as class taps. Then, based on the class tap, a class classification for obtaining a class of the processing data of interest is performed, and a tap coefficient of the class of the processing data of interest,
By performing a predetermined prediction operation using the conversion data and the conversion data, a prediction value of the processing data of interest is obtained. Therefore, it is possible to efficiently obtain desired processing data from the converted data.

According to the second data processing apparatus, data processing method, and recording medium of the present invention, at least student data used to classify the teacher data of interest into one of several classes is included. , Are extracted from blocks other than the block corresponding to the teacher data of interest and output as class taps. Further, a class classification for obtaining a class of the teacher data of interest is performed based on the class tap. Learning is performed so that the prediction error of the predicted value of the teacher data obtained by performing the prediction operation using the tap coefficient for each class and the student data is statistically minimized. Required for each. Therefore, by using the tap coefficients, it is possible to efficiently obtain desired data from the orthogonally transformed or frequency transformed data.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining conventional JPEG encoding / decoding.

FIG. 2 is a diagram illustrating a configuration example of an embodiment of an image transmission system to which the present invention has been applied.

FIG. 3 is a block diagram illustrating a configuration example of a decoder 22 in FIG. 2;

FIG. 4 is a flowchart illustrating a process of a decoder 22 in FIG. 3;

FIG. 5 is a block diagram illustrating a first configuration example of a coefficient conversion circuit 32 in FIG. 3;

FIG. 6 is a diagram illustrating an example of a class tap.

FIG. 7 is a block diagram illustrating a configuration example of a class classification circuit 43 in FIG. 5;

FIG. 8 is a diagram for explaining processing of a power calculation circuit 51;

FIG. 9 is a flowchart illustrating a process of a coefficient conversion circuit 32 of FIG. 5;

FIG. 10 is a flowchart illustrating details of a process in step S12 of FIG. 9;

FIG. 11 is a diagram for explaining a class classification method.

FIG. 12 is a flowchart illustrating a process of a class classification circuit 43 of FIG. 5;

FIG. 13 is a diagram for explaining another method of class classification.

FIG. 14 is a block diagram showing another configuration example of the class classification circuit 43 of FIG. 5;

FIG. 15 is a flowchart illustrating a process of a class classification circuit 43 of FIG. 14;

FIG. 16 is a block diagram illustrating a configuration example of a first embodiment of a learning device that learns tap coefficients.

FIG. 17 is a flowchart illustrating a process of the learning device in FIG. 16;

18 is a block diagram illustrating a second configuration example of the coefficient conversion circuit 32 in FIG.

FIG. 19 is a block diagram illustrating a configuration example of a second embodiment of the learning device.

20 is a block diagram illustrating a third configuration example of the coefficient conversion circuit 32 in FIG. 3;

FIG. 21 is a block diagram illustrating a configuration example of a third embodiment of the learning device.

FIG. 22 is a block diagram illustrating a fourth configuration example of the coefficient conversion circuit 32 in FIG. 3;

FIG. 23 is a block diagram illustrating a configuration example of a fourth embodiment of the learning device.

FIG. 24 is a block diagram illustrating a configuration example of a computer according to an embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 21 encoder, 22 decoder, 23 recording medium, 24 transmission medium, 31 entropy decoding circuit, 32 coefficient conversion circuit, 33 block decomposition circuit, 41 prediction tap extraction circuit, 42 class tap extraction circuit, 43 class classification circuit, 44 coefficient table storage Unit, 45 product-sum operation circuit, 51 power operation circuit, 52 class code generation circuit, 53 threshold table storage unit, 61 blocking circuit, 62 DCT
Circuit, 63 quantization circuit, 64 prediction tap extraction circuit, 65 class tap extraction circuit, 66 class classification circuit, 67 normal equation addition circuit, 68 tap coefficient determination circuit, 69 coefficient table storage section, 71,
81 inverse quantization circuit, 91 inverse quantization circuit, 101
Inverse DCT circuit, 151 difference operation circuit, 152
Class code generation circuit, 153 threshold table storage unit, 201 bus, 202 CPU, 203 ROM,
204 RAM, 205 Hard disk, 206
Output unit, 207 input unit, 208 communication unit, 20
9 drives, 210 input / output interfaces, 2
11 Removable recording medium

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Takeharu Nishikata 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Masashi Uchida 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F term (reference) 5C059 KK01 MA00 MA02 MA23 PP01 SS06 SS11 SS20 TA41 TB07 TB14 TC03 TC06 TD02 TD06 TD12 TD13 UA05 UA38 5C078 AA04 BA21 BA32 BA42 BA57 DA02

Claims (20)

[Claims] 1. Prediction of processing data obtained by subjecting data obtained by subjecting the converted data to desired processing at least from block-based converted data obtained by performing orthogonal transformation processing or frequency conversion processing on the data in predetermined block units. A data processing device for obtaining a value, wherein an obtaining means for obtaining a tap coefficient obtained by performing learning; and a processing data of interest, which is processing data of interest, in one of several classes. Class tap extracting means for extracting the converted data used for class classification at least from blocks other than the block corresponding to the target processing data and outputting it as a class tap, and the target processing data based on the class tap. Class classifying means for performing class classification for obtaining a class of -Up factor, and using the conversion data, by performing a predetermined prediction calculation, the data processing apparatus characterized by comprising a calculating means for calculating a predicted value of the target processing data. 2. The data processing apparatus according to claim 1, wherein said calculation means performs a linear primary prediction calculation using said tap coefficients and conversion data. 3. The data processing apparatus according to claim 1, further comprising a storage unit that stores the tap coefficient, wherein the obtaining unit obtains the tap coefficient from the storage unit. 4. The data processing apparatus according to claim 1, wherein the converted data is at least a discrete cosine transform of the data. 5. The class classification means obtains a class of the processing data of interest based on DC component or AC component power of discrete cosine transformed data, which is the transformation data serving as the class tap. The data processing device according to claim 4, wherein: 6. The data processing apparatus according to claim 1, wherein the class tap extracting unit extracts the conversion data to be the class tap from a block around a block corresponding to the processing data of interest. . 7. The data processing apparatus according to claim 1, wherein the class tap extracting unit extracts the conversion data as the class tap from a block corresponding to the processing data of interest. 8. The tap coefficient is such that a prediction error of a prediction value of the processing data obtained by performing a predetermined prediction operation using the tap coefficient and the conversion data is statistically minimized. The data processing apparatus according to claim 1, wherein the data processing apparatus is obtained by performing learning. 9. The data processing device according to claim 1, wherein the data is image data of a moving image or a still image. 10. Prediction of processing data obtained by subjecting the converted data to a desired process from at least block-based converted data obtained by performing orthogonal transform processing or frequency transform processing on the data in predetermined block units. A data processing method for obtaining a value, wherein an obtaining step of obtaining a tap coefficient obtained by performing learning; and a process data of interest, which is the process data of interest, in one of several classes. A class tap extraction step of extracting the conversion data used for class classification at least from blocks other than the block corresponding to the target processing data and outputting the class tap as a class tap; A class classification step of performing a class classification for obtaining a class of The tap coefficients of the class, and using the conversion data, by performing a predetermined prediction computation, data processing method characterized by comprising a calculation step of obtaining a prediction value of the target processing data. 11. Prediction of processing data obtained by subjecting the converted data to a desired processing from at least block-based converted data obtained by performing orthogonal transformation processing or frequency conversion processing on the data in predetermined block units. An acquisition step of acquiring a tap coefficient obtained by performing learning on a recording medium in which a program for causing a computer to perform data processing for obtaining a value; A class tap extracting step of extracting at least the conversion data used to classify data into any of several classes from blocks other than the block corresponding to the processing data of interest, and outputting as a class tap A class for obtaining the class of the processing data of interest based on the class tap. A class classification step of performing a Lass classification; and a calculation step of calculating a predicted value of the target processing data by performing a predetermined prediction calculation using the tap coefficient and the conversion data of the class of the target processing data. A recording medium on which a program is recorded. 12. A desired processing is performed on the converted data by predictive calculation from at least a block-by-block converted data obtained by performing an orthogonal transform process or a frequency transform process on the data in a predetermined block unit. A data processing device that learns tap coefficients used for obtaining processing data, wherein the conversion data in block units is generated as student data to be a student during learning. As teacher data to be a teacher, attention teacher data, which is the teacher data of interest,
Class tap extracting means for extracting the student data used for classifying into any one of several classes from at least blocks other than the block corresponding to the noted teacher data, and outputting as class taps; Class classification means for performing class classification for obtaining a class of the teacher data of interest based on class taps; prediction of the teacher data obtained by performing a prediction operation using the tap coefficient for each class and student data A data processing apparatus comprising: learning means for performing learning so that a prediction error of a value is statistically minimized, and obtaining the tap coefficient for each class. 13. The learning means so that a prediction error of a prediction value of the teacher data obtained by performing a linear primary prediction operation using the tap coefficient and the student data is statistically minimized. The data processing apparatus according to claim 12, wherein the data processing is performed. 14. The data processing apparatus according to claim 12, wherein the generation unit generates the student data by at least performing a discrete cosine transform on the data. 15. The class classifying means obtains a class of the teacher data of interest based on a DC component or an AC component of discrete cosine transformed data, which is student data serving as the class tap. The data processing device according to claim 14, wherein: 16. The data processing apparatus according to claim 12, wherein the class tap extracting means extracts the student data as the class tap from blocks around a block corresponding to the teacher data of interest. . 17. The data processing apparatus according to claim 12, wherein the class tap extracting means extracts the student data as the class tap from a block corresponding to the teacher data of interest. 18. The data processing device according to claim 12, wherein the data is image data of a moving image or a still image. 19. At least a desired process is performed on the converted data by a prediction operation from the converted data in a block unit obtained by performing at least an orthogonal transform process or a frequency transform process on the data in a predetermined block unit. A data processing method for learning tap coefficients used to determine processing data, wherein the conversion data in block units is generated as student data to be a student during learning. As teacher data to be a teacher, attention teacher data, which is the teacher data of interest,
A class tap extracting step of extracting at least the student data used for classifying into any of several classes from blocks other than the block corresponding to the noted teacher data, and outputting as a class tap; A class classification step of performing a class classification for obtaining a class of the noted teacher data based on a class tap; and a prediction of the teacher data obtained by performing a prediction operation using the tap coefficient and the student data for each class. A learning step of performing learning so that a value prediction error is statistically minimized, and obtaining a tap coefficient for each class. 20. By performing a prediction operation on data, at least a desired processing is performed on the converted data in a block unit obtained by performing an orthogonal transform process or a frequency transform process on a predetermined block basis. A recording medium on which a program for causing a computer to perform a data process of learning tap coefficients used for obtaining processing data is recorded, wherein the conversion data in block units is generated as student data to be a student during learning. And generating the processing data, as teacher data to be a teacher at the time of learning, teacher data of interest as teacher data of interest,
A class tap extracting step of extracting at least the student data used for classifying into any of several classes from blocks other than the block corresponding to the noted teacher data, and outputting as a class tap; A class classification step of performing a class classification for obtaining a class of the noted teacher data based on a class tap; and a prediction of the teacher data obtained by performing a prediction operation using the tap coefficient and the student data for each class. And a learning step of learning so as to statistically minimize the value prediction error and obtaining the tap coefficient for each class.