JP2001320587A - Data processor and data processing method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processor that can efficiently decode data coded in compliance with the JPEG into an image with high image quality. SOLUTION: Entropy decoding is applied to the data coded in compliance with the JPEG to obtain a quantized DCT coefficient, which is fed to a prediction tap extract circuit 41 and a cluster tap extract circuit 42. The prediction tap extract circuit 41 and the cluster tap extract circuit 42 extract required data from the DCT coefficient to respectively configure a prediction tap and a cluster tap. A class classification circuit 43 classifies classes on the basis of the class tap and a coefficient table storage section 44 supplies the tap coefficient corresponding to the class obtained from the result of classification to a product-sum arithmetic circuit 45. The product-sum arithmetic circuit 45 uses the tap coefficient and the prediction tap to conduct a linear prediction arithmetic operation and to obtain image data where number of lateral and longitudinal pixels is twice that of an original image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing apparatus, a data processing method, and a recording medium, and more particularly to a data processing apparatus suitable for use, for example, when decoding a compressed image into a high-quality image. And data processing methods,
And a recording medium.

[0002]

2. Description of the Related Art For example, digital image data has a large data amount.
A large-capacity recording medium and transmission medium are required. Therefore, in general, recording and transmission are performed after compressing and encoding image data to reduce the data amount.

As a method of compressing and encoding an image, for example, a JPEG (Joint Photo) which is a compression encoding method of a still image is used.
There is an MPEG (Moving Picture Experts Group) method, which is a moving image compression encoding method, and the like.

For example, encoding / decoding of image data according to the JPEG system is performed as shown in FIG.

[0005] That is, FIG. 1A shows a configuration of an example of a conventional JPEG encoding device.

[0006] The image data to be encoded is input to a blocking circuit 1, which divides the input image data into blocks of 64 8x8 pixels. Each block obtained by the blocking circuit 1 is supplied to a DCT (Discrete Cosine Transform) circuit 2. The DCT circuit 2 performs a DCT (Discrete Cosine Transform) process on the block from the blocking circuit 1, and performs one DC (Direct Current) component and 63 frequency components (horizontal and vertical). AC (Alter
(Nating Current) component) into a total of 64 DCT coefficients. The 64 DCT coefficients for each block are D
The signal is supplied from the CT circuit 2 to the quantization circuit 3.

The quantization circuit 3 quantizes the DCT coefficient from the DCT circuit 2 according to a predetermined quantization table, and uses the quantization result (hereinafter, appropriately referred to as a quantized DCT coefficient) for quantization. The information is supplied to the entropy encoding circuit 4 together with the quantization table.

FIG. 1B shows an example of a quantization table used in the quantization circuit 3. Generally, quantization tables take into account human visual characteristics,
The low-frequency DCT coefficients of high importance are finely quantized,
A quantization step for coarsely quantizing the DCT coefficient of a low-frequency high frequency is set, and thereby the image quality of the image is suppressed from being degraded, and the compression is performed efficiently.

The entropy coding circuit 4 performs an entropy coding process such as Huffman coding on the quantized DCT coefficient from the quantization circuit 3 and adds a quantization table from the quantization circuit 3. Then, the resulting encoded data is output as a JPEG encoded result.

Next, FIG. 1C shows the JPE of FIG.
1 shows a configuration of an example of a conventional JPEG decoding device that decodes encoded data output from a G encoding device.

The encoded data is transmitted to an entropy decoding circuit 1
1, the entropy decoding circuit 11 separates the encoded data into entropy-encoded quantized DCT coefficients and a quantization table. Further, the entropy decoding circuit 11 performs the entropy-encoded quantization D
The CT coefficients are entropy-decoded, and the resulting quantized DCT coefficients are supplied to an inverse quantization circuit 12 together with a quantization table. The inverse quantization circuit 12 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 11 in accordance with the quantization table from the entropy decoding circuit 11, and converts the resulting DCT coefficient into the inverse DCT coefficient.
It is supplied to the circuit 13. The inverse DCT circuit 13 performs an inverse DCT process on the DCT coefficient from the inverse quantization circuit 12, and supplies the resulting 8 × 8 pixel decoded block to the block decomposition circuit 14. The block decomposition circuit 14 calculates the inverse D
By unblocking the decoded block from the CT circuit 13, a decoded image is obtained and output.

[0012]

SUMMARY OF THE INVENTION The JPEG shown in FIG.
In the encoding device, the quantization circuit 3 can reduce the data amount of the encoded data by increasing the quantization step of the quantization table used for quantizing the block. That is, high compression can be realized.

However, when the quantization step is increased, the so-called quantization error is also increased.
(C) The image quality of the decoded image obtained by the JPEG decoding device is degraded. That is, blur, block distortion, mosquito noise, and the like appear conspicuously in the decoded image.

Therefore, in order to prevent the image quality of the decoded image from deteriorating while reducing the data amount of the encoded data, or to improve the image quality of the decoded image while maintaining the data amount of the encoded data. It is necessary to perform some kind of processing for improving image quality after JPEG decoding.

However, performing processing for improving the image quality after JPEG decoding complicates the processing.
The time until a decoded image is finally obtained also increases.

The present invention has been made in view of such a situation, and is intended to efficiently obtain a high-quality decoded image from a JPEG-coded image or the like. .

[0017]

A first data processing apparatus according to the present invention comprises: an acquiring means for acquiring a tap coefficient obtained by performing learning; and a predetermined predictive operation using a tap coefficient and conversion data. And decoding means for decoding the converted data into the original data, and obtaining processing data obtained by subjecting the original data to a predetermined process.

In the first data processing device, the operation means can perform a linear primary prediction operation using the tap coefficients and the converted data.

The first data processing device may further include a storage unit for storing the tap coefficients. In this case, the obtaining unit may cause the obtaining unit to obtain the tap coefficients from the storage unit.

In the first data processing device, the transformed data can be obtained by subjecting the original data to orthogonal transform or frequency transform and then quantizing.

The first data processing apparatus may further include an inverse quantization means for inversely quantizing the transformed data,
The calculation means can perform a prediction calculation using the inversely quantized transform data.

In the first data processing device, the transformed data may be at least discrete cosine transformed from the original data.

The first data processing apparatus further includes a prediction tap extracting means for extracting conversion data used together with a tap coefficient for predicting the target data of interest among the processing data, and outputting the converted data as a prediction tap. In this case, the calculation means can perform the prediction calculation using the prediction tap and the tap coefficient.

The first data processing apparatus includes a class tap extracting means for extracting converted data used to classify the target data into any of several classes and outputting the converted data as class taps; Classification means for performing a class classification for obtaining the class of the data of interest based on the taps may be further provided. In this case, the calculating means includes a prediction tap and a prediction coefficient using a tap coefficient corresponding to the class of the data of interest. An operation can be performed.

In the first data processing device, the calculation means performs predetermined prediction calculation to obtain processed data obtained by performing processing for improving the quality of the original data.

In the first data processing apparatus, the tap coefficient is such that the prediction error of the predicted value of the processed data obtained by performing a predetermined prediction operation using the tap coefficient and the converted data is statistically minimized. Thus, it can be obtained by learning.

In the first data processing device, the original data can be moving image or still image data.

In the first data processing device, the calculation means performs predetermined prediction calculation to obtain processed data obtained by performing a process for improving image quality on image data.

In the first data processing device, the arithmetic means can obtain processing data with improved resolution in the time or space direction of the image data.

In the first data processing method of the present invention, the conversion step is performed by performing a predetermined prediction operation using the tap coefficient and the conversion data by obtaining the tap coefficient obtained by performing the learning. A decoding step of decoding the data into the original data and obtaining processed data obtained by subjecting the original data to a predetermined process.

According to the first recording medium of the present invention, the conversion step is performed by obtaining a tap coefficient obtained by performing learning, and by performing a predetermined prediction operation using the tap coefficient and the conversion data. Is decoded into original data and an operation step of obtaining processed data obtained by subjecting the original data to predetermined processing is recorded.

The second data processing apparatus according to the present invention performs processing based on predetermined processing on teacher data as a teacher, and generates a quasi-teacher data generating means for obtaining quasi-teacher data. The conversion error or the frequency conversion is used to generate student data to be students, and the prediction error of the prediction value of the teacher data obtained by performing the prediction operation using the tap coefficient and the student data is statistically different. And learning means for performing learning so as to minimize tapping and obtaining tap coefficients.

In the second data processing apparatus, the learning means statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the linear primary prediction operation using the tap coefficients and the student data. Learning can be performed as follows.

In the second data processing apparatus, the student data generating means can generate the student data by subjecting the quasi-teacher data to orthogonal transformation or frequency transformation and further quantizing it.

In the second data processing device, the student data generating means quantizes the quasi-teacher data by orthogonal transform or frequency transform, and further quantizes the quasi-teacher data by inverse quantization.
Student data can be generated.

In the second data processing apparatus, the student data generating means can generate the student data by subjecting the quasi-teacher data to at least discrete cosine transform.

The second data processing apparatus includes a prediction tap extracting means for extracting student data used together with a tap coefficient for predicting the noted teacher data of interest from the teacher data, and outputting the extracted student data as a prediction tap. In this case, the learning means performs learning so that the prediction error of the prediction value of the teacher data obtained by performing the prediction operation using the prediction tap and the tap coefficient is statistically minimized. Can be done.

The second data processing device includes class tap extracting means for extracting student data used to classify the teacher data of interest into one of several classes and outputting the data as class taps; Classification means for performing class classification for obtaining the class of the teacher data of interest based on the class taps may be further provided. In this case, the learning means may include tap coefficients corresponding to the prediction tap and the class of the teacher data of interest. The learning is performed so that the prediction error of the prediction value of the teacher data obtained by performing the prediction calculation using the statistical calculation is statistically minimized, and the tap coefficient for each class can be obtained.

In the second data processing device, the student data generating means stores the quasi-teacher data for each predetermined unit.
At least student data can be generated by performing orthogonal transformation processing or frequency transformation.

In the second data processing apparatus, the quasi-teacher data generating means can generate the quasi-teacher data by subjecting the teacher data to a process of deteriorating the quality of the teacher data.

In the second data processing device, the teacher data can be moving image or still image data.

In the second data processing apparatus, the quasi-teacher data generating means can generate the quasi-teacher data by performing a process of deteriorating the image quality of the image data.

In the second data processing device, the quasi-teacher data generating means can generate the quasi-teacher data in which the resolution in the time or space direction of the image data is deteriorated.

According to a second data processing method of the present invention, a quasi-teacher data generating step of performing a process based on a predetermined process on teacher data to be a teacher to obtain quasi-teacher data; A conversion error or a frequency conversion generates a student data to be a student, and a prediction error of a prediction value of teacher data obtained by performing a prediction operation using the tap coefficient and the student data is statistically different. And a learning step of obtaining a tap coefficient by performing learning so as to minimize it.

The second recording medium according to the present invention performs a quasi-teacher data generating step of performing a process based on a predetermined process on teacher data as a teacher to obtain quasi-teacher data. Or, by performing frequency conversion, a student data generation step of generating student data to be a student, and a prediction error of a prediction value of teacher data obtained by performing a prediction operation using the tap coefficient and the student data are statistically reduced. And a learning step of performing learning so as to minimize it and obtaining a tap coefficient.

In the first data processing apparatus, the data processing method, and the recording medium of the present invention, tap coefficients obtained by performing learning are obtained, and a predetermined prediction is performed using the tap coefficients and converted data. By performing the operation, the converted data is decoded into the original data,
In addition, processed data obtained by subjecting the original data to predetermined processing is obtained.

In the second data processing apparatus, the data processing method, and the recording medium according to the present invention, a process based on a predetermined process is performed on teacher data to be a teacher, and the resulting quasi-teacher data is converted into at least By performing orthogonal transformation or frequency transformation, student data to be a student is generated. Learning is performed so that the prediction error of the prediction value of the teacher data obtained by performing the prediction operation using the tap coefficient and the student data is statistically minimized, and the tap coefficient is obtained.

[0048]

FIG. 2 shows a configuration example of an embodiment of an image transmission system to which the present invention is applied.

The image data to be transmitted is
The encoder 21 encodes the image data supplied thereto, for example, by JPEG encoding to obtain encoded data. That is, the encoder 21 is configured, for example, in the same manner as the above-described JPEG encoding apparatus shown in FIG. 1A, and JPEG-encodes image data. Encoded data obtained by the encoder 21 performing JPEG encoding includes, for example, a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape,
It is recorded on a recording medium 23 such as a phase change disk or the like.
(Cable Television) is transmitted via a transmission medium 24 such as a network, the Internet, or a public line.

The decoder 22 receives the encoded data provided via the recording medium 23 or the transmission medium 24,
Decode to high quality image data. The decoded high-quality image data is supplied to, for example, a monitor (not shown) and displayed.

FIG. 3 shows an example of the configuration of the decoder 22 shown in FIG.

The encoded data is sent to the entropy decoding circuit 3
1, the entropy decoding circuit 31 performs entropy decoding of the encoded data, and obtains a quantized DCT coefficient Q for each block obtained as a result.
It is supplied to the coefficient conversion circuit 32. Note that the coded data includes the entropy-coded quantized DCT as in the case described in the entropy decoding circuit 11 in FIG.
In addition to the coefficients, a quantization table is also included. This quantization table includes a quantization DC
It can be used for decoding the T coefficient.

The coefficient conversion circuit 32 performs a predetermined prediction operation using the quantized DCT coefficient Q from the entropy decoding circuit 31 and a tap coefficient obtained by performing learning, which will be described later. DCT
The coefficient is decoded into an original block of 8 × 8 pixels, and data is further processed to improve the image quality of the original block. That is, although the original block is composed of 8 × 8 pixels, the coefficient conversion circuit 32 performs a prediction operation using the tap coefficient to thereby reduce the spatial resolution in the horizontal and vertical directions of the block of 8 × 8 pixels. In each case, a block composed of 16 × 16 pixels is obtained. Accordingly, here, the coefficient conversion circuit 32 converts a block composed of 8 × 8 quantized DCT coefficients into a 16 × 16 block as shown in FIG.
The data is decoded into a block composed of pixels and output.

The block decomposition circuit 33 includes a coefficient conversion circuit 3
By unblocking the 16 × 16 pixel block obtained in 2 above, a decoded image with improved spatial resolution is obtained and output.

Next, referring to the flowchart of FIG.
The processing of the decoder 22 in FIG. 3 will be described.

The encoded data is supplied to the entropy decoding circuit 3
1, the entropy decoding circuit 31 entropy-decodes the encoded data in step S1, and supplies the quantized DCT coefficient Q for each block to the coefficient transforming circuit 32. The coefficient conversion circuit 32 determines in step S2
In, the quantized DCT coefficient Q for each block from the entropy decoding circuit 31 is decoded into a pixel value for each block by performing a prediction operation using a tap coefficient,
In addition, a so-called high-resolution block having an improved spatial resolution of the block is obtained and supplied to the block decomposition circuit 33. In step S3, the block decomposition circuit 33 performs block decomposition for deblocking the block of the pixel value with the improved spatial resolution from the coefficient conversion circuit 32, and outputs a high-resolution decoded image obtained as a result. ,
The process ends.

Next, the coefficient conversion circuit 32 in FIG. 3 decodes the quantized DCT coefficients into pixel values using, for example, class classification adaptive processing, and further obtains an image whose spatial resolution is improved. be able to.

The class classification adaptation process includes a class classification process and an adaptation process. The class classification process classifies data into classes based on their properties, and performs an adaptation process for each class. Is based on the following method. Note that, here, for simplicity of description, the adaptive processing will be described by taking as an example a case where a quantized DCT coefficient is decoded into an original image.

In this case, in the adaptive processing, for example, by a linear combination of the quantized DCT coefficient and a predetermined tap coefficient,
By obtaining the predicted value of the original pixel, the quantized DCT coefficient is decoded into the original pixel value.

More specifically, for example, a certain image is used as teacher data, and the image is subjected to DCT processing in block units and further quantized to obtain a quantized DCT.
Using the coefficients as student data, a prediction value E [y] of a pixel value y of a pixel as teacher data is converted to some quantized DCT coefficients x
_1, x _2, a set of ..., predetermined tap coefficients w _1, w _2,
.. Are considered by a linear first-order combination model defined by the linear combination. In this case, the predicted value E [y]
Can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (1)

To generalize equation (1), a matrix W consisting of a set of tap coefficients w _j , a matrix X consisting of a set of student data x _ij , and a matrix Y consisting of a set of predicted values E [y _j ] '
To

(Equation 1) Defines the following observation equation.

XW = Y ′ (2) Here, the component x _ij of the matrix X is a set of i-th student data (a set of student data used for predicting the _i-th teacher data y _i ). Means the j-th student data in the matrix W, and the component w _{j of the} matrix W represents a tap coefficient by which a product with the j-th student data in the set of the student data is calculated. Also, y
_i represents the i-th teacher data, and therefore, E [y _i ]
Represents the predicted value of the i-th teacher data. Note that y on the left side of the equation (1) is obtained by omitting the suffix i of the component y _i of the matrix Y. Further, x ₁ , x ₂ ,. The suffix i of the component x _ij is omitted.

Then, it is considered that a least square method is applied to this observation equation to obtain a predicted value E [y] close to the original pixel value y. In this case, a true pixel value y serving as teacher data
And a predicted value E for a pixel value y
A matrix E consisting of a set of residuals e of [y] is

[0065]

(Equation 2) From equation (2), the following residual equation is established.

XW = Y + E (3)

In this case, the predicted value E close to the original pixel value y
The tap coefficient w _j for obtaining [y] is a square error

(Equation 3) Can be obtained by minimizing.

Therefore, the above square error is calculated by tap coefficient w _j
Is zero, that is, a tap coefficient w _j that satisfies the following equation is an optimum value for obtaining a predicted value E [y] close to the original pixel value y.

[0069]

(Equation 4) ... (4)

Therefore, first, the equation (3) is changed to the tap coefficient w
By differentiating with _j , the following equation is established.

[0071]

(Equation 5) ... (5)

From equations (4) and (5), equation (6) is obtained.

[0073]

(Equation 6) ... (6)

Further, considering the relationship among the student data x _ij , the tap coefficient w _j , the teacher data y _i , and the residual e _{i in} the residual equation of the equation (3), the following equation is obtained from the equation (6). A normal equation can be obtained.

[0075]

(Equation 7) ... (7)

The normal equation shown in equation (7) is obtained by converting a matrix (covariance matrix) A and a vector v into

(Equation 8) If the vector W is defined as shown in Expression 1, it can be expressed by the following expression: AW = v (8)

Each normal equation in the equation (7) is prepared by preparing a certain number of sets of the student data x _ij and the teacher data y _i to form the same number as the number J of the tap coefficients w _{j to} be obtained. And therefore equation (8)
Is solved for the vector W (however, in order to solve the equation (8), the matrix A in the equation (8) needs to be regular) to obtain an optimal tap coefficient (here, the square error is minimized). Tap coefficient) w _j can be obtained. In addition,
In solving equation (8), for example, the sweeping method (Ga
uss-Jordan elimination method) can be used.

As described above, the optimum tap coefficient w _j
The adaptive processing is to obtain a predicted value E [y] close to the original pixel value y by using the tap coefficient _wj and using the equation (1).

For example, as teacher data, JPE
While using an image of the same image quality as the image to be G-coded,
When DCT and quantized DCT coefficients obtained by quantizing the teacher data are used as student data, a prediction error is used as a tap coefficient when decoding JPEG-coded image data into original image data. However, a statistically minimum one is obtained.

Therefore, even if the compression rate at the time of performing JPEG encoding is increased, that is, even if the quantization step used for quantization is made coarse, according to the adaptive processing, the prediction error is statistically minimized. Will be performed, and in effect,
A decoding process of a JPEG-coded image and a process of improving the image quality (hereinafter, appropriately referred to as an improvement process) are simultaneously performed. As a result, even if the compression ratio is increased, the image quality of the decoded image can be maintained.

For example, as teacher data, JPE
An image having a higher image quality than the image to be G-encoded is used, and the image quality of the teacher data is used as student data according to JP.
The image quality is degraded to the same image quality as the image to be EG-coded.
When a quantized DCT coefficient obtained by CT and quantization is used, a prediction error is statistically minimized as a tap coefficient when decoding JPEG-coded image data into high-quality image data. Things will be obtained.

Therefore, also in this case, according to the adaptive processing,
The decoding processing of the JPEG-coded image and the improvement processing for further improving the image quality are performed at the same time. As described above, by changing the image quality of the image serving as the teacher data or the student data, it is possible to obtain a tap coefficient for setting the image quality of the decoded image to an arbitrary level.

FIG. 6 shows a first configuration example of the coefficient conversion circuit 32 shown in FIG. 3 for decoding quantized DCT coefficients into pixel values by the above-described class classification adaptive processing.

The quantized DCT coefficients for each block output from the entropy decoding circuit 31 (FIG. 3) are supplied to the prediction tap extraction circuit 41 and the class tap extraction circuit 42.

The prediction tap extracting circuit 41 outputs a block of a high-quality pixel value corresponding to a block of 8 × 8 quantized DCT coefficients supplied thereto (hereinafter referred to as a DCT block as appropriate) (block of this pixel value). Are not present at this stage, but are assumed virtually) (hereinafter, appropriately referred to as high image quality blocks) (in the present embodiment, as described above, blocks of 16 × 16 pixels) are sequentially focused. A high-quality block, and further, each pixel constituting the noted high-quality block is, for example, in a so-called raster scan order,
The target pixel is sequentially set as the target pixel. Further, a prediction tap extraction circuit 4
1 is a quantization D used to predict the pixel value of the pixel of interest.
The CT coefficients are extracted and used as prediction taps.

That is, as shown in FIG. 7, for example, as shown in FIG. 7, all the quantized DCT coefficients of the DCT block corresponding to the high-quality block to which the pixel of interest belongs, The quantized DCT coefficients are extracted as prediction taps. Therefore, in the present embodiment, the same prediction tap is formed for all pixels of a certain high-quality block. However, the prediction tap can be configured with different quantized DCT coefficients for each target pixel.

The prediction tap for each pixel constituting the high image quality block, that is, 256 sets of prediction taps for each of 16 × 16 256 pixels, obtained by the prediction tap extraction circuit 41, is supplied to the product-sum operation circuit 45. Is done. However, in the present embodiment, as described above,
Since the same prediction tap is configured for all the pixels of the high-quality block, in practice, one set of prediction taps is assigned to the product-sum operation circuit 4 for one high-quality block.
5 may be supplied.

The class tap extracting circuit 42 extracts a quantized DCT coefficient used for class classification for classifying the target pixel into one of several classes, and sets it as a class tap.

In JPEG encoding, an image is 8 ×
8-pixel block (hereinafter, appropriately referred to as a pixel block)
Since each pixel is encoded (DCT processing and quantization), all pixels belonging to a high-quality block obtained by improving a certain pixel block are classified into the same class, for example. Therefore, the class tap extraction circuit 42
Constitutes the same class tap for each pixel of a certain high image quality block. That is, the class tap extraction circuit 4
2, for example, as in the case of the prediction tap extraction circuit 41, as shown in FIG. 7, all 8 × 8 quantized DCT coefficients of the DCT block corresponding to the high image quality block to which the pixel of interest belongs, Extract as a class tap.

Here, classifying all pixels belonging to a high image quality block into the same class is equivalent to classifying the high image quality block. Therefore, the class tap extraction circuit 42 does not include 256 sets of class taps for classifying each of the 16 × 16 total 256 pixels constituting the high-quality block of interest, but includes 1 for classifying the high-quality block of interest. For this reason, the class tap extracting circuit 42 performs, for each high-quality block, 64 DCT blocks corresponding to the high-quality block in order to classify the high-quality block. Are extracted and used as class taps.

The quantized DCT coefficients constituting the prediction taps and the class taps are not limited to the above-mentioned patterns.

The class tap of the high-quality block of interest obtained in the class tap extracting circuit 42 is supplied to the class classifying circuit 43. The class classifying circuit 43 outputs the class tap from the class tap extracting circuit 42. , The high-quality block of interest is classified into classes, and a class code corresponding to the resulting class is output.

Here, as a method of performing the class classification,
For example, ADRC (Adaptive Dynamic Range Coding) or the like can be adopted.

In the method using ADRC, the quantized DCT coefficients constituting the class tap are subjected to ADRC processing, and the class of the high-quality block of interest is determined according to the ADRC code obtained as a result.

In the K-bit ADRC, for example,
The maximum value MAX of the quantized DCT coefficient constituting the class tap
And the minimum value MIN is detected, and DR = MAX-MIN is set as the local dynamic range of the set.
, The quantized DCT coefficients making up the class tap are re-quantized to K bits. That is, from among the quantized DCT coefficients forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided (quantized) by DR / 2 ^K. Then, a bit string obtained by arranging the K-bit quantized DCT coefficients constituting the class tap in a predetermined order, which is obtained as described above, is output as an ADRC code. Therefore,
When the class tap is subjected to, for example, 1-bit ADRC processing, the quantized DCT coefficients constituting the class tap are obtained by subtracting the minimum value MIN from the maximum value MAX and the minimum value MI.
Divided by the average value with N, so that each quantized DCT
The coefficient is one bit (binarized). Then, a bit string in which the 1-bit quantized DCT coefficients are arranged in a predetermined order is output as an ADRC code.

The class classification circuit 43 includes, for example,
It is also possible to output the pattern of the level distribution of the quantized DCT coefficients constituting the class taps as it is as the class code. In this case, however, the class tap is constituted by N quantized DCT coefficients, and Assuming that K bits are assigned to the DCT coefficient, the number of class codes output from the classifying circuit 43 is (2
^N) becomes ^K Street, an enormous number exponentially proportional to the number of bits K of quantized DCT coefficients.

Therefore, in the classification circuit 43,
It is preferable to perform the class classification after compressing the information amount of the class tap by the above-described ADRC processing or vector quantization.

By the way, in this embodiment, the class tap is composed of 64 quantized DCT coefficients as described above. Therefore, for example, even if class classification is performed by performing 1-bit ADRC processing on a class tap, the number of class codes has a large value of 2 ⁶⁴ .

Therefore, in the present embodiment, the classifying circuit 43 uses the quantized DCT constituting the class tap.
A feature amount having high importance is extracted from the coefficient, and the class is classified based on the feature amount, thereby reducing the number of classes.

That is, FIG. 8 shows the classification circuit 43 of FIG.
Is shown.

The class taps are supplied to a power calculation circuit 51. The power calculation circuit 51 divides the quantized DCT coefficients constituting the class taps into those of several spatial frequency bands. Calculate the power in the frequency band.

That is, the power calculation circuit 51 converts the 8 × 8 quantized DCT coefficients constituting the class tap into, for example, four spatial frequency bands S ₀ , S ₁ , S ₂ , as shown in FIG.
Divided into S _3.

Here, each of the 8 × 8 quantized DCT coefficients constituting a class tap is represented by an alphabet x,
As shown in FIG. 7, when the raster frequency is represented by adding a sequential integer from 0 in the raster scan order, the spatial frequency band S ₀ has four quantized DCT coefficients x ₀ , x ₁ ,
consists x _8, x _9, the spatial frequency band S ₁ is 12 quantized DCT coefficients _{x 2, x 3, x 4} , x 5, x 6, x 7, x
_{10, x 11, x 12,} x 13, composed of x _14, x _15. Further, the spatial frequency band S ₂ is 12 quantized DCT coefficients _{x 16, x 17, x 24} , x 25, x 32, x 33, x 40, x 41, x
_48, is composed of _{x 49, x 56, x 57} , the spatial frequency band S
_3, 36 quantized DCT coefficients _{x 18, x 1 9, x} 20, x
_{21, x 22, x 23,} x 26, x 27, x 28, x 29, x 30,
_{x 31, x 34, x 3} 5, x 36, x 37, x 38, x 39, x 42, x
_{43, x 44, x 45,} x 46, x 47, x 50, x 5 1, x 52,
_{x 53, x 54, x 55} , x 58, x 59, x 60, x 61, x 62, x
Consists of ₆₃ .

Further, the power calculation circuit 51 performs quantization DC for each of the spatial frequency bands S ₀ , S ₁ , S ₂ , and S _3.
Calculate the power P ₀ , P ₁ , P ₂ , P ₃ of the AC component of the T coefficient,
Output to the class code generation circuit 52.

That is, the power operation circuit 51 operates in the spatial frequency band.
Area S₀For the above four quantized DCT coefficients x₀,
x₁, X₈, X₉AC component x of₁, X₈, X₉Sum of squares
x₁ ^Two+ X₈ ^Two+ X₉ ^TwoAnd calculate the power P₀As
Output to the Las code generation circuit 52. Also, the power calculation time
The road 51 corresponds to the above-mentioned 12 for the spatial frequency band S1.
AC components of the quantized DCT coefficients, ie, all 12
Is calculated, and the sum of squares of the quantized DCT coefficients of₁
To the class code generation circuit 52. Further
In addition, the power operation circuit 51 has a spatial frequency band S_TwoAnd S_ThreeNitsu
The spatial frequency band S₁As in
And each power P_TwoAnd P_ThreeAnd generate class code
Output to the circuit 52.

The class code generation circuit 52 uses the powers P ₀ , P ₁ , P ₂ , and P ₃ from the power calculation circuit 51 as corresponding threshold values TH 0, TH stored in the threshold value table storage unit 53.
1, TH2, and TH3, respectively, and outputs a class code based on the magnitude relation. That is, the class code generation circuit 52 compares the power P ₀ and the threshold value TH0, obtain one-bit code representing the magnitude relationship. Similarly, the class code generation circuit 52 calculates the power P ₁ and the threshold T
H1, the power P ₂ and the threshold value TH2, the power P ₃ and the threshold value TH3,
By comparing each, a 1-bit code is obtained for each. Then, the class code generation circuit 5
2 is a 4-bit code obtained by arranging the four 1-bit codes obtained as described above in, for example, a predetermined order (accordingly, any value from 0 to 15). Output as a class code representing the class of the high image quality block. Therefore, in the present embodiment, the high quality image block of interest is classified into any one of 2 ⁴ (= 16) classes.

The threshold table storage unit 53 stores thresholds TH0 to TH3 to be compared with the powers P _{0 to} P ₃ of the spatial frequency bands S _{0 to} S ₃ , respectively.

In the above case, the DC component x ₀ of the quantized DCT coefficient is not used in the class classification process. However, the class classification process can be performed using the DC component x _0. .

Returning to FIG. 6, the class code output from the classifying circuit 43 as described above is stored in the coefficient table storage unit 4.
4 is given as an address.

The coefficient table storage unit 44 stores a coefficient table in which tap coefficients obtained by performing a learning process described later are registered, and stores an address corresponding to the class code output from the class classification circuit 43. Is output to the product-sum operation circuit 45.

Here, in the present embodiment, one class code is obtained for the high-quality block of interest. On the other hand, in the present embodiment, the high image quality block is 16 × 16
Since the target high-quality block is composed of 256 pixels, 256 sets of tap coefficients for decoding each of the 256 pixels constituting the block are necessary.
Therefore, the coefficient table storage unit 44 stores 256 sets of tap coefficients for an address corresponding to one class code.

The product-sum operation circuit 45 stores the prediction tap output from the prediction tap extraction circuit 41 and the coefficient table storage 44
Obtains the tap coefficients output by the above, performs the linear prediction operation (product-sum operation) shown in Expression (1) using the prediction taps and the tap coefficients, and obtains the 16 × The pixel values (predicted values) of the 16 pixels are output to the block decomposition circuit 33 (FIG. 3) as decoding results of the corresponding DCT blocks.

Here, in the prediction tap extraction circuit 41, as described above, each pixel of the high-quality block of interest is
The target pixel is sequentially set as the target pixel, and the product-sum operation circuit 45 operates in an operation mode (hereinafter, appropriately referred to as a pixel position mode) corresponding to the position of the target pixel in the target high image quality block. Perform processing.

That is, for example, among the pixels of the high-quality block of interest, the i-th pixel in the raster scan order is represented by p _i, and when the pixel p _i is the pixel of interest, the product-sum operation circuit 45 Performs pixel position mode #i processing.

More specifically, as described above, the coefficient table storage unit 44 stores 256 high-quality blocks of interest.
Although outputs 256 tap coefficients set for decoding the respective pixels, to represent the set of tap coefficients for decoding the pixel p _i of which the W _i, sum-of-products operation circuit 45
The operation mode is, when the pixel position mode #i, using the prediction taps and a set W _i of the tap coefficients 256 sets, performs product sum computation of the formula (1), the product-sum operation result _Is the decoding result of the pixel p _i .

Next, the processing of the coefficient conversion circuit 32 of FIG. 6 will be described with reference to the flowchart of FIG.

The quantized DCT coefficients for each block output from the entropy decoding circuit 31 (FIG. 3) are sequentially received by the prediction tap extraction circuit 41 and the class tap extraction circuit 42, and the prediction tap extraction circuit 41 is supplied thereto. High-quality blocks corresponding to blocks of quantized DCT coefficients (DCT blocks) are sequentially set as high-quality blocks of interest.

Then, in step S11, the class tap extracting circuit 42 extracts, from the quantized DCT coefficients received there, one used for classifying the high-quality block of interest, and forms a class tap. It is supplied to the class classification circuit 43.

The class classification circuit 43 classifies the high-quality block of interest using the class tap from the class tap extraction circuit 42 in step S12, and outputs the resulting class code to the coefficient table storage unit 44. I do.

That is, in step S12, as shown in the flowchart of FIG.
, The power calculation circuit 51 of the class classification circuit 43 (FIG. 8) uses 8 × 8 quantized DCs forming a class tap.
The T coefficient is divided into the four spatial frequency bands S _{0 to} S ₃ shown in FIG. 9, and the respective powers P _{0 to} P ₃ are calculated.
The powers P _{0 to} P ₃ are output from the power calculation circuit 51 to the class code generation circuit 52.

The class code generation circuit 52 determines in step S
At 22, the threshold value TH0 is stored in the threshold value table storage unit 53.
To TH3 and read the power P from the power calculation circuit 51.
₀ to the P _3, respectively, compared with the threshold TH0 to TH3 respectively, to generate a class code based on the respective magnitude relation, the process returns.

Returning to FIG. 10, the class code obtained as described above in step S12 is given from the class classification circuit 43 to the coefficient table storage unit 44 as an address.

When the coefficient table storage unit 44 receives the class code as the address from the class classification circuit 43, in step S13, it sets 256 tap coefficients (256 corresponding to the class of the class code) stored at the address. Read out the tap coefficient of the set)
Output to the product-sum operation circuit 45.

Then, the process proceeds to a step S14, where the prediction tap extracting circuit 41 selects one of the pixels of the high quality image block of interest.
In the raster scan order, a pixel that has not yet been set as a target pixel is set as a target pixel, and a quantized DCT coefficient used for predicting a pixel value of the target pixel is extracted and configured as a prediction tap. The prediction tap is supplied from the prediction tap extraction circuit 41 to the product-sum operation circuit 45.

Here, in the present embodiment, the same prediction tap is formed for every pixel of each high-quality block for each high-quality block. For high quality blocks,
If the process is performed only on the pixel which is initially set as the target pixel, it is not necessary to perform the process on the remaining 255 pixels.

In step S15, the product-sum operation circuit 45 acquires a set of tap coefficients corresponding to the pixel position mode for the target pixel from among the 256 sets of tap coefficients output from the coefficient table storage unit 44 in step S13. Using the set of tap coefficients and the prediction tap supplied from the prediction tap extraction circuit 41 in step S14, the product-sum operation shown in Expression (1) is performed to obtain a decoded value of the pixel value of the target pixel.

Then, proceeding to step S16, the prediction tap extracting circuit 41 determines whether or not all pixels of the high quality image block of interest have been processed as the pixel of interest.
If it is determined in step S16 that all the pixels of the high-quality block of interest have not been processed yet as the pixel of interest, the process returns to step S14, and the prediction tap extraction circuit 41 returns In the raster scan order, a pixel which has not yet been set as a target pixel is set as a new target pixel, and the same processing is repeated thereafter.

If it is determined in step S16 that all pixels of the high-quality block of interest have been processed as the pixel of interest, that is, if the decoded values (8 × 8 quantum pixels) of all the pixels of the high-quality block of interest have been processed. The normalized DCT coefficient is 8
X8 pixels, and the 8x8 pixels are converted to 16x
If a 16-pixel high-quality image is obtained, the product-sum operation circuit 45 outputs the high-quality block composed of the decoded value to the block decomposition circuit 33 (FIG. 3), and ends the processing. .

The process according to the flowchart of FIG. 10 is repeated each time the prediction tap extracting circuit 41 sets a new high-quality block of interest.

Next, FIG. 12 shows an example of the configuration of an embodiment of a learning device for performing a learning process of tap coefficients stored in the coefficient table storage section 44 of FIG.

[0131] One or more pieces of learning image data are supplied to the thinning circuit 60 as teacher data to be a teacher at the time of learning. , The coefficient conversion circuit 3 of FIG.
The product-sum operation circuit 45 in 2 performs a process based on the enhancement process performed by performing the product-sum operation using the tap coefficients. That is, here, the improvement processing is processing for converting 8 × 8 pixels into high-quality 16 × 16 pixels (having an improved resolution) whose horizontal and vertical spatial resolutions are doubled. Then, the thinning circuit 60 thins out the pixels of the image data as the teacher data, and generates image data (hereinafter, appropriately referred to as quasi-teacher data) in which both the number of horizontal and vertical pixels are halved.

The image data as the quasi-teacher data has the same image quality (resolution) as the image data to be subjected to the JPEG encoding in the encoder 21 (FIG. 1). Assuming that the target image is an SD (Standard Density) image, the image to be used as teacher data is an HD (High Density) image in which the number of horizontal and vertical pixels of the SD image is doubled. Must be used.

The blocking circuit 61 converts one or more SD images generated by the thinning circuit 60 as quasi-teacher data into
As in the case of JPEG encoding, the image data is divided into 8 × 8 pixel blocks.

The DCT circuit 62 sequentially reads out the pixel blocks formed by the blocking circuit 61 and performs DCT processing on the pixel blocks to obtain DCT coefficient blocks. This block of DCT coefficients is supplied to the quantization circuit 63.

The quantization circuit 63 quantizes the block of DCT coefficients from the DCT circuit 62 in accordance with the same quantization table used for JPEG encoding in the encoder 21 (FIG. 2), and obtains the resulting quantization. Chemical D
The block of the CT coefficient (DCT block) is sequentially supplied to the prediction tap extracting circuit 64 and the class tap extracting circuit 65.

The prediction tap extracting circuit 64 calculates a prediction pixel in FIG. 6 for a pixel which is a pixel of interest out of 16 × 16 pixels constituting a high image quality block to be a high image quality block of interest by a normal equation adding circuit 67 described later. Tap extraction circuit 4
1 constitutes the same prediction tap as the quantization circuit 63
, By extracting necessary quantized DCT coefficients from the output. The prediction tap is supplied from the prediction tap extraction circuit 64 to the normal equation addition circuit 67 as student data to be a student during learning.

The class tap extracting circuit 65 calculates the same class tap as that constituted by the class tap extracting circuit 42 in FIG. It is composed by extracting. This class tap is a class tap extraction circuit 6
5 is supplied to the classifying circuit 66.

The class classification circuit 66 performs the same processing as the class classification circuit 43 of FIG. 6 using the class tap from the class tap extraction circuit 65, thereby classifying the high-quality image block of interest and obtaining the result. The supplied class code is supplied to the normal equation adding circuit 67.

To the normal equation adding circuit 67, the same HD image as that supplied to the thinning circuit 60 as teacher data is supplied.
7 divides the HD image into high-quality blocks of 16 × 16 pixels, and sequentially sets the high-quality blocks as high-quality blocks of interest. Further, the normal equation adding circuit 6
7, among the 16 × 16 pixels constituting the target high-quality image block, for example, in the raster scan order, those not yet set as the target pixel are sequentially set as the target pixel, and (the pixel value of) the target pixel. The addition is performed on the prediction tap (the quantized DCT coefficient constituting the prediction tap) from the prediction tap configuration circuit 64.

That is, the normal equation addition circuit 67 uses the prediction taps (student data) for each class corresponding to the class code supplied from the class classification circuit 66 to generate each component in the matrix A of the equation (8). Multiplication (x _in x _im ) between the student data, and an operation corresponding to summation (Σ).

Further, the normal equation addition circuit 67 uses the prediction tap (student data) and the pixel of interest (teacher data) for each class corresponding to the class code supplied from the class classification circuit 66, and calculates the equation (8). )), Multiplication (x _in y _i ) of student data and teacher data, which are components in the vector v, and an operation corresponding to summation (Σ) are performed.

Note that the normal equation addition circuit 67
The above-described addition is performed for each class in each pixel position mode for the target pixel.

The normal equation adding circuit 67 performs the above-mentioned addition with all the pixels constituting the HD image serving as teacher data supplied thereto as target pixels, and thereby, for each class, for each pixel position mode. The normal equation shown in equation (8) is established.

The tap coefficient determination circuit 68 solves the normal equation generated for each class (and for each pixel position mode) in the normal equation addition circuit 67, thereby obtaining 256 sets of tap coefficients for each class. The coefficients are supplied to addresses corresponding to each class in the coefficient table storage unit 69.

Note that, depending on the number of images prepared as learning images, the contents of the images, and the like, the normal equation adding circuit 67 does not provide a class in which the necessary number of normal equations for obtaining the tap coefficients cannot be obtained. In such a case, the tap coefficient determination circuit 68 outputs, for example, a default tap coefficient for such a class.

The coefficient table storage section 69 stores 256 sets of tap coefficients for each class supplied from the tap coefficient determination circuit 68.

Next, the processing (learning processing) of the learning apparatus of FIG. 12 will be described with reference to the flowchart of FIG.

The HD image, which is image data for learning, is supplied to the thinning circuit 60 as teacher data. In step S30, the thinning circuit 60 thins out the pixels of the HD image as the teacher data, S as quasi-teacher data in which the number of vertical pixels is halved
Generate a D image.

Then, in step S31, the blocking circuit 61 converts the SD image as the quasi-teacher data obtained by the thinning circuit 60 into 8 × 8 in the same manner as in the case of JPEG encoding by the encoder 21 (FIG. 2). The process is divided into pixel blocks of pixels, and the process proceeds to step S32.
In step S32, the DCT circuit 62 sequentially reads out the pixel blocks that have been blocked by the blocking circuit 61, and performs DCT processing on the pixel blocks to obtain a DCT.
The block is a T coefficient block, and the process proceeds to step S33. In step S33, the quantization circuit 63 sequentially reads out the blocks of the DCT coefficients obtained in the DCT circuit 62, quantizes the blocks according to the same quantization table used for the JPEG encoding in the encoder 21, and performs the quantization DCT. The block is made up of coefficients (DCT block).

On the other hand, the HD image as teacher data is also supplied to the normal equation addition circuit 67, and the normal equation addition circuit 67 divides the HD image into high-quality blocks of 16 × 16 pixels. Of the high image quality blocks, those which have not yet been set as the high quality image block of interest are set as high quality image blocks of interest. Further, in step S34, the class tap extraction circuit 65
However, among the pixel blocks blocked by the blocking circuit 61, the quantized DCT coefficients used for classifying the high-quality block of interest are obtained by the D
A class tap is extracted from the CT block, and is supplied to a class classification circuit 66. The classification circuit 66
In step S35, the high-quality block of interest is classified into classes using the class taps from the class tap extraction circuit 65, as in the case described with reference to the flowchart of FIG. Then, the process proceeds to step S36.

In step S36, the normal equation adding circuit 67 sets, as the target pixel, a pixel which has not been set as the target pixel in the raster scan order among the pixels of the target high image quality block, and sets the prediction tap extracting circuit 64 as For the pixel of interest, the same prediction tap as that formed by the prediction tap extraction circuit 41 in FIG. 6 is configured by extracting necessary quantized DCT coefficients from the output of the quantization circuit 63. And
The prediction tap extraction circuit 64 supplies the prediction tap for the pixel of interest to the normal equation addition circuit 67 as student data, and proceeds to step S37.

In step S37, the normal equation adding circuit 67 applies the matrix A and the vector of the equation (8) to the target pixel as teacher data and the prediction tap (the quantized DCT coefficient constituting the student data) as student data. v is added as described above. This addition is performed for each class corresponding to the class code from the class classification circuit 66 and for each pixel position mode for the target pixel.

Then, the process proceeds to a step S38, where the normal equation adding circuit 67 determines whether or not all the pixels of the high quality image block of interest have been added as the pixel of interest. If it is determined in step S38 that all the pixels of the high-quality block of interest have not been added yet as the pixel of interest, the process returns to step S36, and the normal equation adding circuit 67 returns to the normal equation adding circuit 67. Of these, a pixel that has not been set as the pixel of interest in the raster scan order is newly set as the pixel of interest, and the same processing is repeated thereafter.

If it is determined in step S38 that all pixels of the high-quality block of interest have been added as the pixel of interest, the flow advances to step S39.
The normal equation adding circuit 67 determines whether or not all the high-quality blocks obtained from the image as the teacher data have been processed as the high-quality blocks of interest. In step S39, if it is determined that all the high-quality blocks obtained from the image as the teacher data have not been processed yet as the high-quality block of interest, the process returns to step S34, and is determined as the high-quality block of interest. A high-quality block that has not been used is newly set as a high-quality block of interest, and the same processing is repeated thereafter.

On the other hand, if it is determined in step S39 that all the high-quality blocks obtained from the image as the teacher data have been processed as the high-quality blocks of interest, that is, the normal equation adding circuit 67 When the normal equation for each pixel position mode is obtained, the process proceeds to step S40, where the tap coefficient determination circuit 68 solves the normal equation generated for each class of pixel position mode, so that for each class, 256 corresponding to each of the 256 pixel position modes of the class
The tap coefficient of the set is determined, and the coefficient table storage unit 69
Of the class is supplied to the address corresponding to each class and stored,
The process ends.

As described above, the coefficient table storage unit 6
The tap coefficients for each class stored in 9 are stored in the coefficient table storage unit 44 in FIG.

Therefore, the tap coefficients stored in the coefficient table storage unit 44 are such that the prediction error (here, the square error) of the prediction value of the original pixel value obtained by performing the linear prediction operation is statistically minimized. Thus, according to the coefficient conversion circuit 32 shown in FIG. 6, the JPEG encoded image is not limited to the image quality of the HD image used as the teacher data. It can be decoded into a close high-quality image.

Further, according to the coefficient conversion circuit 32, as described above, the decoding processing of the JPEG-coded image
Since the improvement processing for improving the image quality is performed at the same time, the JPEG encoded image is
A high-quality decoded image can be obtained efficiently.

Next, FIG. 14 shows the coefficient conversion circuit 32 of FIG.
2 shows a second configuration example. In the figure, parts corresponding to those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 32 in FIG. 14 has basically the same configuration as that in FIG. 6 except that an inverse quantization circuit 71 is newly provided.

In the embodiment shown in FIG. 14, a quantized DCT coefficient for each block obtained by entropy decoding encoded data in entropy decoding circuit 31 (FIG. 3) is supplied to inverse quantization circuit 71. .

As described above, the entropy decoding circuit 31 converts the coded data from the quantized DCT
In addition to the coefficients, a quantization table can be obtained. In the embodiment shown in FIG. 14, this quantization table is also supplied from the entropy decoding circuit 31 to the inverse quantization circuit 71.

The inverse quantization circuit 71 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 31 in accordance with the quantization table from the entropy decoding circuit 31, and converts the resulting DCT coefficient into a prediction tap extraction circuit. 41 and a class tap extraction circuit 42.

Therefore, in the prediction tap extraction circuit 41 and the class tap extraction circuit, not the quantized DCT coefficient but
A prediction tap and a class tap are respectively configured for DCT coefficients, and thereafter, for DCT coefficients,
The same processing as in FIG. 6 is performed.

As described above, in the embodiment of FIG. 14, since the processing is performed not on the quantized DCT coefficient but on the DCT coefficient, the tap coefficients stored in the coefficient table storage section 44 are the same as those in FIG. It needs to be different.

FIG. 15 shows an example of the configuration of an embodiment of a learning device for performing a learning process of tap coefficients stored in the coefficient table storage section 44 of FIG. In the figure, parts corresponding to those in FIG. 12 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device in FIG. 15 is configured basically in the same manner as in FIG. 12 except that an inverse quantization circuit 81 is newly provided at the subsequent stage of the quantization circuit 63.

In the embodiment shown in FIG. 15, the inverse quantization circuit 81 inversely quantizes the quantized DCT coefficient output from the inverse quantization circuit 63 in the same manner as the inverse quantization circuit 71 shown in FIG.
The DCT coefficients obtained as a result are extracted by the prediction tap extraction circuit 6.
4 and the class tap extraction circuit 65.

Therefore, in the prediction tap extraction circuit 64 and the class tap extraction circuit 65, not the quantized DCT coefficient but
A prediction tap and a class tap are respectively configured for DCT coefficients, and thereafter, for DCT coefficients,
The same processing as in FIG. 12 is performed.

As a result, a tap coefficient that reduces the influence of a quantization error caused by the DCT coefficient being quantized and further dequantized is obtained.

Next, FIG. 16 shows the coefficient conversion circuit 32 of FIG.
3 shows a third configuration example. In the figure, parts corresponding to those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 32 of FIG.
The configuration is basically the same as that in FIG. 6 except that the class classification circuit 43 is not provided.

Therefore, in the embodiment shown in FIG. 16, there is no concept of a class. However, since it is considered that there is only one class, the coefficient table storage unit 44 stores only one class of tap coefficients. The processing is performed using this.

As described above, in the embodiment of FIG. 16, the tap coefficients stored in the coefficient table storage section 44 are:
This is different from the case in FIG.

FIG. 17 shows an example of the configuration of an embodiment of the learning apparatus for performing the learning process of the tap coefficients stored in the coefficient table storage section 44 of FIG. In the figure, parts corresponding to those in FIG. 12 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device in FIG. 17 is basically configured in the same manner as in FIG. 12 except that the class tap extraction circuit 65 and the class classification circuit 66 are not provided.

Therefore, in the learning apparatus of FIG. 17, the above-described addition is performed for each pixel position mode in the normal equation addition circuit 67 regardless of the class. Then, the tap coefficient is determined in the tap coefficient determination circuit 68 by solving the normal equation generated for each pixel position mode.

Next, FIG. 18 shows the coefficient conversion circuit 32 of FIG.
4 shows a fourth configuration example. In the drawings, portions corresponding to those in FIG. 6 or FIG. 14 are denoted by the same reference numerals, and a description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 32 shown in FIG. 18 is basically the same as that shown in FIG. 6 except that the class tap extraction circuit 42 and the class classification circuit 43 are not provided and the inverse quantization circuit 71 is newly provided. Are configured in the same manner as in the above.

Therefore, in the embodiment of FIG. 18, as in the case of the above-described embodiment of FIG. 16, only one class of tap coefficients is stored in the coefficient table storage section 44. The processing is performed.

Further, in the embodiment shown in FIG.
Similarly to the case of the embodiment, in the prediction tap extraction circuit 41, prediction taps are configured not for the quantized DCT coefficients but for the DCT coefficients output from the inverse quantization circuit 71. Processing is performed as an object.

Therefore, also in the embodiment of FIG. 18, the tap coefficients stored in the coefficient table
It is different from the case of.

FIG. 19 shows an example of the configuration of an embodiment of the learning apparatus for performing the learning process of the tap coefficients stored in the coefficient table storage section 44 of FIG. In the drawings, the same reference numerals are given to portions corresponding to the case in FIG. 12 or FIG. 15, and the description thereof will be appropriately omitted below. That is, the learning apparatus of FIG. 19 is basically the same as that of FIG. 12 except that the class tap extraction circuit 65 and the class classification circuit 66 are not provided, and the inverse quantization circuit 81 is newly provided. It is configured similarly.

Therefore, in the learning device of FIG. 19, in the prediction tap extracting circuit 64, instead of the quantized DCT coefficient,
Prediction taps are configured for DCT coefficients, and thereafter, processing is performed for DCT coefficients. further,
In the normal equation adding circuit 67, the above addition is
The tap coefficient is obtained by solving a normal equation generated independently of the class in the tap coefficient determination circuit 68 regardless of the class.

Next, in the above description, JPEG-encoded images for compressing and encoding still images are used. However, the present invention relates to compression-encoding of moving images, for example, MPEG-encoded images. It is also possible to use

That is, FIG. 20 shows an example of the configuration of the encoder 21 shown in FIG. 2 when MPEG coding is performed.

The frames (or fields) constituting the moving image to be MPEG-encoded are sequentially supplied to a motion detection circuit 91 and a computing unit 92.

The motion detection circuit 91 detects a motion vector of a frame supplied thereto in units of macroblocks of 16 × 16 pixels, and the entropy encoding circuit 9
6 and the motion compensation circuit 100.

The arithmetic unit 92 determines that the image supplied thereto is
If the picture is an I (Intra) picture, it is supplied to the blocking circuit 93 as it is, and the picture is P (Predictive) or B (Bidirectional).
ly predictive) picture, the motion compensation circuit 100
, And the difference value is supplied to the blocking circuit 93.

A blocking circuit 93 divides the output of the arithmetic unit 92 into 8 × 8 pixel blocks, and performs DCT.
The signal is supplied to a circuit 94. The DCT circuit 94 performs a DCT process on the pixel block from the blocking circuit 93, and supplies a DCT coefficient obtained as a result to the quantization circuit 95. The quantization circuit 95 receives the D from the DCT circuit 93 in block units.
The CT coefficients are quantized in a predetermined quantization step, and the resulting quantized DCT coefficients are
6 The entropy encoding circuit 96 entropy-encodes the quantized DCT coefficient from the quantization circuit 95, adds the motion vector from the motion detection circuit 91 and other necessary information, and obtains encoded data ( For example, an MPEG transport stream)
Output as PEG encoding result.

Of the quantized DCT coefficients output by the quantization circuit 95, I-pictures and P-pictures need to be locally decoded to be used as reference pictures for P-pictures and B-pictures to be encoded later. In addition to the entropy encoding circuit 96, the signal is also supplied to an inverse quantization circuit 97.

The inverse quantization circuit 97 inversely quantizes the quantized DCT coefficient from the quantization circuit 95, thereby obtaining the DCT
The coefficients are supplied to the inverse DCT circuit 98. The inverse DCT circuit 98 converts the DCT coefficient from the inverse quantization circuit 97 into an inverse DCT
Process and output to the arithmetic unit 99. The arithmetic unit 99 has an inverse D
In addition to the output of the CT circuit 98, a reference image output by the motion compensation circuit 100 is also supplied.
When the output of the inverse DCT circuit 98 is that of a P picture, the output of the inverse DCT circuit 98 is added to the output of the motion compensation circuit 100 to decode the original image, and the motion compensation circuit 1
Supply to 00. The arithmetic unit 99 includes the inverse DCT circuit 9
If the output of No. 8 is that of an I picture, the output is a decoded image of the I picture, so that the output is supplied to the motion compensation circuit 100 as it is.

The motion compensation circuit 100 performs motion compensation on the locally decoded image supplied from the arithmetic unit 99 according to the motion vector from the motion detection circuit 91, and outputs the image after the motion compensation. The reference images are supplied to computing units 92 and 99.

Here, FIG. 21 shows the above MPEG format.
1 shows an example of a configuration of a conventional MPEG decoder for decoding encoded data obtained as a result of encoding.

The coded data is transmitted to the entropy decoding circuit 1
The entropy decoding circuit 111 entropy decodes the encoded data to obtain quantized DCT coefficients, motion vectors, and other information. And the quantized DC
The T coefficient is supplied to the inverse quantization circuit 112, and the motion vector is supplied to the motion compensation circuit 116.

The inverse quantization circuit 112 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 111 to obtain a DCT coefficient, which is supplied to the inverse DCT circuit 113. The inverse DCT circuit 113 performs an inverse DCT process on the DCT coefficient from the inverse quantization circuit 112 and outputs the result to the computing unit 114. To the arithmetic unit 114, in addition to the output of the inverse quantization circuit 113, the already decoded I
A picture or a P picture is converted into an entropy decoding circuit 1
When the output of the inverse DCT circuit 113 is that of a P or B picture, the arithmetic unit 114 performs the motion compensation on the basis of the motion vector from the motion vector 11 and supplies the reference image. By adding the output and the output of the motion compensation circuit 100, the original image is decoded and supplied to the block decomposition circuit 115. If the output of the inverse DCT circuit 113 is that of an I picture, the operation unit 114 outputs the decoded image of the I picture.
5

The block disassembly circuit 115 includes a computing unit 114
By deblocking the decoded image supplied in pixel block units from, a decoded image is obtained and output.

On the other hand, the motion compensation circuit 116
4 receives the I-picture and the P-picture of the decoded image, and performs motion compensation according to the motion vector from the entropy decoding circuit 111. Then, the motion compensation circuit 116 supplies the image after the motion compensation to the arithmetic unit 114 as a reference image.

The decoder 22 shown in FIG. 3 can also efficiently decode encoded data encoded by MPEG into a high-quality image as described above.

That is, the encoded data is supplied to the entropy decoding circuit 31, and the entropy decoding circuit 31 performs entropy decoding on the encoded data. The quantized DCT coefficients, motion vectors, and other information obtained as a result of the entropy decoding are supplied from the entropy decoding circuit 31 to the coefficient conversion circuit 32.

The coefficient transforming circuit 32 performs a predetermined prediction operation using the quantized DCT coefficient Q from the entropy decoding circuit 31 and the tap coefficient obtained by performing the learning. By performing motion compensation according to the motion vector as needed, the quantized DCT coefficient is decoded into a high-quality pixel value,
The high image quality block including the high image quality pixel value is supplied to the block decomposition circuit 33.

The block decomposition circuit 33 includes a coefficient conversion circuit 3
By deblocking the high-quality block obtained in step 2, a high-quality decoded image whose number of pixels in both the horizontal and vertical directions is, for example, twice as large as that of the MPEG encoded image is obtained and output. .

Next, FIG.
4 shows a configuration example of the coefficient conversion circuit 32 in FIG. 3 when decoding PEG-encoded data. In the figure, the same reference numerals are given to the portions corresponding to the case in FIG. 18 or FIG. 21, and the description thereof will be appropriately omitted below. That is, the coefficient conversion circuit 32 shown in FIG.
The configuration is basically the same as that in FIG. 18 except that an arithmetic unit 114 and a motion compensation circuit 116 in FIG. 21 are provided at the subsequent stage of the product-sum operation circuit 45.

Therefore, in the coefficient conversion circuit 32 of FIG.
The quantized DCT coefficient is inversely quantized in the inverse quantization circuit 71, and the prediction tap is formed in the prediction tap extraction circuit 41 using the DCT coefficient obtained as a result. Then, the product-sum operation circuit 45 performs a prediction operation using the prediction tap and the tap coefficient stored in the coefficient table storage unit 44, so that both the horizontal and vertical pixel numbers are equal to those of the original image. Outputs twice as high quality data.

The arithmetic unit 114 is provided with the product-sum operation circuit 4
5 is added to the output of the motion compensation circuit 116 as necessary, thereby decoding a high-quality image in which the number of horizontal and vertical pixels is twice the original image, and Output to the decomposition circuit 33 (FIG. 3).

That is, for the I picture, the output of the product-sum operation circuit 45 is a high-quality image in which the number of pixels in both the horizontal and vertical directions is twice that of the original image. 114 is the output of the product-sum operation circuit 45,
Output to the block decomposition circuit 33.

For a P or B picture, the product-sum operation circuit 45 outputs a high-quality image in which the number of horizontal and vertical pixels is twice the original image and a high-quality reference image. The arithmetic unit 114 calculates the difference from the image
By adding the output of the product-sum operation circuit 45 to the high-quality reference image supplied from the motion compensation circuit 116, the number of horizontal and vertical pixels is double that of the original image. The image is decoded into an image and output to the block decomposition circuit 33.

On the other hand, the motion compensation circuit 116
4 receives the I and P pictures among the high-quality decoded images output by using the motion vectors from the entropy decoding circuit 31 (FIG. 3) for the high-quality decoded images of the I or P pictures. By performing motion compensation, a high-quality reference image is obtained and supplied to the arithmetic unit 114.

Here, since the number of horizontal and vertical pixels of the decoded image is twice as large as that of the original image, the motion compensating circuit 116 uses the motion vector from the entropy decoding circuit 31, for example. The motion compensation is performed in accordance with a motion vector whose size in both the horizontal and vertical directions is doubled.

Next, FIG. 23 shows an example of the configuration of an embodiment of a learning device for learning tap coefficients stored in the coefficient table storage section 44 of FIG. In FIG. 1, FIG.
Parts corresponding to those in FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

The HD image for learning is input to the thinning circuit 120 as teacher data, and the thinning circuit 120
For example, similar to the decimation circuit 60 in FIG. 12, the pixels of the HD image as the teacher data are decimated, and the quasi-teacher data which is an SD image in which both the horizontal and vertical pixels are halved is generated. Then, S as the quasi-teacher data
The D image is obtained from the motion vector detection circuit 121 and the arithmetic unit 1
22.

The motion vector detecting circuit 121, the arithmetic unit 12
2. Blocking circuit 123, DCT circuit 124, quantization circuit 125, inverse quantization circuit 127, inverse DCT circuit 12
8, the arithmetic unit 129 or the motion compensation circuit 130
0 motion vector detection circuit 91, arithmetic unit 92, blocking circuit 93, DCT circuit 94, quantization circuit 95, inverse quantization circuit 97, inverse DCT circuit 98, arithmetic unit 99, or motion compensation circuit 100, respectively. The processing is performed, so that the quantization circuit 125 of FIG.
5 is output.

The quantized DCT output from the quantization circuit 125
The coefficient is supplied to the inverse quantization circuit 81 and the inverse quantization circuit 8
1 dequantizes the quantized DCT coefficient from the quantization circuit 125, converts it into a DCT coefficient, and
4 The prediction tap extraction circuit 64 forms a prediction tap from the DCT coefficient from the inverse quantization circuit 81 and supplies the prediction tap to the normal equation addition circuit 67 as student data.

On the other hand, the HD image as teacher data is supplied not only to the thinning circuit 120 but also to an arithmetic unit 132. Arithmetic unit 132 outputs H as teacher data.
If necessary, the output of the interpolation circuit 131 is subtracted from the D image and supplied to the normal equation addition circuit 67.

That is, the interpolation circuit 131 is the motion compensation circuit 1
A high-quality reference image in which the number of horizontal and vertical pixels of the reference image of the SD image output by 30 is doubled,
Feed to 2.

If the HD picture supplied thereto is an I picture, the arithmetic unit 132 outputs the HD picture of the I picture.
The image is supplied as it is to the normal equation adding circuit 67 as teacher data. When the HD image supplied thereto is a P or B picture, the arithmetic unit 132 outputs the HD image of the P or B picture and the interpolation circuit 1.
By calculating a difference between the reference image and the high-quality image output by the output unit 31, a high-quality difference of the SD image (quasi-teacher data) output by the calculator 122 is obtained, and this is used as teacher data. , To the normal equation addition circuit 67.

In the interpolation circuit 131, for example, the number of pixels can be increased by simple interpolation. Further, in the interpolation circuit 131, for example, the number of pixels can be increased by a class classification adaptive process. further,
The arithmetic unit 132 converts the HD image as the teacher data into the MP
It is possible to use the EG-encoded and local-decoded motion-compensated image as a reference image.

The normal equation adding circuit 67 includes an arithmetic unit 132
Is used as teacher data and the inverse quantization circuit 81
As described above, the addition is performed using the prediction tap from the as the student data, thereby generating a normal equation.

Then, the tap coefficient determination circuit 68 solves the normal equation generated by the normal equation addition circuit 67 to obtain the tap coefficient, and supplies the tap coefficient to the coefficient table storage section 69 for storage.

The product-sum operation circuit 45 shown in FIG. 22 decodes the coded data obtained by the MPEG coding using the tap coefficients thus obtained.
The decoding process of the G-encoded image and the process of improving the image quality can be performed at the same time.
From the PEG-encoded image, it is possible to efficiently obtain a decoded image of high image quality, that is, an HD image in which the number of pixels in both the horizontal and vertical directions is doubled in this embodiment.

Note that the coefficient conversion circuit 32 in FIG. 22 can be configured without providing the inverse quantization circuit 71. In this case, the learning device in FIG. 23 may be configured without the inverse quantization circuit 81.

The coefficient conversion circuit 32 shown in FIG.
As in the case of the above, it is possible to provide and configure the class tap extraction circuit 42 and the class classification circuit 43. In this case, the learning device in FIG. 23 may be provided with a class tap extraction circuit 65 and a class classification circuit 66 as in the case of FIG.

Further, in the case described above, the decoder 22
In FIG. 3, a decoded image in which the spatial resolution of the original image is doubled is obtained. In the decoder 22, however, a decoded image in which the spatial resolution of the original image is set to an arbitrary multiple, and furthermore, It is also possible to obtain a decoded image in which the time resolution of the original image is improved.

That is, for example, when an image to be MPEG-encoded has a low temporal resolution as shown in FIG.
The G-encoded data can be decoded into an image whose temporal resolution is doubled as shown in FIG. 24B. Further, for example, MP
When an image to be EG-encoded is an image of 24 frames / second used in a movie as shown in FIG. 25A, the decoder 22 outputs encoded data obtained by MPEG-encoding the image. As shown in FIG. 25 (B), the time resolution of the original image is increased by 60/24 times,
It is possible to decode the image into seconds. In this case, so-called 2-3 pull-down can be easily performed.

Here, as described above, when the temporal resolution is to be improved in the decoder 22, the prediction taps and the class taps are composed of, for example, DCT coefficients of two or more frames as shown in FIG. It is possible to do so.

Further, the decoder 22 can obtain a decoded image in which not only one of the spatial resolution or the temporal resolution but also both of them is improved.

Next, the above-described series of processing can be performed by hardware or can be performed by software. When a series of processing is performed by software, a program constituting the software is
Installed on a general-purpose computer.

FIG. 27 shows a configuration example of an embodiment of a computer in which a program for executing the above-described series of processing is installed.

The program is stored in a hard disk 205 or a ROM 2 as a recording medium built in the computer.
03 can be recorded in advance.

Alternatively, the program is stored in a floppy (registered trademark) disk, CD-ROM (Compact Disc Read Onl
y Memory), MO (Magneto optical) disc, DVD (Digita
l Versatile Disc), a magnetic disk, a semiconductor memory, etc., can be temporarily or permanently stored (recorded) in a removable recording medium 211. Such a removable recording medium 211 can be provided as so-called package software.

The program may be installed in the computer from the removable recording medium 211 as described above, or may be wirelessly transferred from a download site to the computer via a digital satellite broadcasting artificial satellite, or transmitted to a LAN (Local Area). Network) or the Internet, and the program can be transferred to the computer by wire, and the computer can receive the transferred program by the communication unit 208 and install the program on the built-in hard disk 205.

The computer has a CPU (Central Processing).
Unit 202. The CPU 202 has a bus 2
01, the input / output interface 210 is connected. The CPU 202 operates the input unit 207 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 210. When a command is input, the ROM (Read O
(Nly Memory) 203 is executed. Alternatively, the CPU 202 may execute a program stored in the hard disk 205, a program transferred from a satellite or a network, received by the communication unit 208 and installed on the hard disk 205, or a removable recording medium 211 mounted on the drive 209. The program read and installed on the hard disk 205 is stored in a RAM (Random Access Memory).
y) Load to 204 and execute. This allows the CPU 20
2 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 202 transmits the processing result as necessary, for example, via the input / output interface 210.
An output is made from an output unit 206 including an LCD (Liquid CryStal Display), a speaker, or the like, or transmitted from a communication unit 208, and further recorded on the hard disk 205.

Here, in this specification, processing steps for writing a program for causing a computer to perform various processes do not necessarily need to be processed in a time-series manner in the order described in the flowchart. Alternatively, it also includes processing executed individually (for example, parallel processing or processing by an object).

Further, the program may be processed by one computer, or may be processed in a distributed manner by a plurality of computers. Further, the program may be transferred to a remote computer and executed.

Although the present embodiment is directed to image data, the present invention is also applicable to, for example, audio data.

In the present embodiment, at least D
Although JPEG encoding for performing CT processing and decoding of MPEG-encoded data are performed, the present invention is applicable to decoding of data transformed by other orthogonal transformation or frequency transformation. That is, the present invention is also applicable to, for example, decoding subband encoded data or Fourier transformed data.

Further, in the present embodiment, the decoder 22
In, the tap coefficients used for decoding are stored in advance, but the tap coefficients can be included in the encoded data and provided to the decoder 22.

In this embodiment, the decoding is performed by the linear primary prediction operation using the tap coefficients. However, the decoding may be performed by a second-order or higher-order prediction operation. It is.

[0234]

According to the first data processing apparatus, the data processing method, and the recording medium of the present invention, the tap coefficient obtained by performing the learning is obtained, and the tap coefficient and the conversion data are used to obtain the tap coefficient. By performing a predetermined prediction operation, the converted data is decoded into the original data,
In addition, processed data obtained by subjecting the original data to predetermined processing is obtained. Therefore, efficiently decode the converted data,
In addition, a predetermined process can be performed on the decoded data.

According to the second data processing apparatus, the data processing method, and the recording medium of the present invention, teacher data to be a teacher is subjected to processing based on a predetermined process, and the resulting quasi-teacher data is At least orthogonal transformation or frequency transformation generates student data to be students. Learning is performed so that the prediction error of the prediction value of the teacher data obtained by performing the prediction operation using the tap coefficient and the student data is statistically minimized, and the tap coefficient is obtained. Therefore, by using the tap coefficients, it is possible to efficiently decode the orthogonally or frequency-converted data and perform a predetermined process on the decoded data.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining conventional JPEG encoding / decoding.

FIG. 2 is a diagram illustrating a configuration example of an embodiment of an image transmission system to which the present invention has been applied.

FIG. 3 is a block diagram illustrating a configuration example of a decoder 22 in FIG. 2;

FIG. 4 is a diagram showing how an 8 × 8 DCT coefficient is decoded into 16 × 16 pixels.

FIG. 5 is a flowchart illustrating a process of a decoder 22 in FIG. 3;

FIG. 6 is a block diagram illustrating a first configuration example of a coefficient conversion circuit 32 of FIG. 3;

FIG. 7 is a diagram illustrating an example of a prediction tap and a class tap.

8 is a block diagram illustrating a configuration example of a class classification circuit 43 in FIG. 6;

FIG. 9 is a diagram for explaining processing of a power calculation circuit 51 of FIG. 6;

FIG. 10 is a flowchart illustrating processing of a coefficient conversion circuit 32 in FIG. 6;

FIG. 11 is a flowchart illustrating details of a process in step S12 of FIG. 10;

FIG. 12 is a block diagram illustrating a configuration example of a first embodiment of a learning device to which the present invention has been applied.

FIG. 13 is a flowchart illustrating a process of the learning device in FIG. 12;

FIG. 14 is a block diagram showing a second configuration example of the coefficient conversion circuit 32 of FIG. 3;

FIG. 15 is a block diagram illustrating a configuration example of a second embodiment of a learning device to which the present invention has been applied.

FIG. 16 is a block diagram illustrating a third configuration example of the coefficient conversion circuit 32 in FIG. 3;

FIG. 17 is a block diagram illustrating a configuration example of a third embodiment of a learning device to which the present invention has been applied.

FIG. 18 is a block diagram illustrating a fourth configuration example of the coefficient conversion circuit 32 in FIG. 3;

FIG. 19 is a block diagram illustrating a configuration example of a fourth embodiment of a learning device to which the present invention has been applied.

20 is a block diagram illustrating a configuration example of an encoder 21 in FIG.

FIG. 21 is a block diagram illustrating a configuration example of an MPEG decoder.

FIG. 22 is a block diagram illustrating a fifth configuration example of the coefficient conversion circuit 32 of FIG. 3;

FIG. 23 is a block diagram illustrating a configuration example of a fifth embodiment of a learning device to which the present invention has been applied.

FIG. 24 is a diagram showing an image with improved temporal resolution.

FIG. 25 is a diagram showing an image with improved temporal resolution.

FIG. 26 is a diagram illustrating that class taps and prediction taps are configured from DCT coefficients of two or more frames.

FIG. 27 is a block diagram illustrating a configuration example of a computer according to an embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 21 encoder, 22 decoder, 23 recording medium, 24 transmission medium, 31 entropy decoding circuit, 32 coefficient conversion circuit, 33 block decomposition circuit, 41 prediction tap extraction circuit, 42 class tap extraction circuit, 43 class classification circuit, 44 coefficient table storage Unit, 45 product-sum operation circuit, 51 power operation circuit, 52 class code generation circuit, 53 threshold table storage unit, 60 thinning circuit, 61 blocking circuit, 62 DCT circuit, 63 quantization circuit, 64
Prediction tap extraction circuit, 65 class tap extraction circuit, 66 class classification circuit, 67 normal equation addition circuit, 68 tap coefficient determination circuit, 69 coefficient table storage unit, 71, 81 inverse quantization circuit, 114 arithmetic unit, 115 motion compensation circuit , 120 decimation circuit,
121 motion vector detection circuit, 122 arithmetic unit,
123 block circuit, 124 DCT circuit,
125 quantization circuit, 127 inverse quantization circuit, 12
8 inverse DCT circuit, 129 operation unit, 130 motion compensation circuit, 131 interpolation circuit, 132 operation unit, 2
01 bus, 202 CPU, 203 ROM, 204
RAM, 205 hard disk, 206 output unit,
207 input unit, 208 communication unit, 209 drive, 210 input / output interface, 211 removable recording medium

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Hideo Nakaya, Inventor 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Takeharu Nishikata 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Hideki Otsuka 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Takeshi Kunihiro 6-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni (Inc.) (72) Inventor Takafumi Morito 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Masashi Uchida 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation In-house F term (reference) 5C059 KK01 MA00 MA23 MC14 PP01 PP04 PP05 PP06 PP07 SS06 SS11 SS20 TA46 TB07 TC02 TC03 TC12 TD13 UA05 UA37 5C076 AA21 AA22 BA06 BB01 5C078 AA04 BA21 BA57 DA01 DA02

Claims (29)

[Claims] 1. A data processing device that processes at least transform data obtained by performing an orthogonal transform process or a frequency transform process, wherein the acquiring device acquires a tap coefficient obtained by performing learning. Calculating means for performing a predetermined prediction operation using the tap coefficient and the converted data, thereby decoding the converted data into original data, and obtaining processed data obtained by subjecting the original data to predetermined processing; and A data processing device comprising: 2. The data processing apparatus according to claim 1, wherein said calculation means performs a linear primary prediction calculation using said tap coefficients and conversion data. 3. The data processing apparatus according to claim 1, further comprising a storage unit that stores the tap coefficient, wherein the obtaining unit obtains the tap coefficient from the storage unit. 4. The conversion data is obtained by converting the original data into
2. The data processing device according to claim 1, wherein the data is obtained by performing orthogonal transformation or frequency transformation and further performing quantization. 5. The apparatus according to claim 4, further comprising an inverse quantization means for inversely quantizing the transformed data, wherein the arithmetic means performs a prediction operation using the inversely quantized transformed data. The data processing device according to claim 1. 6. The conversion data is obtained by converting the original data into
2. The data processing apparatus according to claim 1, wherein the data processing apparatus performs at least discrete cosine transform. 7. A prediction tap extracting means for extracting the conversion data used together with the tap coefficient for predicting target data of interest among the processing data and outputting the conversion data as a prediction tap, The data processing apparatus according to claim 1, wherein the means performs a prediction operation using the prediction tap and the tap coefficient. 8. A class tap extracting means for extracting the converted data used to classify the data of interest into one of several classes and outputting the converted data as a class tap, based on the class tap. Classifying means for performing class classification for obtaining the class of the data of interest, wherein the calculating means performs a prediction operation using the prediction tap and the tap coefficient corresponding to the class of the data of interest. The data processing device according to claim 7, wherein 9. The data processing apparatus according to claim 1, wherein the calculation unit obtains the processed data obtained by performing a process for improving the quality of the original data by performing the predetermined prediction calculation. Data processing equipment. 10. The tap coefficient is such that a prediction error of a predicted value of the processing data obtained by performing a predetermined prediction operation using the tap coefficient and the conversion data is statistically minimized. The data processing apparatus according to claim 1, wherein the data processing apparatus is obtained by performing learning. 11. The data processing apparatus according to claim 1, wherein the original data is moving image or still image data. 12. The image processing apparatus according to claim 11, wherein the calculation unit obtains the processed data obtained by performing the processing for improving the image quality of the image data by performing the predetermined prediction calculation. Data processing device. 13. The data processing apparatus according to claim 11, wherein said calculating means obtains the processed data in which the resolution of the image data in a temporal or spatial direction is improved. 14. A data processing method for processing at least transform data obtained by performing an orthogonal transform process or a frequency transform process, comprising: an acquiring step of acquiring a tap coefficient obtained by performing learning; An operation step of performing a predetermined prediction operation using the tap coefficients and the conversion data, thereby decoding the conversion data into original data, and obtaining processed data obtained by performing predetermined processing on the original data; A data processing method comprising: 15. A recording medium on which a program for causing a computer to perform at least data processing for processing conversion data obtained by performing an orthogonal transformation process or a frequency transformation process is provided. An obtaining step of obtaining the obtained tap coefficient, using the tap coefficient and the converted data, by performing a predetermined prediction operation, the converted data is decoded into the original data, and the original data A recording medium characterized by recording a program comprising: an operation step of obtaining processed data subjected to a predetermined process. 16. A data processing device that decodes at least transform data obtained by performing an orthogonal transform process or a frequency transform process, and learns tap coefficients used in a prediction operation for performing a predetermined process on the decoded result. By performing a process based on the predetermined process on teacher data to be a teacher, a quasi-teacher data generating means for obtaining quasi-teacher data, and at least orthogonal transform or frequency transforming the quasi-teacher data, Student data generating means for generating student data to be students, such that a prediction error of a prediction value of the teacher data obtained by performing a prediction operation using the tap coefficient and the student data is statistically minimized. A data processing device comprising: learning means for performing learning and obtaining the tap coefficient. 17. The learning means according to claim 1, wherein a prediction error of a predicted value of the teacher data obtained by performing a linear primary prediction operation using the tap coefficient and the student data is statistically minimized. 17. The data processing apparatus according to claim 16, wherein the data processing is performed. 18. The data according to claim 16, wherein said student data generating means generates said student data by subjecting said quasi-teacher data to orthogonal transformation or frequency transformation and further quantizing. Processing equipment. 19. The student data generation means quantizes the quasi-teacher data by orthogonal transformation or frequency transformation,
17. The data processing apparatus according to claim 16, wherein the student data is generated by performing inverse quantization. 20. The data processing apparatus according to claim 16, wherein said student data generating means generates said student data by subjecting said quasi-teacher data to at least discrete cosine transform. 21. The apparatus according to claim 21, further comprising: a prediction tap extracting unit configured to extract the student data used together with the tap coefficient to predict the noted teacher data of interest from the teacher data, and output the extracted student data as a prediction tap. The learning means performs learning so that a prediction error of a prediction value of the teacher data obtained by performing a prediction operation using the prediction tap and the tap coefficient is statistically minimized. 17. The data processing device according to item 16. 22. Class tap extracting means for extracting the student data used to classify the noted teacher data into one of several classes and outputting the student data as class taps, based on the class taps. Classifying means for performing class classification for obtaining a class of the teacher data of interest, wherein the learning means performs a prediction operation using the prediction tap and a tap coefficient corresponding to the class of the teacher data of interest. 22. The data processing apparatus according to claim 21, wherein learning is performed so that a prediction error of a prediction value of the teacher data obtained by the above is statistically minimized, and the tap coefficient is obtained for each class. 23. The student data generating unit generates the student data by subjecting the quasi-teacher data to at least orthogonal transformation processing or frequency conversion for each predetermined unit. A data processing device according to claim 1. 24. The data processing apparatus according to claim 16, wherein the quasi-teacher data generating means generates the quasi-teacher data by performing a process of degrading the quality of the teacher data. . 25. The data processing apparatus according to claim 16, wherein the teacher data is moving image or still image data. 26. The data processing apparatus according to claim 25, wherein the quasi-teacher data generating unit generates the quasi-teacher data by performing a process of deteriorating the image quality of the image data. . 27. The quasi-teacher data generating unit generates the quasi-teacher data in which the resolution of the image data in the temporal or spatial direction is degraded.
A data processing device according to claim 1. 28. A data processing method for decoding at least transform data obtained by performing an orthogonal transform process or a frequency transform process, and learning tap coefficients used in a prediction operation for performing a predetermined process on the decoded result. By performing a process based on the predetermined process on teacher data to be a teacher, a quasi-teacher data generation step of obtaining quasi-teacher data, and at least orthogonal transform or frequency transforming the quasi-teacher data, A student data generating step of generating student data to be a student, and a prediction error of a prediction value of the teacher data obtained by performing a prediction operation using the tap coefficient and the student data so as to be statistically minimized. A learning step of performing learning and obtaining the tap coefficient. 29. A data process for decoding at least transform data obtained by performing an orthogonal transform process or a frequency transform process and learning tap coefficients used for a prediction operation for performing a predetermined process on the decoded result. A recording medium in which a program to be executed by a computer is recorded, wherein a quasi-teacher data generating step of performing a process based on the predetermined process on teacher data serving as a teacher to obtain quasi-teacher data; At least, by orthogonal transformation or frequency transformation, a student data generating step of generating student data to be a student, and a prediction value of the teacher data obtained by performing a prediction operation using the tap coefficient and the student data. Learning so as to statistically minimize the prediction error of A recording medium characterized by recording a program comprising: