JP4496494B2 - Image coding apparatus and image coding method - Google Patents

Description

  The present invention relates to an image encoding device and an image encoding method. In particular, the present invention relates to an image encoding apparatus and an image encoding method for encoding an image by thinning out the image, for example, so that a decoded image almost identical to the original image is obtained.

  Various image encoding methods have been proposed in the past, and one of them is, for example, a method of compressing and encoding an image by subsampling its pixels, for example. .

  However, when the thinned and compressed image is simply expanded by interpolation, the resolution of the resulting decoded image deteriorates.

  Here, as a cause of the deterioration of the resolution of the decoded image in this way, first, the thinned image does not contain the high frequency component included in the original image, and secondly, the image after the thinning is performed. It is conceivable that the pixel values of the pixels constituting the are not necessarily suitable for restoring the original image.

  The present invention has been made in view of such circumstances, and enables compression coding by thinning out an image so that a decoded image that is the same (substantially the same) as the original image is obtained. is there.

An image encoding device according to one aspect of the present invention is an image encoding device that encodes an image, and compresses the original data by reducing the number of pixels of the original image, and corrects the compressed data. An operation using a correction value is applied to the compressed data to correct the compressed data and output the corrected data, and the correction data is classified into a predetermined class according to the nature of the data A prediction unit that predicts the original image by obtaining a prediction value by linear combination of a classification coefficient, a prediction coefficient corresponding to the class, and the correction data, and outputs the prediction value; Prediction error calculation means for calculating a prediction error of the prediction value, determination means for determining whether the prediction error is equal to or less than a predetermined value, and determination means as to whether the prediction error is equal to or less than a predetermined value When it is determined, the correction data, and output means for outputting as a coded result of the original image, when the prediction error is determined the determining means that no less than a predetermined value, part of the compressed data The correction value is changed so that the prediction error becomes small, and the processing of the correction unit, the prediction unit, the calculation unit, and the determination unit is repeated.

An image encoding method according to an aspect of the present invention is an image encoding method for encoding an image, wherein compressed data is generated by reducing the number of pixels of an original image, and correction for correcting the compressed data is performed. An operation using a value is performed on the compressed data to correct the compressed data, and output corrected data. The corrected data is classified into a predetermined class according to its property and corresponds to the class. Predicting the original image by obtaining a prediction value by linear combination of a prediction coefficient and the correction data, outputting the prediction value, calculating a prediction error of the prediction value for the original image, and calculating the prediction When it is determined whether the error is equal to or less than a predetermined value and it is determined that the prediction error is not equal to or less than the predetermined value, the correction value for correcting the compressed data is changed so that the prediction error is small, and the prediction error is If it is determined that the value or less, the said correction data for a portion of said compressed data, and outputs the encoding result of the original image.

In one aspect of the present invention, compressed data is generated by reducing the number of pixels of the original image, and the compressed data is corrected by performing an operation using a correction value for correcting the compressed data on the compressed data. Correction data is output. In addition, the correction data is classified into a predetermined class according to its property, and the original image is predicted by obtaining the prediction value by linear combination of the prediction coefficient corresponding to the class and the correction data, and the prediction value is output. Then, the prediction error of the prediction value for the original image is calculated. Then, it is determined whether the prediction error is equal to or less than a predetermined value, and when it is determined that the prediction error is not equal to or less than the predetermined value, a correction value for correcting the compressed data for a part of the compressed data is a prediction error Is changed to be smaller. On the other hand, when it is determined that the prediction error is equal to or less than the predetermined value, the correction data is output as an encoding result of the original image.

  According to one aspect of the present invention, it is possible to obtain a decoded image that is substantially the same as the original image by using the appropriate correction data.

Embodiments of the present invention will be described below .

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

  FIG. 1 shows the configuration of an embodiment of an image processing apparatus to which the present invention is applied.

  The transmission apparatus 1 is supplied with digitized image data. The transmission device 1 compresses and encodes input image data (decreasing the number of pixels) and encodes the resulting encoded data, for example, an optical disc, a magneto-optical disc, a magnetic tape, a phase It is recorded on the recording medium 2 formed of a change disk or the like, or is transmitted through, for example, a terrestrial wave, a satellite line, a telephone line, a CATV network, the Internet, or another transmission path 3.

  In the receiving device 4, the encoded data recorded on the recording medium 2 is reproduced or the encoded data transmitted via the transmission path 3 is received, and the encoded data is expanded and decoded. Then, the decoded image obtained as a result is supplied to a display (not shown) and displayed.

  The image processing apparatus as described above is, for example, an optical disk apparatus, a magneto-optical disk apparatus, a magnetic tape apparatus, or other apparatus for recording / reproducing an image, or, for example, a videophone apparatus or a television set. The present invention is applied to a broadcasting system, a CATV system, and other devices that transmit images. As will be described later, since the amount of encoded data output from the transmission apparatus 1 is small, the image processing apparatus in FIG. 1 has a low transmission rate, such as a mobile phone or other portable terminal that is convenient for movement. It is also applicable to.

  FIG. 2 shows a configuration example of the transmission apparatus 1 of FIG.

  The I / F (InterFace) 11 performs reception processing of image data supplied from the outside and transmission processing of encoded data to the transmitter / recording device 16. A ROM (Read Only Memory) 12 stores a program for IPL (Initial Program Loading) and others. A RAM (Random Access Memory) 13 stores system programs (OS (Operating System)) and application programs recorded in the external storage device 15, and data necessary for the operation of a CPU (Central Processing Unit) 14. It is made to memorize. In accordance with the IPL program stored in the ROM 12, the CPU 14 expands the system program and the application program from the external storage device 15 to the RAM 13, and executes the application program under the control of the system program. The image data supplied from the image data is encoded as described later. The external storage device 15 is, for example, a magnetic disk device or the like, and stores system programs and application programs executed by the CPU 14 as well as data necessary for the operation of the CPU 14 as described above. The transmitter / recording device 16 records the encoded data supplied from the I / F 11 on the recording medium 2 or transmits it via the transmission path 3.

  The I / F 11, the ROM 12, the RAM 13, the CPU 14, and the external storage device 15 are connected to each other via a bus.

  In the transmission apparatus 1 configured as described above, when image data is supplied to the I / F 11, the image data is supplied to the CPU 14. The CPU 14 encodes the image data and supplies the encoded data obtained as a result to the I / F 11. When the encoded data is received, the I / F 11 supplies the encoded data to the transmitter / recording device 16. In the transmitter / recording device 16, the encoded data from the I / F 11 is recorded on the recording medium 2 or transmitted via the transmission path 3.

  FIG. 3 shows an example of the functional configuration of the transmitter 1 of FIG. 2 excluding the transmitter / recorder 16.

  The image data to be encoded is supplied to the compression unit 21, the local decoding unit 22, and the error calculation unit 23. The compression unit 21 compresses the image data by, for example, simply thinning out the pixels, and compresses the resulting data (image data after the thinning is performed) according to control from the determination unit 24. It is made to correct. Correction data obtained as a result of correction in the compression unit 21 is supplied to the local decoding unit 22 and the determination unit 24.

  The local decoding unit 22 predicts the original image based on the correction data from the compression unit 21 and supplies the predicted value to the error calculation unit 23. As will be described later, the local decoding unit 22 uses the correction data and the original image data, and calculates a prediction coefficient for each predetermined class for calculating a prediction value by linear combination with the correction data. A process for obtaining is performed, and an adaptive process for obtaining a predicted value is performed based on the prediction coefficient. As described above, the local decoding unit 22 supplies the prediction value to the error calculation unit 23 and also supplies the prediction coefficient for each class obtained at that time to the determination unit 24.

  The error calculation unit 23 is configured to calculate the prediction error of the prediction value from the local decoding unit 22 with respect to the original image data (original image) input thereto. This prediction error is supplied to the determination unit 24 as error information.

  Based on the error information from the error calculation unit 23, the determination unit 24 determines the appropriateness of using the correction data output from the compression unit 21 as the encoding result of the original image. If the determination unit 24 determines that the correction data output from the compression unit 21 is not appropriate as the encoding result of the original image, the determination unit 24 controls the compression unit 21 and further corrects the compressed data. Thus, new correction data obtained as a result is output. In addition, when the determination unit 24 determines that the correction data output from the compression unit 21 is appropriate as the encoding result of the original image, the determination unit 24 uses the correction data supplied from the compression unit 21 as the optimum result. The compressed data (hereinafter referred to as optimum compressed data as appropriate) is supplied to the multiplexing unit 25, and the prediction coefficient for each class supplied from the local decoding unit 22 is supplied to the multiplexing unit 25.

  The multiplexing unit 25 multiplexes the optimum compressed data (correction data) from the determination unit 24 and the prediction coefficient for each class, and the multiplexing result is used as encoded data in the transmitter / recording device 16 (FIG. 2). ).

  Next, the operation will be described with reference to the flowchart of FIG. When image data is supplied to the compression unit 21, the compression unit 21 compresses the image data by thinning out in step S <b> 1. At first, without performing correction, the local decoding unit 22 and the determination unit are compressed. 24. In the local decoding unit 22, in step S2, the correction data from the compression unit 21 (initially, the compressed data obtained by simply thinning out the image data as described above) is locally decoded.

  That is, in step S2, the prediction coefficient for each class for calculating the predicted value of the original image is calculated by linear combination with the correction data using the correction data from the compression unit 21 and the original image data. Processing to obtain is performed, and a prediction value is obtained based on the prediction coefficient. The prediction value obtained in the local decoding unit 22 is supplied to the error calculation unit 23, and the prediction coefficient for each class is supplied to the determination unit 24.

  Here, the image composed of the predicted values output from the local decoding unit 22 is the same as the decoded image obtained on the receiving device 4 (FIG. 1) side.

  When receiving the predicted value of the original image from the local decoding unit 22, the error calculating unit 23 calculates the prediction error of the predicted value from the local decoding unit 22 with respect to the original image data in step S3, and as error information , Supplied to the determination unit 24. When the determination unit 24 receives the error information from the error calculation unit 23, in step S4, based on the error information, the appropriateness of using the correction data output from the compression unit 21 as the encoding result of the original image. Determine.

  That is, in step S4, it is determined whether the error information is equal to or less than a predetermined threshold value ε. If it is determined in step S4 that the error information is not equal to or less than the predetermined threshold ε, it is recognized that the correction data output from the compression unit 21 is not appropriate to be the encoded data of the original image, and the process proceeds to step S5. The determination unit 24 controls the compression unit 21 and thereby corrects the compressed data. The compression unit 21 changes the correction amount (correction value Δ described later) under the control of the determination unit 24 to correct the compressed data, and outputs the correction data obtained as a result to the local decoding unit 22 and the determination unit 24. To do. And it returns to step S2 and the same process is repeated hereafter.

  On the other hand, if it is determined in step S4 that the error information is equal to or less than the predetermined threshold ε, it is recognized that it is appropriate to use the correction data output from the compression unit 21 as the encoding result of the original image. The determination unit 24 outputs the correction data when error information equal to or less than the predetermined threshold ε is obtained to the multiplexing unit 25 as optimal compression data together with the prediction coefficient for each class. In step S6, the multiplexing unit 25 multiplexes the optimum compressed data from the determination unit 24 and the prediction coefficient for each class, outputs the encoded data obtained as a result, and ends the process.

  As described above, the correction data obtained by correcting the compressed data when the error information is equal to or less than the predetermined threshold value ε is used as the encoding result of the original image. Based on the correction data, it is possible to obtain an image substantially identical to the original image (original image).

  Next, FIG. 5 shows a configuration example of the compression unit 21 of FIG.

  The image data to be encoded is input to the thinning circuit 31. The thinning circuit 31 thins the input image data to 1 / N. Accordingly, the thinning circuit 31 outputs compressed data obtained by compressing image data to 1 / N. This compressed data is supplied from the thinning circuit 31 to the correction circuit 32.

  The correction circuit 32 gives an address to the correction value ROM 33 in accordance with a control signal from the determination unit 24 (FIG. 3), thereby reading the correction value Δ. Then, the correction circuit 32 generates correction data by adding, for example, the correction value Δ from the correction value ROM 33 to the compressed data from the thinning circuit 31 and supplies the correction data to the local decoding unit 22 and the determination unit 24. It is made to do. The correction value ROM 33 stores various combinations of correction values Δ (for example, combinations of correction values for correcting compressed data for one frame) for correcting the compressed data output by the thinning circuit 31. The combination of correction values Δ corresponding to the addresses supplied from the correction circuit 32 is read out and supplied to the correction circuit 32.

  Next, the processing of the compression unit 21 in FIG. 5 will be described with reference to FIG.

  For example, when image data for one frame or the like is supplied to the thinning circuit 31, the thinning circuit 31 thins the image data to 1 / N in step S11, and the compressed data obtained as a result is corrected to the correction circuit 32. Output to.

  Here, as shown in FIG. 7, the thinning circuit 31 thins out image data to 1/9, for example. That is, the thinning-out circuit 31 extracts 9 × 3 (horizontal × vertical) 9 pixels as one unit, extracts only the pixel value of the center pixel of each unit (the portion indicated by ● in the figure), Is deleted (the part indicated by a circle in the figure). Note that the thinning circuit 31 performs the above-described processing, for example, in units of one frame (field). Accordingly, compressed data obtained by thinning out one frame of image data to 1/9 is supplied from the thinning circuit 31 to the correction circuit 32. However, the thinning process in the thinning circuit 31 can also be performed in units of blocks by dividing an image of one frame into several blocks.

  When receiving the compressed data from the thinning circuit 31, the correction circuit 32 determines whether or not a control signal has been received from the determination unit 24 (FIG. 3) in step S12. If it is determined in step S12 that the control signal has not been received, steps S13 and S14 are skipped and the process proceeds to step S15. The correction circuit 32 directly uses the compressed data from the thinning circuit 31 as the correction data. It outputs to the decoding part 22 and the determination part 24, and returns to step S12.

  That is, as described above, the determination unit 24 is configured to control the compression unit 21 (correction circuit 32) based on the error information, and immediately after the compressed data is output from the thinning circuit 31, Since error information is not obtained (because error information is not output from the error calculation unit 23), no control signal is output from the determination unit 24. For this reason, immediately after the compressed data is output from the thinning circuit 31, the correction circuit 32 does not correct the compressed data (corrected by adding 0), and directly uses the local decoding unit 22 and the determination as the corrected data. To the unit 24.

  On the other hand, if it is determined in step S12 that the control signal from the determination unit 24 has been received, the process proceeds to step S13, and the correction circuit 32 outputs the address according to the control signal to the correction value ROM 33. As a result, in step S 13, a combination (set) of correction values Δ for correcting the compressed data for one frame stored in the address is read from the correction value ROM 33 and supplied to the correction circuit 32. The When the correction circuit 32 receives the combination of the correction values Δ from the correction value ROM 33, in step S14, the correction circuit 32 adds the corresponding correction value Δ to each compressed data of one frame, thereby correcting the corrected data by correcting the compressed data. calculate. Thereafter, the process proceeds to step S15, and the correction data is output from the correction circuit 32 to the local decoding unit 22 and the determination unit 24, and the process returns to step S12.

  As described above, the compression unit 21 repeatedly outputs correction data obtained by correcting the compressed data into various values according to the control of the determination unit 24.

  Note that, for example, when the encoding of one frame image is completed, the determination unit 24 supplies a control signal indicating that to the compression unit 21. In step S12, the compression unit 21 It is also determined whether such a control signal has been received. If it is determined in step S12 that a control signal indicating that the encoding of one frame image has been completed is received, the compression unit 21 ends the processing for the frame (field) and the next frame is supplied. The process returns to step S11 and repeats the processes of steps S11 to S15.

  In the above-described case, the thinning circuit 31 is made to generate compressed data by extracting only pixel data (pixel value) for the center pixel of 3 × 3 pixels. It is also possible to calculate an average value of 3 × 3 pixels and generate compressed data using the average value as the pixel value of the center pixel of the 3 × 3 pixels.

  Next, FIG. 8 shows a configuration example of the local decoding unit 22 of FIG.

  The correction data from the compression unit 21 is supplied to the class classification blocking circuit 41 and the predicted value calculation blocking circuit 42. The class classification blocking circuit 41 is configured to block the correction data into a class classification block centered on the target correction data, which is a unit for classifying the correction data into a predetermined class according to its property. .

That is, in FIG. 7, assuming that the i-th correction data (compressed data) (or pixel) (or pixel) (the part indicated by ● in the figure) from the top and j-th from the left is represented as X _ij , The blocking circuit 41 has eight pixels X _{(i−1) (j−1)} , X _{(i− )} adjacent to the upper left, upper, upper right, left, right, lower left, lower, lower right of the attention correction data X _{ij. 1) j} , X _{(i-1) (j + 1)} , Xi _(j-1) , Xi _{(j + 1)} , X _{(i-1) (j-1)} , X _{(i-1) A} class classification block including a total of 9 pixels including _j and X _{(i + 1) (j + 1)} is configured. This class classification block is supplied to the class classification adaptive processing circuit 43.

  In this case, the class classification block is configured by a square block of 3 × 3 pixels. However, the shape of the class classification block does not have to be a square, for example, a rectangle Or a cross-shaped or any other shape. Further, the number of pixels constituting the class classification block is not limited to 3 × 3 9 pixels.

The prediction value calculation blocking circuit 42 blocks the correction data into prediction value calculation blocks centering on the target correction data, which is a unit for calculating the prediction value of the original image. . That is, in FIG. 7, the pixel value of 3 × 3 9 pixels in the original image (original image) centered on the correction data X _ij (the portion indicated by the mark ● in the figure) from the leftmost. Y _ij (1), Y _ij (2), Y _ij (3), Y _ij (4), Y _ij (5), Y _ij (6), Y _ij (7) ), Y _ij (8), Y _ij (9), the predicted value calculation blocking circuit 42, for example, calculates the predicted values of the pixels Y _ij (1) to Y _ij (9). , 25 pixels X of 5 × 5 centering on the target correction data _{X ij (i-2) (} j-2), X (i-2) (j-1), X (i-2) j, X ( _{i-2) (j + 1)} , X _{(i-2) (j + 2)} , X _{(i-1) (j-2)} , X _{(i-1) (j-1)} , X _{(i- 1) j} , X _{(i-1) (j + 1)} , X _{(i-1) (j + 2)} , Xi _(j-2) , Xi _(j-1) , _Xij , Xi _{( j + 1)} , Xi _{(j + 2)} , X _{(i + 1) (j-2)} , X _{(i + 1) (j-1)} , X _{(i + 1) j} , X _{(i + 1) ) (j + 1)} , X _{(i + 1) (j + 2)} , X _{(i + 2) (j-2)} , X _{(i + 2) (j-1)} , X _{(i + 2) j} , X _{(i + 2) (j + 1)} , X _{(i + 2) (j + 2)} The predicted value calculation block is configured.

Specifically, for example, in order to calculate the predicted values of nine pixels Y ₃₃ (1) to Y ₃₃ (9) in the original image surrounded by a rectangle in FIG. 7, the pixels X ₁₁ , X ₁₂ , _{X 13, X 14, X 15} , X 21, X 22, X 23, X 24, X 25, X 31, X 32, X 33, X 34, X 35, X 41, X 42, X 43, X 44 , X ₄₅ , X ₅₁ , X ₅₂ , X ₅₃ , X ₅₄ , and X ₅₅ constitute a predicted value calculation block (in this case, the attention correction data is X ₃₃ ).

  The predicted value calculation block obtained in the predicted value calculation blocking circuit 42 is supplied to the class classification adaptive processing circuit 43.

  Note that the number of pixels and the shape of the prediction value calculation block are not limited to those described above, as in the case of the class classification block. However, it is desirable that the number of pixels constituting the prediction value calculation block is larger than the number of pixels constituting the class classification block.

  Further, when the above-described blocking is performed (the same applies to processes other than blocking), there may be no corresponding pixel near the image frame of the image. In this case, for example, the image frame The processing is performed on the assumption that the same pixel as that constituting the pixel exists outside.

  The class classification adaptive processing circuit 43 includes an ADRC (Adaptive Dynamic Range Coding) processing circuit, a class classification circuit 45, and an adaptive processing circuit 46, and performs class classification adaptive processing.

  Class classification adaptation processing classifies input signals into several classes based on their characteristics, and applies appropriate adaptation processing to the input signals of each class. It is divided into processing.

  Here, the class classification process and the adaptation process will be briefly described.

  First, the class classification process will be described.

Now, for example, as shown in FIG. 9 (A), a pixel of interest and three adjacent pixels form a 2 × 2 pixel block (class classification block). It is expressed by 1 bit (takes a level of 0 or 1). In this case, the 2 × 2 4 pixel block including the target pixel can be classified into 16 (= (2 ¹ ) ⁴ ) patterns as shown in FIG. 9B, depending on the level distribution of each pixel. . Therefore, in this case, the target pixel can be classified into 16 patterns, and such pattern classification is a class classification process and is performed in the class classification circuit 45.

  The class classification processing can be performed in consideration of the activity (complexity of the image) (severity of change) of the image (image in the block).

Here, normally, for example, about 8 bits are assigned to each pixel. In the present embodiment, as described above, the class classification block is composed of 9 pixels of 3 × 3. Therefore, if the class classification process is performed on such a class classification block, it is classified into an enormous number of classes of (2 ⁸ ) ⁹ .

  Therefore, in the present embodiment, the ADRC processing circuit 44 is configured to perform ADRC processing on the class classification block, whereby the number of bits of the pixels constituting the class classification block is set. By reducing the size, the number of classes is reduced.

  That is, for example, in order to simplify the description, as shown in FIG. 10A, when a block composed of four pixels arranged on a straight line is considered, in ADRC processing, the maximum value of the pixel value is considered. MAX and minimum value MIN are detected. Then, DR = MAX−MIN is set as the local dynamic range of the block, and the pixel values of the pixels constituting the block are requantized to K bits based on the dynamic range DR.

That is, from each pixel value in the block, subtracts the minimum value MIN, dividing the subtracted value by DR / 2 ^K. Then, it is converted into a code (ADRC code) corresponding to the division value obtained as a result. Specifically, for example, when K = 2, as shown in FIG. 10 (B), the division value belongs to which range obtained by dividing the dynamic range DR into 4 (= 2 ² ) equally. And the division value belongs to the range of the lowest level, the range of the second level from the bottom, the range of the third level from the bottom, or the range of the highest level, respectively, It is coded into 2 bits such as 00B, 01B, 10B, or 11B (B represents a binary number). On the decoding side, the ADRC code 00B, 01B, 10B, or 11B is the center value L _{00 of} the lowest level range obtained by equally dividing the dynamic range DR into four, and the second level range from the bottom. center value L ₀₁ of the is converted into a center value L ₁₁ of the central value L ₁₀ or most of the upper level range, the range of the third level from the bottom, to the value, decoded by the minimum value MIN is added Is done.

  Here, such ADRC processing is called non-edge matching.

  The details of the ADRC processing are disclosed in, for example, Japanese Patent Application Laid-Open No. 3-53778 filed by the applicant of the present application.

  By applying ADRC processing that performs requantization with a smaller number of bits than the number of bits allocated to the pixels constituting the block, as described above, the number of classes can be reduced. This is performed in the ADRC processing circuit 44.

  In the present embodiment, the class classification circuit 45 performs the class classification process based on the ADRC code output from the ADRC processing circuit 44. However, the class classification process may be, for example, DPCM (predictive coding). ), BTC (Block Truncation Coding), VQ (Vector Quantization), DCT (Discrete Cosine Transform), Hadamard Transform, and the like.

  Next, the adaptation process will be described.

For example, now, the predicted value E [y] of the pixel value y of the original image is changed to pixel values (hereinafter referred to as learning data) x ₁ , x ₂ ,. Consider a linear primary combination model defined by a linear combination of predetermined prediction coefficients w ₁ , w ₂ ,. In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +...
... (1)

  Therefore, in order to generalize, a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y],

で定義すると、次のような観測方程式が成立する。

Then, the following observation equation holds.

XW = Y '
... (2)

  Then, it is considered to apply the least square method to this observation equation to obtain a predicted value E [y] close to the pixel value y of the original image. In this case, a matrix Y consisting of a set of pixel values y of the original image (hereinafter referred to as teacher data as appropriate) y and a set of residuals e of predicted values E [y] for the pixel values y of the original image. E

で定義すると、式（２）から、次のような残差方程式が成立する。

From the equation (2), the following residual equation is established.

XW = Y + E
... (3)

In this case, the prediction coefficient w _i for obtaining the predicted value E [y] close to the pixel value y of the original image is a square error.

を最小にすることで求めることができる。

Can be obtained by minimizing.

Accordingly, when the above-mentioned square error differentiated by the prediction coefficient w _i becomes 0, that is, the prediction coefficient w _i satisfying the following equation obtains the prediction value E [y] close to the pixel value y of the original image. Therefore, it is an optimum value.

・・・（４）

... (4)

Therefore, first, the following equation is established by differentiating the equation (3) by the prediction coefficient w _i .

・・・（５）

... (5)

  From equations (4) and (5), equation (6) is obtained.

・・・（６）

... (6)

  Further, considering the relationship among the learning data x, the prediction coefficient w, the teacher data y, and the residual e in the residual equation of Equation (3), the following normal equation can be obtained from Equation (6). .

・・・（７）

... (7)

  The normal equation of Expression (7) can be established by the same number as the number of prediction coefficients w to be obtained. Therefore, the optimal prediction coefficient w can be obtained by solving Expression (7). In solving equation (7), for example, a sweep-out method (Gauss-Jordan elimination method) or the like can be applied.

  As described above, the optimum prediction coefficient w is obtained for each class, and further, the prediction value E [y] close to the pixel value y of the original image is obtained by the equation (1) using the prediction coefficient w. Is adaptive processing, and this adaptive processing is performed in the adaptive processing circuit 46.

  Note that the adaptive processing is different from the interpolation processing in that a component included in the original image that is not included in the thinned image is reproduced. That is, the adaptive process is the same as the interpolation process using a so-called interpolation filter as long as only Expression (1) is seen, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter uses the teacher data y. In other words, since it is obtained by learning, the components included in the original image can be reproduced. From this, it can be said that the adaptive process is a process having an image creating action.

  Next, the processing of the local decoding unit 22 in FIG. 8 will be described with reference to the flowchart in FIG.

  In the local decoding unit 22, first, in step S21, the correction data from the compression unit 21 is blocked. In other words, in the class classification blocking circuit 41, the correction data is blocked into 3 × 3 pixel class classification blocks centered on the target correction data, and is supplied to the class classification adaptive processing circuit 43, and the predicted value In the calculation blocking circuit 42, the correction data is divided into 5 × 5 pixel prediction value calculation blocks centered on the target correction data and supplied to the class classification adaptive processing circuit 43.

  As described above, the class classification adaptive processing circuit 43 is supplied with original image data in addition to the class classification block and the prediction value calculation block. The class classification block is the ADRC processing unit 44. The predicted value calculation block and the original image data are supplied to the adaptive processing circuit 46.

  Upon receiving the class classification block, the ADRC processing circuit 44 performs, for example, 1-bit ADRC (ADRC that performs re-quantization by 1 bit) on the class classification block in step S22, thereby The correction data is converted (encoded) into 1 bit and output to the class classification circuit 45. In step S23, the class classification circuit 45 performs class classification processing from the class classification block that has been subjected to ADRC processing. That is, the state of the level distribution of each pixel constituting the class classification block subjected to ADRC processing is detected, and the class to which the class classification block belongs (attention correction data constituting the class classification block (located at the center) Determined correction data) class). The class determination result is supplied to the adaptive processing circuit 46 as class information.

In the present embodiment, since class classification processing is performed on a class classification block composed of 9 pixels of 3 × 3 subjected to 1-bit ADRC processing, each class classification block Is classified into ^{one of} 512 (= (2 ¹ ) ⁹ ) classes.

  In step S24, the adaptive processing circuit 46 performs adaptive processing for each class on the basis of the class information from the class classification circuit 45. Thus, the prediction coefficient for each class and the original image of one frame are processed. A predicted value of data (original image data) is calculated.

  That is, in the present embodiment, for example, 25 × 9 prediction coefficients for each class are calculated from the original image data for one frame and the correction data. Furthermore, when attention is paid to a certain correction data, prediction is made for a total of nine pixels, that is, a pixel of the original image corresponding to the correction data of interest and eight pixels of the original image adjacent to the pixel. An adaptive process is performed using 25 × 9 prediction coefficients whose values correspond to the class information of the target correction data and a prediction value calculation block having 5 × 5 pixels centered on the target correction data. Is calculated by

Specifically, for example, 3 × 3 correction data X ₂₂ , X ₂₃ , X ₂₄ , X ₃₂ , X ₃₃ , X ₃₄ centered on the correction data (target correction data) X ₃₃ shown in FIG. , X ₄₂ , X ₄₃ , and X ₄₄ , the class information C regarding the class classification block is output from the class classification circuit 45, and the correction data X is used as a prediction value calculation block corresponding to the class classification block. ₃₃ of 5 × 5 pixels centered corrected data _{X 11, X 12, X 13} , X 14, X 15, X 21, X 22, X 23, X 24, X 25, X 31, X 32, X 33 , X ₃₄ , X ₃₅ , X ₄₁ , X ₄₂ , X ₄₃ , X ₄₄ , X ₄₅ , X ₅₁ , X ₅₂ , X ₅₃ , X ₅₄ , X ₅₅ are predicted value calculation blocks. Assuming that the data is output from the conversion circuit 42, first, correction data constituting the prediction value calculation block is learned. With the over data, in the original image, the corrected pixel value of the 3 × 3 pixels centered on the data X ₃₃ (portion that is enclosed by a rectangle in FIG. 7) Y ₃₃ (1) to Y ₃₃ (9), As the teacher data, the normal equation shown in Equation (7) is established.

Further, for example, a normal equation is similarly calculated for other prediction value calculation blocks corresponding to class classification blocks classified into the same class information C in one frame as a predetermined period. Prediction coefficients w ₁ (k) to w for obtaining the predicted value E [Y ₃₃ (k)] of the pixel value Y ₃₃ (k) (here, k = 1, 2,..., 9). ₂₅ (k) (in this embodiment, 25 pieces of learning data are used to obtain one prediction value, and accordingly, 25 prediction coefficients w are also required correspondingly). When a normal equation of a number is obtained (therefore, until the normal equation of such a number is obtained, in step S24, the process of forming a normal equation is performed), the class information is solved by solving the normal equation. For C, the pixel value Y ₃₃ (k ) Prediction coefficients w ₁ (k) to w ₂₅ (k) that are optimal for obtaining the predicted value E [Y ₃₃ (k)] are calculated. This process is performed for each class, whereby 25 × 9 prediction coefficients are calculated for each class (in order to obtain nine predicted values using 25 correction data, one class The number of prediction coefficients for is 25 × 9).

Then, using the prediction coefficient for the class information C and the prediction value calculation block, the prediction value E [Y ₃₃ (k)] is obtained according to the following equation corresponding to Equation (1).

E [Y ₃₃ (k)] = w ₁ (k) X ₁₁ + w ₂ (k) X ₁₂ + w ₃ (k) X ₁₃
+ W ₄ (k) X ₁₄ + w ₅ (k) X ₁₅ + w ₆ (k) X ₂₁
+ W ₇ (k) X ₂₂ + w ₈ (k) X ₂₃ + w ₉ (k) X ₂₄
+ W ₁₀ (k) X ₂₅ + w ₁₁ (k) X ₃₁
+ W ₁₂ (k) X ₃₂ + w ₁₃ (k) X ₃₃
+ W ₁₄ (k) X ₃₄ + w ₁₅ (k) X ₃₅
+ W ₁₆ (k) X ₄₁ + w ₁₇ (k) X ₄₂
+ W ₁₈ (k) X ₄₃ + w ₁₉ (k) X ₄₄
+ W ₂₀ (k) X ₄₅ + w ₂₁ (k) X ₅₁
+ W ₂₂ (k) X ₅₂ + w ₂₃ (k) X ₅₃
+ W ₂₄ (k) X ₅₄ + w ₂₅ (k) X ₅₅
... (8)

  In step S24, a 25 × 9 prediction coefficient is obtained for each class as described above, and using the prediction coefficient for each class, the pixel of the 3 × 3 original image centered on the target correction data is used. A predicted value is obtained.

  Thereafter, the process proceeds to step S25, where the 25 × 9 prediction coefficient for each class is supplied to the determination unit 24, and the 3 × 3 prediction value is supplied to the error calculation unit 23. Then, the process returns to step S21, and the same processing is repeated for each frame as described above, for example.

  Next, FIG. 12 shows a configuration example of the error calculation unit 23 of FIG.

  The original image data is supplied to the blocking circuit 51, where the blocking circuit 51 outputs nine image data corresponding to the predicted values output from the local decoding unit 22. The block is divided into units, and the resulting 3 × 3 pixel block (for example, a 3 × 3 pixel block as shown in a rectangle in FIG. 7) is output to the square error calculation circuit 52. . As described above, the square error calculation unit 52 is supplied with blocks from the block forming circuit 51 and is also supplied with prediction values from the local decoding unit 22 in units of 9 (3 × 3 pixel block units). The square error calculation circuit 52 calculates a square error as a prediction error of the prediction value for the original image, and supplies the square error to the integrating unit 55.

  That is, the square error calculation circuit 52 is composed of computing units 53 and 54. The computing unit 53 subtracts the corresponding predicted value from each of the blocked image data from the blocking circuit 51 and supplies the subtracted value to the computing unit 54. The computing unit 54 squares the output of the computing unit 53 (difference between the original image data and the predicted value) and supplies the square to the integrating unit 55.

  When receiving the square error from the square error calculation circuit 52, the accumulating unit 55 reads the stored value in the memory 56, adds the stored value and the square error, and repeatedly supplies the stored value to the memory 56 for storage. Thus, an integrated value (error variance) of the square error is obtained. Furthermore, when the integration of the square error for a predetermined amount (for example, for one frame) is completed, the integration unit 55 reads the integration value from the memory 56 and supplies it to the determination unit 24 as error information. ing. The memory 56 is configured to store the output value of the integrating unit 55 while clearing the stored value every time processing for one frame is completed.

  Next, the operation will be described with reference to the flowchart of FIG. In the error calculation unit 23, first, in step S31, the stored value of the memory 56 is cleared to, for example, 0, and the process proceeds to step S32. In the blocking circuit 51, the image data is blocked as described above. The block obtained as a result is supplied to the square error calculation circuit 52. In step S33, the square error calculation circuit 52 calculates the square error between the image data of the original image (original image) and the prediction value supplied from the local decoding unit 22 constituting the block supplied from the blocking circuit 51. Is calculated.

  That is, in step S <b> 33, the computing unit 53 subtracts the corresponding predicted value from each of the blocked image data supplied from the blocking circuit 51, and supplies it to the computing unit 54. Further, in step S <b> 33, the computing unit 54 squares the output of the computing unit 53 and supplies it to the integrating unit 55.

  When the square error is received from the square error calculation circuit 52, the integrating unit 55 reads the stored value of the memory 56 and adds the stored value and the square error in step S34, thereby obtaining an integrated value of the square error. The integrated value of the square error calculated in the integrating unit 55 is supplied to the memory 56 and stored by being overwritten on the previous stored value.

  Then, in step S35, the integrating unit 55 determines whether or not the integration of the square error for one frame, for example, as a predetermined amount has ended. If it is determined in step S35 that the square error accumulation for one frame has not been completed, the process returns to step S32, and the processing from step S32 is repeated again. If it is determined in step S35 that the integration of the square error for one frame has been completed, the process proceeds to step S36, where the integration unit 55 stores the integrated value of the square error for one frame stored in the memory 56. Is output to the determination unit 24 as error information. Then, the process returns to step S31, waits for the supply of the original image and the predicted value for the next frame, and repeats the processing from step S31 again.

Therefore, in the error calculation unit 23, when the original image data is Y _ij (k) and the predicted value is E [Y _ij (k)], the calculation according to the following equation is performed. Error information Q is calculated.

Q = Σ (Y _ij (k) −E [Y _ij (k)]) ²
However, Σ means summation for one frame.

  Next, FIG. 14 shows a configuration example of the determination unit 24 of FIG.

  The prediction coefficient memory 61 is configured to store the prediction coefficient supplied from the local decoding unit 22. The correction data memory 62 stores correction data supplied from the compression unit 21.

  In the correction data memory 62, when the compressed data is newly corrected in the compression unit 21 and new correction data is supplied as a result, the correction data memory 62 stores the correction data already stored (previous correction data). Instead, new correction data is stored. In addition, at the timing when the correction data is updated to a new one in this way, the local decoding unit 22 outputs a set of prediction coefficients for each new class corresponding to the new correction data. Also in the prediction coefficient memory 61, when a prediction coefficient for each new class is supplied in this way, the prediction coefficient for each class already stored (prediction coefficient for each previous class) is replaced with the new prediction coefficient. A prediction coefficient for each class is stored.

  The error information memory 63 is configured to store error information supplied from the error calculation unit 23. The error information memory 63 stores the error information supplied last time from the error calculation unit 23 in addition to the error information supplied this time (even if new error information is supplied, Until new error information is supplied, the already stored error information is held). The error information memory 63 is cleared every time processing for a new frame is started.

  The comparison circuit 64 compares the current error information stored in the error information memory 63 with a predetermined threshold value ε, and further compares the current error information with the previous error information as necessary. Has been made. The comparison result in the comparison circuit 64 is supplied to the control circuit 65.

  Based on the comparison result in the comparison circuit 64, the control circuit 65 determines the appropriateness (optimum) of using the correction data stored in the correction data memory 62 as the encoding result of the original image. When recognized (determined), a control signal requesting output of new correction data is supplied to the compression unit 21 (correction circuit 32) (FIG. 5). When the control circuit 65 recognizes that the correction data stored in the correction data memory 62 is optimal to be the encoding result of the original image, the control circuit 65 stores the class stored in the prediction coefficient memory 61. Each prediction coefficient is read out and output to the multiplexing unit 25, and the correction data stored in the correction data memory 62 is read out and supplied to the multiplexing unit 25 as optimum compressed data. Further, in this case, the control circuit 65 outputs a control signal indicating that the encoding of one frame image has been completed to the compression unit 21, and as described above, the control circuit 65 causes the compression unit 21 to The process for the frame is started.

  Next, the operation of the determination unit 24 will be described with reference to FIG.

  In the determination unit 24, first, in step S41, whether or not the error information is received from the error calculation unit 23 is determined by the comparison circuit 64. If it is determined that the error information is not received, the process proceeds to step S41. Return. If it is determined in step S41 that the error information has been received, that is, if the error information is stored in the error information memory 63, the process proceeds to step S42, and the comparison circuit 64 stores the error information in the error information memory 63. The determined error information (current error information) is compared with a predetermined threshold ε to determine which is larger.

  If it is determined in step S42 that the current error information is greater than or equal to the predetermined threshold ε, the previous error information stored in the error information memory 63 is read in the comparison circuit 64. In step S43, the comparison circuit 64 compares the previous error information with the current error information and determines which is larger.

  Here, when the processing for one frame is started and error information is supplied for the first time, the error information memory 63 does not store the previous error information. The processing after step S43 is not performed, and a control signal for controlling the correction circuit 32 (FIG. 5) is output so that the control circuit 65 outputs a predetermined initial address to the correction value ROM 33. .

  If it is determined in step S43 that the current error information is equal to or less than the previous error information, that is, if the error information is reduced by correcting the compressed data, the process proceeds to step S44, where the control circuit 65 A control signal instructing to change the correction value Δ in the same manner as the previous time is output to the correction circuit 32, and the process returns to step S41. If it is determined in step S43 that the current error information is larger than the previous error information, that is, if the error information has increased by correcting the compressed data, the process proceeds to step S45, where the control circuit 65 Then, a control signal for instructing to change the correction value Δ opposite to the previous time is output to the correction circuit 32, and the process returns to step S41.

  When the error information that has continued to decrease starts to increase at a certain timing, the control circuit 65 sets the correction value Δ to the previous value, for example, at a magnitude that is 1/2. Conversely, a control signal instructing to change is output.

  Then, by repeating the processes of steps S41 to S45, the error information decreases, and when it is determined in step S42 that the current error information is smaller than the predetermined threshold value ε, the process proceeds to step S46, and control is performed. The circuit 65 reads out the prediction coefficient for each class stored in the prediction coefficient memory 61, reads out one frame of correction data stored in the correction data memory 62 as optimum compressed data, and supplies the data to the multiplexing unit 25. Then, the process ends.

  After that, waiting for the error information for the next frame to be supplied, the process according to the flowchart shown in FIG. 15 is repeated again.

  The correction circuit 32 can correct the compressed data for all the compressed data of one frame or only a part of the compressed data. In the case of performing correction only for a part of the compressed data, for example, the control circuit 65 can detect pixels having a strong influence on the error information and correct only the compressed data for such pixels. . A pixel having a strong influence on error information can be detected as follows, for example. That is, first, the error information is obtained by performing processing using the compressed data for the pixels remaining after thinning out as they are. Then, a control signal for performing a process of correcting the compressed data for the pixels remaining after the thinning out one by one by the same correction value Δ is output from the control circuit 65 to the correction circuit 32 and obtained as a result. The error information may be compared with the error information obtained when the compressed data is used as it is, and a pixel whose difference is a predetermined value or more may be detected as a pixel having a strong influence on the error information.

  As described above, the correction of the compressed data is repeated until the error information is made smaller (below) than the predetermined threshold ε, and the correction data when the error information becomes smaller than the predetermined threshold ε is the code of the image. Therefore, in the receiving device 4 (FIG. 1), the original pixel value of the pixels constituting the image after thinning is corrected from the correction data having the most appropriate value for restoring the original image. It is possible to obtain a decoded image that is the same (substantially the same) as the image.

  In addition to being compressed by thinning processing, the image is also compressed by ADRC processing, class classification adaptation processing, and the like, so that encoded data with a very high compression rate can be obtained. Note that the encoding process as described above in the transmission apparatus 1 achieves high-efficiency compression by using so-called organic integration of the compression process by thinning and the class classification adaptation process, From this, it can be said that it is an integrated encoding process.

  Next, FIG. 16 shows a configuration example of the receiving device 4 of FIG.

  In the receiver / reproducing apparatus 71, the encoded data recorded on the recording medium 2 is reproduced or the encoded data transmitted via the transmission path 3 is received and supplied to the separation unit 72. In the separation unit 72, correction data (optimum compressed data) and a prediction coefficient for each class are extracted from the encoded data. The correction data is supplied to the class classification blocking circuit 73 and the prediction value calculation blocking circuit 77, and the prediction coefficient for each class is supplied to the prediction circuit 76 and stored in the built-in memory 76A.

  The class classification blocking circuit 73, the ADRC processing circuit 74, the class classification circuit 75, or the predicted value calculation blocking circuit 77 are the class classification blocking circuit 41, the ADRC processing circuit 44, the class classification circuit 45, FIG. Alternatively, each block is configured in the same manner as each of the predicted value calculation block forming circuits 42. Therefore, in these blocks, the same processing as that in FIG. 8 is performed. The prediction value calculation block is output, and the class classification circuit 75 outputs class information. These prediction value calculation blocks and class information are supplied to the prediction circuit 76.

  The prediction circuit 76 reads the 25 × 9 prediction coefficient corresponding to the class information supplied from the class classification circuit 75 from the memory 76 A, and supplies the 25 × 9 prediction coefficient and the prediction value calculation blocking circuit 77. 3 × 3 pixel predicted value of the original image is calculated according to the equation (1) using the 5 × 5 pixel predicted value calculation block, and is composed of such predicted values. The image is output as a decoded image, for example, in units of one frame. As described above, this decoded image is almost the same as the original image.

  On the receiving side, even if it is not the receiving device 4 as shown in FIG. 16, the decoding is performed by performing normal interpolation without using the prediction coefficient by the device that decodes the thinned image by simple interpolation. An image can be obtained. However, the decoded image obtained in this case has deteriorated image quality (resolution).

  In the above case, the local decoding unit 22 in FIG. 3 obtains the prediction coefficient and uses this to calculate the prediction value. However, the local decoding unit 22 does not obtain the prediction coefficient ( It is possible to calculate a prediction value using a prediction coefficient obtained by learning in advance).

  That is, FIG. 17 illustrates a second functional configuration example of the transmission device 1 of FIG. In the figure, parts corresponding to those in FIG. 3 are denoted by the same reference numerals. That is, the transmission apparatus 1 is configured in the same manner as in FIG. 3 except that a local decoding unit 1022 is provided instead of the local decoding unit 22.

  However, in FIG. 3, the original image data is supplied to the local decoding unit 22, but in FIG. 17, the original image data is not supplied to the local decoding unit 1022.

  FIG. 18 shows a configuration example of the local decoding unit 1022 of FIG. In the figure, parts corresponding to those in FIG. 8 are denoted by the same reference numerals. That is, the local decoding unit 1022 is configured in the same manner as the local decoding unit 22 in FIG. 8 except that a prediction coefficient ROM 81 and a prediction circuit 82 are provided instead of the adaptive processing circuit 46.

  The prediction coefficient ROM 81 stores a prediction coefficient for each class obtained by performing learning (described later) in advance, receives the class information output from the class classification circuit 44, and sets the address corresponding to the class information. The stored prediction coefficient is read and supplied to the prediction circuit 82.

  The prediction circuit 82 uses the prediction value calculation block of 5 × 5 pixels from the prediction value calculation block forming circuit 42 and the prediction coefficient of 25 × 9 from the prediction coefficient ROM 81 to express Formula (1) (specifically For example, a linear linear expression shown in Expression (8)) is calculated, and thereby a predicted value of 3 × 3 pixels of the original image is calculated.

  Therefore, according to the class classification adaptive processing circuit 43 of FIG. 18, the predicted value is calculated without using the original image (original image). For this reason, as described above, the original image is not supplied to the local decoding unit 1022.

  Next, the processing of the local decoding unit 1022 in FIG. 18 will be further described with reference to the flowchart in FIG.

  In the local decoding unit 1022, first, in steps S 51 to S 53, the same processing as in steps S 21 to S 23 of FIG. 11 is performed, whereby class information is output from the class classification circuit 45. The This class information is supplied to the prediction coefficient ROM 81.

  When receiving the class information, the prediction coefficient ROM 81 reads out 25 × 9 prediction coefficients corresponding to the class information from the stored prediction coefficients for each class and supplies them to the prediction circuit 82 in step S54.

  The prediction circuit 82 is supplied with a 25 × 9 prediction coefficient from the prediction coefficient ROM 81, and is also supplied with a 5 × 5 pixel prediction value calculation block from the prediction value calculation blocking circuit 42. . In step S55, the prediction circuit 82 uses the 25 × 9 prediction coefficient from the prediction coefficient ROM 81 and the 5 × 5 pixel prediction value calculation block from the prediction value calculation blocking circuit 42 to perform an adaptive process. Is performed, that is, specifically, the calculation according to the equation (1) (or the equation (8)) is performed, so that the correction data of interest (here, the center of the prediction value calculation block) is obtained. A predicted value of a pixel of a 3 × 3 original image centered on (pixel) is obtained.

  Thereafter, for example, when a prediction value for one frame is obtained, the process proceeds to step S56, and 25 × 9 prediction coefficients for each class stored in the prediction coefficient ROM 81 are read and supplied to the determination unit 24. The predicted value obtained in step S55 is supplied to the error calculator 23. Then, the process returns to step S51, and the same processing is repeated for each frame, for example.

  In this embodiment, the prediction coefficient stored in the prediction coefficient ROM 81 is used as the prediction coefficient for each class. Therefore, since the value does not change, the prediction coefficient for each class is determined in step S25 by the determination unit 24. Basically, it is not necessary to supply again after supplying once.

  Further, even when the transmission apparatus 1 is configured as shown in FIG. 17, the reception apparatus 4 can obtain a decoded image that is substantially the same as the original image by being configured as shown in FIG. it can.

  Next, FIG. 20 shows a configuration example of an image processing apparatus that performs learning for obtaining a prediction coefficient stored in the prediction coefficient ROM 81 of FIG.

  The learning block circuit 91 and the teacher block circuit 92 are supplied with image data for learning (learning image) for obtaining prediction coefficients applicable to all images.

  The learning blocking circuit 91 extracts, for example, 25 pixels (5 × 5 pixels) in the positional relationship indicated by ● in FIG. 7 from the input image data, The learning block is supplied to the ADRC process 93 and the learning data memory 96.

  Further, the teacher block generation circuit 92 generates, for example, a block composed of 3 × 3 9 pixels from the input image data, and the block composed of 9 pixels is used as a teacher block as a teacher block. It is supplied to the data memory 98.

  For example, when the learning block forming circuit 91 generates a learning block composed of 25 pixels in the positional relationship indicated by ● in FIG. A teacher block of 3 × 3 pixels indicated by a circle is generated.

  The ADRC processing circuit 93 extracts, for example, 9 pixels (3 × 3 pixels) at the center from 25 pixels constituting the learning block, and the ADRC processing circuit 44 of FIG. As in the case of, 1-bit ADRC processing is performed. The 3 × 3 pixel block subjected to the ADRC process is supplied to the class classification circuit 94. In the class classification circuit 94, the blocks from the ADRC processing circuit 93 are subjected to class classification processing as in the case of the class classification circuit 45 of FIG. 18, and the class information obtained thereby is learned via the terminal a of the switch 95. The data is supplied to the data memory 96 and the teacher data memory 98.

  In the learning data memory 96 or the teacher data memory 98, the learning block from the learning blocking circuit 91 or the teacher block from the teacher blocking circuit 92 is respectively assigned to the address corresponding to the class information supplied thereto. Remembered.

  Accordingly, in the learning data memory 96, for example, if a block of 5 × 5 pixels indicated by ● in FIG. 7 is stored as a learning block at a certain address, the same address is used in the teacher data memory 98. A 3 × 3 pixel block surrounded by a rectangle in the figure is stored as a teacher block.

  Thereafter, the same processing is repeated for all learning images prepared in advance, whereby the learning block and the local decoding unit 1022 in FIG. 18 are the same as the 25 pixels constituting the learning block. A teacher block composed of 9 pixels, for which a predicted value is obtained using a predicted value calculation block composed of 25 correction data having a positional relationship, includes a learning data memory 96, a teacher data memory 98, Are stored at the same address.

  In the learning data memory 96 and the teacher data memory 98, a plurality of information can be stored at the same address, whereby a plurality of learning blocks and a teacher data are stored at the same address. Blocks can be stored.

  When the learning blocks and the teacher blocks for all the learning images are stored in the learning data memory 96 and the teacher data memory 98, the switch 95 that has selected the terminal a is switched to the terminal b. The output of the counter 97 is supplied to the learning data memory 96 and the teacher data memory 98 as addresses. The counter 97 counts a predetermined clock and outputs the count value. In the learning data memory 96 or the teacher data memory 98, a learning block or a teacher block stored at an address corresponding to the count value is stored. It is read out and supplied to the arithmetic circuit 99.

  Accordingly, the arithmetic circuit 99 is supplied with a set of learning blocks of a class corresponding to the count value of the counter 97 and a set of teacher blocks.

  When the arithmetic circuit 99 receives a set of learning blocks and a set of teacher blocks for a certain class, the arithmetic circuit 99 uses them to calculate a prediction coefficient that minimizes the error by the method of least squares.

That is, for example, the pixel values of the pixels constituting the learning block are now x ₁ , x ₂ , x ₃ ,... And the prediction coefficients to be obtained are w ₁ , w ₂ , w ₃ ,. Then, in order to obtain the pixel value y of a certain pixel constituting the teacher block by these linear linear combinations, the prediction coefficients w ₁ , w ₂ , w ₃ ,... There is.

y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +...

Therefore, the arithmetic circuit 99 calculates a square error of the predicted value w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +... With respect to the true value y from the learning block of the same class and the corresponding teacher block. The minimum prediction coefficients w ₁ , w ₂ , w ₃ ,... Are obtained by building and solving the normal equation shown in the above equation (7). Therefore, by performing this process for each class, 25 × 9 prediction coefficients are generated for each class.

  The prediction coefficient for each class obtained by the arithmetic circuit 99 is supplied to the memory 100. In addition to the prediction coefficient from the arithmetic circuit 99, the memory 100 is supplied with a count value from the counter 97. Thus, in the memory 100, the prediction coefficient from the arithmetic circuit 99 becomes the count value from the counter 97. Stored at the corresponding address.

  As described above, the memory 100 stores 25 × 9 prediction coefficients optimum for predicting 3 × 3 pixels of a block of the class at an address corresponding to each class.

  The prediction coefficient ROM 81 in FIG. 18 stores the prediction coefficient for each class stored in the memory 100 as described above.

  Next, FIG. 21 illustrates a third functional configuration example of the transmission device 1 of FIG. In the figure, parts corresponding to those in FIG. 3 are denoted by the same reference numerals. That is, the transmission apparatus 1 is provided with a local decoding unit 2022 or a determination unit 2024 instead of the local decoding unit 22 or the determination unit 24, respectively, except that the multiplexing unit 25 is not provided. The configuration is the same as in FIG.

  However, also in this case, the original image data is not supplied to the local decoding unit 2022 as in the case of FIG. Furthermore, in the embodiment of FIG. 21, the local decoding unit 2022 is configured not to output the prediction coefficient to the determination unit 2024.

  Next, the operation will be described with reference to the flowchart of FIG.

  When the image data is supplied to the compression unit 21, the compression unit 21 compresses the image data by thinning out the image data in step S61, and initially, without performing correction, the local decoding unit 2022 and the determination unit. Output to 2024. In step S62, the local decoding unit 2022 locally decodes the correction data from the compression unit 21 (initially, compressed data obtained by simply thinning out image data as described above).

  That is, in step S62, the correction data from the compression unit 21 is locally decoded, for example, in the same manner as in the local decoding unit 1022 shown in FIG. 18, and thereby the predicted value of the original image is calculated. This predicted value is supplied to the error calculation unit 23.

  In step S63, the error calculation unit 23 calculates a prediction error in the same manner as in step S3 of FIG. 4, and supplies the prediction error to the determination unit 2024 as error information. When receiving the error information from the error calculation unit 23, the determination unit 2024 determines whether or not the error information from the error calculation unit 23 is equal to or less than a predetermined threshold ε in step S64, as in step S4 of FIG. judge. If it is determined in step S64 that the error information is not equal to or less than the predetermined threshold value ε, the process proceeds to step S65, and the determination unit 2024 controls the compression unit 21, thereby correcting the compressed data. Then, the compression unit 21 corrects the compressed data and waits for new correction data to be output, and then returns to step S62.

  On the other hand, if it is determined in step S64 that the error information is equal to or smaller than the predetermined threshold ε, the determination unit 2024 uses the correction data when error information equal to or smaller than the predetermined threshold ε is obtained as the optimum compressed data. The data is output to the transmitter / recording device 16 (FIG. 2), and the process ends.

  Accordingly, the prediction coefficient for each class is not output here.

  Next, FIG. 23 shows a configuration example of the local decoding unit 2022 of FIG. In the figure, portions corresponding to those in FIG. 18 are denoted by the same reference numerals. That is, the local decoding unit 2022 is basically configured similarly to the local decoding unit 1022 in FIG.

  However, in FIG. 18, the prediction coefficient ROM 81 supplies the prediction coefficient for each class stored therein to the determination unit 24, but in FIG. 23, the prediction coefficient ROM 81 performs the prediction for each class. The coefficient is not supplied to the determination unit 24.

  Next, the operation will be described with reference to the flowchart of FIG.

  In the local decoding unit 2022, the same processing as in steps S51 to S55 in FIG. 18 is performed in steps S71 to S75. In step S76, only the prediction value obtained in step S75 is supplied to the error calculation unit 23, and the process returns to step S71. Thereafter, the same processing is repeated, for example, in units of one frame.

  Next, FIG. 25 illustrates a configuration example of the determination unit 2024 in FIG. In the figure, portions corresponding to those in FIG. 14 are denoted by the same reference numerals. That is, as described above, since the prediction coefficient is not supplied from the local decoding unit 2022, the determination unit 2024 is not provided with the prediction coefficient memory 61 for storing the prediction coefficient. Basically, the configuration is the same as the determination unit 24 of FIG.

  Next, the operation will be described with reference to the flowchart of FIG.

  In the determination unit 2024, the same processing as in steps S41 to S45 of FIG. 15 is performed in steps S81 to 85, and thereby the compression unit 21 is controlled so that the error information is reduced.

  Then, by repeating the processes of steps S81 to S85, the error information decreases, and when it is determined in step S82 that the current error information is smaller than the predetermined threshold value ε, the process proceeds to step S86, and control is performed. The circuit 65 reads out one frame of correction data stored in the correction data memory 62 as optimum compressed data, supplies it to the transmitter / recording device 16, and ends the processing.

  After that, waiting for the error information for the next frame to be supplied, the same processing is repeated again.

  Next, FIG. 27 illustrates a configuration example of the reception device 4 when the transmission device 1 is configured as illustrated in FIG. In the figure, portions corresponding to those in FIG. 16 are denoted by the same reference numerals. That is, the receiving apparatus 4 of FIG. 27 is different from that of FIG. 16 except that the prediction coefficient ROM 101 is newly provided between the class classification circuit 75 and the prediction circuit 76 and the prediction circuit 76 does not include the memory 76A. The configuration is basically the same as that in FIG.

  When the transmission apparatus 1 is configured as shown in FIG. 21, since it has been described above, the encoded data output from the receiver / reproduction apparatus 71 does not include the prediction coefficient for each class. In the separating unit 72, only the correction data (optimum compressed data) is extracted from the encoded data and supplied to the class classification blocking circuit 73 and the prediction value calculation blocking circuit 77.

  In the class classification blocking circuit 73, the ADRC processing circuit 74, the class classification circuit 75, or the predicted value calculation blocking circuit 77, the class classification blocking circuit 41, the ADRC processing circuit 44, the class classification circuit 45 in FIG. Alternatively, the same processing as that performed by each of the prediction value calculation block forming circuits 42 is performed, whereby a prediction value calculation block of 5 × 5 pixels is output from the prediction value calculation block forming circuit 77, and the class classification circuit From 75, class information is output. The prediction value calculation block is supplied to the prediction circuit 76, and the class information is supplied to the prediction coefficient ROM 101.

  The prediction coefficient ROM 101 stores the same prediction coefficient for each class stored in the prediction coefficient ROM 81 of FIG. 23. When class information is supplied from the class classification circuit 75, it corresponds to the class information. The 25 × 9 prediction coefficients are read out and supplied to the prediction circuit 76.

  The prediction circuit 76 uses the 25 × 9 prediction coefficient from the prediction coefficient ROM 101 and the correction data constituting the prediction value calculation block of 5 × 5 pixels supplied from the prediction value calculation blocking circuit 77, and uses the equation According to (1), a predicted value of 3 × 3 pixels of the original image is calculated, and an image of one frame composed of such predicted values is output as a decoded image.

  When the transmission device 1 is configured as shown in FIG. 21 and the reception device 4 is configured as shown in FIG. 27, it is not necessary to transmit and receive 25 × 9 prediction coefficients for each class. The transmission capacity or recording capacity can be reduced by that amount.

  The prediction coefficient ROMs 81 and 101 can store the average value of the pixel values constituting the teacher block for each class, instead of storing the prediction coefficient for each class. In this case, when class information is given, a pixel value corresponding to the class is output. In the local decoding units 1022 and 2022 in FIGS. 82 can be dispensed with. Similarly, in the receiving apparatus 4 of FIG. 27, the predicted value calculation blocking circuit 77 and the prediction circuit 76 need not be provided.

  The image processing apparatus to which the present invention is applied has been described above. Such an image processing apparatus, for example, encodes a standard system television signal such as the NTSC system, or so-called high vision with a large amount of data. This is particularly effective when encoding a television signal of a system.

  In this embodiment, the error sum of squares is used as the error information. However, as the error information, for example, the sum of absolute values of errors, the sum of the third power or more, and the like are used. It is possible to do so. Which one is used as error information can be determined based on, for example, its convergence.

  In the present embodiment, the correction of the compressed data is repeatedly performed until the error information is equal to or less than the predetermined threshold value ε. However, an upper limit may be provided for the number of times of correction of the compressed data. Is possible. That is, for example, when image transmission is performed in real time, processing for one frame needs to be completed within a predetermined period, but error information converges within such a predetermined period. Not always. Therefore, by setting an upper limit on the number of corrections, if the error information does not converge below the threshold ε within a predetermined period, the processing for that frame is terminated (the correction data at that time is used as the encoding result). ), It is possible to start processing for the next frame.

  Furthermore, in this embodiment, blocks such as a class classification block and a prediction value calculation block are configured from one frame image, but the blocks are continuous in time series, for example. It is also possible to configure it from pixels at the same position in a plurality of frames.

  In the present embodiment, the compression unit 21 simply thins out the image, that is, extracts the central pixel in a 3 × 3 pixel block and uses this as compressed data. In addition, for example, the average value of the nine pixels constituting the block is obtained, and the average value is set as the pixel value of the central pixel in the block, thereby reducing the number of pixels (decimation) and compressing this. It is also possible to use data.

  Furthermore, in the present embodiment, for example, a normal equation is created in units of one frame and a prediction coefficient for each class is obtained. However, prediction coefficient calculation processing is performed in other ways, for example, in units of one field or a plurality of units. It is also possible to create a normal equation for each frame. The same applies to other processes.

  Further, in the present embodiment, the CPU 14 configuring the transmission device 1 in FIG. 2 executes application programs stored in the external storage device 15 that also configures the transmission device 1 to perform various encoding processes. However, these encoding processes can also be performed by hardware. Similarly, the processing in the receiving device 4 can be realized by causing a computer to execute a program for performing such processing, or by hardware.

It is a block diagram which shows the structure of one Embodiment of the image processing apparatus to which this invention is applied. It is a block diagram which shows the structural example of the transmitter 1 of FIG. It is a block diagram which shows the 1st functional structural example of the transmitter 1 of FIG. 4 is a flowchart for explaining the operation of the transmission device 1 of FIG. 3. It is a block diagram which shows the structural example of the compression part 21 of FIG. It is a flowchart for demonstrating operation | movement of the compression part 21 of FIG. FIG. 6 is a diagram for explaining processing of the thinning circuit 31 in FIG. 5. It is a block diagram which shows the structural example of the local decoding part 22 of FIG. It is a figure for demonstrating a classification process. It is a figure for demonstrating an ADRC process. It is a flowchart for demonstrating operation | movement of the local decoding part 22 of FIG. It is a block diagram which shows the structural example of the error calculation part 23 of FIG. 13 is a flowchart for explaining an operation of an error calculation unit 23 in FIG. 12. It is a block diagram which shows the structural example of the determination part 24 of FIG. It is a flowchart for demonstrating operation | movement of the determination part 24 of FIG. It is a block diagram which shows the 1st structural example of the receiver 4 of FIG. It is a block diagram which shows the 2nd functional structural example of the transmitter 1 of FIG. FIG. 18 is a block diagram illustrating a configuration example of a local decoding unit 1022 in FIG. 17. 19 is a flowchart for explaining the operation of the local decoding unit 1022 of FIG. It is a block diagram which shows the structure of one Embodiment of the image processing apparatus which calculates the prediction coefficient memorize | stored in the prediction coefficient ROM81 of FIG. It is a block diagram which shows the 3rd functional structural example of the transmitter 1 of FIG. It is a flowchart for demonstrating operation | movement of the transmitter 1 of FIG. It is a block diagram which shows the structural example of the local decoding part 2022 of FIG. 24 is a flowchart for explaining an operation of a local decoding unit 1022 of FIG. It is a block diagram which shows the structural example of the determination part 2024 of FIG. It is a flowchart for demonstrating operation | movement of the determination part 2024 of FIG. It is a block diagram which shows the 2nd structural example of the receiver 4 of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Transmitter, 2 Recording medium, 3 Transmission path, 4 Receiver, 11 I / F, 12 ROM, 13 RAM, 14 CPU, 15 External storage device, 16 Transmitter / recorder, 21 Compression part, 22 Local decoding part , 23 Error calculation unit, 24 determination unit, 25 multiplexing unit, 31 decimation circuit, 32 correction circuit, 33 correction value ROM, 41 class classification blocking circuit, 42 prediction value calculation blocking circuit, 43 class classification adaptive processing Circuit, 44 ADRC processing circuit, 45 class classification circuit, 46 adaptive processing circuit, 51 blocking circuit, 52 square error calculation circuit, 53, 54 computing unit, 55 accumulator, 56 memory, 61 prediction coefficient memory, 62 correction data memory , 63 error information memory, 64 comparison circuit, 65 Control circuit, 71 receiver / reproduction device, 72 separation unit, 73 class classification blocking circuit, 74 ADRC processing circuit, 75 class classification circuit, 76 prediction circuit, 76A memory, 77 prediction value calculation blocking circuit, 81 prediction Coefficient ROM, 82 prediction circuit, 91 learning block circuit, 92 teacher block circuit, 93 ADRC processing circuit, 94 class classification circuit, 95 switch, 96 learning data memory, 97 counter, 98 teacher data memory, 99 arithmetic circuit , 100 memory, 101 prediction coefficient ROM, 1022, 2022 local decoding unit, 2024 determination unit

Claims (4)

An image encoding device for encoding an image, comprising:
Compression means for generating compressed data by reducing the number of pixels of the original image;
Correction means for correcting the compressed data by performing an operation using a correction value for correcting the compressed data, correcting the compressed data, and outputting the corrected data;
Classification means for classifying the correction data into a predetermined class according to the nature of the data;
Prediction means for predicting the original image by obtaining a prediction value by linear combination of the prediction coefficient corresponding to the class and the correction data, and outputting the prediction value;
Prediction error calculation means for calculating a prediction error of the prediction value for the original image;
Determining means for determining whether the prediction error is equal to or less than a predetermined value;
An output unit that outputs the correction data as an encoding result of the original image when the determination unit determines that the prediction error is equal to or less than a predetermined value;
When the determination unit determines that the prediction error is not less than a predetermined value, the correction value for a part of the compressed data is changed so that the prediction error is reduced, and the correction unit, the prediction unit, An image encoding apparatus characterized by repeating the processing of the calculating means and the determining means. When the determination unit determines that the prediction error is not less than a predetermined value, the correction value for a pixel having a relatively strong influence on the prediction error in the compressed data is changed so that the prediction error is small, Repeat the processing of the correction means, the prediction means, the calculation means, and the determination means
The image coding apparatus according to claim 1. An image encoding method for encoding an image, comprising:
Generate compressed data by reducing the number of pixels in the original image,
An operation using a correction value for correcting the compressed data is performed on the compressed data to correct the compressed data, and the corrected data is output.
The correction data is classified into a predetermined class according to its property,
Predicting the original image by obtaining a prediction value by linear combination of the prediction coefficient corresponding to the class and the correction data, and outputting the prediction value,
Calculating a prediction error of the predicted value for the original image;
Determining whether the prediction error is less than or equal to a predetermined value;
When it is determined that the prediction error is not less than or equal to a predetermined value, the correction value for correcting the compressed data for a part of the compressed data is changed so that the prediction error becomes small,
When it is determined that the prediction error is equal to or less than a predetermined value, the correction data is output as an encoding result of the original image. If it is determined that the prediction error is not less than or equal to a predetermined value, the correction value for pixels having a relatively strong influence on the prediction error in the compressed data is changed so that the prediction error is small, and the compressed data is corrected. And repeating the process of predicting the prediction value, calculating the prediction error, and determining the prediction error
The image encoding method according to claim 3.

Images