JPH05336505A - Image hierarchical coding method - Google Patents

Abstract

(57) [Summary] [Structure] In an encoding device for performing orthogonal transform encoding 2 on an image signal in a frame or field and performing quantization,
Hierarchical conversion coefficient layering 4 is performed immediately before or after quantization,
A layered coding apparatus that determines layered order 7 such that the image quality degradation when local decoding 5 is performed using only layered basic layer data is less than or equal to a preset threshold value. [Effect] As in the case of using inter-frame coding, it is possible to suppress an increase in the amount of coded information due to layering and deterioration of coding efficiency, and to discard all extended layer data,
Even if only the data of the basic layer is transmitted, the image quality above a certain level can be secured.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital network capable of transmitting a moving image, such as an image terminal capable of recognizing a plurality of transmission lines having different transmission qualities and selectively using them, for example, a broadband ISDN ATM packet network. The present invention relates to a digital moving image coding system for image terminals.

[0002]

2. Description of the Related Art A digital moving image is coded by dividing image data obtained by high-efficiency encoding into an important portion and a relatively unimportant portion as image decoding information, or by layering them.
In recent years, the movement to transmit is particularly wide band ISDN A
Research has begun for the purpose of TM network transmission. For example, the closest known example to the present invention is the improvement of the coding efficiency by the coefficient separation algorithm in the packet video coding of the Institute of Electronics and Communication Engineers (IE90-38) (NTT HI Research Institute).

As shown in FIG. 1, the above-mentioned prior art is based on the motion-compensated prediction + DCT coding system (H.261), which is a recommended image coding system for videophones and videoconferences, and the frequency after DCT conversion. This is a description of a method in which transform coefficients corresponding to components are separated and hierarchically transmitted. In this case, as described in the literature, the DCT
The coefficient is divided into low frequency component and high frequency component into two layers,
Only low-frequency components important for the image are used as reference data for interframe predictive coding. Therefore, the hierarchical order is determined so that the information amount of the next intra-frame block to be predicted is minimized. That is, since the amount of data used for inter-frame prediction is not perfect, the coding efficiency decreases, so it is difficult to minimize this coding efficiency, and layering is performed so that the amount of generated information is minimized. Become. That is, in this publicly known example, there is a problem in that a hierarchy order determination method is adopted that does not protect the deterioration of image quality by layering and that minimizes the deterioration of coding efficiency due to layering.

[0004]

In the present invention, the hierarchical order determination method, which has been a problem in the above-mentioned conventional example, is improved,
The method of layering is also expanded. That is, by layered transmission,
It is an object to provide a layering method that protects important data and maintains a certain level of image quality even if any data loss occurs on the transmission path. Also, the layering method should not be simply layered into low frequency components and high frequency components of orthogonal transform coefficients, but should be expandable in the way of separating more than three layers and transform coefficients. Is an issue.

[0005]

In order to solve the above-mentioned problems, the present invention firstly considers the hierarchization of transform coefficients on the premise of orthogonal transformation within a field (frame), and the number of hierarchies is currently CCITT. Since it is under consideration by the standardization committee of, the number of layers is not limited to two, but three or more layers have been taken into consideration. Secondly, as a layering method, not only a basic layer and an extended layer are generated in order from a low frequency component in a zigzag scan, but also a plurality of representative values of coefficients in a transform block are used as a basic layer and a coefficient absolute value is set. We propose a case where the error between the value and the representative value is generated as an extended hierarchy. Third, as the layering determination method, since only the basic layer is used for interframe coding as in the conventional case, the order that maximizes the coding efficiency is not selected, but the image quality is determined by a predetermined threshold value. It is decided to be the best within the range of.

[0006]

2 is a conceptual diagram of the hierarchical coding according to the present invention. First, it is premised on intra-field or intra-frame coding, not inter-frame coding. When interframe coding is used, it is premised that the basic layer important as image information is not used for interframe coding. Next, the transmitting side adaptively determines the layering order for each block and generates a plurality of layers. This layer is composed of a base layer and one or more extension layers, which are transmitted through different transmission lines having different transmission qualities. The layering is basically performed immediately after the orthogonal transform coding, but the quantization may be performed immediately before or immediately after the layering. When layering is performed immediately before quantization, the transform coefficient itself is the layering target. When it is performed immediately after the quantization, the quantized data of the transform coefficient becomes the layering target. It can be said that it is more accurate to determine the order for obtaining the desired image quality by using the quantized data when determining the hierarchical order by local decoding, but conversely, in the case immediately before the quantization, There is an advantage that different quantization can be performed in the base layer and the enhancement layer based on the layering result.

[0007]

FIG. 3 shows an embodiment of the layering method according to the present invention. Here, an example of a hierarchical method is shown. In the figure, squares represent vertical / horizontal distributions of orthogonal transform coefficients. In this case, an orthogonal transform such as a discrete cosine transform of 8 pixels × 8 pixels is taken as an example. Therefore, the lower left side indicates the low frequency component, and the lower right side indicates the high frequency component. For example, when four hierarchization is performed, for example, as shown in the figure, the low-frequency / medium-frequency / high-frequency conversion coefficients are separated, and the representative coefficient value or the average value of 3 coefficients of each hierarchy band is set as the basic hierarchy. To do. Next, the error 15 coefficient between the coefficient representative value of the low frequency component and the coefficient absolute value is the first extension layer, the error 34 coefficient between the representative coefficient value of the medium frequency component and the coefficient absolute value is the second extension layer, the representative coefficient of the high frequency component. Error between numerical value and absolute value of coefficient 15
A total of four hierarchies are performed using the coefficient as the third extended hierarchy. Of course, the number of layers is two if the coefficient representative value and the coefficient absolute value are two, and it is needless to say that the number of layers can be freely set only by the number of separated coefficients. Further, the separation position of each layer may be adaptively determined for each block as described later. For example, in a block having a coefficient distribution that is biased toward the low frequency component, the coefficient of the low frequency component can be made smaller and the low frequency component can be finely separated.

Another embodiment of the layering method according to the present invention is shown in FIG. Here, for example, the absolute value a _ij of the low frequency component 15 coefficient and the representative value (average value) b of the remaining high frequency component 49 coefficient are used as the basic hierarchy, and the absolute value b _ij and the representative value b of this 49 coefficient are used.
The error between and is transmitted as an extended layer. If it is divided into three layers, the absolute value of the low frequency component, the representative value b of the medium frequency component and the representative value c of the high frequency component are used as the basic layer, and the absolute value of the coefficient of the medium frequency component of the remaining components is used. b _ij
The error between the representative value b and the representative value b should be in the first hierarchical layer, and the error between the coefficient absolute value c _ij of the high frequency component and the typical value c should be in the second hierarchical layer.

An embodiment of the hierarchical order determination method according to the present invention is shown in FIG. Hierarchization is basically performed on the transmitting side. In the present invention, only the basic layer is not used as reference data for inter-frame coding, but is layered and transmitted only on transmission. Therefore, if the layer order is determined so that the image quality becomes equal to or higher than a predetermined threshold value. Good.

For example, in the case of FIG. 2 (a), once the layering is tried, if only the basic layer can be transmitted on the transmission line without data loss, all the extended layer data are lost.
It is assumed that it has been discarded or an error has occurred. Therefore, if the data of only the basic layer arrives at the receiving side, it is sufficient to determine how much the image quality deteriorates as compared with the case where all the data arrives, as a criterion for determining the layer order. As shown in the figure, the local decoder 5 is provided on the transmitting side to
The sum of squares of the difference between the intra-block decoded data when the decoding is performed using all the data in the enhancement layer and the enhancement layer data is discarded and only the base layer is used, and the decoding data in the block is decoded when the enhancement layer data is zero. Calculate the integrated value. Since this error corresponds to image quality deterioration, the obtained sum of square sums is compared with a threshold value set in advance by the comparator 6, and the lowest coefficient order below the threshold value is used as the basic hierarchy. In other words, the larger the order of the basic layer, the smaller the sum of square sums and the less the image quality deterioration. Therefore, the order may be reduced and the boundary order of layering may be set to the extent that does not exceed the threshold value. become.

Of course, in the case where layering is performed after quantization as shown in FIG. 2 (b), among the transform coefficient data after quantization, the case where local decoding is performed only in the basic layer as described above,
The sum of squared sums of the errors when local decoding is performed using all the data of the extension layer may be used as the criterion.

Another embodiment of the hierarchical order determination method according to the present invention is shown in FIG. In FIG. 5, when layering is performed after orthogonal transformation and immediately before quantization, local decoding is performed using all the data of the base layer and the enhancement layer, that is, input image data before orthogonal transformation is obtained. This is a matter of course if the orthogonal transformation is basically a linear transformation except for calculation error. Therefore, the internal difference calculation of FIG. 5 is not necessary, and only the local decoding calculation of only the enhancement layer is required. The threshold may be the same value as described above.

Another embodiment of the hierarchical order determination method according to the present invention is shown in FIG. As shown in FIGS. 5 and 6, in order to determine the hierarchical order, the order setting is once tried, the error threshold value is compared, and another order is set again, and the error threshold value is compared again. That is, iterative calculation. This repetition is performed for the maximum number of in-block coefficients, which may be very complicated and useless. There is also a problem of absolute delay due to repeated calculation.

In order to solve such a problem, FIG.
In (a), a local decoding operation is performed on the number candidates of the hierarchical order at the same time to determine which order is optimal at one time.

For example, in FIG. 7B, four hierarchical orders are used as candidates, and a, a
Local decoding is performed on the four coefficient patterns b, c, and d. In this case, two or three layers are presumed, although it depends on the layering method. In this method, since the hierarchical order is predetermined, not only the feedback for iterative calculation is unnecessary but also it is necessary to try local decoding 64 times in the case of 8 × 8 blocks. ,
It only requires 4 times, so the amount of calculation is small. Of course, each block is slightly different in the sense that the minimum image quality is maintained, and the transmission cost is not the minimum value, but the image quality does not fall below the specified level. FIG. 7 shows the case of layering before quantization shown in FIG. 6, but it is obvious that the method of FIG. 5 in which layering is performed after quantization may be taken. Similarly, when a plurality of layers are set as shown in FIGS. 3 and 4, some layering methods may be set in advance as candidates.

[0016]

According to the present invention, as shown in the conventional example,
By transmitting only low-frequency components, which are necessary for decoding the transform coefficient after orthogonal transform coding, as a basic layer through a transmission path with good transmission quality, it is possible to suppress the image quality deterioration of the received image. An improvement effect can be expected.

First, in the conventional interframe coding, since only the basic layer is used as the reference data for the interframe coding, the coding efficiency is reduced due to the incompleteness of the reference data. In the literature, about 1.4 times as much coded information was required as compared with the case without layering. On the other hand, the present invention is basically based on the intra-frame coding, and since the hierarchical result is not fed back anywhere, the code amount does not increase due to the hierarchical structure.

Secondly, in the conventional example, the hierarchization is only two hierarchies of the low frequency component and the high frequency component which are important for the image, but the present invention also considers the hierarchization of three or more hierarchies.
Therefore, for example, when an ATM (asynchronous transfer mode) network such as a broadband ISDN is considered as a target transmission line, only the types of hierarchical transmission lines can be used. In addition, the hierarchical method introduces not only the absolute value of the low frequency component but also the representative value (average value) of the transform coefficient component so that the basic layer and the extended layer can be freely configured by combining the absolute value and the representative value. ing.
As a result, even if the transmission quality of the enhancement layer is extremely poor, the high-frequency component of the transform coefficient cannot be decoded at all and the image quality does not have resolution, and a certain amount of high-frequency component can be decoded and the high-frequency component can be effectively decoded. Therefore, even in the basic layers of the same order, there is a high possibility that higher image quality can be obtained on the receiving side depending on the transmission state.

Thirdly, in the present invention, since interframe coding is not involved, layering may be performed immediately before quantization as in the conventional case or immediately before quantization. In the immediately preceding case, not only dequantization is not necessary, but also decoding the transform coefficient as it is reproduces the input signal to the transform coding, so local decoding can be omitted, and The simple hierarchical order determination method can be directly applied.

Fourth, in the conventional hierarchical order determination method, the amount of coded information differs depending on the ratio of the coefficient used for interframe coding and the coefficient used for intraframe coding.
The order that maximizes the coding efficiency is determined as the hierarchical order. Therefore, from the viewpoint of image quality, there is no guarantee that the best image quality will always be obtained for each block. On the other hand, in the hierarchical order determination method according to the present invention,
As shown in FIGS. 5, 6 and 7, the minimum image quality can be ensured even when all the enhancement layer coefficients are discarded and only the basic layer is transmitted.

[Brief description of drawings]

FIG. 1 is a block diagram of a conventional hierarchical coding system.

FIG. 2 is a block diagram of hierarchical encoding.

FIG. 3 is a first explanatory diagram of hierarchical conversion coefficient blocks.

FIG. 4 is a second explanatory diagram of hierarchical conversion coefficient blocks.

FIG. 5 is a first block diagram of a hierarchical order determination method.

FIG. 6 is a second block diagram of a hierarchical order determination method.

FIG. 7 is a third block diagram of a hierarchical order determination method.

[Explanation of symbols]

1 ... Interframe predictive coding, 2 ... Transform coding, 3 ... Quantization, 4 ... Hierarchicalization, 5 ... Local decoding, 6 ... Error sum-of-squares integration calculation and threshold comparison, 7 ... Hierarchical order determination.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yoshinori Miyamoto 1-280 Higashi Koikekubo, Kokubunji, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd.

Claims (6)

[Claims] 1. An apparatus for performing high-efficiency coding by orthogonally transforming an image signal in units of blocks in a field, wherein transform coding coefficients are adaptively hierarchized into a plurality of layers for each block on the transmission side and transmitted. Hierarchical coding method for featured images. 2. The plurality of coefficient absolute values of the intra-block transform coefficient, or the plurality of coefficient representative values including the absolute value in the basic layer, and the coefficient absolute values other than that and the absolute value in claim 1. A hierarchical coding method for an image in which an error between the representative value and the representative value is used as an extension layer and is adaptively determined for each block and transmitted. 3. The method according to claim 1, wherein representative values of coefficients up to the n-th coefficient and coefficients of the (n + 1) -th order or higher are used in the basic hierarchy and also in the (n + 1) th coefficient.
An image hierarchical coding method in which the difference from the representative value of the coefficient of the next or higher order is used as an enhancement layer and the order n is adaptively determined in block units for transmission. 4. A hierarchical coding method for an image according to claim 1, 2 or 3, wherein the adaptive layering is performed immediately before or after quantization after orthogonal transformation. 5. The S / N ratio and immediate decoding according to claim 1, 2, 3 or 4, when only the base layer is received and decoded, and when both the base layer and the enhancement layer are received and decoded. Hierarchical coding method for images that uses error values as a criterion for layering. 6. The method according to claim 1, 2, 3, 4 or 5.
An image layering coding method in which a layering decision is made by locally decoding at the transmitting side using only the base layer, the enhancement layer, and the enhancement layer.