JPH0998434A - Image encoding device and image decoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate picture quality degradation and encoding efficiency decline due to drift in scalable encoding capable of varying resolution and picture quality in multiple hierarchies. SOLUTION: A motion compensation prediction and conversion encoding device using motion compensation prediction in a conversion coefficient region for the respective N×N pieces (N: a natural number) of conversion coefficients is provided with a means 220 for preparing the conversion coefficient pyramid of N hierarchies by selecting the n×n pieces n=1 to N} of locally decoded conversion coefficients from a low band, a means 212 for preparing the reproducing image pyramid of N hierarchies by executing inverse conversion for the respective hierarchies to the conversion coefficient pyramid of N hierarchies, a means 213 for storing the reproducing picture pyramid of N hierarchies for the respective hierarchies, a means 214 for referring to the pictures stored in the storage means 213 and preparing motion compensation prediction signals for the respective hierarchies, a means 215 for converting the motion compensation prediction signals to the conversion coefficients for the respective hierarchies and a means 230 for preparing a motion compensation predicted value by integrating the conversion coefficients.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding apparatus and a decoding apparatus for highly efficiently coding and transmitting / accumulating an image signal, and also for decoding the image signal, and more particularly to an image code having a scalability function. The present invention relates to an encryption device and a decoding device.

[0002]

2. Description of the Related Art Since an image signal has an enormous amount of information, it is generally compressed and encoded for transmission and storage. In order to encode an image signal with high efficiency, an image in a frame unit is divided into blocks in units of a required number of pixels, and each block is subjected to orthogonal transform to separate a spatial frequency of the image into each frequency component, and to perform a transform. It is obtained as a coefficient and encoded.

By the way, as a function of image encoding, image quality (SN
R: Signal to Noise Ratio), spatial resolution, and temporal resolution can be made variable step by step.

FIG. 7 shows that the spatial resolution is variable in N steps,
It is an image of a bitstream having a scalability function in which the image quality is made variable in M steps. Figure 7
By decoding the bit stream indicated by the shaded area in FIG. 1, the spatial resolution is n (= 1 to N) and the image quality is m.
A reproduced image of (= 1 to M) is obtained.

The scalability function is also incorporated in the video part (IS13818-2) of MPEG2, which is a media-integrated video coding standard standardized in ISO / IEC.

This scalability is shown in FIGS.
It is realized by the hierarchical coding method as shown in.
FIG. 15 shows the SNR scalability and FIG.
Shows an example of the encoder of spatial scalability and the configuration of its decoder, respectively.

In FIGS. 15 and 16, D is a delay means for giving a delay until a predicted value is obtained from the base layer, and DCT is means for performing a discrete cosine transform (orthogonal transform). , Q is a quantizer for quantization, IQ is an inverse quantizer for inverse quantization, IDCT is an inverse D
CT means, FM is a frame memory, MC is motion compensated prediction, VLC is variable length coding,
VLD is a means for performing variable length decoding, DS is means for downsampling, US is means for upsampling, and w is a weighting parameter (0, 0.5, 1).

FIG. 15A shows an example of the structure of an encoder for encoding, and FIG. 15B shows an example of the structure of a decoder. The encoder is divided into a base layer, which is a layer with low image quality, and an enhancer, which is a layer with high image quality.

The base layer is encoded by MPEG1 or MPEG2, and the enhancer is
Reproduce the data encoded in the base layer, and
This reproduced value is subtracted from the data, and only the resulting error is quantized and encoded with a quantization step size smaller than the quantization step size of the base layer. That is, it is quantized and encoded more finely. Then, by adding the information of the enhancer to the base layer information, it is possible to improve the definition and to transmit / store a high quality image.

In this way, the image is divided into the base layer and the enhancer, the data encoded in the base layer is reproduced, the reproduced data is subtracted from the original data, and only the resulting error is obtained. SNR scalability is a technique that enables high-definition image encoding / decoding by performing quantization and encoding with a quantization step size smaller than the base layer quantization step size.

In the encoder shown in FIG. 15A, the input image is input to the base layer and the enhancer respectively, and the base layer performs a process of obtaining an error amount between the input image and the motion compensation prediction value obtained from the image of the previous frame. After that, orthogonal transform (DCT) is performed, and the transform coefficient is quantized and variable-length decoded to be a base layer output. Also, the quantized output is
After inverse quantization, inverse DCT is performed, the motion compensation prediction value of the previous frame is added to this to obtain a frame image, and motion compensation prediction is performed from this frame image to obtain the motion compensation prediction value of the previous frame.

On the other hand, in the enhancer, the input image is
After giving a delay until the prediction value from the base layer is obtained, a process for obtaining an error from the motion compensation prediction value in the enhancement layer obtained from the image of the previous frame is performed, and then orthogonal transformation (DCT) is performed. ), The inverse quantization output of the base layer is added to the transform coefficient, and then this is quantized and variable length decoding is performed to output the enhanced layer. Also,
The quantized output is inversely quantized and then inverse DCT is performed by adding the motion compensation prediction value of the previous frame obtained in the base layer, and the motion compensation prediction value of the previous frame obtained in the enhancement layer is added to this to obtain a frame image. Then, motion compensation prediction is performed from this frame image to obtain the motion compensation prediction value of the previous frame in the enhancement layer.

By this means, it is possible to encode a moving image using SNR scalability.

The SNR scalability of FIG. 15 is shown in FIG.
In FIG. 5, the image is represented by two layers, but by further increasing the number of layers, reproduced images with various SNRs can be obtained.

In the decoder shown in FIG. 15 (b), the variable length decoded data of the enhancement layer and the base layer, which are given separately, are subjected to the variable length decoding separately and after dequantization, both are added, and After inverse DCT, the motion compensation prediction value of the previous frame is added to restore the image signal, and the motion compensation prediction is performed from the image one frame before obtained from the restored image signal to obtain the motion compensation prediction value of the previous frame. To do.

The above is an example of encoding and decoding that employs SNR scalability.

On the other hand, the spatial scalability is viewed from the spatial resolution, and is coded separately for a base layer having a low spatial resolution and an enhanced layer having a high spatial resolution. The base layer is encoded using a normal MPEG2 encoding method, and the enhance layer upsamples the image of the base layer (pixels such as an average value are added between pixels of the low resolution image to create a high resolution image). thing)
To create an image of the same size as the enhanced layer and perform adaptive prediction from the motion-compensated prediction from the image of the enhanced layer and the motion-compensated prediction from this up-sampled image to obtain an efficient code. The configuration example of the encoder is as shown in FIG. 16A, and the configuration example of the decoder is shown in FIG.
It can be realized as in (b) of 6.

The spatial scalability shown in FIG. 16 exists for realizing backward compatibility that, for example, a part of the bit stream of MPEG2 can be extracted and decoded by MPEG1, and images of various resolutions are reproduced. It is not a function that enables the operation (reference: “Special feature MPEG” TV magazine, Vol. 49, N.
o. 4, pp. 458-463, 1993).

That is, the moving picture coding technique in MPEG2 aims at high-efficiency coding and high-quality reproduction of a high-quality image, and is capable of reproducing an image faithful to the coded image.

However, with the spread of multimedia, in the system on the reproducing side, in addition to the demand for the reproducing apparatus capable of fully decoding the data of the high-efficiency-encoded high-quality image, the system on the reproducing side is also required. In addition, there are demands such as applications in which the screen can be reproduced regardless of the image quality, and demands for a simplified system in order to suppress the system price.

In order to meet such a demand, for example, when an image is divided into a matrix of 8 × 8 pixels and DCT is performed for each block, an 8 × 8 transform coefficient is obtained. Therefore, if the first low frequency term to the eighth low frequency term should be decoded, the first low frequency term to the fourth low frequency term should be decoded, For example, decoding from the low-frequency term to the sixth low-frequency term, such as decoding, can be simplified by restoring 4x4 or 6x6 information instead of 8x8. You can do it.

However, what is originally 8 × 8 is 4 × 4
For example, if the data is restored with 6 × 6 information, a mismatch between the motion compensation prediction values occurs, and errors are accumulated, so that the image is significantly deteriorated. A major issue is how to overcome such a mismatch between the encoding side and the decoding side.

Although not standardized, orthogonal transformation (eg DCT) is used as a method for transforming the spatial resolution in order to deal with the difference in spatial resolution between the encoding side and the decoding side.
(Discrete Cosine Transform) There is also a method of varying the spatial resolution by inversely transforming a part of the coefficients with an order smaller than the original order.

However, when performing motion-compensated prediction on a resolution-converted image, image quality deterioration called drift caused by motion-compensated prediction occurs in a reproduced image (reference document: Iwahashi et al., "Scalable Decoder Motion Compensation for Drift Reduction ", IEICE Technical Report IE94-97,
1994).

Therefore, there is a problem as a technique for overcoming the mismatch between the encoding side and the decoding side.

As a moving picture coding technique, J.
Y. A. Wang et. al. "Applying
Mid-level Vision Technology
esfor Video Data Compress
Ion and Manipulation ", M.I.
I. T. MediaLab. Tech. ReportN
o. 263, Feb. In 1994, an image coding method belonging to a category called mid-level coding has been proposed.

In this method, assuming that there is an image as shown in FIG. 17A, this is shown in FIGS. 17B and 17C, and the background and the subject (hereinafter, this is referred to as an object). Are encoded separately.

In this system, in order to separately encode the background ((c) of FIG. 17) and the object ((b) of FIG. 17), information for representing the shape of the object and the position on the screen is displayed. The alpha map signal (white pixel in (d) of FIG. 17 indicates the pixel of the object) is required.

The background alpha map signal (see FIG. 17).
(E)) is uniquely obtained from the alpha map signal of the object.

In such an encoding method, it is necessary to encode an image having an arbitrary shape, and resolution conversion must be possible in order to reproduce images having different resolutions.

As a technique for encoding an arbitrary shape image and a resolution converting method, the inventors of the present invention filed Japanese Patent Application No. 7-9707.
There is a method called an orthogonal transformation method of an arbitrarily shaped image signal already proposed in No. 3. In this technique, for an image including a background and a subject, for example, according to a map signal representing the position and shape of an object (subject; (content)) in an encoding device, the image signal is positioned inside the object in the image signal. The block (internal block) encodes the transform coefficient by two-dimensional orthogonal transforming only the signal of the pixel contained in the object, and the block containing the boundary of the object (edge block), respectively The map signal is encoded, and in the decoding device, the orthogonal transform coefficient necessary for reproducing the image of the desired resolution is selected from the decoded orthogonal transform coefficients based on the decoded and resolution-converted map signal, and all internal blocks are selected. , And the edge block inverses only the coefficients contained inside the object. And exchange conversion are those that obtain a reproduced image signal resolution conversion,
As a result, the resolution conversion can be performed on the edge block including the object of an arbitrary shape.

FIG. 18 shows an example of the orthogonal transformation method for the arbitrary shape image signal. When the arbitrary shape image is equally divided into square blocks, the conversion of the edge block including the boundary of the shape and the resolution conversion are performed. Is illustrated.

FIG. 18 is a diagram for explaining the conversion procedure for an edge block including a boundary portion of a shape. As shown in FIG. 18, [i] Among the input edge block signals,
[ii] First, the pixels included in the shaded content are gathered at the left end.

[Iii] Next, the pixels indicated by diagonal lines are subjected to one-dimensional DCT in the horizontal direction. [iv] Next, the conversion coefficients indicated by halftone lines are gathered at the upper end. [v] Finally, one-dimensional DCT is performed in the vertical direction on the transform coefficient indicated by the mesh line.

By following these steps, a two-dimensional conversion coefficient (black part in [v]) of an arbitrary shape can be obtained.

FIG. 19 shows a resolution conversion procedure. In FIG. 19, the original [i] alpha map signal is converted into an [ii] alpha map signal whose resolution has been converted to 5/8 in both horizontal and vertical directions, and [iii] as in the conversion procedure of FIG. 18 (a). Then, by rearranging in the horizontal direction and then rearranging in the [iv] vertical direction, the position of the conversion coefficient necessary to obtain a reproduced image with a resolution of 5/8 in both the horizontal and vertical directions is obtained. [v] Next, the coefficient of the required band is selected using this position information (black portion). A resolution-converted image is obtained by applying the inverse conversion to the conversion means of FIG. 18A to the conversion coefficient selected here according to the resolution-converted alpha map signal.

[0037]

When encoding / decoding a moving image, there is a demand for decoding at a resolution lower than the resolution on the encoding side, depending on the form of use. However, if the resolution on the encoding side and the resolution on the decoding side are different, there will be deterioration of the reproduced image due to mismatch, and this can be suppressed, and efficient encoding is possible on the encoding side. It is necessary to develop the technology to do so.

Further, there is a coding technique for coding the background and the object separately, but such a coding technique also requires scalable coding capable of varying the resolution and the image quality.

However, there is still no technology that can meet these demands.

Therefore, the object of the present invention is to
First, even if the resolution on the encoding side and the resolution on the decoding side are different, a mismatch does not occur, a high-quality image can be encoded / decoded, and the encoding efficiency is maintained. Another object of the present invention is to provide an image encoding / decoding device capable of performing the above.

A second object of the present invention is to make it possible to make the resolution and the image quality variable without causing a mismatch in the coding technique for separately coding the background and the object. Image coding /
It is to provide a decoding device.

[0042]

In order to achieve the first object of the present invention, firstly, N × N (N: natural number)
In the motion-compensated prediction + transform coding apparatus in which the motion-compensated prediction in the transform coefficient region is used for each transform coefficient of n, n × n (n = 1 to N) locally decoded transform coefficients are selected from the low band. Thus, means for creating a conversion coefficient pyramid of N layers, means for creating a reproduced image pyramid of N layers by performing inverse conversion of the conversion coefficient pyramid of N layers for each layer, and reproduced image pyramid of N layers For each layer, a unit for creating a motion compensation prediction signal for each layer by referring to the image stored in the storage unit, and a conversion coefficient for each motion compensation prediction signal for each layer. There is provided a moving picture coding apparatus having a means for converting to, and a means for creating a motion compensation prediction value by integrating the conversion coefficient.

In order to achieve the first object of the present invention, secondly, from the encoded bit stream encoded by the encoding device of the first configuration, the nth one is selected.
Means for extracting codes up to layers (n = 1 to N), means for creating transform coefficient pyramids for n layers from decoded n × n transform coefficients, and transform coefficient pyramids for n layers for each hierarchy By performing an inverse transformation on, the means for creating a reproduced image pyramid of n layers, the means for accumulating the reproduced image pyramid of n layers for each layer, and the image accumulated in the accumulating means are referred to, There are means for creating a motion compensation prediction signal for each layer, means for converting the motion compensation prediction signal into a transform coefficient for each layer, and means for creating a motion compensation prediction value by integrating the transform coefficients. Then, a moving picture decoding device is provided which reproduces a reproduced image of the nth layer.

Further, in order to achieve the first object, the present invention thirdly realizes MNR (M: natural number) SNR scalability using the encoding device of the first configuration. An encoding device, which is the m-th layer (m = 2 to N)
Means for obtaining a difference signal between the prediction error signal of the M-th layer and the local reproduction value of the prediction error signal of the (M-1) -th layer, and the difference signal is smaller than the quantization step size of the (M-1) -th layer in the m-th layer. A unit for quantizing with the step size, the dequantized difference signal, and the local reproduction value of the prediction error signal of the m-1 th layer are added to obtain the local reproduction value of the prediction error signal of the m th layer. Provided is a moving picture coding device characterized by obtaining.

Further, in order to achieve the first object of the present invention, fourthly, an m-th layer (from the coded bit stream coded by the coding device of the third structure) (m = 1 to M), means for extracting the codes of the respective layers up to the m-th layer, means for dequantizing the quantized value decoded by the means in each layer, The means for adding the inverse quantized values up to m layers is the second
A moving picture decoding device added to the above configuration.

Further, in order to achieve the above-mentioned second object, the present invention, fifthly, motion-compensated prediction + transform code in which motion-compensated prediction in a transform coefficient region is used for each of N × N transform coefficients. In the digitizing device, there is an alpha map signal for identifying the background and the object of the input image, means for encoding the alpha map, means for converting the arbitrarily shaped image into conversion coefficients according to the alpha map, and the means according to the alpha map Provided is an image encoding device having a unit for reproducing an image of an arbitrary shape by inversely transforming a transform coefficient.

In order to achieve the second object of the present invention, sixthly, in the moving picture coding apparatus of the fifth construction, sixthly, the resolution of the alpha map signal is converted to the alpha map of the N layer. A conversion coefficient pyramid of N layers is created by selecting a means for creating a signal pyramid and a conversion coefficient locally decoded according to an alpha map signal for n layers (n = 1 to N) for each hierarchy. By creating means and inverse transforming the transform coefficient pyramid of N layers according to the alpha map signal for each layer,
A means for creating a reproduced image pyramid of N layers, a means for accumulating the reproduced image pyramid of N layers for each layer, and an image stored in the accumulating means are referred to an alpha map signal for each layer. Therefore, by combining means for creating a motion-compensated prediction signal, means for converting the motion-compensated prediction signal into transform coefficients according to an alpha map signal for each layer, and integration of the transform coefficients according to an alpha map signal pyramid, Provided is a moving picture coding device having means for creating a compensation prediction value.

In order to achieve the second object of the present invention, seventhly, moving picture decoding for decoding the coded bitstream coded by the coding apparatus of the fifth configuration is provided. An apparatus for reproducing an arbitrary shape image by decoding an alpha map, converting an arbitrary shape image into a conversion coefficient according to the alpha map, and inversely converting the conversion coefficient according to the alpha map. There is provided an image decoding device characterized by having means.

In order to achieve the second object of the present invention, eighthly, from the coded bit stream coded by the coding device of the sixth configuration, the nth coded stream is selected.
Means for extracting codes up to the hierarchy (n = 1 to N), means for decoding the alpha map signal, means for converting the resolution of the decoded alpha map signal to create an alpha map signal pyramid for the N hierarchy, and decoding A means for creating a conversion coefficient pyramid of n layers from the converted conversion coefficients according to the alpha map signal pyramid, and an inverse conversion of the conversion coefficient pyramid of the n layers according to the alpha map signal for each layer, A means for creating a reproduced image pyramid, a means for accumulating reproduced image pyramids of n layers for each layer, and a motion compensation according to an alpha map signal for each layer by referring to the images accumulated in the accumulating means. Means for creating a prediction signal, and a conversion coefficient for the motion-compensated prediction signal for each layer according to an alpha map signal A moving picture decoding apparatus having a converting means and a means for creating a motion-compensated prediction value by integrating the conversion coefficients according to the alpha map signal pyramid, and reproducing a reproduced image of the nth layer. I will provide a.

In order to achieve the second object, the present invention, ninthly, realizes SNR scalability of M layers (M: natural number) using the encoding device of the fifth configuration. An encoding device, which is the m-th layer (m = 2 to N)
Means for obtaining a difference signal between the prediction error signal of the M-th layer and the local reproduction value of the prediction error signal of the (M-1) -th layer, and the difference signal is smaller than the quantization step size of the (M-1) -th layer in the m-th layer. A unit for quantizing with the step size, the dequantized difference signal, and the local reproduction value of the prediction error signal of the m-1 th layer are added to obtain the local reproduction value of the prediction error signal of the m th layer. Provided is a moving picture coding device characterized by obtaining.

In order to achieve the second object of the present invention, tenthly, from the coded bit stream coded by the coding device of the ninth configuration, the m-th layer ( (m = 1 to M), means for extracting the codes of the respective layers up to the m-th layer, means for dequantizing the quantized value decoded by the means in each layer, There is provided a moving picture decoding device having a configuration in which means for adding dequantized values up to m layers is added to the seventh configuration.

In order to achieve the second object, the present invention, eleventhly, realizes MNR (M: natural number) SNR scalability using the encoding device of the sixth configuration. An encoding device, wherein the m-th layer (m = 2 to
N) means for obtaining a difference signal between the prediction error signal and the local reproduction value of the prediction error signal of the (m-1) th layer, and in the mth layer, the difference signal is based on the quantization step size of the (m-1) th layer. And a means to quantize with a small step size,
A moving image code characterized by obtaining a local reproduction value of the prediction error signal of the m-th layer by adding the dequantized difference signal and the local reproduction value of the prediction error signal of the (m-1) th layer. To provide a rectification device.

In the twelfth aspect of the present invention, in order to achieve the second object, the twelfth layer from the encoded bit stream encoded by the encoding apparatus having the eleventh configuration is (m = 1 to M), means for extracting the codes of the respective layers up to the m-th layer, means for dequantizing the quantized value decoded by the means in each layer, There is provided a moving picture decoding device characterized in that means for adding dequantized values up to m layers is added to the eighth configuration.

In order to achieve the first object, the present invention, thirteenthly, is motion compensation prediction + conversion code in which motion compensation prediction in a conversion coefficient region is used for every N × N conversion coefficients. In the coding device, the coding device realizes SNR scalability of the M layer, which is the m-th layer (m = 2 to m = 2).
M) means for obtaining the prediction value of the m-th layer by switching the motion compensation prediction value and the local reproduction value of the (m-1) -th layer for each transform coefficient, and the quantum of the prediction error signal in the (m-1) -th layer. The conversion coefficient having an absolute value of the converted value less than or equal to the threshold value has a selector for outputting the motion-compensated prediction value of the m-th layer, and the conversion coefficient having the absolute value of the absolute value greater than or equal to the threshold value has a selector for outputting the local reproduction value of the m-1th layer. (EN) Provided is a moving picture coding device.

Further, in order to achieve the first object of the present invention, fourteenth, from the coded bit stream coded by the coding apparatus of the thirteenth structure, the m-th layer (m = 2 to M), means for decoding the codes of each layer up to the m-th layer, and means for dequantizing the quantized value of the prediction error signal decoded by the means in each layer. , M-th layer motion compensation prediction value and m-th layer
Means for obtaining the prediction value of the m-th layer by switching the reproduction value of one layer for each transform coefficient, and the absolute value of the quantized value of the prediction error signal in the (m-1) -th layer becomes less than or equal to the threshold value. There is provided a moving picture decoding device characterized in that it has a selector for outputting a motion compensation prediction value of the m-th layer as a transform coefficient and a reproduction value of the m-1 th layer as a transform coefficient that is equal to or more than a threshold value.

According to the present invention having such a configuration, N × N
When motion compensation is performed in each transform coefficient region, a motion compensation prediction value is obtained for each resolution of N layers to obtain reproduced images with different resolutions without deterioration of image quality due to drift. You can

Further, in the present invention, the encoding device and the SN are
By combining R scalability, scalable coding is realized in which resolution and image quality are divided into multiple layers.

Further, according to the present invention, in the above-mentioned encoding device, a reproduced image in which the resolution and the image quality of the arbitrary shape image are variable is obtained by performing the arbitrary shape orthogonal transformation according to the alpha map signal.

[0059]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A specific example of the present invention will be described below with reference to the drawings. The present invention is directed to image coding / reception in a transmitter / receiver (A and B in FIG. 1) in the image transmission system in FIG.
The present invention relates to a decoding device.

(First Specific Example) A first specific example of the present invention will be described with reference to FIGS. 2, 3 and 4. In the first specific example, mismatches due to the difference in resolution between the encoding side and the decoding side are prevented, and the same predicted value as that of the encoder can be obtained at any resolution, so that a high-quality image without drift can be restored. A system for doing so will be described.

<< Encoding Device of First Specific Example >> FIG. 2A
Is a block diagram on the encoding side of an image encoding / decoding device to which the present invention is applied, and FIG. 2B is a block diagram showing a specific configuration example of a local decoding circuit used in the configuration of FIG. 2A. Is.

First, the image coding apparatus will be described.
FIG. 2A is a block diagram of a motion compensation prediction + orthogonal transform coding device (post-transformation differential configuration) using motion compensation prediction in an orthogonal transform coefficient domain to which the present invention is applied.

In FIG. 2A, reference numeral 100 is an orthogonal transformation circuit, 110 is a difference circuit, 120 is a quantization circuit, 130 is a variable length coding circuit, 140 is an inverse quantization circuit, and 200 is a local decoding circuit.

Of these, the orthogonal transform circuit 100 is for performing an orthogonal transform process on the image signal, and divides the image signal supplied through the line 10 into N × N pixel blocks.
In this block unit, for example, DCT (discrete cosine transform) is used for orthogonal transform to obtain N × N transform coefficients.

The difference circuit 110 is the orthogonal transformation circuit 1
Orthogonal transform coefficient supplied from 00 and the local decoding circuit 20.
The prediction error is calculated from the prediction values of the N × N transform coefficients supplied through the twisted line 0. The quantizing circuit 120 quantizes the prediction error obtained by the difference circuit 110, and the variable length coding circuit 130 variable length codes the prediction error signal quantized by the quantizing circuit 120. That is, the quantized value of the prediction error signal is variable length coded and is output as a coded image signal via the line 30.

The inverse quantizer circuit 140 includes the quantizer circuit 120.
Is a circuit for receiving a quantized prediction error signal from the device and dequantizing the prediction error signal to obtain a reproduction value of the prediction error signal.
It is configured to supply 0.

The local decoding circuit 200 includes an inverse quantization circuit 14
The reproduced value of the prediction error signal obtained from 0 and the motion compensated predicted value obtained from the previous image are added to obtain the reproduced value of the transform coefficient, which is inversely transformed to obtain the local decoded signal, and this obtained value is obtained. A motion-compensated prediction value is generated from a locally decoded image signal, and this motion-compensated prediction value is orthogonally transformed for every N × N pixels to obtain N × N transform coefficient predicted values.

The local decoding circuit 200 includes an adding circuit 201,
It is composed of an inverse orthogonal transform circuit 202, a frame memory 203, a motion compensation prediction circuit 204, and an orthogonal transform circuit 205. Then, in the local decoding circuit 200, the reproduction value of the prediction error signal obtained from the inverse quantization circuit 140 and the line 2
The predicted value supplied via 0 is added in the adder circuit 201 to obtain the reproduction value of the transform coefficient, and the inverse orthogonal transform circuit 202 inversely transforms the transform coefficient obtained in the adder circuit 201 to obtain N. The locally decoded signal for each × N pixel is obtained, and the frame memory 203 holds the locally decoded image by accumulating the locally decoded signal for each N × N pixel supplied from the inverse orthogonal transform circuit 202. Further, the motion compensation prediction circuit 204 is for generating a motion compensation prediction value using the image signal of the locally decoded image held in the frame memory 203, and the orthogonal transformation circuit 205 is for the motion compensation prediction circuit 204. This is a configuration in which the motion-compensated prediction value generated by is orthogonally transformed for each N × N pixel, and the transformation coefficient is output via the line 20.

In the image coding apparatus having such a configuration, when an image signal is supplied via the line 10, the image signal is orthogonally transformed for each line N × N pixels by the orthogonal transformation circuit 100. As a result, N × N transform coefficients are obtained.
The obtained conversion coefficient is input to the difference circuit 110.

In the difference circuit 110, the orthogonal transformation circuit 100
A prediction error between the orthogonal transform coefficient supplied from the local decoding circuit 200 and the predicted value of the N × N transform coefficients supplied from the local decoding circuit 200 via the line 20 is calculated. Then, the calculation result is supplied to the quantization circuit 120. The quantization circuit 120 quantizes this prediction error value. The prediction error signal quantized by the quantization circuit 120 is supplied to the variable length coding circuit 130 and the inverse quantization circuit 140.

In the variable length coding circuit 130, the quantized value of the prediction error signal is variable length coded and output via the line 30. The dequantization circuit 140 dequantizes the prediction error signal to obtain a reproduced value of the prediction error signal, and then supplies the reproduced value to the local decoding circuit 200 via the line 40.

In the local decoding circuit 200, the reproduction value of the conversion coefficient is added by adding the reproduction value of the prediction error signal supplied via the line 40 and the prediction value supplied via the line 20 in the addition circuit 201. Is obtained and then supplied to the inverse orthogonal transform circuit 202. The inverse orthogonal transform circuit 202 inversely transforms the transform coefficient supplied from the adder circuit 201 and outputs a locally decoded signal.

In the frame memory 203, the locally decoded signal for each N × N pixel supplied from the inverse orthogonal transform circuit 202 is accumulated to obtain a locally decoded image. Motion compensation prediction circuit 204
Then, the motion-compensated prediction value is generated using the locally decoded image signal stored in the frame memory 203, and is supplied to the orthogonal transform circuit 205. The orthogonal transformation circuit 205 orthogonally transforms the motion-compensated prediction value for each N × N pixel, and transforms the transform coefficient into line 2
Output through 0.

In this way, when the image signal is compression-encoded, after orthogonal transformation, the local decoding circuit 200 generates a motion compensation prediction value using the locally decoded image signal,
The difference between this and a transform coefficient obtained by orthogonally transforming the image signal is obtained, a prediction error is obtained, the prediction error is quantized, and then variable length coding is performed.

Next, a concrete example of the local decoding circuit 200 is shown in FIG.

In FIG. 2B, 211 is an adder circuit,
220 is a coefficient selection circuit, 212 is an inverse orthogonal transform circuit, 21
3 is a frame memory, 214 is a motion compensation prediction circuit, 21
5 is an orthogonal transformation circuit, and 230 is a coefficient integration circuit.

Inverse orthogonal transform circuit 212, frame memory 2
13, the motion compensation prediction circuit 214, and the orthogonal transform circuit 215 each obtain a transform coefficient of “1 × 1” to “N × N”, assuming that the transform coefficient has an N × N configuration. To be able to do so, for "1x1", "2x2"
, .About. "N-1.times.N-1", and "N.times.N" independent systems are prepared for a total of N systems.

In the local decoding circuit 200 of FIG. 2B, the adder circuit 211 uses the reproduction value of the prediction error signal supplied via the line 40 and the prediction value (motion compensation prediction value) supplied via the line 20. ) Is added to the motion-compensated conversion coefficient reproduction value ((A) in FIG. 3), and the coefficient selection circuit 220 is a reproduction value of the motion-compensated conversion coefficient in FIG. ), From the N × N conversion coefficients, the low-frequency n × n (n = 1 to N) conversion coefficients are selected,
A pyramid of N layers of “1 × 1” to “N × N” shown in FIG. 3B is formed, and it has a function of supplying the transform coefficient of each layer to the inverse orthogonal transform circuit 212 of the corresponding layer. It is a thing.

That is, among the N × N transform coefficients of FIG. 3A, N × N transform coefficient sets, N−1 × N−1 transform coefficient sets, and N−2 × N−2 transforms. A total of N types of conversion coefficient sets, that is, a coefficient set, a 2 × 2 conversion coefficient set, and a 1 × 1 conversion coefficient set, are obtained, and each of the N groups of inverse orthogonal transform circuits 212 has a corresponding level. Input to the inverse orthogonal transform circuit of the system (note that the set of transform coefficients is at least better than N. For example, “N × N”, “3N / 4 × 3N / 4”, “N / 2 ×”).
(N / 2 ”,“ N / 4 × N / 4 ”, and“ 1 × 1 ”, that is, a set of five conversion coefficients in total).

This is sufficient by simply extracting the corresponding coefficient portion from the N × N conversion coefficients shown in FIG. For example, the 1 × 1 transform coefficient set is given to the inverse orthogonal transform circuit 212 (IOT ₁ ) of the 1 × 1 system, and 2 × 2.
The transform coefficient set of is the inverse orthogonal transform circuit 21 of the 2 × 2 system.
2 (IOT ₂ ), and the N−1 × N−1 transform coefficient set is the inverse orthogonal transform circuit 212 of the N−1 × N−1 system.
(IOT _N-1 ), the set of N × N transform coefficients is N
It is given to the inverse orthogonal transform circuit 212 (IOT _N ) of the × N system.

The inverse orthogonal transform circuit 212 for each system obtains a locally decoded signal by inversely transforming the transform coefficient supplied to itself from the coefficient selection circuit 220 for each hierarchy, and the locally decoded signal for each system is obtained. Is shown in FIG. 3 (C).
The locally decoded signals obtained in each of the 1 to N systems will be collectively referred to as a locally decoded signal pyramid.
This locally decoded signal pyramid ((C) in FIG. 3) corresponds to a Gaussian pyramid constructed by using orthogonal transform (reference on Gaussian pyramid: PJ.
Burt et. al "The Laplacian
Pyramidas a Compact Image
Code ”, IEEE Trans. COM Vo
l. 31, No. 4, pp. 532-540, Apri
1983).

Frame memory 2 for each system of 1 to N systems
Reference numeral 13 is a unit for accumulating locally decoded signals of the relevant system supplied from the inverse orthogonal transform circuit 212 to obtain a locally decoded image of its own system, and each frame memory 213 of 1 to N systems.
The locally decoded image for each layer obtained by accumulating in
Together, they will be called locally decoded image pyramids.

As a result, the set of 1 × 1 transform coefficients is 1 ×
The local decoded signal (local decoded signal of the first low frequency term) of only the DC component is obtained by accumulating in the frame memory 213 (FM ₁ ) of the system for 1 and the 2 × 2 transform coefficient set is 2 × 2. A local decoded signal (locally decoded signal composed of first and second low-frequency terms), which is stored in the frame memory 213 (FM ₂ ) for use in the video signal and has the lowest frequency component of the DC component and the AC component, is obtained. The conversion coefficient set of × N is stored in the frame memory 213 (FM _N ) for N × N to generate the DC component and N.
A locally decoded signal composed of AC components up to the -1st order (a locally decoded signal composed of the first low-frequency term to the Nth low-frequency term) is obtained.

The motion compensation prediction circuit 214 generates a motion compensation prediction value for each layer by using the locally decoded image signal stored in the frame memory 213. The compensation prediction circuit 214 is configured to generate a motion-compensated prediction value of a layer corresponding to its own system by using the locally decoded image signals stored in the frame memory 213 of its own system.

The orthogonal transformation circuit 215 orthogonally transforms the motion compensation prediction value for each layer and supplies the transformation coefficient of the hatched portion in FIG. 3D to the coefficient integration circuit 230. That is, the orthogonal transform circuit 2 for each system of 1 to N systems
Reference numeral 15 denotes an orthogonal transformation circuit that receives a motion compensation prediction value generated by a system corresponding to each system motion compensation prediction circuit 214 and performs orthogonal transformation, for example, by the first system orthogonal transformation circuit 215 (OT ₁ ). If there is, the motion compensation prediction value of the frequency band of the DC component (first low frequency term) is the second frequency band of the DC component (second low frequency) if the orthogonal transformation circuit 215 (OT ₂ ) of the second system. If the motion compensation prediction value of the third system is the orthogonal transformation circuit 215 (OT ₃ ) of the third system, the motion compensation prediction value of the next frequency band of the DC component (third low frequency term) is the orthogonal system of the Nth system. If it is the conversion circuit 215 (OT _N ), it outputs the motion compensation prediction value of the frequency band of the highest order term (Nth frequency term).

The coefficient integration circuit 230 uses the orthogonal transformation circuits 2
15 receives the conversion coefficient by the orthogonal transformation of the motion compensation prediction value of each layer output from 15 and receives N × N conversion coefficient prediction values ((E) in FIG. 3) integrated for each band via the line 20. It is what is output.

The operation of the local decoding circuit 200 having such a configuration is as follows. The addition circuit 211 adds the reproduction value of the prediction error signal supplied via the line 40 and the prediction value (motion compensation prediction value) supplied via the line 20 to reproduce the motion-compensated conversion coefficient. A value ((A) in FIG. 3) is obtained. The reproduction value of the motion-compensated conversion coefficient is supplied to the coefficient selection circuit 220, and in the coefficient selection circuit 220, the low-frequency n is selected from the N × N conversion coefficients of FIG.
The conversion coefficients of × n (n = 1 to N) are selected, the pyramids of N layers of “1 × 1” to “N × N” shown in FIG. The signal is supplied to the inverse orthogonal transform circuit 212.

That is, from among the N × N transform coefficients of FIG. 3A, N × N transform coefficient sets, N−1 × N−1 transform coefficient sets, and N−2 × N−2 transforms. A total of N types of conversion coefficient sets, that is, a coefficient set, a 2 × 2 conversion coefficient set, and a 1 × 1 conversion coefficient set are obtained. From the N × N conversion coefficients in FIG.
It suffices to simply extract the relevant coefficient part.

The inverse orthogonal transform circuit 212 inversely transforms the transform coefficient supplied from the coefficient selection circuit 220 for each layer and outputs a local decoded signal pyramid ((C) in FIG. 3).

This locally decoded signal pyramid ((C) of FIG. 3) corresponds to a Gaussian pyramid constructed by using orthogonal transform.

In the frame memory 213, the locally decoded signal pyramid supplied from the inverse orthogonal transform circuit 212 is accumulated for each layer to obtain a locally decoded image pyramid.

The motion compensation prediction circuit 214 generates a motion compensation prediction value for each layer using the locally decoded image signal stored in the frame memory 213, and the orthogonal transformation circuit 21.
5 The orthogonal transform circuit 215 orthogonally transforms the motion compensation prediction value for each layer, and supplies the transform coefficient in the shaded area in FIG. 3D to the coefficient integration circuit 230.

In the coefficient integration circuit 230, the N × N conversion coefficient prediction values obtained by integrating the conversion coefficients of the respective layers for each band are drawn in line 2.
Output through 0. The motion vector used for motion compensation may be obtained for each layer, or even if the motion vector obtained in the Nth layer is reduced to n / N and used in the nth layer, no drift occurs. In addition, point A in FIG.
3E correspond to (A) to (E) of FIG. 3, respectively.

In this way, when the image signal is compression-encoded, after orthogonal transformation, the local decoding circuit 200 generates a motion compensation prediction value using the locally decoded image signal, and orthogonally transforms this and the image signal. The difference from the obtained transform coefficient is obtained to obtain the prediction error, and the prediction error is quantized and then variable-length coded. In particular, the locally decoded image signal is
The image signal is divided into blocks by N × N pixels and orthogonally transformed,
When compression-encoded, 1 × 1, 2 × 2, 3 × 3, ... N
For each layer consisting of × N transform coefficients, the transform coefficient is inversely transformed to obtain a locally decoded signal pyramid, which is stored in a frame memory for each layer to obtain a locally decoded image for each layer. The motion compensation prediction value for the component of the maximum frequency term in the hierarchy is obtained for each hierarchy, and the motion compensation prediction value in the hierarchy having the N × N transform coefficient configuration is obtained by orthogonally transforming and integrating the motion compensation prediction values. I chose
Therefore, the motion-compensated prediction value for each layer and the inverse orthogonal transform output corresponding to the n × n corresponding layer can be reproduced without any mismatch (however, n = 1 to N is a natural number).

<< Decoding Device of First Specific Example >> FIG. 4 is a block diagram of a decoding device for decoding a coded bit stream to obtain a reproduced image in the coding device of FIG.

In FIG. 4A, 150 is a variable length decoding circuit, 160 is an inverse quantization circuit, and 300 is a decoding circuit. The decoding circuit 300 includes an adder circuit 301, an inverse orthogonal transform circuit 302, a frame memory 303, and a motion compensation prediction circuit 3.
04, the orthogonal transformation circuit 305.

The variable length decoding circuit 150 decodes the coded bit stream into a prediction error signal, and the dequantization circuit 160 dequantizes the decoded prediction error signal to reproduce the prediction error signal. The decoding circuit 300 adds the reproduction value of this prediction error signal and the prediction value of the prediction error obtained from the previous frame to obtain the reproduction value of the transform coefficient, and then orthogonalizes this. The signal obtained by performing the inverse conversion of the conversion is output as a decoded signal.

Specifically, the decoding circuit 300 adds the reproduced value of the prediction error signal supplied from the dequantization circuit 160 and the prediction value supplied from the orthogonal transformation circuit 305 by the addition circuit 301. After obtaining the reproduced value of the transform coefficient, the signal obtained by inversely transforming the reproduced value of the transform coefficient in the inverse orthogonal transform circuit 302 is output as a decoded signal, and the decoded signal is accumulated in the frame memory 303, In 303, a decoded image is obtained by accumulating the decoded signal for each N × N pixel, and in the motion compensation prediction circuit 304, a motion compensated prediction value is generated using the decoded image signal accumulated in the frame memory 303, The orthogonal transformation circuit 305 performs orthogonal transformation for each N × N pixels, and the obtained transformation coefficient is supplied to the addition circuit 301.

The operation of this structure will be described. When the bitstream coded by the coding apparatus of FIG. 2 is supplied to the variable length decoding circuit 150 via the line 50, the coding bitstream is predicted by the variable length decoding circuit 150. After being decoded into a signal, it is supplied to the inverse quantization circuit 160. The dequantization circuit 160 dequantizes the prediction error signal to obtain a reproduction value of the prediction error signal, and then supplies it to the decoding circuit 300 via the line 60.
In the decoding circuit 300, after the reproduction value of the prediction error signal supplied via the line 60 and the prediction value supplied from the orthogonal transformation circuit 305 are added by the addition circuit 301, the reproduction value of the transform coefficient is obtained. , To the inverse orthogonal transform circuit 302.

In the inverse orthogonal transformation circuit 302, the addition circuit 301
The transform coefficient supplied by the inverse transform is applied to the decoded signal on the line 70.
Output via. The frame memory 303 accumulates the decoded signal for each N × N pixel supplied from the inverse orthogonal transform circuit 302 to obtain a decoded image. The motion compensation prediction circuit 304 generates a motion compensation prediction value using the decoded image signal stored in the frame memory 303, and the orthogonal transformation circuit 30
5 The orthogonal transformation circuit 305 orthogonally transforms the motion-compensated predicted value for each N × N pixels, and adds the transformation coefficient to the addition circuit 3.
01.

<< Configuration Example of Decoding Circuit 300 in First Specific Example >> FIG. 4B is a specific example of the decoding circuit 300 corresponding to the local decoding circuit 200 which is a specific example of the present invention.
In this specific example, out of the data hierarchically divided into N layers, data for n layers from the low frequency band is decoded to obtain n in both horizontal and vertical directions.
A case of obtaining a reproduced image with a resolution of / N will be described.

As shown in FIG. 4B, the decoding circuit 300
Is composed of an addition circuit 311, a coefficient selection circuit 320, an inverse orthogonal transformation circuit 312, a frame memory 313, a motion compensation prediction circuit 314, an orthogonal transformation circuit 315, and a coefficient integration circuit 330.

In this example, each of the inverse orthogonal transform circuit 312, the frame memory 313, the motion compensation prediction circuit 314, and the orthogonal transform circuit 315 converts the data of the lower layer to the nth layer among the data layered in the N layer. When decoding is performed to obtain a reproduced image with a resolution of n / N in both horizontal and vertical directions, conversion coefficients are "1 × 1" to "n × n" (where n = 1.
~ N), so that each of the configurations can be acquired, "1x1", "2x2", "n-1xn-"
Independent systems for "1" and "n × n" are prepared, and a total of n systems are provided.

The adder circuit 311 is provided with the reproduction value of the prediction error signal supplied from the inverse quantization circuit 160 and the coefficient integration circuit 3
The predicted value supplied from 30 is added to obtain the reproduction value of the conversion coefficient, and the coefficient selection circuit 320
Is to organize the reproduced values of the transform coefficient obtained by the adder circuit 311 into pyramids of n layers and distribute them to each layer. In this specific example, image decoding is performed using the first layer to the nth layer. "1 x 1" to "n" because we aim to
In this configuration, each layer of “× n” is separated and distributed.

The inverse orthogonal transform circuit 312 is for performing an inverse orthogonal transform on the transform coefficient, is provided for each hierarchical layer, and of the hierarchically separated layers distributed by the coefficient selection circuit 320 for each hierarchical layer, the corresponding hierarchical layer. Is inversely orthogonally transformed and decoded.

That is, the coefficient selection circuit 320 sets the value to "1".
The ones in each layer of "x1" to "nxn" are distributed,
The “1 × 1” hierarchy is given to the inverse orthogonal transformation circuit 312 (IOT ₁ ) for the 1 × 1 system, and the “2 × 2” hierarchy is the inverse of the 2 × 2 system. Orthogonal transformation circuit 312
(IOT ₂ ), the one of the hierarchy of “n−1 × n−1” is the inverse orthogonal transform circuit 31 of the system for n−1 × n−1.
2 (IOT _N-1 ) and the "n × n" hierarchy is the inverse orthogonal transform circuit 312 (IO) of the n × n system.
T _N ).

In the inverse orthogonal transform circuit 312 having n systems,
The transform signal supplied from the coefficient selection circuit 320 is inversely transformed for each layer to convert the decoded signal pyramid to the frame memory 31.
3, the inverse orthogonal transform circuit 31 of the n × n system is supplied.
The decoded signal which is the inverse transform output of 2 (IOT _N ) is the final image signal output via line 70.

The frame memory 313 for n systems accumulates the decoded signals supplied from the inverse orthogonal transform circuit 312 of the corresponding system for each layer to obtain a decoded image pyramid.

That is, the decoded signal of the "1 × 1" layer is the frame memory 313 (FM ₁ ) of the 1 × 1 system.
The image decoded signal (first
A decoded signal composed of a low frequency term) is obtained, and the decoded signal of the “2 × 2” hierarchy is the 2 × 2 frame memory 313 (F
The decoded signal (first and _second ) of the image, which is accumulated in M ₂ ) and has the lowest frequency component of the DC component and the AC component
A decoded signal composed of low-frequency terms is obtained, and the decoded signal of the “n × n” hierarchy is the frame memory 31 of the n × n system.
3 (FM _N ) to obtain a decoded signal (decoded signal including the first low-frequency term to the n-th low-frequency term) including the components from the direct current component to the alternating current component up to the (n-1) th order.

The motion compensation prediction circuit 314 generates a motion compensation prediction value for each layer using the decoded image signal stored in the frame memory 313.
The motion compensation prediction circuit 314 for each system of the system is configured to generate a motion compensation prediction value of a hierarchy corresponding to the system using the decoded image signal accumulated in the frame memory 313 of the system.

The orthogonal transformation circuit 315 orthogonally transforms the motion compensation prediction value for each layer and supplies the transformation coefficient of the area of the hatched display portion in FIG. 3D to the coefficient integration circuit 330. That is, the 1- to n-system orthogonal transform circuits 315 receive the motion-compensated prediction values generated by the respective systems of the motion-compensation prediction circuits 314 and perform orthogonal transform. In the case of the first system orthogonal transformation circuit 315 (OT ₁ ), the motion compensation prediction value of the frequency band of the DC component (first low frequency term) is calculated in the case of the second system orthogonal transformation circuit 315 (OT ₂ ). , The motion compensation prediction value of the frequency band next to the DC component (second low frequency term)
In the case of the orthogonal transformation circuit 315 (OT ₃ ) of the system, the motion compensation prediction value of the next frequency band of the DC component (third low frequency term) is calculated by the orthogonal transformation circuit 315 (OT _N ) of the n-th system. The motion compensation prediction value in the frequency band of the nth term (nth low frequency term) is output.

The coefficient integrating circuit 330 supplies n × n transform coefficient prediction values, which are obtained by integrating the transform coefficients of each layer for each band, to the adding circuit 311.

In such a configuration, the adder circuit 311
Then, the reproduction value of the prediction error signal supplied via the line 60 and the prediction value supplied from the coefficient integration circuit 330 are added to obtain the reproduction value of the conversion coefficient, and then the coefficient selection circuit 320 is supplied. Supply. In the coefficient selection circuit 320, “1
An inverse orthogonal transform circuit 3 which constitutes a pyramid of n layers of "x1" to "nxn", and transform coefficients of each layer are provided for each layer.
Feed to the corresponding of twelve.

The inverse orthogonal transform circuit 312 inversely transforms the transform coefficient supplied from the coefficient selection circuit 320 for each layer and supplies the decoded signal pyramid to the corresponding frame memory 313 for each layer, and also for the nth layer. Decode signal on line 7
It is output as a restored image signal via 0.

In the frame memory 313 for each hierarchical layer, the decoded signal supplied from the inverse orthogonal transform circuit 312 of the corresponding hierarchical layer of its own system is accumulated to obtain a decoded image for each hierarchical layer, and a decoded image pyramid is obtained. obtain.

In the motion compensation prediction circuit 314 for each layer,
Motion compensation prediction values are generated using the decoded image signals stored in the corresponding frame memory 313 of the own system, and the motion compensation prediction values for each layer are obtained. Then, this is supplied to the orthogonal transformation circuit of the corresponding layer among the orthogonal transformation circuits 315 for each layer. Orthogonal transform circuit 3 for each layer
In FIG. 15, the motion compensation prediction value of the corresponding layer is received and orthogonally transformed to obtain the conversion coefficient of the area of the shaded display portion in FIG.
Supply to 30.

The coefficient integration circuit 330 obtains n × n conversion coefficient prediction values in which the conversion coefficients for each layer are integrated for each band, and supplies this to the addition circuit 311. Also, FIG.
Points A to E in (b) correspond to (A) to (E) in FIG. 3, respectively, as in FIG. 2 (b). The image output from the decoding circuit 300 via the line 70 may be only the reproduced image of the nth layer.

In this way, when the image signal is divided into blocks by N × N pixels and orthogonally transformed, and the bit stream of the compression-encoded signal is decoded by n × n smaller than N × N, The reproduction value of the prediction error signal obtained from the stream is distributed so as to have a form corresponding to the hierarchy of the transform coefficient configuration of 1 × 1 to n × n, and each is subjected to inverse orthogonal transform, and the hierarchy corresponding to n × n of these is obtained. The inverse orthogonal transform output corresponding to is used as a decoded signal for use in image reproduction.

Further, with respect to the transform coefficient corresponding to each layer, outputs obtained by inverse orthogonal transform are accumulated to obtain a frame image corresponding to each layer, and a motion compensation prediction value is generated for each layer, respectively. A motion compensation prediction value for each layer is obtained, orthogonal transformation is performed for each layer to obtain a motion compensation prediction value for a component of the maximum frequency term in that layer for each layer, and the motion compensation prediction values for each layer are integrated. The motion compensation prediction value in the hierarchy of n transform coefficient structures is obtained. Then, the motion compensation prediction value is compensated for the reproduction value of the prediction error signal.

Therefore, motion compensation is performed for the component of the maximum frequency term in each layer, and the reproduction value (motion compensated) of the prediction error signal corresponds to the layer of n × n transform coefficient structure. By performing inverse orthogonal transform only on the transform coefficients to be used and using the output for image reproduction, there is no mismatch due to the difference in resolution between the encoding side and the decoding side. That is, it is possible to prevent image quality deterioration due to the difference in the order of the orthogonal transform low frequency terms used on the encoding side and the decoding side.

On the encoding side, when the image signal is compression-encoded, the local decoding circuit 200 is used after orthogonal transformation.
By using the locally decoded image signal to generate a motion-compensated prediction value, obtain a difference between this and a transform coefficient obtained by orthogonally transforming the image signal, obtain a prediction error, and quantize this prediction error. Variable length coding is used. In particular, the locally decoded image signal is 1 × 1, 2 × 2, 3 × when the image signal is divided into blocks by N × N pixels, orthogonally transformed, and compression-coded.
For each layer consisting of 3, to N × N transform coefficients, the transform coefficient is inversely transformed to obtain a locally decoded signal pyramid, which is accumulated in a frame memory for each layer to obtain a locally decoded image for each layer. From this, the motion compensation prediction value for the component of the maximum frequency term in that hierarchy is obtained for each hierarchy, and the motion compensation prediction value in the hierarchy of the N × N transform coefficient configuration is obtained by orthogonally transforming and integrating the motion compensation prediction values. As described above, it is possible to reproduce the motion compensation prediction value and the inverse orthogonal transform output corresponding to the n × n corresponding layer for each layer without causing a mismatch (however, n = 1 to N is a natural number).

(Second Specific Example) A second specific example of the present invention will be described with reference to FIGS. 5 and 6. The second specific example relates to SNR scalability, in which the image quality is improved by making the quantization step coarse initially and gradually finer.

FIG. 5 shows a motion-compensated prediction + orthogonal-transform coding apparatus (differential structure after transformation) using motion-compensated prediction in the orthogonal transform coefficient domain to which the present invention is applied, and FIG. 6 shows this coding apparatus. It is a block diagram of the decoding apparatus which implement | achieves SNR scalability from the bit stream obtained by.

FIG. 5 shows an example of a coding device for performing quantization by dividing into M layers. In FIG. 5, 100 is an orthogonal transform circuit, 121, 122 and 123 are quantization circuits, and 13 is a quantization circuit.
1 to 133 are variable length coding circuits, 420 and 421 are addition circuits, 200a and 200b, to 200M are local decoding circuits, 400 and 401 are delay circuits, 111, 112 and 11
3, 410, 411 are difference circuits, 132, 141, 14
Reference numerals 2 and 143 are inverse quantization circuits.

First layer L1 having local decoding circuit 200a
Is for obtaining the base layer encoded signal, and has the second layer L2 having the local decoding circuit 200b.
The constituent element of is for obtaining the encoded signal of the enhancement layer, and the constituent element of the Mth layer LM having the local decoding circuit 200M is for obtaining the encoded signal of the enhancement layer.

In the encoding device having the configuration as shown in FIG. 5, the image signal is first orthogonally transformed by the orthogonal transformation circuit 100, and the image signal to be encoded is supplied via the line 10. The supplied image signal is orthogonally transformed for each N × N pixel in the orthogonal transformation circuit 100 to obtain N × N.
Are obtained. This orthogonal transformation coefficient is for each layer L
1 to LM.

In the first layer L1, the orthogonal transformation circuit 1
The orthogonal transform coefficient from 00 is input to the difference circuit 111. Then, in the difference circuit 111, the orthogonal transformation circuit 1
Orthogonal transform coefficient supplied from 00 and the local decoding circuit 20.
A prediction error with respect to the prediction values of the N × N transform coefficients supplied via the line 21 from 0a is calculated and supplied to the quantization circuit 121. The prediction error signal quantized by the quantization circuit 121 is supplied to the variable length coding circuit 131 and the dequantization circuit 14.
1 is supplied.

In the variable length coding circuit 131, the quantized value of the prediction error signal is variable length coded and output via the line 31. The dequantization circuit 141 dequantizes the prediction error signal to obtain a reproduction value of the prediction error signal, and then supplies it via the line 41 to the local decoding circuit 200a and the second layer L2.

In the second layer L2, the delay circuit 400 operates in the orthogonal transform circuit 1 until the reproduction value of the prediction error signal of the block in the first layer L1 is obtained via the line 41.
Delays the timing at which the orthogonal transform coefficient supplied from 00 is supplied to the difference circuit 112.

The difference circuit 112 calculates a prediction error between the orthogonal transform coefficient supplied from the delay circuit 400 and the predicted value of the transform coefficient supplied from the local decoding circuit 200b via the line 22, and the difference circuit 410 calculates it. Supplied. In the difference circuit 410, the second layer L supplied from the difference circuit 112.
The difference between the prediction error at 2 and the reproduction value of the prediction error at the first layer L1 supplied via the line 41 is calculated and supplied to the quantization circuit 122, where the difference is quantized. .

The difference between the prediction error signals quantized by the quantization circuit 122 is supplied to the variable length coding circuit 132 and the inverse quantization circuit 142. In the variable length coding circuit 132, the quantized value of the difference of the prediction error signal is variable length coded, and the line 3
2 is output.

The dequantization circuit 142 dequantizes the difference between the prediction error signals to obtain the reproduced value of the difference between the prediction error signals, and then the addition circuit 420 supplies the reproduced value via the line 41 of the first layer L1. The reproduction value of the prediction error signal is added, and the second
After obtaining the reproduction value of the prediction error signal of the layer L2, it is supplied to the local decoding circuit 200b via the line 42.

In the Mth layer LM, the delay circuit 401
Then, until the reproduction value of the prediction error signal of the block in the (M-1) th layer LM-1 is obtained via the line 43, the orthogonal transform coefficient supplied from the orthogonal transform circuit 100 remains unchanged.
The timing supplied to 13 is delayed. Then, the difference circuit 113 calculates a prediction error between the orthogonal transform coefficient supplied from the delay circuit 401 and the predicted value of the transform coefficient supplied from the local decoding circuit 200M via the line 23,
It is supplied to the difference circuit 411.

In the difference circuit 411, the difference between the prediction error in the M-th layer supplied from the difference circuit 113 and the reproduced value of the prediction error in the (M-1) -th layer LM-1 supplied via the line 43. Is calculated and supplied to the quantization circuit 123, where it is quantized. Then, the difference between the prediction error signals quantized by the quantization circuit 123 is the variable length coding circuit 13.
3 and the inverse quantization circuit 143.

In the variable length coding circuit 133, the quantized value of the difference of the prediction error signal is variable length coded and output via the line 33. The dequantization circuit 143 dequantizes the difference between the prediction error signals to obtain a reproduced value of the difference between the prediction error signals, and then the addition circuit 421 supplies the reproduced value to the M-1th layer LM supplied via the line 43. The reproduction value of the prediction error signal of the M-th layer LM is obtained by adding the reproduction value of the prediction error signal of −1, and this is supplied to the local decoding circuit 200M via the line 44.

Here, the quantization step size in the m-th (m = 1 to M) layer Lm is smaller than that in the (m-1) -th layer Lm-1. That is, the quantization step size is made smaller than that of the previous stage layer. However, it is better to use the same motion vector for each layer in motion compensation.
The variable length codes used in the variable length coding circuits 131, 132, 133 may be the same or different.

In this way, in the second and higher layers, by subtracting the locally decoded signal of each layer from the self to the first and lower layers from the transform coefficient obtained from the orthogonal transform circuit 100, the order conversion corresponding to the own layer is performed. By obtaining the prediction error signal value of the highest-order coefficient among the coefficients, that is, the frequency-term component of the highest-order area in that hierarchy for each hierarchy, quantizing this, and performing variable-length coding for output, the M-layer A bitstream is obtained by coding the prediction error signal value for the component of the maximum frequency term in each layer.

The bit stream for each layer is, for example, multiplexed and output when it is used for transmission or the like. Then, on the decoding side, this is separated and returned to a bit stream for each layer for use.

FIG. 6 is a block diagram of a decoding device for decoding a bit stream up to the m-th layer from a bit stream divided into M layers and encoded by the encoding device of FIG. 5 to obtain a reproduced image. It is a figure.

In FIG. 6, 151, 152 and 153 are variable length decoding circuits, 161, 162 and 163 are dequantization circuits, 430 and 431 are addition circuits,
300 is.

Variable length decoding circuit 151 and inverse quantization circuit 1
At 61, the bit stream of the first layer L1 is decoded, at the variable length decoding circuit 152 and the inverse quantization circuit 162, the bit stream of the second layer L2 is decoded, and at the variable length decoding circuit 153 and the inverse quantization circuit 163. The bitstream of the nth layer Ln is decoded.

In such a configuration, the coded bit stream corresponding to each layer coded by the coding device is transmitted via the lines 51, 52 and 53 to the variable length decoding circuits 151, 152 for the corresponding layers. 153. And
The coded bitstreams of the corresponding layers respectively supplied are
These variable length decoding circuits 151, 152 and 153 respectively decode the prediction error signal or the difference between the prediction error signals, and then the inverse quantization circuits 161, 16 of the corresponding layers.
2, 163.

The dequantization circuits 162 and 163 dequantize the difference between the prediction error signals to obtain the reproduced value of the difference between the prediction error signals. Then, in the adder circuit 430, the reproduction values of the differences in the prediction error from the m-th layer to the second layer are added and supplied to the adder circuit 431. In addition, the inverse quantization circuit 1
In 61, the prediction error signal of the first layer is inversely quantized to obtain a reproduction value of the prediction error signal, which is then supplied to the adding circuit 431. Then, in this addition circuit 431, the reproduction value of the total prediction error signal for m layers is added with the addition value of the reproduction value of the difference in the prediction error from the m-th layer to the second layer obtained by the addition circuit 430. It is determined and is provided to decoding circuit 300 via line 60.

Here, the local decoding circuits 200a, 200
b, ~ 200M-1 and decoding circuit 300 according to the first embodiment of the present invention.
If a specific example of is applied, a bit stream whose image quality is divided into M layers and resolution is divided into N layers is formed, and by decoding a part of the bit stream, a reproduced image with a desired image quality m and resolution n can be obtained. It can be obtained (see FIG. 7).

(Third Concrete Example) FIGS. 8, 9 and 10
The third specific example of the present invention will be described with reference to FIG. The third specific example is a technique in which only the image of the part of the image of interest in the image can be encoded with a desired resolution. In this specific example, the first specific example is an alpha map. It is applied to an image of an arbitrary shape indicated by a signal.

FIG. 8A shows an example of the configuration of an encoding apparatus for encoding an image of an arbitrary shape. In the figure, 180 is an alpha map encoding circuit, 181 is a multiplexing circuit, 105 is an orthogonal transformation circuit, and 115. Is a difference circuit, 125 is a quantization circuit, 135 is a variable length decoding circuit, 145 is an inverse quantization circuit, 500 is a local decoding circuit, 501 is an addition circuit, 50
2 is an inverse orthogonal transform circuit, 503 is a frame memory, 504
Is a motion compensation prediction circuit, and 505 is an orthogonal transformation circuit.

In this specific example, in addition to the image signal, alpha map information (information indicating the position of the image, for example, the image being binarized) corresponding to the image of this image signal is also created and the present system is created. Shall be entered in.

The alpha map encoding circuit 180 receives the alpha map information of the image as an input, encodes it and outputs it to the line 82, and has a function of decoding the encoded alpha map signal. It has a function of outputting the locally decoded signal of the alpha map signal decoded by this through the line 81.

The orthogonal transformation circuit 105 receives the image signal and the locally decoded signal of the alpha map signal supplied via the line 81, and refers to the locally decoded signal of the alpha map signal to determine the portion of the image to be extracted. The image signal is orthogonally transformed and output.

The alpha map is binary data indicating a target portion of the image, and by referring to this, it is a mechanism to know which portion of the image is the target portion.

The local decoding circuit 500 includes the orthogonal transformation circuit 10
The signal of the prediction error value (prediction error signal), which is the difference obtained by orthogonally transforming in 5 and subtracting the motion compensation prediction value, is compensated from the image for which the prediction value has been compensated, based on the locally decoded signal of the alpha map. Is obtained, subjected to orthogonal transformation, and output as a predicted value.

The multiplexing circuit 181 multiplexes the coded signal of the alpha map information of the image output from the alpha map coding circuit 180 and the coded signal of the image error signal output from the variable length decoding circuit 135. It is what is output.

With such a configuration, the alpha map coding circuit 180 codes the input alpha map information. Then, the encoded alpha map signal is output via the line 82, the encoded alpha map signal is decoded, and the decoded alpha map signal is used as the local decoded signal of the alpha map signal via the line 81. 5
00 and the orthogonal transformation circuit 105.

On the other hand, in the orthogonal transform circuit 105, an image signal is input via the line 10, but this image signal is orthogonally transformed based on the locally decoded signal of the alpha map supplied via the line 81. . Then, the coefficient obtained by this orthogonal transformation is given to the difference circuit 115.

In the difference circuit 115, the orthogonal transformation circuit 105
The prediction error between the orthogonal transform coefficient supplied from the local decoding circuit 500 and the predicted value of the transform coefficient supplied from the local decoding circuit 500 via the line 25 is calculated, supplied to the quantization circuit 125, and quantized here. .

The prediction error signal quantized by the quantization circuit 125 is supplied to the variable length coding circuit 135 and the dequantization circuit 145. Variable length decoding circuit 135
Then, the quantized value of the prediction error signal is variable length coded. Then, the variable-length coded signal is output to the line 35.

On the other hand, the dequantization circuit 145 dequantizes the prediction error signal to obtain the reproduced value of the prediction error signal, and then supplies it to the local decoding circuit 500 via the line 45.

In the local decoding circuit 500, the reproduction value of the conversion coefficient is reproduced by adding the reproduction value of the prediction error signal supplied via the line 45 and the prediction value supplied via the line 25 in the addition circuit 501. After obtaining the value, the inverse orthogonal transform circuit 502
To supply.

The inverse orthogonal transform circuit 502 inversely transforms the transform coefficient supplied from the adder circuit 501 based on the locally decoded signal of the alpha map supplied via the line 81, outputs the locally decoded signal, and outputs it to the frame memory. Give to 503.

Then, the frame memory 503 stores the locally decoded image supplied from the inverse orthogonal transform circuit 502. The motion compensation prediction circuit 504 uses the locally decoded image signal stored in the frame memory 503, and from this, based on the locally decoded signal of the alpha map supplied via the line 81, the motion compensation for only the image portion of interest is performed. A predicted value is generated and supplied to the orthogonal transformation circuit 505. The orthogonal transform circuit 505 orthogonally transforms the motion compensation prediction value based on the locally decoded signal of the alpha map supplied via the line 81, and outputs the transform coefficient via the line 25.

The orthogonal transform circuits 105 and 505 and the inverse orthogonal transform circuit 502 are provided in, for example, Japanese Patent Application No. 7-97.
It is advisable to apply the orthogonal transform method for arbitrary shape image signals, which is the technique disclosed in No. 073.

The encoded alpha map signal is line 82.
Via the line 35, the coded transform coefficients are supplied to the multiplexing circuit 181 to be multiplexed and then the line 8
5 is output as a bit stream.

In this manner, the target image portion extracted and variable-length coded and the encoded alpha map signal indicating the target image portion are multiplexed to form a bit stream.

FIG. 8B shows a specific example of the local decoding circuit 500 which enables the motion-compensated predicted value of the target image to be accurately obtained at the target resolution. here,
An error signal is obtained for each layer and integrated at the end to obtain a highly accurate predicted value.
Is an addition circuit, 512 is an inverse orthogonal transformation circuit, 513 is a frame memory, 514 is a motion compensation prediction circuit, 515 is an orthogonal transformation circuit, 520 is a coefficient selection circuit, 530 is a coefficient integration circuit, and 540 is a resolution conversion circuit.

Inverse orthogonal transform circuit 512, frame memory 5
13, the motion compensation prediction circuit 514 has a conversion coefficient of N ×
If the configuration is N, the conversion coefficients are “1 × 1” to “N”.
In order to be able to acquire each of the × N ”configurations, for“ 1 × 1 ”,“ 2 × 2 ”, to“ N−1 × N− ”.
Independent systems for "1" and "N × N" are prepared, and the total N systems (N layers) are configured.

The resolution conversion circuit 540 converts the resolution of the locally decoded signal of the alpha map given through the line 81 to n / N times (n = 1 to N) both in the horizontal and vertical directions, and the line 83 as a signal of the N layer pyramid. Is output to.

The adder circuit 511 is a circuit for adding the reproduction value of the prediction error signal supplied via the line 45 and the prediction value supplied via the line 25, and the reproduction value of the conversion coefficient is obtained by this addition. It is a thing.

The coefficient selection circuit 520 receives the reproduction value of the conversion coefficient from the addition circuit 511, selects the conversion coefficient according to the alpha map signal pyramid of the N layer supplied through the line 83, and selects the first to the first conversion coefficients. By obtaining the corresponding transform coefficient for each of the N layers, the N layer pyramid is obtained.

The inverse orthogonal transform circuit 512 inversely orthogonally transforms and outputs the transform coefficient of the corresponding layer among the transform coefficients of the respective layers. In the inverse orthogonal transform circuit 512 for each layer, A locally decoded signal pyramid is obtained by inversely transforming the transform coefficient supplied from the coefficient selection circuit 520 to obtain a locally decoded signal according to the alpha map signal pyramid supplied via the line 83 for each layer.

The frame memory 513 of each layer accumulates the locally decoded signal supplied from the inverse orthogonal transform circuit 512 of the corresponding layer to obtain a locally decoded image. The motion compensation prediction circuit 514 of each layer uses the locally decoded image signal accumulated in the frame memory 513 of the corresponding layer, and according to the alpha map signal pyramid supplied via the line 83 for each layer, The motion compensation prediction value in each layer is generated and supplied to the orthogonal transformation circuit 515 of the corresponding layer.

Further, the orthogonal transformation circuit 515 of each layer
Is a line 8 for the motion compensation prediction value of the corresponding layer.
The orthogonal transform is performed in accordance with the alpha map signal supplied via 3, and the transform coefficient at the maximum frequency term in the hierarchy among the transform coefficients subjected to the orthogonal transform is supplied to the coefficient integrating circuit 530. .

The coefficient integrating circuit 530 integrates the transform coefficients output from the orthogonal transform circuit 515 of each layer and outputs the integrated result to the line 25.

That is, the orthogonal transformation circuit 515 for each layer of the first to Nth layers is the motion compensation prediction circuit 514 for each layer.
Of the DC layers of the DC component in the frequency band of the DC component, for example, in the case of the orthogonal transformation circuit 515 (OT ₁ ) of the system for the first layer. The motion compensation prediction value of (first low frequency term) is converted into the orthogonal transformation circuit 515 (OT ₂ ) of the system for the second layer.
If so, the motion compensation prediction value in the frequency band (second low frequency term) next to the DC component is converted to the orthogonal transformation circuit 5 of the system for the third layer.
If it is 15 (OT ₃ ), the motion compensation prediction value of the next frequency band (third low frequency term) of the DC component is the highest if it is the orthogonal transformation circuit 515 (OT _N ) of the system for the Nth layer. The motion compensation prediction value of the term frequency band (Nth frequency term) is output.

Then, the coefficient integrating circuit 530 receives the transform coefficients by the orthogonal transform of the motion compensation prediction values of the respective layers outputted from the orthogonal transform circuits 515, and integrates them for each band.
It outputs the × N transform coefficient prediction values via the line 25.

In such a configuration, the resolution converting circuit 5 is connected via the line 81 from the alpha map encoding circuit 180.
The locally decoded signal of the alpha map supplied to the unit 40 is resolution-converted in the resolution conversion circuit 540, and the resolution is converted to n / N times (n = 1 to N) both in the horizontal and vertical directions, and the first layer to the Nth layer. By obtaining the conversion coefficients corresponding to the respective layers up to, a pyramid of N layers of conversion coefficients is created.

The resolution-converted N-layer pyramid has a motion-compensated prediction circuit 514 (MC
_{1 to} MC _N ) via line 83. Also, line 8
The N-layered pyramid output via the 3 is a coefficient selection circuit 520, an inverse orthogonal transformation circuit 512, and an orthogonal transformation circuit 51.
5, also input to the coefficient integration circuit 530.

On the other hand, the output (reproduction value of the prediction error signal) dequantized by the dequantization circuit 145 is the coefficient integration circuit 53.
The conversion coefficient prediction value output from 0 (conversion coefficient prediction value obtained by integrating the conversion coefficients of the respective layers for each band) is added by the addition circuit 511 to obtain a reproduction value of the conversion coefficient. Then, the reproduction value of the conversion coefficient thus obtained is
It is supplied to the coefficient selection circuit 520.

The coefficient selection circuit 520 selects transform coefficients in accordance with the alpha map signal pyramid of the N layers supplied via the line 83 to form pyramids of the N layers, and the transform coefficients of each layer are converted into the respective layers. The signal is supplied to the corresponding inverse orthogonal transform circuit 512. Inverse orthogonal transform circuit 512 of each layer
Then, according to the alpha map signal pyramid supplied via the line 83 for each layer, the coefficient selection circuit 520
A locally decoded signal pyramid is obtained by inversely transforming the supplied transform coefficient to obtain a locally decoded signal.

The local decoded signals are applied to the frame memories 513 of the corresponding layers, and in these frame memories 513, the local decoded signals supplied from the inverse orthogonal transform circuit 512 of the corresponding layers are accumulated to form a locally decoded image. obtain. As a result, the locally decoded signal pyramid can be accumulated for each layer to obtain the locally decoded image pyramid.

The locally decoded image pyramid is given to the motion compensation prediction circuit 514. Motion compensation prediction circuit 5 for each layer
14, the locally decoded image signal stored in the frame memory 513 of the corresponding layer is used, and the line 83 is used for each layer.
In accordance with the alpha map signal pyramid supplied via, the motion compensation prediction value is generated and supplied to the orthogonal transform circuit 515 of the corresponding layer.

The orthogonal transform circuit 515 of each layer orthogonally transforms the input motion-compensated prediction value according to the alpha map signal to obtain a transform coefficient for each layer.
That is, in the orthogonal transform circuit 515, orthogonal transform is performed according to the alpha map signal pyramid supplied via the line 83 for each layer, and the transform coefficients in the highest-order frequency terms obtained in each layer by this transform are integrated. Supply to the circuit 530. The coefficient integration circuit 530 outputs a conversion coefficient prediction value obtained by integrating the conversion coefficients of the respective layers for each band through the line 25.

The orthogonal transformation circuit 515, the inverse orthogonal transformation circuit 512, and the coefficient selection circuit 520 are provided in Japanese Patent Application No.
It is advisable to apply the orthogonal transformation method of the arbitrary shape image signal capable of resolution conversion, which is the technology disclosed in No. 97073.

The transform coefficient prediction value obtained by integrating the transform coefficients of the respective layers output from the coefficient integrating circuit 530 for each band is output as the output of the local decoding circuit 500 via the line 25 as shown in FIG.
By giving it to the difference circuit 115 of (a), in the difference circuit 115, the orthogonal transform coefficient supplied from the orthogonal transform circuit 105 and the predicted value of the transform coefficient supplied from the local decoding circuit 500 via the line 25. The prediction error of
It is supplied to the quantization circuit 125 and is quantized here.

The prediction error signal quantized by the quantization circuit 125 is supplied to the variable length coding circuit 135 and the dequantization circuit 145, and the variable length decoding circuit 135 quantizes the prediction error signal. The value is variable length coded and the line 35
Is output via.

On the other hand, in the dequantization circuit 145, after dequantizing the prediction error signal to obtain the reproduced value of the prediction error signal, it is supplied to the local decoding circuit 500 via the line 45. In addition, the local decoding circuit 500 performs motion compensation prediction to obtain a transform coefficient prediction value, which is returned to the difference circuit 115.

In this way, the image-of-interest portion of the image is extracted, and the error amount between the motion-compensated prediction value of only the image-of-interest portion and the motion-compensated prediction value of the image-of-interest portion of the preceding frame screen is obtained. The variable-length coded signal and the coded alpha map signal indicating the image of interest are multiplexed, converted into a bit stream, and output.

To reproduce the bit stream, the following is done.

FIG. 9 is a block diagram of a decoding device for decoding the bit stream coded by the coding device of FIG. 8 to obtain a reproduced image.

In FIG. 9A, reference numeral 190 is a separation circuit, 191 is an alpha map decoding circuit, 155 is a variable length decoding circuit, 165 is an inverse quantization circuit, and 600 is a decoding circuit. Of these, the separation circuit 190 separates the code relating to the alpha map and the code relating to the transform coefficient, and the alpha map decoding circuit 191 reproduces the separated alpha map signal and decodes it via the line 92. It is supplied to the circuit 600.

The variable length decoding circuit 155 is the separation circuit 1
The coded bit stream of the code relating to the prediction error signal separated and supplied at 90 is decoded into a prediction error signal, and the dequantization circuit 165 dequantizes the decoded prediction error signal to perform prediction. The decoder circuit 600 obtains a reproduced value of the error signal, and the decoding circuit 600 obtains and outputs a reproduced value based on the reproduced value of the prediction error signal and the decoded signal of the alpha map.

The decoding circuit 600 includes an adder circuit 601, an inverse orthogonal transform circuit 602 (IOT _N ), a frame memory 603.
(FM _N ), a motion compensation prediction circuit 604 (MC _N ) and an orthogonal transformation circuit 605 (OT _N ).

The adder circuit 601 is a circuit for adding the signal given through the line 65 and the output of the orthogonal transform circuit 605 (OT _N ), and the inverse orthogonal transform circuit 602 (IOT _N ) is
The output of the adder circuit 601 is inversely orthogonally transformed according to the alpha map from the alpha map decoding circuit 191 to obtain a reproduction signal, which is output to the line 75.

The frame memory 603 (FM _N )
Is for accumulating signals from the inverse orthogonal transform circuit 602 (IOT _N ) to obtain a frame image, and the motion compensation prediction circuit 604 (MC _N ) is for performing motion compensation prediction from this frame image. Orthogonal transformation circuit 605 (OT _N )
Is a value that is obtained by the motion compensation prediction and is orthogonally transformed according to the alpha map signal to obtain a transformation coefficient, which is given to the addition circuit 601.

In such a configuration, the multiplexed coded bit stream output from the multiplexing circuit 181 of FIG. 8 is supplied to the demultiplexing circuit 190 via the line 90.

Then, the separating circuit 190 separates the coded bit stream into a code related to the alpha map and a code related to the transform coefficient. The code relating to the alpha map is supplied to the alpha map decoding circuit 191 via the line 91, and the code relating to the prediction error signal is supplied to the variable length decoding circuit 155 via the line 55.

The alpha map decoding circuit 191 reproduces the alpha map signal from the code relating to the alpha map and supplies it to the decoding circuit 600 via the line 92.

On the other hand, the variable length decoding circuit 1 is connected via the line 55.
The encoded bit stream supplied to 55 is decoded into a prediction error signal here and then supplied to the inverse quantization circuit 165. The dequantization circuit 165 dequantizes the prediction error signal to obtain a reproduction value of the prediction error signal, and then supplies it to the decoding circuit 600 via the line 65. Then, the decoding circuit 60
At 0, the reproduction value is obtained based on the decoded signal of the alpha map supplied via the line 92 and is output via the line 75.

A concrete example of the decoding circuit 600 is shown in FIG. In the figure, 640 is a resolution conversion circuit, 610 is a coefficient selection circuit, 611 is an addition circuit, 612 is an inverse orthogonal transformation circuit, 613 is a frame memory, 514 is a motion compensation prediction circuit, 615 is an orthogonal transformation circuit, and 630 is a coefficient integration circuit. Is.

Of these, each of the inverse orthogonal transform circuit 612, the frame memory 613, the motion compensation prediction circuit 514, and the orthogonal transform circuit 615 has a transform coefficient of N × N on the encoder side. It is assumed that the decoding restores the desired configuration “n × n” (n = 1 to N; N is a natural number) of these, and in this case, the conversion coefficient is “1 × 1” to “n × n”. To be able to get each
Independent systems for “1 × 1”, “2 × 2”, and “n × n” are prepared, and have a total of N systems (N layers).

The resolution conversion circuit 640 converts the resolution of the locally decoded signal of the alpha map given through the line 92 to n / N times (n = 1 to N) both in the horizontal and vertical directions and inversely orthogonalizes it as a signal of the n-layer pyramid. The data is output to the conversion circuit 612 and the orthogonal conversion circuit 615. Inverse orthogonal transform circuit 61
The two-orthogonal transformation circuit 615 is provided for each layer, and therefore the resolution-converted signal is input to the corresponding layer corresponding to the signal.

The adder circuit 611 is a circuit for adding the signal given through the line 65 and the output of the coefficient integration circuit 630. The coefficient selection circuit 610 receives the reproduction value of the conversion coefficient from the adder circuit 611 and converts the resolution. According to the N-layer alpha map signal pyramid supplied from the circuit 640, the N-layer pyramid is obtained by selecting the transform coefficient and obtaining the corresponding transform coefficient of each of the first to N-th layers.

Also, the inverse orthogonal transform circuit 612 for each layer receives the corresponding transform coefficient of the corresponding transform coefficients of the first to Nth layers provided from the coefficient selecting circuit 610, and inversely transforms each transform coefficient. Then, the reproduction signal is restored to obtain the reproduction signal, and in this system, the output of the layer corresponding to the target resolution is used as the final reproduction signal.

The frame memory 613 of each layer obtains the output of the inverse orthogonal transform circuit of the self-corresponding layer of the inverse orthogonal transform circuit 612 of each layer, accumulates the output, and stores the frame image of the resolution corresponding to the layer. The motion compensation prediction circuit 514 obtains the image from the frame memory for the layer corresponding to each layer of the frame memories 613 for each layer and obtains the motion compensation prediction value of the image in the layer from this. The orthogonal transform circuit 615 is provided for each hierarchical layer, and the motion compensation prediction value of the corresponding hierarchical layer is orthogonally transformed, and among the orthogonally transformed transform coefficients, the transform coefficient at the maximum frequency term in the hierarchical layer is obtained. Is output.

The coefficient integrating circuit 630 integrates the transform coefficients output from the orthogonal transform circuits 615 of the respective layers and outputs the integrated coefficients to the adder circuit 611.

That is, the orthogonal transformation circuit 615 for each layer for the first to Nth layers is the motion compensation prediction circuit 61 for each layer.
4 is for receiving the motion-compensated prediction values generated by the respective corresponding layers of the four layers, and performing orthogonal transformation, and outputting the transform coefficient of the maximum frequency term in that layer. For example, the orthogonal transformation of the system for the first layer. In the case of the circuit 515 (OT ₁ ), the motion compensation prediction value in the frequency band (first low frequency term) of the DC component is set to the second value.
If it is the orthogonal transformation circuit 515 (OT ₂ ) of the hierarchy system, the motion compensation prediction value of the frequency band (second low frequency term) next to the DC component is converted to the orthogonal transformation circuit 515 of the hierarchy system of the third hierarchy.
If it is (OT ₃ ), the motion compensation prediction value of the next frequency band of DC components (third low frequency term) is the highest term if it is the orthogonal transformation circuit 515 (OT _N ) of the system for the Nth layer. The motion compensation prediction value in the frequency band (N-th frequency term) is output.

Then, the coefficient integrating circuit 630 receives the transform coefficients by the orthogonal transform of the motion compensation prediction values of the respective layers outputted from the orthogonal transform circuits 515, and integrates them for each band.
This is to provide × n transform coefficient prediction values to the adder circuit 611.

In such a configuration, the resolution conversion circuit 640 converts the locally decoded signal of the alpha map given through the line 92 to horizontal / vertical resolution n / N times and inversely orthogonally converts it as an n-layer pyramid signal. Circuit 612
It outputs to the orthogonal transformation circuit 615. Inverse orthogonal transform circuit 61
The two-orthogonal transformation circuit 615 is provided for each layer, and therefore, the resolution-converted signal is input to the corresponding layer corresponding to the signal.

On the other hand, the addition circuit 611 receives the signal given from the dequantization circuit 165 and the output of the coefficient integration circuit 630 via the line 65, and the addition circuit 611 adds both signals to obtain the reproduction value of the conversion coefficient. The coefficient selection circuit 610
Give to. The coefficient selection circuit 610 receives the reproduction value of the conversion coefficient from the addition circuit 611, selects the conversion coefficient in accordance with the N-layer alpha map signal pyramid supplied from the resolution conversion circuit 640, and selects each of the first to Nth layers. By obtaining the corresponding transform coefficient of, an N hierarchical pyramid is obtained. This N-layer pyramid is input to the corresponding layer of the inverse orthogonal transform circuit 612 for each layer.

That is, the inverse orthogonal transform circuit 612 for each layer.
Then, among the corresponding conversion coefficients of the first to Nth layers provided from the coefficient selection circuit 610, those of the corresponding layers are received, and the received conversion coefficients are inversely transformed to obtain a reproduction signal. In this system, the output of the layer corresponding to the target resolution is used as the final reproduction signal.

The output of the inverse orthogonal transform circuit 612 for each layer is
Further, it is input to the corresponding layer of the frame memory 613 provided for each layer. As a result, the frame memory 613 for each layer obtains and stores the output of the inverse orthogonal transform circuit of the self-corresponding layer of the inverse orthogonal transform circuits 612 of each layer, and stores the output. Get the image.

The motion compensation prediction circuit 514 for each layer obtains an image from the frame memory for the layer corresponding to each layer among the frame memories 613 for each layer and obtains the motion compensation prediction value of the image in that layer from this. . Then, this is input to the corresponding layer of the orthogonal transformation circuit 615 provided for each layer. The orthogonal transform circuit 615 for each layer orthogonally transforms the motion compensation prediction value of the corresponding layer, and the transform coefficient at the maximum frequency term in that layer among the transformed coefficients of this orthogonal transform is the coefficient integration circuit 630.
Output to

Then, the coefficient integrating circuit 630 integrates the transform coefficients output from the orthogonal transform circuit 615 of each layer and outputs the integrated transform coefficient to the adder circuit 611.

As described above, with regard to the configuration of FIG. 9B, the reproduced images up to the nth layer of the N layer pyramid are obtained by the same process as that of FIG. 8B. Then, if the resolution of the desired reproduced image is compatible with the nth layer, the output for the nth layer among the outputs of the inverse orthogonal transform circuit 612 for each layer is used as the reproduced signal.

[0214] Note that, as a technique that can be used for the reduction / enlargement conversion in the resolution conversion circuit 540 and the resolution conversion circuit 640, for example, "Onoe: Image Processing Handbook,
p. 630, Shokoido ”,“ binary image resolution conversion method ”may be used.

In the above third specific example, only the image of the image of interest can be coded at a desired resolution from the image, and at the reproducing side, an image at a resolution equal to or lower than this can be obtained. You will be able to get it.

(Fourth Specific Example) Next, a fourth specific example of the present invention will be described with reference to FIG. The fourth specific example is a technique that enables an image of an arbitrary shape to be encoded in the technique of the second specific example described with reference to FIG.

FIG. 10 shows an SN to which the fourth specific example is applied.
It is a block diagram which shows the structure of the encoding circuit part for implement | achieving R scalability. In the figure, 105 is an orthogonal transformation circuit, 180 is an alpha map coding circuit, 181 is a multiplexing circuit, 126, 127 and 128 are quantization circuits, 13
6, 137, 138 are variable length coding circuits, 500a, 5
00b, to 500M are local decoding circuits, 405 to 408 are delay circuits, 116, 117, 118, 415 and 416 are difference circuits, 146, 147 and 148 are inverse quantization circuits, 4
25 and 426 are adder circuits.

The alpha map encoding circuit 180 receives the alpha map information of the image as an input, encodes it and outputs it to the line 82, and has a function of decoding the encoded alpha map signal. It has a function of outputting the locally decoded signal of the alpha map signal decoded by this through the line 81.

The constituent elements of the first layer L1 having the local decoding circuit 500a are for obtaining the encoded signal of the base layer, and the constituent elements of the second layer L2 having the local decoding circuit 500b are the enhanced ones. This is for obtaining a coded signal of a layer, and a constituent element of the M-th layer LM having the local decoding circuit 500M is for obtaining a coded signal of an enhanced layer.

The orthogonal transform circuit 105 of FIG.
, And the locally decoded signal of the alpha map is supplied via line 81. Then, the orthogonal transformation circuit 105 orthogonally transforms the image signal based on the locally decoded signal of the alpha map.

The alpha map encoding circuit 180 of FIG.
Is input with the alpha map code via line 80,
On the other hand, the image signal is supplied to the orthogonal transformation circuit 105 via the line 10. Then, the alpha map encoding circuit 1
80 encodes this and outputs it to the multiplexing circuit 181, decodes the encoded alpha map, and gives it to the orthogonal transformation circuit 105 via the line 81.

The multiplexing circuit 181 multiplexes the alpha map encoded output from the alpha map encoding circuit 180 and the output from the variable length encoding circuit 136 and outputs the multiplexed result.

In the orthogonal transformation circuit 105, the image signal supplied via the line 10 is orthogonally transformed based on the locally decoded signal of the alpha map via the line 81, and the orthogonal transformation obtained by this orthogonal transformation is performed. The coefficients are calculated by using the difference circuit 116 of the first layer L1 and the delay circuits 405, 40 of the second layer L2.
6 to the delay circuits 407 and 408 of the Mth layer LM.

The differential circuit 1 in the first layer L1
In 16, the prediction error between the orthogonal transform coefficient supplied from the orthogonal transform circuit 105 and the predicted value of the transform coefficient supplied from the local decoding circuit 500 a via the line 26 is calculated and supplied to the quantization circuit 126. . And this quantization circuit 1
It is quantized at 26. The quantized prediction error signal is
It is supplied to the variable length coding circuit 136 and the dequantization circuit 146. In the variable length coding circuit 136, the quantized value of the prediction error signal is variable length coded and output via the line 36.

Further, the dequantization circuit 146 dequantizes the prediction error signal to obtain the reproduced value of the prediction error signal, and then supplies it via the line 46 to the local decoding circuit 500 and the second layer L2. Then, in the second layer, first, the delay circuit 40
At 6, the orthogonal transform coefficient supplied from the orthogonal transform circuit 105 continues until the reproduction value of the prediction error signal of the block in the first layer L1 is obtained via the line 46.
Delay the timing supplied to.

In the delay circuit 405, the delay circuit 40
Similarly to 6, after delaying the alpha map signal supplied via the line 81, it is supplied to the local decoding circuit 500 of the second layer L2 via the line 86.

The difference circuit 117 calculates a prediction error between the orthogonal transform coefficient supplied from the delay circuit 406 and the predicted value of the transform coefficient supplied from the local decoding circuit 500b through the line 27, and the difference circuit 415 calculates the prediction error. Supplied. And
In the difference circuit 415, the difference between the prediction error in the second layer L2 supplied from the difference circuit 117 and the reproduced value of the prediction error in the first layer L1 supplied via the line 46 is calculated and quantized. It is supplied to the circuit 127. Then, the quantizing circuit 127 quantizes this.

The difference between the prediction error signals quantized by the quantization circuit 127 is supplied to the variable length coding circuit 137 and the inverse quantization circuit 147.

In the variable length coding circuit 137, the quantized value of the difference between the prediction error signals is variable length coded and is output as the variable length coded signal of the second layer L2 via the line 37.

Further, the dequantization circuit 147 which has received the quantized output of the difference of the prediction error signal dequantizes it and returns it to the reproduced value of the difference of the prediction error signal, and then the addition circuit 425.
The reproduction value of the prediction error signal of the first layer L1 is added by the line 46 to obtain the reproduction value of the prediction error signal of the second layer. Then, the reproduced value of the prediction error signal of the second layer is supplied to the local decoding circuit 5 via the line 47.
Supply to 00b.

In the M-th layer LM, the output of orthogonal transform circuit 105 is first delayed by delay circuit 408 for a predetermined time. That is, the delay amount here is a delay time corresponding to the time when the reproduction value of the prediction error signal of the block in the (M-1) th layer LM-1 is obtained via the line 48, and is supplied from the orthogonal transform circuit 105. The timing until the selected orthogonal transform coefficient is supplied to the difference circuit 118 is delayed.

In the delay circuit 407, the delay circuit 40
After delaying the alpha map signal supplied via the line 81 as in the case of 8, it is supplied to the local decoding circuit 500M of the Mth layer LM via the line 87.

The difference circuit 118 calculates the prediction error between the orthogonal transform coefficient supplied from the delay circuit 408 and the predicted value of the transform coefficient supplied from the local decoding circuit 500M via the line 28, and the difference circuit 416 calculates the error. Supplied. And
The difference circuit 416 calculates the difference between the prediction error in the Mth layer LM supplied from the difference circuit 118 and the reproduction value of the prediction error in the M−1th layer LM−1 supplied via the line 48. And is supplied to the quantization circuit 128 where it is quantized.

The difference between the prediction error signals quantized by the quantization circuit 128 is supplied to the variable length coding circuit 138 and the dequantization circuit 148. In the variable length coding circuit 138, the quantized value of the difference of the prediction error signal is variable length coded,
It will be output via the line 38 as a variable length coded signal in the layer LM.

On the other hand, in the dequantization circuit 148, after the difference of the prediction error signal is dequantized to obtain the reproduction value of the difference of the prediction error signal, it is supplied to the adder circuit 426 via the line 48. The reproduction value of the prediction error signal of the M−1th layer is added to obtain the reproduction value of the prediction error signal of the Mth layer LM, and then the reproduction value is supplied to the local decoding circuit 500M via the line 49.

In this way, in the technique of the second specific example, it becomes possible to code an image of an arbitrary shape.

Next, the decoding device will be described.

FIG. 11 is a block diagram of an apparatus for decoding the coded signal in the fourth specific example. In the figure,
Reference numeral 190 is a separation circuit, 191 is an alpha map decoding circuit, 156, 157 and 158 are variable length decoding circuits, 16
6, 167 and 168 are inverse quantization circuits, 435 and 436 are addition circuits, and 600 is a decoding circuit.

The demultiplexing circuit 190 demultiplexes the multiplexed signal of the coded signal of the first layer and the coded signal of the alpha map multiplexed by the multiplexing circuit 181, and separates the coded signal of the first layer and the alpha signal. The alpha map decoding circuit 191 is for returning to the encoded signal of the map.
The encoded signal of the alpha map separated by 0 is decoded to obtain the original alpha map, and the variable length decoding circuit 156 decodes the encoded signal of the first layer separated by the separation circuit 190. Inverse quantization circuit 166
Is for dequantizing the decoded signal to return it to the original error value. The inverse quantization circuit 167.
Dequantizes this to return to the original error value for the second layer L2, and the variable length decoding circuit 158 encodes it with the variable length encoding circuit 138 of the mth layer Lm on the decoding device side. The inverse quantization circuit 168
Is to dequantize this and return it to the original error value for the m-th layer Lm.

The adder circuit 435 is for adding the original error value for the third layer L3 and the original error value for the second layer L2, and the adder circuit 436 is the output of the adder circuit 435 and the first error value. The original error value for the layer L1 is added.

The decoding circuit 600 decodes and outputs the reproduction signal of the image portion of interest from the output of the addition circuit 436 and the alpha map output from the alpha map decoding circuit 191.

In FIG. 11, the coded bit stream of the first layer L1 supplied to the demultiplexing circuit 190 via the line 90 is separated into a code relating to the alpha map and a code relating to the transform coefficient. It is output via 56. The encoded bit streams supplied to the variable length decoding circuits 156, 157, 158 via the lines 56, 57, 58 are decoded into the prediction error signal or the difference between the prediction error signals, and then the dequantization circuit 166. 167,1
68 respectively.

The dequantization circuits 167 and 168 dequantize the difference between the prediction error signals to obtain the reproduced value of the difference between the prediction error signals. Then, the adder circuit 435 adds the reproduced values of the differences in the prediction errors from the m-th layer Lm to the second layer L2 and supplies them to the adder circuit 436. The dequantization circuit 166 for the first layer L1 dequantizes the prediction error signal of the first layer L1 to obtain a reproduction value of the prediction error signal, and then supplies it to the adder circuit 436 where the mth layer. The reproduction values of the prediction error signal for Lm to the second layer L2 are added. From the m-th layer Lm obtained by the adder circuit 436 to the first-layer L
The sum of the reproduced values of the prediction error signal up to 1 is supplied to the decoding circuit 600 via the line 65.

Then, the decoding circuit 600 obtains the reproduction signal of the image of the target image portion based on the sum of these reproduction values and the alpha map.

In this way, an image of arbitrary shape can be coded and decoded.

(Fifth Concrete Example) A fifth concrete example of the present invention will be described with reference to FIGS. 12, 13 and 14. The fifth specific example is a technique for improving the coding efficiency of the m-th layer.

This example is the same as the second example and the fourth example.
In the specific example of, the prediction signal in the m-th layer is
The coding efficiency of the m-th layer is improved by adaptively switching between the decoded signal of the hierarchical layer and the motion-compensated prediction signal of the m-th layer.

In the following, an example in which the present specific example is applied to the second specific example in the case of two layers of a base layer and an enhancement layer will be shown. The same applies to the fourth specific example.

<< Example of Configuration of Encoding Device in Fifth Specific Example >> FIG. 12 is a block diagram of an encoding device of the present invention. This encoding device includes an orthogonal transformation circuit 100, local decoding circuits 200 and 700, a delay circuit 409, and a difference circuit 11.
0 and 119, quantization circuits 120 and 129, variable length coding circuits 130 and 139, dequantization circuit 140
And 149.

The local decoding circuit 700 includes an adder circuit 701, an inverse orthogonal transform circuit (IOT _N ), a frame memory 703.
(FM _N ), a motion compensation prediction circuit 704 (MC _N ), an orthogonal transformation circuit 705 (OT _N ), and a selector 706.

In the orthogonal transform circuit 100, the image signal supplied through the line 10 is orthogonally transformed for each N × N pixel, and N × N transform coefficients are obtained. The base layer has the same configuration as in the first and third concrete examples, and the locally decoded signal 2
00, the reproduced signal of the transform coefficient of the block which is the output signal of the adder circuit 201 and the quantized value of the motion compensation prediction error signal of the transform coefficient of the block which is the output of the quantization circuit 120 are respectively the line BD and the line BD. It is supplied to the enhancement layer via PQ.

In the enhancement layer, in the delay circuit 409 in the layer, the orthogonal transform coefficient supplied from the orthogonal transform circuit 100 corresponds to the difference circuit 119 for the time until the reproduction signal of the block is obtained via the line BD.
Delay the timing supplied to.

In the difference circuit 119, the orthogonal transformation circuit 100
A prediction error between the orthogonal transform coefficient supplied from the local decoding circuit 700 and the predicted values of the N × N transform coefficients supplied from the local decoding circuit 700 via the line 29 is calculated and supplied to the quantization circuit 129. The prediction error signal quantized by the quantization circuit 129 is supplied to the variable length coding circuit 139 and the inverse quantization circuit 149.

In the variable length coding circuit 139, the quantized value of the prediction error signal is variable length coded and output via the line 39. The dequantization circuit 149 supplies the reproduced value of the prediction error signal obtained by dequantizing the prediction error signal to the local decoding circuit 700.

In the local decoding circuit 700, the inverse quantization circuit 1
The reproduction value of the prediction error signal supplied from S.49 and the prediction value supplied via the line 29 are added by the adder circuit 701 to obtain the reproduction value of the transform coefficient, which is supplied to the inverse orthogonal transform circuit 702. Supply.

In the inverse orthogonal transformation circuit 702, the addition circuit 701 is used.
The supplied transform coefficient is inversely transformed and a locally decoded signal is output. Then, the frame memory 703 accumulates the locally decoded signal for each N × N pixel supplied from the inverse orthogonal transform circuit 702 to obtain a locally decoded image. Motion compensation prediction circuit 7
In 04, a motion-compensated prediction value is generated using the locally decoded image signal stored in the frame memory 703, and is supplied to the orthogonal transform circuit 705.

The orthogonal transform circuit 705 orthogonally transforms the motion compensation prediction value for each N × N pixel, and outputs the transform coefficient to the selector 706 via the line EMC. The selector 706 adaptively switches the transform coefficient supplied via the line BD and the line EMC according to the quantized value of the transform coefficient of the motion compensation prediction error signal in the base layer supplied via the line PQ. .

FIG. 13 shows a document (TK Tan et. Al. “A Frequ” applied to the selector 706.
engy Scalable Coding Sche
meEmploying Pyramid and S
UBUBAN TECHNIQUES ”, IEEE T
rans. CAS for Video Techno
logic, Vol. 4, No. 2, Apr. 1994)
It is an example of the switching means described in.

In FIG. 13, PQ is the quantization circuit 120.
, BD is an adder circuit 2 in the local decoding circuit 200.
The output of 01, EMC is the output of the orthogonal transformation circuit 705 in the local decoding circuit 700, and the output P of the quantization circuit 120.
Among the quantized values that are Q, the coefficients that are not “0” (enclosed by white circles) are the coefficients that motion compensation prediction did not hit. Here, since the motion-compensated prediction circuit 704 performs motion-compensated prediction using the same motion vector as that of the base layer, motion-compensated prediction with the same coefficient does not apply to the enhanced layer.

On the other hand, if the coding of the base layer is completed before the coding of the enhancement layer, the reproduction signal of the base layer can be used. Therefore, in the quantized value of the output PQ in FIG. 13, the coefficient surrounded by the white circle causes the reproduction signal of the base layer to be selected by the selector 706 and output via the line 29. Incidentally, the point that the selector 706 is switched for each coefficient by using the output PQ is the same as the above-mentioned document. However, this specific example is different in that the reproduction of the base layer is used as the prediction value.

<< Structural Example of Decoding Device in Fifth Specific Example >> FIG. 14 is a diagram for obtaining a reproduced image by decoding the bit stream coded by being divided into two layers in the encoding device of FIG. It is a block diagram of a decoding device. This decoding device includes variable length decoding circuits 150 and 159, dequantization circuits 160 and 169, and decoding circuits 300 and 80.
0.

The enhancement layer decoding circuit 800 includes an adder circuit 801, an inverse orthogonal transform circuit 802, a frame memory 803, a motion compensation prediction circuit 804, and an orthogonal transform circuit 80.
5, and a selector 806.

In FIG. 14, the base layer has the same structure as the first and third concrete examples, and the reproduction signal BD of the conversion coefficient of the block which is the output signal of the adder circuit 301 and the variable length decoding circuit 150. The quantized value PQ of the motion compensation prediction error signal of the transform coefficient of the block, which is the output, is supplied to the selector 806 of the enhancement layer.

In the enhancement layer, the coded bit stream supplied to the variable length decoding circuit 159 via the line 59 is decoded into a prediction error signal and then supplied to the inverse quantization circuit 169. The dequantization circuit 169 dequantizes the prediction error signal to obtain a reproduced value of the prediction error signal, and then supplies it to the decoding circuit 800 via the line 69.

In the decoding circuit 800, the reproduction value of the conversion coefficient is obtained by adding the reproduction value of the prediction error signal supplied via the line 69 and the prediction value supplied from the selector 806 in the addition circuit 801. Then, it is supplied to the inverse orthogonal transform circuit 802. Then, the inverse orthogonal transform circuit 802 inversely transforms the transform coefficient supplied from the adder circuit 801, and outputs the decoded signal via the line 79.

In the frame memory 803, the decoded signal for each N × N pixel supplied from the inverse orthogonal transform circuit 802 is accumulated to obtain a decoded image. The motion compensation prediction circuit 804 uses the decoded image signal stored in the frame memory 803 to generate a motion compensation prediction value and supplies it to the orthogonal transform circuit 805.

The orthogonal transform circuit 805 orthogonally transforms the compensation prediction value for every N × N pixels, and outputs the transform coefficient via the line EMC. In the selector 806, the reproduced signal BD and the transform coefficient EMC output from the orthogonal transform circuit 805 are used as the quantized value P of the transform coefficient of the motion compensation prediction error signal in the base layer.
It is adaptively switched according to Q (output of the variable length decoding circuit 150). Here, the selector 806 is the selector 7
The same operation as 06 is performed.

As described above, in the present concrete example, in the second concrete example and the fourth concrete example, the prediction signal in the m-th layer is the decoded signal in the m-1 th layer and the motion-compensated prediction signal in the m-th layer. Are obtained by adaptively switching between and, and by this, it becomes possible to improve the coding efficiency of the m-th layer.

In the above specific example, an example in which the conversion bases do not overlap between blocks has been shown.

On the other hand, “Reference: Nyozawa et al., Image Coding Using Motion Compensation Filter Bank Structure, PCSJ92, 8-
5, 1992 "proposes an encoding method using a motion compensation filter bank structure in which a reduction in encoding efficiency is small by adopting a differential structure after conversion even when bases overlap. As described above, since the idea of the above-mentioned document can be applied to the predictive coding apparatus (differential structure after conversion) in the orthogonal transform coefficient domain, even if the motion compensation filter bank structure is applied to the first to fifth specific examples. good.

Although various examples have been described above, the present invention is a scalable coding method capable of varying resolution and image quality in multiple hierarchies without causing deterioration of image quality due to drift and significant reduction of coding efficiency. The purpose of the present invention is to provide a moving picture coding / decoding device, and N × N (N:
N × n (n = 1 to N) locally decoded transform coefficients are selected from the low frequency band in the motion compensation prediction + transform coding using the motion compensation prediction in the transform coefficient region for each transform coefficient (natural number). As a result, an N-layer conversion coefficient pyramid is created, and the N-layer conversion coefficient pyramid is inversely converted for each layer to create an N-layer reproduction image pyramid. Each frame is accumulated to obtain each frame image, and with reference to each frame image, a motion compensation prediction signal is created for each layer, and the motion compensation prediction signal is converted into a transform coefficient for each layer, The motion-compensated prediction value is created by extracting the highest-order transform coefficients in each layer and integrating them. Then, this is encoded.

Decoding is performed by extracting, from the transform coefficients obtained by decoding, lower-order transform coefficients including the highest-order transform coefficient in the hierarchy corresponding to the required resolution and lower, and inversely transforming them. Thus, the motion compensation prediction value in the layer corresponding to the required resolution is obtained and used as the reproduction signal.

Therefore, even when decoding is performed at an arbitrary resolution lower than the resolution on the encoding side, a scalable encoding method capable of varying resolution and image quality in multiple layers without causing a mismatch Therefore, it is possible to obtain a moving image coding / decoding device without deterioration of image quality due to drift and significant reduction of coding efficiency.

[0274]

As described above, according to the present invention, the scalable coding in which the resolution and the image quality of an arbitrarily shaped image can be changed in multiple stages is realized without the influence of drift and the deterioration of the coding efficiency.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the present invention, showing an example of an image transmission system to which an image encoding device and an image decoding device according to the present invention are applied.

[Fig. 2] Fig. 2 is a diagram for explaining the present invention and is a block diagram showing a configuration example of an encoding device in a first concrete example of the present invention.

FIG. 3 is a diagram for explaining the present invention and is a diagram for explaining a local decoding circuit in the first example of the present invention.

FIG. 4 is a diagram for explaining the present invention and is a block diagram showing a configuration example of a decoding device in the first example of the present invention.

FIG. 5 is a diagram for explaining the present invention and is a block diagram showing a configuration example of a second specific example of the present invention.

6 is a block diagram of a decoding device that decodes bitstreams up to the m-th layer from the bitstreams that have been divided into M layers and encoded by the encoding device in FIG. 5 to obtain a reproduced image.

FIG. 7 is a diagram for explaining scalability.

[Fig. 8] Fig. 8 is a diagram for explaining the present invention and is a block diagram showing a configuration example of an encoding device in a third concrete example of the present invention.

FIG. 9 is a diagram for explaining the present invention and is a block diagram showing a configuration example of a decoding device in a third example of the present invention.

FIG. 10 is a diagram for explaining the present invention and is a block diagram showing a configuration of an encoding circuit unit in a fourth example of the present invention.

FIG. 11 is a diagram for explaining the present invention and is a block diagram showing a configuration example of a decoding circuit unit in a fourth concrete example of the present invention.

[Fig. 12] Fig. 12 is a diagram for explaining the present invention and is a block diagram showing a configuration example of an encoding device in a fifth concrete example of the present invention.

FIG. 13 is a diagram for explaining the present invention, which is a diagram for explaining a prediction value switching method in the fifth example of the present invention.

FIG. 14 is a diagram for explaining the present invention, which is a block diagram showing a configuration example of a decoding device in a fifth example of the present invention.

FIG. 15 is a diagram for explaining a conventional technique, in which MP
Block diagram of SNR scalability of EG2.

FIG. 16 is a diagram for explaining a conventional technique,
Block diagram of the spatial scalability of EG2.

FIG. 17 is a diagram illustrating an alpha map.

FIG. 18 is a diagram illustrating orthogonal transformation of an arbitrarily shaped image, which is a prior art.

FIG. 19 is a diagram for explaining resolution conversion of an arbitrarily shaped image which is a prior art.

[Explanation of symbols]

100, 105, 205, 305, 505, 605, 7
05,805 ... Orthogonal transformation circuit 110-113,115-119,410,411,4
15, 416 ... Difference circuit 120 to 123, 125 to 129 ... Quantization circuit 130 to 133, 135 to 139 ... Variable length coding circuit 140 to 149, 160 to 169 ... Inverse quantization circuit 150 to 153, 155 to 159 ... Variable length decoding circuit 180 ... Alpha map encoding circuit 181 ... Multiplexing circuit 190 ... Separation circuit 191 ... Alpha map decoding circuit 200, 200a to 200M, 500, 500a to 50
0M, 700 ... Local decoding circuit 300, 600, 800 ... Decoding circuit 201, 211, 301, 311, 420, 421, 4
25,426,430,431,435,436,50
1, 511, 601, 611, 701, 801 ... Addition circuit 202, 302, 502, 602, 702, 802 ... Inverse orthogonal transformation circuit 203, 303, 503, 603, 703, 803 ... Frame memory 204, 304, 504 604, 704, 804 ... Motion compensation prediction circuit 212, 312, 512, 612 ... Inverse orthogonal transformation circuit pyramid 213, 313, 513, 613 ... Frame memory pyramid 214, 314, 514, 614 ... Motion compensation prediction circuit pyramid 215, 315 , 515, 615 ... Orthogonal transformation circuit pyramid 220, 320, 520, 620 ... Coefficient selection circuit 230, 330, 530, 630 ... Coefficient integration circuit 400, 401, 405, 406, 407, 408 ... Delay circuit.

Claims (14)

[Claims] 1. N × N pieces (N: natural number) by orthogonal transformation
A prediction error signal obtained by converting the image signal into N × N conversion coefficients by orthogonal transformation using the motion-compensated prediction value in the conversion coefficient region for each conversion coefficient and correcting the motion-compensated prediction value. In a moving picture coding apparatus for obtaining a bit stream by coding the same, the locally decoded transform coefficients are n × n (n = 1 to 1) from the low band.
N) means for creating at least two transform coefficient pyramids among the N hierarchies from the first hierarchy to the N th hierarchy by selecting, and performing inverse transformation for each of the transformation coefficient pyramids of the N hierarchies With reference to the means for creating the N-layer reproduced image pyramid, the means for accumulating the N-layer reproduced image pyramid for each layer, and the image for each layer accumulated in the accumulating means, each layer is referred to. A unit for creating a motion compensation prediction signal for each layer, a unit for converting the motion compensation prediction signal into a transform coefficient for each layer, and a unit for obtaining the transform coefficient of the highest-order area in each layer for each layer, Means for obtaining the motion-compensated prediction value by integrating the transform coefficients of the highest-order area in each layer. 2. A decoding device for decoding the coded bitstream obtained by the coding device according to claim 1, wherein the decoding device decodes the coded bitstream and regenerates it into transform coefficients, and the regenerated transform. Means for obtaining a conversion coefficient in which a motion compensation prediction value is corrected for the coefficient, and conversion coefficients in which the motion compensation prediction value is corrected for each of the first to nth layers (n = 1 to N) A means for creating a conversion coefficient pyramid of n layers by extracting the codes corresponding to the layers and a conversion coefficient pyramid of the n layers is subjected to inverse conversion for each layer to create a reproduced image pyramid of n layers. , Means for making the reproduced image information of the n-th layer a target reproduced image, means for accumulating an n-layer reproduced image pyramid for each layer, and referring to the images accumulated in the accumulating means, Means for creating a motion-compensated prediction signal for each layer, means for converting the motion-compensated prediction signal into transform coefficients for each layer, and obtaining, for each layer, a transform coefficient of the highest-order area in that layer, An image decoding apparatus for reproducing a reproduced image of the n-th layer, which has means for obtaining the motion-compensated prediction value by integrating the obtained conversion coefficients of the highest-order areas. 3. An M using the encoding device according to claim 1.
An encoding device that realizes SNR scalability of a layer (M: natural number), comprising a prediction error signal of the m-th layer (m = 2 to N) and a local reproduction value of the prediction error signal of the (m-1) -th layer. A means for obtaining a difference signal; an encoding means for encoding the difference signal to obtain an m-th layer encoded bit stream; and, in the m-th layer, the difference signal is smaller than the quantization step size of the (m-1) -th layer. Means for quantizing with a step size; adding means for obtaining a local reproduction value of the prediction error signal of the m-th layer by adding a local reproduction value of the prediction error signal of the (m-1) th layer; And a local decoding means for accumulating the local reproduction value to obtain an image, obtaining a motion compensation value of the m-th layer from the image, and obtaining a prediction error signal for the m-th layer. apparatus. 4. A decoding device for decoding an encoded bitstream encoded by the encoding device according to claim 3, wherein among the encoded bitstreams, first to m-th layers (m = 1 to M). Decoding means for obtaining the quantized value for each layer up to the m-th layer, and the quantized value for each layer decoded by this decoding means are respectively dequantized. Inverse quantizing means, means for adding and outputting the dequantized values up to the m-th layer by the dequantizing means, and decoding means for decoding the added output and providing it for image reproduction Video decoding device. 5. A motion compensation prediction value in a transform coefficient region is used for each of N × N transform coefficients by orthogonal transform, the image signal is transformed into N × N transform coefficients by orthogonal transform, and the motion is converted into the transform coefficient. Obtain a prediction error signal that has been compensated for the compensation prediction value, and encode it to obtain a bit stream.In a device that uses moving picture coding, the alpha map signal that identifies the background and object of the input image is received and A unit for encoding, a unit for transforming an image of an arbitrary shape into a transform coefficient by orthogonally transforming an image of a corresponding region according to the alpha map in the input image, and outputting the image according to the alpha map. A moving picture coding / decoding apparatus having means for reproducing an arbitrarily shaped image by inversely transforming a transform coefficient. 6. The apparatus according to claim 5, wherein means for converting the resolution of the input alpha map signal to create an alpha map signal pyramid of N layers, and local decoding for each hierarchy according to the alpha map signal. A conversion coefficient pyramid for N layers is created by selecting the conversion coefficients for n layers (n = 1 to N) from the low range, and the conversion coefficient pyramid for the N layers is converted into an alpha map signal for each layer. Therefore, by performing an inverse transformation, a unit for creating a reproduced image pyramid of N layers, a unit for accumulating the reproduced image pyramid of N layers for each layer, and an image accumulated in the accumulating unit are referred to, Means for creating a motion compensation prediction signal according to an alpha map signal for each layer, and the motion compensation prediction signal for each layer being an alpha map signal And a means for converting the conversion coefficient into a conversion coefficient according to the alpha map signal pyramid to generate a motion compensation prediction value. 7. A motion compensation prediction value in a transform coefficient region is used for each of N × N transform coefficients by orthogonal transform, the image signal is transformed into N × N transform coefficients by orthogonal transform, and the motion is converted into the transform signal. A device that uses a moving image encoding that obtains a prediction error signal that has been corrected by a compensation prediction value and encodes this to obtain a bit stream, which receives an alpha map signal that identifies an object and a background of an input image. This is encoded, and the image of the corresponding region according to the alpha map of the input image is orthogonally transformed to transform the arbitrary shape image into a transformation coefficient, and the transformation coefficient and the alpha map are converted. As a decoding device for decoding the bitstream encoded by the encoding means that is encoded and output as a bitstream, a bitstream is provided. Means for decoding an alpha map from a program, means for decoding a transform coefficient from a bitstream, means for transforming an arbitrary shape image into a transform coefficient according to the alpha map, and inverse the transform coefficient according to the alpha map. An image decoding apparatus having a unit for reproducing an image of an arbitrary shape by conversion. 8. An image decoding apparatus for decoding a coded bitstream coded by the coding apparatus according to claim 6, wherein the nth layer (n = 1 to 1) from the coded bitstream.
N) means for extracting codes up to N), means for decoding an alpha map signal from the coded bit stream, means for converting the resolution of the decoded alpha map signal to create an N layer alpha map signal pyramid, Means for creating a conversion coefficient pyramid of n layers from the converted conversion coefficients according to an alpha map signal pyramid; and an inverse conversion of the conversion coefficient pyramid of the n layers according to an alpha map signal for each of the n layers. Of the reproduced image pyramid, the means for accumulating the reproduced image pyramid of the n layers for each layer, the image accumulated in the accumulating unit, and moving in accordance with the alpha map signal for each layer. Means for creating a compensation prediction signal, and the motion compensation prediction signal for each layer by an alpha map. And a unit for converting the conversion coefficient according to the alpha map signal pyramid to create a motion compensation prediction value according to the alpha map signal pyramid. A video decoding device characterized by: 9. An M using the encoding device according to claim 5.
An encoding device that realizes SNR scalability of a layer (M: natural number), comprising a prediction error signal of the m-th layer (m = 2 to N) and a local reproduction value of the prediction error signal of the (m-1) -th layer. A means for obtaining a difference signal; a means for quantizing the difference signal with a step size smaller than a quantization step size of the m-1 th layer in the m-th layer; the dequantized difference signal; A moving picture coding apparatus, wherein the local reproduction value of the prediction error signal of the m-th layer is obtained by adding the local reproduction value of the prediction error signal of the -1st layer. 10. A decoding device for decoding an encoded bitstream encoded by the encoding device according to claim 9, wherein the m-th layer (m = 1 to 1) from the encoded bitstream.
M) means for extracting codes up to M), decoding means for decoding the extracted codes for each layer up to the m-th layer, and means for inversely quantizing the quantized value decoded by the decoding means in each layer, The moving picture decoding apparatus according to claim 7, further comprising means for adding dequantized values up to the m-th layer. 11. An M using the encoding device according to claim 6.
An encoding device that realizes SNR scalability of a layer (M: natural number), comprising a prediction error signal of the m-th layer (m = 2 to N) and a local reproduction value of the prediction error signal of the (m-1) -th layer. A means for obtaining a difference signal; a means for quantizing the difference signal with a step size smaller than a quantization step size of the m-1 th layer in the m-th layer; the dequantized difference signal; A moving picture coding apparatus, wherein the local reproduction value of the prediction error signal of the m-th layer is obtained by adding the local reproduction value of the prediction error signal of the -1st layer. 12. A decoding device for decoding a coded bitstream coded by the coding device according to claim 11, a unit for decoding an alphamap signal from the coded bitstream, and a decoded alphamap. Means for converting the resolution of the signal to create an N-layer alpha map signal pyramid; means for creating an n-layer transform coefficient pyramid from the decoded transform coefficients according to the alpha map signal pyramid; M-th layer (m = 1 to 1
M) for extracting codes up to M), decoding means for decoding the extracted codes for each layer up to the m-th layer according to the alpha map signal of the corresponding layer of the alpha map signal pyramid, and the decoding means for decoding Dequantizing means for dequantizing the quantized values in each layer, means for adding the dequantized values up to the m-th layer dequantized by the dequantizing means, and decoding the addition output And a decoding means for reproducing the reproduced image of the m-th layer by doing so. 13. A motion compensation prediction in a transform coefficient region is used for each N × N transform coefficients by orthogonal transform, and an image signal is converted into N × N transform coefficients by orthogonal transform and the motion compensation is performed on the image signal. A coding device for a moving picture, which obtains a prediction error signal with a correction for a prediction value and codes the coding error signal to obtain a bit stream, is a coding device which realizes SNR scalability of M layers, and is an m-th layer ( (m = 2 to M) and a means for obtaining a prediction value of the m-th layer by switching the motion compensation prediction value and the local reproduction value of the (m-1) -th layer for each conversion coefficient, The transform coefficient whose absolute value of the quantized value of the error signal is less than or equal to the threshold value outputs the motion compensation prediction value of the m-th layer, and the transform coefficient above the threshold value outputs the local reproduction value of the (m-1) -th layer. Moving image code characterized by having a selector Apparatus. 14. A decoding device for decoding a coded bitstream coded by the coding device according to claim 13, wherein the m-th layer (m = 2 to m) from the coded bitstream.
M) for extracting codes up to M), a decoding unit for decoding the extracted codes for each layer up to the m-th layer, and a quantized value of the prediction error signal decoded by the decoding unit for inverse quantization in each layer. Means, a means for obtaining the predicted value of the m-th layer by switching the motion-compensated predicted value of the m-th layer and the reproduction value of the m-1st layer for each conversion coefficient, and prediction in the m-1st layer. A selector that outputs a motion-compensated prediction value for the m-th layer when the absolute value of the quantized value of the error signal is less than or equal to the threshold value, and a reproduction value for the m-1th layer when the absolute value of the quantized value is less than or equal to the threshold value A moving picture decoding apparatus comprising: