RU2384970C1 - Interlayer forcasting method for video signal - Google Patents

Abstract

FIELD: physics, cinematography.

SUBSTANCE: invention relates to a method for interlayer forecasting when coding/decoding a video signal. The result is achieved due to that the method provides construction of pairs of frame macroblocks from one field macroblock or two vertically adjacent field macroblocks of a base layer and use of texture information of the constructed pair of frame macroblocks in the interlayer forecasting of the texture of pairs of frame macroblocks of the current layer.

EFFECT: design of an interpolation method which takes into account video signal layer scanning modes.

8 cl, 67 dwg

Description

FIELD OF THE INVENTION

The present invention relates to an interlayer prediction method for encoding / decoding a video signal.

State of the art

Scalable video codec (SVC) encodes the video into a sequence of images with the highest possible visual quality and at the same time provides the ability to decode part of the sequence of encoded images (in particular, a partial sequence of frames selected intermittently from the full sequence of frames) and use it to represent video with low visual quality .

Despite the fact that it is possible to present video with low visual quality by receiving and processing part of a sequence of images encoded in a scalable scheme, the problem remains that the visual quality is significantly degraded if the bit rate is reduced (bitrate). One solution to this problem is to provide an auxiliary sequence of images for low bitrates, for example, a sequence of images having a small screen size and / or low frame rate, in the form of at least one layer in a hierarchical structure.

When it is assumed that two sequences are provided, the auxiliary (junior) image sequence is called the base layer, and the main (senior) image sequence is called the enhanced layer or layer of improved quality. The video signals of the base and advanced layers are redundant, since the same video source is encoded in two layers. To improve the coding efficiency of the enhanced layer, the video signal of the enhanced layer is encoded using encoded information (motion information or texture information) of the base layer.

Although a single video source 1 can be encoded into multiple layers with different transfer rates, as shown in FIG. 1a, many video sources 2b in different scan modes that contain the same content 2a can be encoded into respective layers, as shown in FIG. 1b. And in this case, an encoder that encodes the upper layer can improve the coding efficiency by performing inter-layer prediction using the lower layer encoded information, since both sources 2b provide the same content 2a.

Thus, it is necessary to provide a method for interlayer prediction, taking into account the scanning modes of the video signals, when encoding different sources in the corresponding layers. When an interlaced video is encoded, it can be encoded into even and odd fields and can also be encoded into pairs of odd and even macroblocks in one frame. Accordingly, image types for encoding an interlaced video signal should also be taken into account in interlayer prediction.

In general, the enhanced layer provides images with a resolution higher than that of the images of the base layer. Accordingly, if the images of the layers have different resolutions, when different sources are encoded into the corresponding layers, then it is also necessary to perform interpolation to increase the resolution of the image (i.e., the size of the image). Since the closer the video objects in the images of the base layer are for use in interlayer prediction to the video objects in the images of the improved layer for prediction encoding, the higher the encoding speed, it is necessary to create an interpolation method that takes into account the scanning modes of the video signals of the layers.

SUMMARY OF THE INVENTION

The aim of the present invention is to provide a method for performing inter-layer prediction, provided that at least one of the two layers contains interlaced components of the video signal.

Another objective of the present invention is to provide a method for performing interlayer motion prediction in layers containing images with different spatial resolutions (scalability) depending on image types.

Another objective of the present invention is to provide a method for performing interlayer texture prediction in layers containing images with different spatial and / or temporal resolutions (scalability).

One interlayer motion prediction method in accordance with the present invention comprises the steps of establishing intra-mode macroblock (intra-frame coding) motion-related information from motion-related inter-mode macroblock information (inter-frame coding), while the macroblocks intra-mode and inter-mode are two vertically adjacent macroblocks of the base layer; and then, motion information of a pair of vertically adjacent macroblocks is obtained for use in interlayer motion prediction based on these two vertically adjacent macroblocks.

Another interlayer motion prediction method in accordance with the present invention comprises the steps of installing an intra-mode macroblock, which is one of two vertically adjacent macro-blocks of intra-mode and inter-mode of the base layer, according to an inter-mode block containing zero traffic related information; and then, motion information of a pair of vertically adjacent macroblocks is obtained for use in interlayer motion prediction based on these two vertically adjacent macroblocks.

Another interlayer motion prediction method in accordance with the present invention comprises the steps of: deriving motion information of one macroblock from motion information of a pair of vertically adjacent macroblocks of a base layer frame; and using the derived motion information as the prediction information in the motion information of the macroblock field in the current layer or the corresponding motion information of the pair of macroblock fields in the current layer.

Another interlayer motion prediction method in accordance with the present invention comprises the steps of deriving relevant information about the movement of two macroblocks from information about the movement of one macroblock of a field of the base layer or information about the movement of one macroblock of a field selected from a pair of vertically adjacent macroblocks of the fields of the base layer; and using the derived corresponding motion information as the prediction information in the corresponding motion information of the pair of macroblocks of the frame of the current layer.

One interlayer motion prediction method for layers with images with different resolutions in accordance with the present invention comprises the steps of converting an image of a minor layer into a frame image with the same resolution by selectively applying predictive methods to convert to macroblock frames according to types images and types of macroblocks in images; upsampling (oversampling) with respect to the frame image is performed to obtain the same resolution as the resolution of the senior layer; and then an interlayer prediction method is applied that is suitable for types of macroblock frames in an oversampled frame image and types of macroblocks in a senior layer image.

Another interlayer motion prediction method for layers with images with different resolutions in accordance with the present invention comprises the steps of recognizing the types of images of the lower and upper layers and / or the types of macroblocks contained in the images; apply a method for predicting a pair of macroblocks of a frame from one macroblock of the field to the image of the younger layer in accordance with the recognition results in order to build a virtual image having the same aspect ratio as the aspect ratio of the image of the senior layer; oversampling the virtual image; and then an interlayer motion prediction is applied to the senior layer using an oversampling virtual image.

Another interlayer motion prediction method for layers with images with different resolutions in accordance with the present invention comprises the steps of recognizing the types of images of the lower and upper layers and / or the types of macroblocks contained in the images; apply a method for predicting a pair of macroblocks of a frame from one macroblock of the field to the image of the younger layer in accordance with the recognition results in order to build a virtual image having the same aspect ratio as the aspect ratio of the image of the senior layer; and apply interlayer motion prediction to the image of the senior layer using the constructed virtual image.

Another interlayer motion prediction method for layers with images with different resolutions in accordance with the present invention comprises the steps of recognizing the types of images of the lower and upper layers; copy information about the movement of the block in the image of the lower layer to build a virtual image if the image type of the lower layer is interlaced and the image type of the upper layer is progressive; oversampling the virtual image; and applying a method for predicting motion between macroblock frames of an oversampled virtual image and an older layer image.

Another interlayer motion prediction method for layers with images with different resolutions in accordance with the present invention comprises the steps of recognizing the types of images of the lower and upper layers; copy the motion information of the block of the lower layer to build a virtual image if the image type of the lower layer is interlaced and the image type of the upper layer is progressive; and using a virtual image to apply interlayer motion prediction to an older layer image.

In an embodiment of the present invention, interlayer motion prediction sequentially predicts partitioning modes, reference indices, and motion vectors.

In another embodiment of the present invention, reference indices, motion vectors, and partition modes are sequentially predicted.

In another embodiment of the present invention, motion information of a pair of macroblock fields of a virtual base layer to be used for inter-layer motion prediction is derived from motion information of a pair of macroblocks of a frame of the base layer.

In another embodiment of the present invention, motion information of a field macroblock in an even or odd field image of a virtual base layer to be used for interlayer motion prediction is derived from the motion information of a pair of macroblocks of a base layer frame.

In another embodiment of the present invention, a macroblock is selected from a pair of macroblock fields of the base layer fields, and motion information of the pair of macroblock fields of the virtual base layer to be used for interlayer motion prediction is derived from the motion information of the selected macroblock.

In another embodiment of the present invention, motion information of a pair of macroblocks of a virtual base layer frame to be used for inter-layer motion prediction is derived from motion information of a field macroblock in an even or odd field image of the base layer.

In another embodiment of the present invention, the information of the field macroblock in the even or odd field image of the base layer is copied to further construct a virtual field macroblock, and the motion information of the pair of macroblocks of the virtual base layer frame to be used for interlayer motion prediction is derived from the pair motion information macroblocks of fields constructed in the aforementioned manner.

One interlayer texture prediction method in accordance with the present invention comprises the steps of constructing a pair of field macroblocks from a pair of vertically adjacent macroblocks of a base layer frame; and use the corresponding texture information of the constructed pair of field macroblocks as the corresponding texture prediction information of the field macroblock pair of fields of the current layer.

Another interlayer texture prediction method in accordance with the present invention comprises the steps of constructing one field macroblock from a pair of vertically adjacent macroblocks of a base layer frame; and use the texture information of the constructed one field macroblock as the field macroblock texture prediction information of the current layer.

Another interlayer texture prediction method in accordance with the present invention comprises the steps of constructing a pair of macroblock frames from one macroblock field or a pair of vertically adjacent macroblock fields of the base layer; and use the corresponding texture information of the constructed pair of macroblock frames as the corresponding texture prediction information of the pair of macroblock frames of the current layer.

Another interlayer texture prediction method in accordance with the present invention comprises the steps of constructing N pairs of macroblock frames from a pair of vertically adjacent macroblock fields of the base layer fields, wherein N is an integer greater than 1; and use the corresponding texture information of the constructed N pairs of macroblock frames as the corresponding texture prediction information of N pairs of macroblock frames located at different time positions in the current layer.

Another interlayer texture prediction method in accordance with the present invention comprises the steps of dividing each frame of the younger layer into multiple field images so that the younger layer can have the same temporal resolution as the temporal resolution of the older layer; oversampling each of the split field images in the vertical direction to stretch each of the split field images in the vertical direction; and then each of the oversampling field images is used in the interlayer texture prediction of each frame of the senior layer.

Another interlayer texture prediction method in accordance with the present invention comprises the steps of oversampling each field image of a younger layer in a vertical direction to stretch each field image in a vertical direction; and using each of the oversampling field images in an interlayer texture prediction of each frame of the senior layer.

Another method for interlayer texture prediction in accordance with the present invention comprises the steps of dividing each frame of the senior layer into a plurality of field images; perform downsampling (sub-sampling) in relation to the images of the younger layer to reduce the images of the younger layer in the vertical direction; and then sub-sampled images are used in inter-layer texture prediction obtained by separating field images of the older layer.

The method for encoding a video signal using interlayer prediction in accordance with the present invention comprises the steps of determining whether to use, in the interlayer prediction of the texture, the corresponding texture information of 2N blocks constructed by alternately selecting rows of 2N blocks in an arbitrary image of the base layer, and then by placing the selected rows in the order of choice, or the corresponding texture information of 2N blocks constructed by interpolating one block selected from 2N blocks of the base layer; and include information showing the result of the determination in the encoded information.

A method for decoding a video signal using interlayer prediction in accordance with the present invention comprises the steps of checking whether or not specific indicative information is contained in the received signal; and determining, based on the check result, whether to use in the interlayer texture prediction the corresponding texture information of 2N blocks constructed by alternately selecting rows of 2N blocks in an arbitrary image of the base layer, and then placing the selected rows in the selection order, or the corresponding texture information of 2N blocks, constructed by interpolating one block selected from 2N blocks of the base layer.

In an embodiment of the present invention, each frame of a high or low layer is divided into two field images.

In an embodiment of the present invention, if special indicative information is not contained in the received signal, then the case is regarded as similar to the case in which a signal containing indicative information set to zero is received and blocks whose corresponding texture information is to be used in the interlayer prediction are determined .

One way to use the base layer video signal in an interlayer texture prediction in accordance with the present invention comprises the steps of separating the interlaced video signal of the base layer into components of even and odd fields; increase each of the components of the even and odd fields in the vertical and / or horizontal direction; and then combine and use the increased components of the even and odd fields in the interlayer texture prediction.

Another way to use the base layer video signal in the interlayer texture prediction in accordance with the present invention comprises the steps of dividing the progressive video signal of the base layer into a group of even lines and a group of odd lines; increase each of the groups of even and odd lines in the vertical and / or horizontal direction; and combine and use the combined enlarged groups of even and odd lines in the interlayer texture prediction.

Another way to use the base layer video signal in the interlayer texture prediction in accordance with the present invention comprises the steps of increasing the interlaced video signal of the base layer in the vertical and / or horizontal direction to provide the same resolution as the resolution of the progressive video signal of the older layer; and perform interlayer prediction of the texture of the video signal of the senior layer based on the enlarged video signal.

Another way of using the base layer video signal in the interlayer texture prediction in accordance with the present invention comprises the steps of increasing the progressive video signal of the base layer in the vertical and / or horizontal direction to provide the same resolution as the resolution of the interlaced video signal of the older layer; and perform interlayer prediction of the texture of the video signal of the senior layer based on the enlarged video signal.

In an embodiment of the present invention, video separation and enlargement is performed at the macroblock level (or based on macroblocks).

In another embodiment of the present invention, separation and magnification of the video signal is performed at the image level.

In another embodiment of the present invention, separation and magnification of the video signal is performed if the image formats of the two layers to which the interlayer texture prediction should be applied, i.e. if one layer contains progressive images and the other layer contains interlaced images.

In another embodiment of the present invention, separation and magnification of the video signal is performed if both images of the two layers to which the interlayer texture prediction is to be applied are interlaced.

List of drawings

Figa and 1b is a visual representation of the methods of encoding a single video source into multiple layers;

Figures 2a and 2b are an abbreviated configuration view of a video encoding device to which an inter-layer prediction method in accordance with the present invention is applied;

Figs. 2c and 2d are types of image sequences for encoding an interlaced video signal;

3a and 3b are a schematic diagram of a procedure by which an image of a base layer is constructed and deblocking filtering is performed for interlayer texture prediction in accordance with an embodiment of the present invention;

Figures 4a-4f are a schematic diagram of a procedure in which information about motion in a macroblock of a field of a virtual base layer to be used for interlayer prediction of motion in a macroblock of a field in an MBAFF frame (frame with macroblock adaptive field coding) is obtained using information about movement in a macroblock of a frame in accordance with an embodiment of the present invention;

FIG. 4g is a schematic diagram of a procedure in which texture information in a pair of macroblocks is used to predict texture in a pair of field macroblocks in an MBAFF frame in accordance with an embodiment of the present invention;

Fig. 4h is a visual representation of a method for converting a pair of macroblocks of a frame into a pair of macroblocks of a field in accordance with an embodiment of the present invention;

5a and 5b are a representation of a procedure for deriving a reference index and motion information in accordance with another embodiment of the present invention;

6a-6c are a schematic diagram of procedures in which motion information in a macroblock of a virtual base layer field is obtained using motion information in a macroblock of a frame in accordance with an embodiment of the present invention;

6d is a schematic diagram of a procedure in which texture information in a pair of macroblock frames is used to predict texture in a macroblock of a field in a field image in accordance with an embodiment of the present invention;

7a and 7b are a representation of a procedure for deriving a reference index and motion information in accordance with another embodiment of the present invention;

FIGS. 8a-8c are a schematic diagram of procedures in which motion information in a macroblock of a frame from a macroblock of a virtual base layer field to be used for inter-layer motion prediction is output using motion information of a macroblock of a field in an MBAFF frame in accordance with an embodiment of the present inventions;

Fig. 8d is a schematic diagram of a procedure in which texture information of a pair of field macroblocks in an MBAFF frame is used to predict texture of a pair of macroblocks of a frame in accordance with an embodiment of the present invention;

Fig. 8e is a diagram of a method for converting a pair of field macroblocks into a pair of frame macroblocks in accordance with an embodiment of the present invention;

8f-8g is a schematic diagram of procedures in which texture information of a pair of field macroblocks in an MBAFF frame is used in inter-layer prediction of a pair of macroblocks of a frame when only one of the pair of field macroblocks is an inter-mode macroblock in accordance with an embodiment of the present invention;

Fig. 8h is a schematic diagram of a procedure in which texture information of a pair of field macroblocks in an MBAFF frame is used in predicting the texture of a plurality of pairs of macroblock frames in accordance with an embodiment of the present invention;

Figa-9b is a diagram of the output procedures of reference indices and motion information in accordance with another embodiment of the present invention;

10a-10c are a schematic diagram of procedures in which macroblock motion information of a virtual base layer frame to be used for interlayer motion prediction is derived using field macroblock motion information in a field image in accordance with an embodiment of the present invention;

10d is a schematic diagram of a procedure in which field macroblock texture information in a field image is used in texture prediction of a pair of macroblock frames in accordance with an embodiment of the present invention;

11 is a diagram of a procedure for deriving reference indices and motion information in accordance with another embodiment of the present invention;

12a and 12b are a schematic diagram of procedures in which macroblock motion information of a virtual base layer frame to be used for inter-layer motion prediction is derived using field macroblock motion information in a field image in accordance with another embodiment of the present invention;

Figa-13d is a schematic representation, separately by type of image, of the procedures by which the motion information of the macroblock field of the virtual base layer to be used for interlayer motion prediction is derived using the motion information of the macroblock field in accordance with an embodiment of the present invention;

Figa-14k is a schematic representation, separately by image type, of methods for performing inter-layer motion prediction when the spatial resolutions of the layers differ, in accordance with many different embodiments of the present invention;

15a and 15b are a schematic diagram of procedures in which an image with a different spatial resolution is used in an interlayer texture prediction when the enhanced layer is progressive and the base layer is interlaced, in accordance with an embodiment of the present invention;

FIGS. 16a and 16b are a schematic diagram of procedures in which, to use a base layer image in an interlayer texture prediction, a pair of macroblocks in an image are divided into macroblocks, and the obtained macroblocks are enlarged in accordance with an embodiment of the present invention;

17a and 17b are a schematic diagram of procedures in which an image with a different spatial resolution is used in an interlayer texture prediction when the enhanced layer is interlaced and the base layer is progressive in accordance with an embodiment of the present invention;

FIG. 18 is a schematic diagram of procedures in which an image with a different spatial resolution is used in an interlayer prediction when both the enhanced and the base layers are interlaced in accordance with an embodiment of the present invention;

FIG. 19a is a schematic diagram of a procedure in which inter-layer prediction is applied when an enhanced layer is a sequence of progressive frames, and image types and temporal resolutions of two layers are different in accordance with an embodiment of the present invention;

Fig. 19b is a schematic diagram of a procedure in which inter-layer prediction is applied when the enhanced layer is a sequence of progressive frames and the two layers contain different types of images and have the same resolution in accordance with an embodiment of the present invention;

FIG. 20 is a schematic diagram of a procedure in which inter-layer prediction is applied when a base layer is a sequence of progressive frames, and image types and temporal resolutions of two layers are different in accordance with an embodiment of the present invention; and

FIG. 21 is a schematic diagram of a procedure in which inter-layer prediction is applied when the base layer is a sequence of progressive frames and the two layers contain different types of images and have the same resolution in accordance with an embodiment of the present invention.

Embodiments of the invention

The following is a detailed description of embodiments of the present invention with reference to the accompanying drawings.

Fig. 2a schematically shows the building blocks of a video encoding device to which the inter-layer prediction method in accordance with the present invention is applied. Although the device shown in FIG. 2a is configured to encode an input video signal in two layers, the principles of the present invention described below are also applicable to interlayer processing even when the video signal is encoded in three or more layers.

An interlayer prediction method in accordance with the present invention in an Enhanced Layer Encoder 20 (EL encoder) in the apparatus shown in FIG. 2a. The coded information (motion information and texture information) is received in the encoder 21 of the base layer (BL encoder). Interlayer texture prediction or motion prediction is performed based on the received information. If necessary, the received information is decoded, and prediction is performed based on the decoded information. Of course, in the present invention, the input video signal can be encoded using the video source 3 of the base layer, which is already encoded, as shown in fig.2b. In this case, the same interlayer prediction method as described below is used.

In the case shown in FIG. 2a, there are two possible ways in which the BL encoder 21 encodes the interlaced video signal or by which the encoded video source 3 shown in FIG. 2b is encoded. In particular, in one of the two ways, the interlaced video signal is simple, field by field, encoded into a sequence of fields, as shown in Fig. 3a, and in another way, frames are encoded into a sequence of frames by constructing each frame in a sequence of pairs of macroblocks of two (even and odd ) fields, as shown in fig.3b. The upper macroblock of the pair of macroblocks in the frame encoded in this way is called the “upper macroblock”, and the lower macroblock is called the “lower macroblock”. If the upper macroblock consists of a video component of an even (or odd) field, then the lower macroblock consists of a video component of an odd (or even) field. The construction of a frame by the above method is called a frame with macroblock adaptive field coding (MBAFF). An MBAFF frame can contain not only pairs of macroblocks containing, each, macroblocks of odd and even fields, but also pairs of macroblocks, each containing two macroblocks of a frame.

Accordingly, when the macroblock in the image contains an interlaced video component, it can be a macroblock in a field and can also be a macroblock in a frame. Each macroblock containing an interlaced video component is called a field macroblock, while each macroblock containing a progressive (progressive scan) video component is called a frame macroblock.

Thus, the interlayer prediction method must be determined by determining whether each of the types of macroblock to be encoded in the EL encoder 20 and the macroblock of the base layer to be used in the interlayer prediction of the macroblock are the type of macroblock of the frame or the type of macroblock of the field. If the macroblock is a field macroblock, then the inter-layer prediction method must be determined by determining whether the macroblock is a field macroblock in a field or in an MBAFF frame.

The method will be described separately for each case. Before the description, an assumption is made that the resolution of the current layer is equal to the resolution of the base layer. That is, it is assumed that the type of spatial scalability (SpatialScalabilityType ()) is zero. A description of the case when the resolution of the current layer is higher than the resolution of the base layer will be given later. In the following description and drawings, the terms “upper” and “even” (or odd) are used interchangeably, and the terms “lower” and “odd” (or even) are used interchangeably.

In order to perform inter-layer prediction for encoding or decoding an enhanced layer using a base layer, it is first necessary to decode the base layer. Therefore, the following first describes the decoding of the base layer.

When a base layer is decoded, not only motion information in the base layer is decoded, for example, split modes, reference indices and motion vectors, but also the texture of the base layer.

When the base layer texture is decoded for interlayer texture prediction, not all sample data of the video objects of the base layer is decoded to reduce decoder load. In macroblocks processed in the so-called intra-mode (intra-frame coding mode, hereinafter sometimes referred to as intra-macroblocks), the data of video object samples are decoded, and in macroblocks processed in the so-called inter-mode (interframe coding mode, hereinafter sometimes inter macroblocks), only residual data, which is the error data between the sample data of video objects, is decoded without motion compensation with respect to neighboring images.

In addition, decoding the texture of the base layer for interlayer texture prediction is performed using images, rather than macroblocks, to construct images of the base layer that coincide in time with the images of the enhanced layer. The image of the base layer is constructed according to the data of video object samples reconstructed from intra-macroblocks and residual data decoded from inter-macroblocks, as described above.

Compensation and motion conversion in intra mode or inter mode, for example, DCT (discrete cosine transform) and quantization are performed on blocks of video objects, for example, 16 × 16 macroblocks or 4 × 4 sub-blocks. This causes image artifacts to become distorted at block boundaries. To suppress said blocking artifacts, deblocking filtering is used. The deblocking filter smooths outlines of blocks of video objects to improve the quality of video frames.

Whether or not to apply deblocking filtering to suppress blocking distortions depends on the brightness of the blocks of video objects at the borders and the gradients of pixels near the borders. The power or degree of the deblocking filter is determined by the quantization parameter, intra-mode, inter-mode, blocking mode, indicating the size of blocks or something like that, motion vector, pixel value before deblocking filtering, etc.

The interlayer prediction deblocking filter is applied to the intra macroblock in the base layer image, which is the basis for texture prediction in the enhanced layer macroblock processed in the intra intra mode (intra layer coding mode of the base layer (intraBL mode) or the interframe coding mode with interlayer prediction )

When each of the two layers to be encoded in accordance with the interlayer prediction method is encoded into a sequence of field images, as shown in FIG. 2c, each of the two layers is considered to have a frame format, so that, based on the encoding processes for the frame format, it can be easily output encoding / decoding processes, including deblocking filtering.

A method for performing deblocking filtering in accordance with an embodiment is described below for the case where the image format of the base layer is different from the image format of the enhanced layer, i.e. for the case when the enhanced layer has a frame (or progressive) format and the base layer has a field (or interlaced format) format, for the case when the enhanced layer has a field format and the base layer has a frame format, or for the case when one from the enhanced and base layers, it is encoded into a sequence of field images and the other is encoded as an MBAFF frame, although both the enhanced and base layers have a field format, as shown in FIGS. 2c and 2d.

3a and 3b schematically illustrate the procedures by which an image of a base layer is constructed in order to perform deblocking filtering for interlayer texture prediction in accordance with embodiments of the present invention.

FIG. 3a shows an embodiment in which the enhanced layer has a frame format and the base layer has a field format, and FIG. 3b shows an embodiment in which the enhanced layer has a field format and the base layer has a frame format.

In these embodiments, for interlayer texture prediction, the texture of the inter-macroblock and intra-macroblock of the base layer is decoded to construct an image of the base layer containing sample data of video objects and residual data, and the constructed image is oversampled in accordance with the relationship between resolution (or screen size ) the base layer and the resolution (or screen size) of the enhanced layer after the constructed image is processed by a deblocking filter to suppress artifact in blocking.

The first method (method 1) shown in FIGS. 3a and 3b is a method in which the base layer is divided into two field images to perform deblocking filtering. In this method, when an enhanced layer is created using a base layer encoded in a different image format, the image of the base layer is divided into an even field image and an odd field image, and two field images are subjected to deblocking (i.e., deblocking filtering) and oversampling. Then the two images are combined into one image, and the interlayer texture prediction is performed on one image.

The first method contains the following three steps.

In the separation step (step 1), the image of the base layer is divided into the image of the upper field (or even field) containing even lines and the image of the lower field (or odd field) containing odd lines. The image of the base layer is a video image containing residual data (inter-mode data) and sample data of video objects (intra-mode data), which is restored from the base layer data stream by means of motion compensation.

At the deblocking step (step 2), the field images broken up at the separation step are released by the deblocking filter. In this case, a conventional deblocking filter is applicable as a deblocking filter.

When the resolution of the enhanced layer differs from the resolution of the base layer, the unlocked field images are oversampled in accordance with the relationship between the resolution of the enhanced layer and the resolution of the base layer.

In the combining step (step 3), the oversampled image of the upper field and the oversampled image of the lower field are interlaced alternately to combine into one image. Then, texture prediction of the enhanced layer based on a single image is performed.

According to the second method (method 2) shown in FIGS. 3a and 3b, in which an enhanced layer is created using a base layer encoded in a different image format, the image of the base layer is directly unlocked and subjected to oversampling without splitting into two field images, and interlayer prediction The texture is based on the resulting image.

According to the second method, the image of the base layer, which corresponds to the image of the enhanced layer to be encoded by means of inter-layer texture prediction, is released directly, without dividing it into images of the upper and lower fields, and then is subjected to oversampling. Then, texture prediction of the enhanced layer from the oversampled image is performed.

The de-blocking filter for processing the image of the base layer, constructed for inter-layer motion prediction, is applied only to the region containing sample data of video objects decoded from intra-macroblocks, without applying to the region containing residual data.

In the case where the base layer is encoded in the field format shown in Fig. 3a, i.e. when the base layer is encoded into a sequence of field images, as shown in Fig. 2c, or in an MBAFF frame, as shown in Fig. 2d, to apply the second method, it is necessary to perform the process of interlacing interleaving lines of images of the upper and lower fields to combine them into one image (in the case shown in FIG. 2c), or interlaced alternating rows of upper and lower macroblocks from pairs of field macroblocks to combine them into one image (in the case shown in FIG. 2d). The above process is further described in detail with reference to fig.8d and 8e. Images of the upper and lower fields, or the upper and lower macroblocks subject to interlacing, are images or macroblocks of fields containing residual data (inter-mode data) and sample data of video objects (intra-mode data) restored with motion compensation.

In addition, in the case where the upper and lower macroblocks from the pair of field macroblocks (base layer) in the MBAFF frame shown in Fig. 2d differ in modes, and intra-mode blocks are selected from the macroblocks to be used for interlayer prediction of the texture of macroblock pairs improved layer (in the case described below, shown in fig.8g), in the case when any frame (image) in the base layer encoded into pairs of macroblock fields in the MBAFF frame, as shown in fig.2d, does not coincide in time with the image improved layer (in the description below 8h), or in the case where the texture of the enhanced layer in pairs of macroblocks is predicted from the base layer of macroblock fields in field images, as shown in FIG. 2c (in the case described below in FIG. 10d), the selected macroblock from the macroblocks of the fields, it is oversampled into an intermediate pair of macroblocks (“841” in FIG. 8g and “851” and “852” in FIG. 8h) or two intermediate macroblocks (“1021” in FIG. 10d), and the deblocking filter processes intra- macroblocks from said macroblocks.

The interlayer texture prediction explained in the context of the plurality of the following embodiments is based on the unlocked base layer images described in connection with the embodiment shown in FIGS. 3a and 3b.

The following is a description of the interlayer prediction method separately for each case, classified according to the type of macroblocks in the current layer to be encoded and the type of macroblocks in the base layer to be used for interlayer prediction of macroblocks of the current layer. The description assumes that the spatial resolution of the current layer is equal to the spatial resolution of the base layer, as described above.

I. Case: MB (macroblock) of a frame -> MB field in an MBAFF frame

In this case, macroblocks in the current layer (EL) are encoded into field macroblocks in an MBAFF frame, and macroblocks in the base layer to be used for interlayer prediction of macroblocks of the current layer are encoded into macroblocks of frames. The video components contained in both the upper and lower macroblocks in the base layer are the same as the video components contained in a pair of aligned macroblocks in the current layer. The upstream and downstream (upper and lower) macroblocks are hereinafter referred to as a pair of macroblocks, and the term “pair” in the following description refers to a pair of vertically adjacent blocks. The following is a first description of the interlayer motion prediction.

As the split modes of the current macroblock, the EL encoder 20 applies macroblock split modes obtained by combining a pair of macroblocks 410 of the base layer into one macroblock (by compressing to half the size in the vertical direction). Fig. 4a shows a detailed example of said process. As shown, firstly, the corresponding pair of macroblocks 410 of the base layer is combined into one macroblock (S41), and the partitioning modes of the macroblock obtained by combining are reproduced in another to build a pair of macroblocks 411 (S42). Then, the respective split modes of the pair of macroblocks 411 are applied to the pair of macroblocks 412 of the virtual base layer (S43).

However, when the corresponding pair of macroblocks 410 are combined into one macroblock, it is possible to create a partition area that is prohibited in the partition mode. To prevent this, the EL encoder 20 determines the split mode according to the following rules.

1) Two, upper and lower, 8 × 8 blocks (“B8_0” and “B8_2” in Fig. 4a) in a pair of macroblocks of the base layer are combined into one 8 × 8 block. However, if any of the corresponding 8 × 8 blocks is not subdivided, then they are combined into two 8 × 4 blocks, and if any of the corresponding 8 × 8 blocks are subdivided, then they are combined into four 4 × 4 blocks (" 401 "in Fig. 4a).

2) An 8 × 16 block of the base layer is compressed to an 8 × 8 block, a 16 × 8 block is converted to two adjacent 8 × 4 blocks, and a 16 × 16 block is converted to a 16 × 8 block.

If at least one of the pair of corresponding macroblocks is intra-mode encoded, then the EL encoder 20 first performs the following processing processes before the combining process.

If only one of the two macroblocks is encoded in intra-mode, then information about the movement of the inter-macroblock, for example, macroblock partition modes, reference indices and motion vectors, is copied to the intra-macroblock, as shown in Fig. 4b, or the intra-macroblock is considered inter a 16 × 16 macroblock with zero motion vectors and zero reference indices, as shown in FIG. 4c. Alternatively, as shown in FIG. 4d, reference indices of an intra-macroblock are assigned by copying reference indices of an inter-macroblock to an intra-macroblock, and zero motion vectors are assigned to the intra-macroblock. Then, the aforementioned combining process is performed, and then the derivation of reference indices and motion vectors is performed in the order described below.

The EL encoder 20 performs the following process to derive reference indices of the current pair of macroblocks 412 from the reference indices of the corresponding pair of macroblocks 410.

If each of a pair of 8 × 8 blocks of the base layer corresponding to the current 8 × 8 block is subdivided into the same number of parts, then the reference index of one (upper or lower block) of the 8 × 8 block pair is determined equal to the reference index of the current 8 × 8 block. Otherwise, the reference index of one of the pair of 8 × 8 blocks, which is divided into fewer parts, is determined to be equal to the reference index of the current 8 × 8 block.

In another embodiment of the present invention, the smaller of the reference indices assigned to the pair of 8 × 8 blocks of the base layer corresponding to the current 8 × 8 block is determined to be the reference index of the current 8 × 8 block. Such a determination method, shown in the example of FIG. 4e, can be expressed as follows:

reference index of the current B8_0 = min (reference index of B8_0 of the upper MB frame of the base layer, reference index of B8_2 of the upper MB frame of the base layer),

reference index of the current B8_1 = min (reference index of B8_1 of the upper MB of the base layer frame, reference index of B8_3 of the upper MB of the base layer frame),

a reference index of the current B8_2 = min (a reference index of B8_0 of the lower MB of the base layer frame, a reference index of B8_2 of the lower MB of the base layer frame), and

reference index of the current B8_3 = min (reference index from B8_1 of the lower MB of the base layer frame, reference index of B8_3 of the lower MB of the base layer frame).

The above procedure for deriving reference indices is applicable to both upper and lower macroblocks of fields. The reference index of each 8 × 8 block thus found is multiplied by 2, and the multiplied reference index is determined to be equal to its final reference index. The reason for this multiplication is that, when decoding, the number of images is twice the number of images in the sequence of frames, since the macroblocks of the fields belong to images divided into even and odd fields. Depending on the decoding algorithm, the final reference index of the lower macroblock of the field can be determined by multiplying its reference index by 2 and then adding 1 to the multiplied reference index.

The following describes the procedure by which the EL encoder 20 receives the motion vectors of a pair of macroblocks of a virtual base layer.

The motion vectors are determined by 4 × 4 blocks, and therefore, the corresponding 4 × 8 block of the base layer is determined as shown in FIG. 4f. If the corresponding 4 × 8 block is subdivided, then the motion vector of its upper or lower 4 × 4 block is determined equal to the motion vector of the current 4 × 4 block. Otherwise, the motion vector of the corresponding 4 × 8 block is determined equal to the motion vector of the current 4 × 4 block. The found motion vector, the vertical component of which is divided into 2, is used as the final motion vector of the current 4 × 4 block. The reason for this division is that the video component contained in two macroblocks of the frame corresponds to the video component of one macroblock of the field, so that the size of the field video object is halved in the vertical direction.

After the motion information in the pair of macroblock 412 fields of the virtual base layer is defined as described above, the motion information is used to interlayerly predict the motion of the desired pair of macroblocks 413 of the enhanced layer fields. In addition, in the following description, after the motion information in the macroblock or macroblock pair of the virtual base layer is determined, the motion information is used for interlayer motion prediction in the corresponding macroblock or the corresponding macroblock pair of the current layer. In the following description, it is assumed that such a process is applied even without a reminder that motion information in a macroblock or macroblock pair of a virtual base layer is used for interlayer motion prediction in the corresponding macroblock or corresponding macroblock pair of the current layer.

Fig. 5a schematically shows how traffic information in a pair of macroblocks 500 of a virtual base layer field to be used for inter-layer prediction is derived from traffic information in a pair of macroblocks 500 of a base layer frame corresponding to the current pair of macroblocks, in accordance with another embodiment of the present invention. In this embodiment, as shown, the reference index of the upper or lower block 8 × 8 of the upper macroblock from the pair of macroblocks of the base layer frame is used as the reference index of the upper block 8 × 8 of each of the pair of macroblocks 500 of the fields of the virtual base layer, and the reference index of the upper or lower block 8 × 8 of the lower macroblock of the base layer is used as the reference index of the lower block 8 × 8 of each macroblock in a pair of macroblocks of 500 fields. On the other hand, as shown, the motion vector of the topmost 4 × 4 block of the upper macroblock from the pair of macroblocks of the base layer frame is usually applied to the topmost 4 × 4 block of each macroblock in the macroblock pair of 500 fields of the virtual base layer, the motion vector of the third block is 4 × 4 of the upper macroblock from a pair of macroblocks of the base layer frame is usually used for the second 4 × 4 block of each macroblock in a pair of macroblocks of 500 fields, the motion vector of the topmost block 4 × 4 of the lower macroblock from a pair of macroblocks of the frame of the base layer of the base layer It is usually applied for the fourth 4x4 block of each macroblock in a pair of macroblocks of 500 fields, and the motion vector of the third 4x4 block of the lower macroblock from a pair of macroblocks of the base layer frame is usually applied for the fourth 4x4 block of 4x4 of each macroblock in a pair of macroblocks of 500 fields.

5a, the upper 4 × 4 501 block and the lower 4 × 4 502 block in an 8 × 8 block in a pair of macroblocks 500 fields constructed for use in inter-layer prediction use the motion vectors of 4 × 4 blocks in different 8 × blocks 8 511 and 512 base layers. Said motion vectors may be motion vectors that use different reference images. That is, different 8 × 8 511 and 512 blocks may have different reference indices. Accordingly, in this case, in order to construct a pair of macroblocks 500 of the virtual base layer, the EL encoder 20 typically uses the motion vector of the corresponding 4 × 4 503 block selected for the upper 4 × 4 501 block as the motion vectors of the second 4 × 4 502 virtual block the base layer, as shown in fig.5b (521).

In the embodiment described with reference to FIGS. 4a-4f, in order to create motion information of a virtual base layer for predicting motion information in the current pair of macroblocks, the EL encoder 20 sequentially obtains partition modes, reference indices, and motion vectors based on the information about the movement of the corresponding pair of macroblocks of the base layer. However, in the embodiment described with reference to FIGS. 5a and 5b, the EL encoder 20 first obtains reference indices and motion vectors of the macroblock pair of the virtual base layer based on the motion information of the corresponding macroblock pair of the base layer and, finally, then determines the partitioning modes pairs of macroblocks of the virtual base layer based on the obtained values. When split modes are determined, segments of 4x4 blocks with the same received motion vectors and reference indices are combined, and if the combined block mode is a valid split mode, the split modes are set to the combined mode, otherwise the split modes are set to the pre-join modes.

In the above embodiment, if both macroblocks of the corresponding pair of macroblocks 410 of the base layer are intra-mode macroblocks, then only the intra-frame prediction of the base layer is performed on the current pair of macroblocks 413. In this case, motion prediction is not performed. Of course, a pair of macroblocks of the virtual base layer is not built in the case of texture prediction. If only one of the corresponding pair of macroblocks 410 of the base layer is an intra-macroblock, then the motion information of the inter-macroblock is copied to the intra-macroblock, as shown in Fig. 4b, the motion vectors and reference indices of the intra-macroblock are set to zero, as shown in Fig. 4c, or reference indices of the intra-macroblock are set by copying the reference indices of the inter-macroblock to the intra-macroblock, and the motion vectors of the intra-macroblock are set to zero, as shown in Fig. 4d. Then, the motion information of the pair of macroblocks of the virtual base layer is obtained as described above.

After constructing a pair of macroblocks of a virtual base layer for inter-layer motion prediction, as described above, the EL encoder 20 predicts and encodes information about the motion in the current pair of field macroblocks 413 using motion information in the constructed pair of macroblocks.

The following is a description of the interlayer texture prediction. FIG. 4g shows an example of an interlayer texture prediction method in the case of “MB frame -> MB field in MBAFF frame”. The EL encoder 20 determines the block modes of the corresponding pair of macroblocks 410 of the base layer frame. If both macroblocks of the corresponding pair of macroblocks 410 of the frame are macroblocks of either intra- or inter-mode, then the EL encoder 20 modifies (or converts) the corresponding pair of macroblocks 410 of the base layer into an intermediate pair of macroblocks 421 fields to perform intra-frame prediction from the base layer of the current pairs of field macroblocks 413 (when both frame macroblocks 410 are intra-mode macroblocks), or to perform prediction on the residual data in the manner described below (when both frame macroblocks 410 are inter-mode macroblocks ima). When both macroblocks from the corresponding pair of macroblocks 410 are intra-mode macroblocks, the intermediate pair of field macroblocks 421 contains data released (i.e., subjected to deblocking filtering) after decoding in the case of intra-mode is completed, as described previously. The same is true for an intermediate pair of macroblocks obtained from macroblocks of the base layer for use in the texture prediction process set forth in the following description of many different embodiments.

However, interlayer texture prediction is not performed when only one of the two macroblocks is an inter-mode macroblock. A pair of base layer macroblocks 410 for use in interlayer texture prediction contains source video data that is not encoded (or decoded video data) if the macroblocks are intra-mode macroblocks and contain encoded residual data (decoded residual data) if the macroblocks are inter-mode macroblocks . The same is true for a pair of macroblocks of the base layer in the following description of texture prediction.

Fig. 4h shows a method for converting a pair of macroblocks of a frame into a pair of macroblocks of fields to be used for interlayer texture prediction. As shown, the even lines of the pair of macroblocks A and B of the frame are sequentially selected to construct the upper macroblock A 'of the field, and the odd lines of the pair of macroblocks A and B of the frame are sequentially selected to construct the lower macroblock A' of the field. When one macroblock of a field is filled with rows, it is first filled with even (or odd) rows of the upper block A (A_even or A_odd) and then filled with the odd (or even) rows of the lower block B (B_even or B_odd).

II. Case: MB frame -> MB fields in a field image

In this case, macroblocks in the current layer are encoded into macroblocks of fields in the field image, and macroblocks in the base layer to be used for interlayer prediction of macroblocks of the current layer are encoded into macroblocks of frames. The video components contained in a pair of macroblocks in the base layer are the same as the video components contained in a combined macroblock in an even or odd field in the current layer. The following is a first description of the interlayer motion prediction.

As the partition modes of the even or odd macroblock of the virtual base layer, the EL encoder 20 applies the macroblock partition modes obtained by combining a pair of macroblocks of the base layer into one macroblock (by compressing to half the size in the vertical direction). Fig. 6a shows a detailed example of the process. As shown, firstly, the corresponding pair of macroblocks 610 of the base layer is combined into one macroblock 611 (S61), and the macroblock partitioning modes obtained by this combination are applied to the macroblock of the virtual base layer to be used for interlayer prediction of the motion of the current macroblock 613 (S62 ) The association rules are similar to the rules in the previous case I. The processing method when at least one of the corresponding pair of macroblocks 610 is intra-mode encoded is similar to the method in the previous case I.

The procedure for deriving reference indices and motion vectors is also carried out in the same manner as described above in the previous case I. In case I, the same derivation procedure is applied to the upper and lower macroblocks, since one frame contains pairs of even and odd macroblocks. However, this case II differs from case I in that the output procedure applies to only one macroblock of the field, as shown in FIGS. 6b and 6c, since only one macroblock corresponding to a pair of macroblocks 610 of the base layer is present in the current field image to be encoded .

In the above embodiment, for predicting motion information in a macroblock of a virtual base layer, the EL encoder 20 sequentially obtains partition modes, reference indices, and macroblock motion vectors based on the motion information of the corresponding macroblock pair of the base layer.

In another embodiment of the present invention, the EL encoder 20 first obtains reference indices and motion vectors of the macroblock of the virtual base layer based on the motion information of the corresponding pair of macroblocks of the base layer and finally determines the block modes of the macroblock of the virtual base layer based on the obtained values. On figa and 7b schematically shows the receipt of reference indices and motion vectors of the macroblock field of the virtual base layer. The acquisition operations in the present case are similar to the operations in case I described with reference to FIGS. 5a and 5b, except that the motion information of the upper or lower macroblock is output using the motion information in the macroblock pair of the base layer.

When the split modes are finally determined, the segments of 4 × 4 blocks with the same received motion vectors and reference indices are combined, and if the combined block mode is a valid split mode, the split modes are set to the combined mode, and, otherwise, the split modes are set to associations.

In the above embodiments, if both macroblocks of the corresponding pair of macroblocks of the base layer are intra-mode macroblocks, then motion prediction is not performed, and motion information in the pair of macroblocks of the virtual base layer is also not generated, and if only one of the two macroblocks is an intra- macroblock mode, then motion prediction is performed as described above in the present case.

The following is a description of the interlayer texture prediction. FIG. 6d shows an example of an interlayer texture prediction method in the case of “MB frame -> MB field in the field image”. The EL encoder 20 determines the block modes of the corresponding pair of macroblocks 610 of the base layer. If both macroblocks of the corresponding pair of macroblocks are macroblocks of either intra- or inter-mode, then the EL encoder 20 constructs an intermediate field macroblock 621 from one pair of macroblocks 610 of the frame. If the current macroblock 613 belongs to an even field image, then the EL encoder 20 builds an intermediate field macroblock 621 from even lines of the corresponding pair of macroblocks 610. If the current macroblock 613 belongs to an odd field image, then EL encoder 20 builds an intermediate field macroblock 621 from odd lines pairs of macroblocks 610. The construction method is similar to the method of constructing one macroblock A 'or B' of the field in Fig. 4h.

After the intermediate field macroblock 621 is built, the EL encoder 20 performs an intra-frame basic prediction of the current field macroblock 613 (when both macroblocks of the corresponding pair of macroblocks 610 are intra-mode macroblocks) based on the texture information in the field macroblock 621 or perform its prediction by the residual data (when both macroblocks of the corresponding pair of macroblocks 610 are inter-mode macroblocks).

The EL encoder 20 does not perform inter-layer texture prediction only if one of the corresponding pair of macroblocks 610 is an inter-mode macroblock.

III. Case: MB in MBAFF frame -> MB frame

In this case, macroblocks in the current layer are encoded into macroblock frames, and macroblocks in the base layer to be used for interlayer prediction of macroblock frames of the current layer are encoded into field macroblocks in an MBAFF frame. The video components contained in the macroblock of the field in the base layer are the same as the video components contained in a pair of aligned macroblocks in the current layer. The following is a first description of the interlayer motion prediction.

As modes of splitting a pair of macroblocks in a virtual base layer, the EL encoder 20 applies macroblock splitting modes obtained by stretching (twice, in the vertical direction) an upper or lower macroblock from a pair of macroblocks of a base layer. On figa shows a detailed example of the above process. Although the upper field macroblock is selected in the following description and drawings, everything described below is applicable when the lower field macroblock is selected.

As shown in FIG. 8a, the upper macroblock of the field in the corresponding pair of macroblocks 810 of the base layer is doubled to construct two macroblocks 811 (S81), and the split modes obtained by stretching are applied to the pair of macroblocks 812 of the virtual base layer (S82).

However, when the corresponding macroblock of the field is stretched twice in the vertical direction, it is possible to create a partition mode (or scheme), which is forbidden (a) in macroblock partition modes. To prevent this, the EL encoder 20 determines the split modes depending on the split split modes in accordance with the following rules.

1. Blocks 4 × 4, 8 × 4 and 16 × 8 of the base layer after stretching become, by definition, blocks 4 × 8, 8 × 8 and 16 × 16, obtained by increasing them twice in the vertical direction.

2. Each of the blocks 4 × 8, 8 × 8 and 16 × 16 of the base layer after stretching is composed, by definition, of two, upper and lower, blocks of the same size. As shown in FIG. 8a, an 8 × 8 block of B8_0 becomes, by definition, two 8 × 8 blocks (801). The reason the 8 × 8 B8_0 block is not specified as an 8 × 16 block after stretching is because the adjacent stretched block on the left or right side may not be an 8 × 16 block, and in this case , no macroblock splitting mode is supported.

If one of the corresponding pair of macroblocks 810 is encoded in intra-mode, then the EL encoder 20 selects the macroblock of the upper or lower field encoded in inter-mode instead of intra-mode and performs the expansion process on it to determine the split modes of the pair of macroblocks 812 in the virtual base layer.

If both macroblocks of the corresponding pair of macroblocks 810 are intra-mode macroblocks, then the EL encoder 20 performs only interlayer texture prediction, without determining the partitioning modes, during the above-described stretching process and the following process of obtaining reference indices and motion vectors.

To obtain reference indices of a pair of macroblocks of the virtual base layer from reference indices of the corresponding macroblock of the field, the EL encoder 20 determines the reference index of the corresponding 8 × 8 B8_0 base layer block equal to the reference index of each of the two upper and lower 8 × 8 blocks, as shown in FIG. .8b, and divides the found reference index of each 8 × 8 block by 2 to obtain its final reference index. The reason for this division is that, to apply to the frame sequence, the number of images must be halved, since the number of reference images of macroblock fields are set based on images divided into even and odd fields.

Upon receipt of the motion vectors of a pair of macroblocks 812 frames of the virtual base layer, the EL encoder 20 determines the motion vector of the corresponding 4 × 4 base layer block to be equal to the motion vector of the 4 × 8 block in the pair of macroblocks 812 of the virtual base layer, as shown in FIG. 8c, and uses, as the final motion vectors, the motion vector found, the vertical component of which is multiplied by 2. The basis for such a multiplication is that the video component contained in one field macroblock corresponds to the video component of two akroblokov frame, so that the size of the video object HR multiplied twice in the vertical direction.

In the above embodiment, in order to predict motion information of a pair of macroblocks of a virtual base layer, the EL encoder 20 sequentially obtains partition modes, reference indices and macroblock motion vectors based on the motion information of the corresponding macroblock of the base layer field.

In another embodiment of the present invention, when acquiring motion information of a pair of macroblocks of a virtual base layer to be used for inter-layer prediction of the current pair of macroblocks, the EL encoder 20 first obtains reference indices and motion vectors of the pair of macroblocks of the virtual base layer based on the motion information of the corresponding macroblock field base layer and, finally, then determines the block mode in each of the pair of macroblocks of the virtual base layer based on the obtained values, as azano 9a. When the split modes are finally defined, the segments of 4x4 blocks with the same received motion vectors and reference indices are combined, and if the combined block mode is a valid split mode, then the split modes are set to the combined mode, and, otherwise, the split modes are set to modes before unification.

Below is a more detailed description of the embodiment shown in figa. As shown, the macroblock of the inter-mode field of the base layer is selected, and the motion vectors and reference indices of the selected macroblock are used to obtain reference indices and motion vectors of a pair of macroblocks of a virtual base layer frame to be used to predict the motion of the current pair of macroblocks. If both macroblocks are inter-mode macroblocks, then an arbitrary macroblock is selected from the upper and lower macroblocks (901 or 902), and information about the motion vectors and reference indices of the selected macroblock is used. As shown, to obtain reference indices, the corresponding value of the upper 8 × 8 block of the selected macroblock is copied to the reference indices of the upper and lower blocks 8 × 8 of the upper macroblock of the virtual base layer, and the corresponding value of the lower 8 × 8 block of the selected macroblock is copied to the reference indices of the upper and lower blocks 8 × 8 of the lower macroblock of the virtual base layer. As shown, to output the motion vectors, the corresponding value of each 4 × 4 block of the selected macroblock is usually used as the motion vector of the corresponding pair of vertically adjacent 4 × 4 blocks in a pair of macroblocks of the virtual base layer. In another embodiment of the present invention, the motion information of the corresponding pair of macroblocks of the base layer can be mixed and used to obtain motion vectors and reference indices of the pair of macroblocks of the frame of the virtual base layer, which differs from the embodiment shown in Fig. 9a. Fig. 9b shows a procedure for obtaining motion vectors and reference indices in accordance with such an embodiment. A detailed description of the combination of copied reference indices and motion vectors of subunits (8 × 8 blocks and 4 × 4 blocks) in a pair of macroblocks of a virtual base layer is omitted in this application, since it can be intuitively understood from the above description of the procedure for obtaining motion information and the drawing in FIG. 9b.

However, since the motion information of both macroblocks from a pair of macroblocks of the base layer fields is used in the embodiment shown in FIG. 9b, if one of the pair of macroblocks of the base layer fields is encoded in intra-mode, then intra-macroblock motion information is output using motion information of another macroblock, which is an inter-mode macroblock. In particular, information on motion vectors and reference indices of a pair of macroblocks of a virtual base layer can be obtained, as shown in Fig. 9b, after the motion vectors and reference indices of the intra-macroblock are constructed by copying the corresponding information of the inter-macroblock to the intra-macroblock, as shown in FIG. 4b, or after the intra-macroblock is accepted as an inter-macroblock with zero motion vectors and zero reference indices, as shown in FIG. 4c, or after the intra-macroblock reference indices are set by copying the the indices of the inter-macroblock into the intra-macroblock, and its motion vectors are set to zero, as shown in Fig. 4d. After the information on the motion vectors and reference indices in the pair of macroblocks of the virtual base layer is obtained, the block modes in the pair of macroblocks are determined based on the received information, as described above.

On the other hand, if both macroblocks from the corresponding pair of macroblocks of the base layer fields are intra-mode macroblocks, then motion prediction is not performed.

The following is a description of the interlayer texture prediction. Fig. 8d shows an example of an interlayer texture prediction method in the case of "MB field in MBAFF frame -> MB frame". The EL encoder 20 determines the block modes of the corresponding pair of macroblocks 810 fields of the base layer. If both macroblocks of the corresponding pair of macroblocks 810 of the field are macroblocks of either intra- or inter-mode, then the EL encoder 20 converts the corresponding pair of macroblocks 810 of the fields of the base layer into an intermediate pair of macroblocks 821 of the frame to perform intra-frame prediction from the base layer of the current pair of macroblocks 813 frame (when both frame macroblocks 810 are intra-mode macroblocks) or to predict it from residual data in the manner described below (when both frame macroblocks 810 are inter-mode macroblocks). When both macroblocks from the corresponding pair of macroblocks 810 are intra-mode macroblocks, the pair of macroblocks 810 contains data that has been decoded, and the deblocking filter is applied to the pair of macroblocks 821, as described above.

Fig. 8e shows a method for converting a pair of field macroblocks into a pair of frame macroblocks. As shown, the rows of a pair of macroblocks A and B fields are selected alternately (A-> B-> A-> B-> A->, ...) sequentially from the top of each of the macroblocks and then sequentially placed in the order of selection from above to build a pair of macroblocks A 'and B' frame. Since the rows of a pair of field macroblocks are rearranged in the mentioned order, the upper macroblock A 'of the frame is constructed from the rows of the upper half of the pair of macroblocks A and B of the fields, and the lower macroblock B' of the frame is constructed from the rows of the lower half.

On the other hand, if only one of the corresponding pair of macroblocks 810 of the base layer fields is an inter-mode macroblock, then one block is selected from the pair of macroblocks 810 of the base layer in accordance with the block modes of the current pair of macroblocks 813 of the frame, and the selected block is used for interlayer texture prediction . Alternatively, before determining the block modes of the current pair of macroblocks 813 of the frame, each of the methods described below is applicable to perform interlayer prediction, and then the block modes in the pair of macroblocks 813 can be determined.

8f and 8g show an example in which one block is selected to perform inter-layer prediction. In the case where the current pair of macroblocks 813 of the frame is inter-mode encoded (or when its inter-frame prediction is performed), as shown in FIG. 8f, from the pair of macroblocks 810 of the base layer fields, an inter-mode block 810a is selected and the selected block is oversampled direction to create two corresponding macroblocks 831. Then, two macroblocks 831 are used to predict from the residual data the current pair of macroblocks 813 of the frame. In the case where the current pair of frame macroblocks 813 is not inter-mode encoded (or when its intra-frame prediction is performed), as shown in FIG. 8g, an intra-mode block 810b is selected from the pair of macroblocks 810 of the base layer fields, and the selected block is oversampled in in the vertical direction to create two corresponding macroblocks 841. After the deblocking filter is applied to the two macroblocks 841, two macroblocks 841 are used for intra-frame prediction from the base layer of the current pair of macroblocks 813 of the frame.

The method shown in FIGS. 8f and 8g, in which one block is selected and oversampled to create a pair of macroblocks to be used for interlayer texture prediction, is also applicable when the layers have different frame rates. When the frame rate of the enhancement layer is higher than the frame rate of the base layer, some images in the image sequence of the enhancement layer may not have a time-appropriate image in the base layer. Interlayer prediction of the texture of a pair of macroblocks of a frame contained in an image of an enhanced layer that does not have a time-appropriate image in the base layer can be performed using one of a pair of spatially aligned macroblock fields in the previous time image in the base layer.

On fig.8h presents an example of the above method in the case where the frame rate of the enhanced layer is two times higher than the frame rate of the base layer.

As shown, the frame rate of the enhanced layer is two times higher than the frame rate of the base layer. Therefore, one of every two images of the enhanced layer, for example, an image with an image sequence number (POC) of "n2", does not have an image with the same image sequence number (POC) in the base layer. In this case, the same POC indicates a coincidence in time.

When there is no time-matching image in the base layer (for example, when the current POC is n2), the lower field macroblock 802 contained in a pair of spatially aligned field macroblocks in the previous image (i.e., in an image with a POC smaller than the current POC by 1), is vertically downsampled to create an intermediate pair of macroblocks 852 (S82), and then an intermediate pair of macroblocks 852 is used to perform an interlayer prediction of the texture of the current pair of macroblocks 815. When the base layer has a matching n time image (for example, when the current POC is n1), the upper block macroblock 801 contained in a pair of spatially aligned field macroblocks in a time-matching image is vertically downsampled to create an intermediate pair of macroblocks 851 (S82), and then an intermediate pair of macroblocks 851 (S82) is used to perform interlayer prediction of the texture of the current pair of macroblocks 814. When the intermediate pair of macroblocks 851 or 852 created by oversampling contains a pair of macroblocks decoded from intr a-macroblock, a pair of macroblocks is used to inter-layer texture prediction after a deblocking filter processes a pair of macroblocks.

In another embodiment of the present invention, when a time-matching image is present in the base layer (when the current POC is n1 in the example of FIG. 8h), a pair of macroblocks of a frame can be created from a pair of macroblocks of fields in accordance with the embodiment shown in FIG. 8d, instead of applying the method shown in FIG. 8h, and then can be used for interlayer texture prediction. In addition, when the current image does not contain a time-matching image in the base layer (when the current POC is n2 in the example of FIG. 8h), interlayer texture prediction can be performed as in FIG. 8h, or, conversely, interlayer texture prediction may not run on macroblocks in the current image.

Accordingly, in an embodiment of the present invention, a “field_base_flag” flag is assigned to indicate whether the inter-layer texture prediction is performed in accordance with the method shown in FIG. 8d or in accordance with the method shown in FIG. 8h, the flag is included in the encoding information . For example, said flag is set to “0” when the texture prediction is performed in accordance with the method shown in FIG. 8d, and is set to “1” when the texture prediction is made in accordance with the method shown in FIG. 8h. This flag is set in the sequence parameter set in the enhanced layer, the sequence parameter in the scalable extension, the image parameter set, the image parameter set in the scalable extension, the slice header, the slice header in the scalable extension, the macroblock layer or the macroblock layer in the scalable extension to be transmitted to the decoder .

IV. Case: MB fields in a field image -> MB frame

In this case, the macroblocks in the current layer (EL) are encoded into macroblock frames, and the macroblocks in the base layer (BL) to be used for interlayer prediction of macroblock frames of the current layer are encoded into macroblocks of fields in the field image. The video components contained in the macroblock of the field in the base layer are the same as the video components contained in a pair of aligned macroblocks in the current layer. The following is a first description of the interlayer motion prediction.

As the macroblock splitting modes in the virtual base layer, the EL encoder 20 applies the splitting modes obtained by stretching the macroblock in an even or odd field of the base layer (twice, in the vertical direction). Fig. 10a presents a detailed example of the process. The procedure shown in Fig. 10a differs from the procedure in case III, when the macroblock of the upper or lower field in the MBAFF frame is selected, in that a spatially aligned field macroblock 1010 in an even or odd field is usually used, and is similar to the procedure in case III in that the combined macroblock 1010 of the field is stretched, and the split modes of the two macroblocks obtained by stretching are applied to the pair of macroblocks 1012 of the virtual base layer. When the corresponding macroblock 1010 of the field is stretched twice in the vertical direction, it is possible to create a partition mode (or pattern), which is prohibited (a) in macroblock partition modes. To prevent this, the EL encoder 20 determines the split modes depending on the split split modes in accordance with the same rules as rules 1 and 2 proposed in case III.

If the corresponding macroblock is encoded in intra-mode, the EL encoder 20 performs only inter-layer texture prediction, without performing the determination of split modes, during the above-described stretching process and the following process for obtaining reference indices and motion vectors. That is, the EL encoder 20 does not perform inter-layer motion prediction.

The procedure for obtaining reference indices and motion vectors is also similar to the procedure described in the previous case III. However, case IV under consideration differs from case III in the following. Since, in case III, the corresponding macroblocks of the base layer are transmitted in pairs of even and odd macroblocks in the frame, one of the upper and lower macroblocks is selected and used in the output procedure. Since, in the case under consideration IV, there is only one macroblock in the base layer that corresponds to the current macroblock to be encoded, information about the movement in the pair of macroblocks 1012 of the virtual base layer is derived from the information about the movement of the corresponding macroblock of the field, without performing the macroblock selection procedure, as 10b and 10c, and the obtained motion information is used for interlayer motion prediction in the current pair of macroblocks 1013.

11 schematically shows the receipt of reference indices and motion vectors of a pair of macroblocks of a virtual base layer in accordance with another embodiment of the present invention. In this case, information about the movement of a pair of macroblocks of the virtual base layer is derived from information about the movement of an even or odd macroblock of the field of the base layer, which is different from the case described above with reference to figa. In the present case, output operations such as those in the case shown in FIG. 9a are applied. However, the process of mixing and using motion information in a pair of macroblocks in the case shown in Fig. 9b is not applicable in case IV under consideration, since, in the case under consideration, there is no pair of upper and lower macroblocks in the corresponding field of the base layer.

In the embodiment described with reference to FIGS. 10a-10c, for predicting motion information in a pair of macroblocks of a virtual base layer, the EL encoder 20 sequentially obtains partitioning modes, reference indices, and motion vectors based on the motion information of the corresponding macroblock of the base layer field . However, in another embodiment shown in FIG. 11, the EL encoder 20 first obtains reference indices and motion vectors of the pair of macroblocks of the virtual base layer based on the motion information of the corresponding pair of macroblocks of the base layer and, finally, then determines the split modes of the pair of virtual macroblocks base layer based on the values obtained. When the split modes are defined, segments of 4x4 blocks with the same received motion vectors and reference indices are combined, and if the combined block mode is a valid split mode, the split modes are set to the combined mode, and, otherwise, the split modes are set to associations.

When texture prediction is performed in the above embodiments, if the corresponding macroblock of the base layer field is an intra-mode macroblock, then intra-frame prediction based on the base layer is performed on the current macroblock. If the corresponding macroblock of the field is an inter-mode macroblock, and if the current macroblock is inter-mode encoded, interlayer prediction of residual data is performed. Of course, in this case, the macroblock of the field for use in prediction is used to predict the texture after it is oversampled in the vertical direction.

In another embodiment of the present invention, a virtual macroblock is created from a macroblock of a field contained in an odd or even field to construct a pair of macroblocks, and then information about the movement of a pair of macroblocks of a virtual base layer is derived from the constructed pair of macroblocks. 12a and 12b are examples of the aforementioned embodiment.

In this embodiment, the reference indices and motion vectors of the corresponding macroblock of the even (or odd) field of the base layer are copied (1201 and 1202) to create a virtual macroblock of the odd (or even) field to build a pair of macroblocks 1211, and the motion information of the constructed pair of macroblocks 1211 is mixed to output information about the movement of a pair of macroblocks 1212 of the virtual base layer (1203 and 1204). In an example method of mixing and using motion information, as shown in FIGS. 12a and 12b, reference indices of the upper 8 × 8 block of the corresponding upper macroblock are applied to the upper 8 × 8 block of the upper macroblock from the pair of macroblocks 1212 of the virtual base layer, the reference indices of the lower block 8 × 8 are applied to the upper 8 × 8 block of the lower macroblock, reference indices of the upper 8 × 8 block of the corresponding lower macroblock are applied to the lower 8 × 8 block of the upper macroblock from a pair of macroblocks 1212 of the virtual base layer, and reference eksy lower block 8 × 8 block are applied to the lower 8 × 8 macroblock of the lower (1203). Motion vectors are applied in accordance with the reference indices (1204). A description of such a process is not given in this application since it can be intuitively understood from FIGS. 12a and 12b.

In the embodiment shown in FIGS. 12a and 12b, split modes of the pair of macroblocks 1212 of the virtual base layer are determined based on the obtained reference indices and motion vectors using a method similar to the above method.

The following is a description of the interlayer texture prediction. 10b shows an example of an interlayer texture prediction method for the case under consideration “MB field in the field image -> MB frame”. First, the EL encoder 20 oversamples the corresponding macroblock 1010 of the base layer field to create two intermediate macroblocks 1021. If the corresponding macroblock 1010 of the field is an intra-mode macroblock, then the EL encoder 20 applies a deblocking filter to the two created intermediate macroblocks 1021 and then performs intra prediction on the base layer of the current pair of macroblocks 1013 of the frame based on two intermediate macroblocks 1021. If the corresponding field macroblock 1010 is an inter-mode macroblock, then the EL encoder 20 performs the prediction an indication of the residual data of the current pair of macroblocks 1013 of the frame based on two created intermediate blocks 1021.

V. Case: MB fields -> MB fields

This case is divided into the following four cases, since the macroblocks of the fields are divided into macroblocks of the fields contained in the field image and macroblocks of the fields contained in the MBAFF frame.

i) The case when the base and advanced layers are MBAFF frames

Such a case is shown in FIG. 13a. As shown, the motion information (partitioning modes, reference indices, and motion vectors) of the corresponding pair of macroblocks of the base layer is used as information about the motion of the pair of macroblocks of the virtual base layer by directly copying the motion information of the corresponding pair of macroblocks to the pair of macroblocks of the virtual base layer. In this case, motion information is copied between macroblocks that are identical in parity. In particular, the motion information of the even-field macroblocks is copied to the even-field macroblocks, and the motion information of the odd-field macroblocks is copied to the odd-field macroblocks to construct a macroblock of a virtual layer, which is used to predict the macroblock of the current layer.

When texture prediction is performed, a known method for interlayer texture prediction is applied.

ii) The case where the base layer contains field images and the enhanced layer contains MBAFF frames

Such a case is shown in FIG. 13b. As shown, the motion information (partitioning modes, reference indices, and motion vectors) of the corresponding macroblock of the base layer field is used as the motion information of each of the pair of macroblocks of the virtual base layer by directly copying the motion information of the corresponding macroblock of the field to each of the pair of macroblocks. In this case, the rule of copying taking into account parity is not applied, since the information about the movement of one macroblock of the field is applied to macroblocks of both the upper and lower fields.

When texture prediction is performed, intra-frame prediction is applied on the base layer (when the corresponding base layer block is an intra-mode block), or residual data prediction (when the corresponding base layer block is the inter-mode block) is applied between the macroblocks of the enhanced and base layers having the same (even or odd) field attributes.

iii) The case where the base layer contains MBAFF frames and the enhanced layer contains field images

Such a case is shown in FIG. 13c. As shown, a macroblock of a field of identical parity is selected from a pair of macroblocks of the base layer corresponding to the current macroblock of the field, and motion information (partitioning modes, reference indices, and motion vectors) of the selected macroblock of the field is used as information about the movement of the macroblock of the field of the virtual base layer by directly copying information about the movement of the selected macroblock of the field into the macroblock of the field of the virtual base layer.

When texture prediction is performed, intra-frame prediction is applied on the base layer (when the corresponding base layer block is an intra-mode block), or residual data prediction (when the corresponding base layer block is the inter-mode block) is applied between the macroblocks of the enhanced and base layers having the same (even or odd) field attributes.

iv) The case where the base and enhanced layers are field images

Such a case is shown in FIG. 13d. As shown, the motion information (partitioning modes, reference indices, and motion vectors) of the corresponding macroblock of the field of the base layer is used as information about the movement of the macroblock of the field of the virtual base layer by directly copying the motion information of the corresponding macroblock of the field to the macroblock of the field of the virtual base layer. In this case, traffic information is also copied between macroblocks that are identical in parity.

When texture prediction is performed, a known method for interlayer texture prediction between macroblock frames is applied.

The above description of interlayer prediction is for the case where the base and enhanced layers have the same resolution. The following is a description of how the image type (progressive frame, MBAFF frame or interlaced field) of each layer and / or the macroblock type in the image is recognized when the resolution of the enhanced layer is higher than the resolution of the base layer (i.e. when the type of spatial scalability (SpatialScalabilityType ()) is above zero), and an interlayer prediction method is applied according to the recognized types. First, the interlayer motion prediction is explained.

M_A). Base layer (progressive frame) -> Advanced layer (MBAFF frame)

On figa shows a processing method for such a case. As shown, first, the motion information of all the macroblocks of the corresponding frame in the base layer is copied to create a virtual frame. Oversampling is then performed. During the said oversampling, interpolation is performed using the texture information of the image of the base layer with an interpolation frequency at which the image resolution (or image size) can be equal to the image resolution of the current layer. In addition, information about the movement of each macroblock of the image enlarged by interpolation is generated based on the information about the movement of each macroblock of the created virtual frame. For the said formation, one of a number of known methods is used. Images of an intermediate base layer constructed in this way have the same resolution as images of the current (improved) layer. Accordingly, in the case under consideration, the above-described interlayer motion prediction is applicable.

In the case under consideration (Fig. 14a), the macroblocks in the image in the base and current layers are macroblocks of a frame and macroblocks of fields in an MBAFF frame, since the base layer contains frames and the current layer contains MBAFF frames. Accordingly, to perform the interlayer motion prediction, the method from the above case I is applicable. However, the same MBAFF frame may contain not only a pair of field macroblocks, but also a pair of macroblocks of a frame, as described above. Accordingly, the known method for predicting motion between macroblock frames is used (inter-frame prediction method), which comprises simply copying motion information when the type of a pair of macroblocks of the current layer corresponding to a pair of macroblocks in the image of the intermediate base layer is recognized as the type of macroblocks of the frame, and not the type of macroblocks fields.

M_B). Base layer (progressive frame) -> Improved layer (interlaced field)

14b shows a processing method for such a case. As shown, first, the motion information of all the macroblocks of the corresponding frame in the base layer is copied to create a virtual frame. Oversampling is then performed. During the said oversampling, interpolation is performed using information about the texture image of the base layer with an interpolation frequency at which the image resolution may be equal to the image resolution of the current layer. In addition, information about the movement of each macroblock of the image enlarged by interpolation is generated based on the information about the movement of each macroblock of the created virtual frame.

To perform the interlayer motion prediction, the method from the above case II is applied, since each macroblock in the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of the field in the field image.

M_C). Base layer (MBAFF frame) -> Improved layer (progressive frame)

Fig. 14c shows a processing method for such a case. As shown, first, the corresponding MBAPF frame of the base layer is converted to a progressive frame. To convert a pair of macroblocks of the fields of the MBAPF frame into a progressive frame, the method from the above case III is applied, and to convert a pair of macroblocks of the fields of the MBAPF frame, the known method of inter-frame conversion is used. Of course, when in the case under consideration the method from case III is applied, the virtual frame and the motion information of each macroblock of the frame are created using data obtained by inter-layer prediction without performing the encoding operation of the difference between the predicted data and the data of the layer actually to be encoded.

After the virtual frame is received, oversampling is performed on the virtual frame. During the mentioned oversampling, interpolation is performed with the interpolation frequency at which the resolution of the base layer can become equal to the resolution of the current layer. In addition, the motion information of each macroblock of the enlarged image is generated based on the motion information of each macroblock of a virtual frame using one of a number of known methods. In this case, a known method for interlayer motion prediction between macroblock frames is performed, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of the frame.

M_D). Base layer (interlaced field) -> Enhanced layer (progressive frame)

On fig.14d depicts one processing method for such a case. In this case, the type of image is such as the type of macroblock image. As shown, first, the corresponding field of the base layer is converted into a progressive frame. The frame obtained by the conversion has the same vertical / horizontal aspect ratio (aspect ratio) as the image of the current layer. To convert an interlaced field to a progressive frame, the oversampling process and the method from case IV described above are applied. Of course, when the case IV method is applied in the case under consideration, the texture data of the virtual frame and the motion information of each macroblock of the frame are created using data obtained by the interlayer prediction without performing the encoding operation of the difference between the predicted data and the data of the layer actually to be encoded.

After the virtual frame is received, oversampling is performed on the virtual frame. During the aforementioned oversampling, interpolation is performed so that the resolution of the virtual frame can become equal to the resolution of the current layer. In addition, the motion information of each macroblock of the interpolated image is generated based on the motion information of each macroblock of a virtual frame using one of a number of known methods. In this case, a known method for interlayer motion prediction between macroblock frames is performed, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of the frame.

14e shows one processing method for the aforementioned case M_D) in accordance with another embodiment of the present invention. As shown, in this embodiment, the odd or even corresponding field is converted to a progressive frame. To convert an interlaced field to a progressive frame, oversampling and the above-described method from case IV are used, as shown in FIG. 14d. After the virtual frame is obtained, the motion prediction method between images having the same aspect ratio, which is one of a number of known methods, is applied to the virtual frame for motion prediction between the image of the current layer and the virtual frame of the intermediate layer to perform information prediction encoding about the movement of each macroblock of a progressive image of the current layer.

The method shown in FIG. 14e differs from the method shown in FIG. 14d in that no intermediate prediction signal is generated.

FIG. 14f shows a processing method for the aforementioned case M_D) in accordance with another embodiment of the present invention. As shown, in this embodiment, motion information of all macroblocks of the corresponding field of the base layer is copied to create a virtual image. Oversampling is then performed. During such oversampling, the texture information of the base layer image is used, and different interpolation frequencies are used for vertical and horizontal interpolation so that the enlarged image has the same size (or resolution) as the image of the current layer. In addition, one of a number of known prediction methods (for example, Advanced Special Scalability (ESS)) is applied to the virtual image to generate a variety of syntax information and motion information of the enlarged image. The motion vectors constructed during the above process are stretched in accordance with the magnification factor. After the oversampling image of the intermediate base layer is constructed, the image is used to perform interlayer prediction of the movement of each macroblock in the image of the current layer to encode motion information of each macroblock of the image of the current layer. In this case, the known method of interlayer motion prediction between macroblock frames is applied.

14g shows a processing method for the aforementioned case M_D) in accordance with another embodiment of the present invention. As shown, in this embodiment, motion information of all the macroblocks of the corresponding field of the base layer is first copied to create a virtual image. After that, the information about the texture of the image of the base layer is used to perform interpolation with different frequencies for vertical and horizontal interpolation. Texture information created during such an operation is used for interlayer texture prediction. In addition, motion information in a virtual image is used to perform interlayer prediction of the movement of each macroblock in the image of the current layer. In this case, one of a number of known methods is used (for example, the method of extended special scalability (ESS) specified in the combined scalable video model (JSVM)) to perform encoding with prediction of the image movement of the current layer.

The method shown in FIG. 14g differs from the method shown in FIG. 14f in that no intermediate prediction signal is generated.

M_E). Base layer (MBAFF frame) -> Advanced layer (MBAFF frame)

FIG. 14h shows a processing method for such a case. As shown, first, the corresponding MBAFF frame of the base layer is converted to a progressive frame. To convert an MBAFF frame to a progressive frame, the method from the above case III is used to convert pairs of macroblock fields of an MBAFF frame, and an inter-frame prediction method is used to convert pairs of macroblocks of an MBAFF frame. Of course, when in the case under consideration the method from case III is applied, the virtual frame and the motion information of each macroblock of the frame are created using data obtained by inter-layer prediction without performing the encoding operation of the difference between the predicted data and the data of the layer actually to be encoded.

After the virtual frame is received, oversampling is performed on the virtual frame. During the mentioned oversampling, interpolation is performed with the interpolation frequency at which the resolution of the base layer can become equal to the resolution of the current layer. In addition, the motion information of each macroblock of the enlarged image is generated based on the motion information of each macroblock of a virtual frame using one of a number of known methods. To perform interlayer motion prediction, the method from the above case I is applied, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of the field in the MBAPF frame. However, the same MBAFF frame may contain not only a pair of macroblock fields, but also a pair of macroblocks of a frame, as described above. Accordingly, a known method for predicting motion between macroblock frames is used (interframe prediction method), which comprises copying motion information when a pair of macroblocks of the current layer corresponding to a pair of macroblocks in an image of an intermediate base layer are macroblocks of a frame, and not macroblocks of fields.

M_F). Base layer (MBAFF frame) -> Improved layer (interlaced field)

On figi presents a processing method for such a case. As shown, first, the corresponding MBAFF frame of the base layer is converted to a progressive frame. In order to convert the MBAFF frame to a progressive frame, the method from case III above is applied to convert the pairs of macroblock fields of the MBAFF frame, and the inter-frame prediction method is used to convert the macroblock pairs of the MBAFF frame. Of course, also, when the method from case III is applied in the case under consideration, the virtual frame and the motion information of each macroblock of the frame are created using data obtained by inter-layer prediction without performing the encoding operation of the difference between the predicted data and the data of the layer actually to be encoded.

After the virtual frame is received, interpolation is performed on the virtual frame with an interpolation frequency at which the resolution can be equal to the resolution of the current layer. In addition, the motion information of each macroblock of the enlarged image is generated based on the motion information of each macroblock of a virtual frame using one of a number of known methods. To perform the interlayer motion prediction, the method from the above case II is applied, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of a field in an even or odd field.

M_G). Base layer (interlaced field) -> Advanced layer (MBAFF frame)

FIG. 14j shows a processing method for such a case. As shown, first the interlaced field of the base layer is converted to a progressive frame. To convert an interlaced field to a progressive frame, oversampling and the method from case IV described above are applied. Of course, also, when in the case under consideration the method from case IV is applied, a virtual frame and motion information of each macroblock of the frame are created using data obtained by inter-layer prediction without performing the encoding operation of the difference between the predicted data and the data of the layer actually to be encoded.

After the virtual frame is received, oversampling is performed on the virtual frame so that the resolution can become equal to the resolution of the current layer. In addition, movement information of each macroblock of the enlarged image is generated using one of a number of known methods. To perform interlayer motion prediction, the method from the above case I is applied, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the frame, and each macroblock of the current layer is a macroblock of the field in the MBAPF frame. However, the same MBAFF frame may contain not only a pair of macroblock fields, but also a pair of macroblocks of a frame, as described above. Accordingly, the known method for predicting motion between macroblock frames is used (interframe prediction method), instead of the prediction method from the above case I, when a pair of macroblocks of the current layer corresponding to a pair of macroblocks in the image of the intermediate base layer contains macroblocks of the frame, and not macroblocks of the fields.

M_H). Base layer (interlaced field) -> Enhanced layer (interlaced field)

14k shows a processing method for such a case. As shown, first, the motion information of all the macroblocks of the corresponding field in the base layer is copied to create a virtual field, and then oversampling on the virtual field is performed. Mentioned oversampling is performed with an oversampling frequency at which the resolution of the base layer may be equal to the resolution of the current layer. In addition, using one of a number of known methods, information about the movement of each macroblock of the enlarged image is generated based on information about the movement of each macroblock of the created virtual field. To perform the interlayer motion prediction, the method from the above case iv) is used in case V, since each macroblock of the image of the intermediate base layer constructed in this way is a macroblock of the field in the field image, and each macroblock of the current layer is also a macroblock of the field in the field image.

Although, for oversampling, the description of the embodiments shown in FIGS. 14a-14k uses texture information of a virtual field or frame of an intermediate layer, instead of information about the texture image of the base layer, the texture information of the image of the base layer can also serve as oversampling. In addition, in the absence of need, the interpolation process using texture information is excluded from the above-described oversampling process when outputting motion information in an image of an intermediate layer to be used for interlayer motion prediction, which is performed in a subsequent step.

On the other hand, although the description of texture prediction is given for the case where the base and improved layers have the same spatial resolution, both layers can have different spatial resolutions, as described above. In the case where the resolution of the enhanced layer is higher than the resolution of the base layer, operations are first performed to ensure that the image resolution of the base layer is equal to the resolution of the image of the enhanced layer to create an image of the base layer having the same resolution as the image of the enhanced layer, and, based on each macroblock in the image, a texture prediction method corresponding to each of the above cases IV is selected to perform predictive coding. The following is a description of the procedure for ensuring that the resolution of the image of the base layer is equal to the resolution of the image of the enhanced layer.

When two layers for interlayer prediction are considered, the number of combinations of image formats (progressive and interlaced formats) for encoding between two layers is four, since there are two methods of scanning a video signal, one method of progressive scanning and another method of interlacing. Therefore, a method for increasing the resolution of base layer images to perform interlayer texture prediction will be described separately for each of the four cases.

T_A). The case when the enhanced layer is progressive and the base layer is interlaced

On figa presents an embodiment of a method of using the image of the base layer for interlayer prediction of the texture in this case. As shown, the base layer image 1501, coinciding in time with the image 1500 of the current (improved) layer, contains even and odd fields that are output at different points in time. Therefore, first, the EL encoder 20 divides the image of the base layer into even and odd fields (S151). Intra macroblocks of the base layer image 1501 contain source video data that is not encoded (or video data that is encoded) and which serve for intra-frame prediction of the base layer, and inter-macroblocks of the image contain encoded residual data (or decoded residual data) that serve to predict residual data. The same is true for macroblocks or base layer images in the texture prediction described below.

After dividing the corresponding image 1501 into field components, the EL encoder 20 interpolates the dividing fields 1501a and 1501b in the vertical and / or horizontal direction to create enlarged even and odd images 1502a and 1502b (S152). When interpolating, one of a number of known methods is used, for example, 6-point filtering and binary-linear filtering. The vertical and horizontal ratios for increasing the resolution (i.e., size) of the image by interpolation are equal to the vertical and horizontal ratios of the image size 1500 of the enhanced layer to the image size 1501 of the base layer. Vertical and horizontal relationships can be equal to each other. For example, if the resolution ratio between the enhanced and base layers is 2, then interpolation is performed on the odd and even fields 1501a and 1501b obtained by separating to create one additional pixel between each pixel in each field in the vertical and horizontal directions.

After the interpolation is completed, the enlarged even and odd fields 1502a and 1502b are combined to create an image 1503 (S153). With this combination, the lines of enlarged even and odd fields 1502a and 1502b are selected alternately (1502a -> 1502b -> 1502a -> 1502b-> ...) and then line up in the selected order to build the combined image 1503. In this case, the block mode of each macroblock in the combined image 1503. For example, the macroblock block mode of the combined image 1503 is set to be identical to the macroblock mode in the base layer image 1501, which contains an area containing the same video component. The aforementioned determination method is applicable in any case of image enlargement described below. Since the combined image 1503 constructed in this way has a spatial resolution equal to the spatial resolution of the current enhanced layer image 1500, texture prediction (for example, inter-frame texture prediction of inter-macroblocks) of the macroblocks in the current progressive image 1500 is performed based on the corresponding macroblocks of the combined image 1503 (S154 )

FIG. 15b shows a method of using a base layer image for interlayer texture prediction in accordance with another embodiment of the present invention. As shown, in this embodiment, the interpolation of the image of the base layer containing the even and odd fields that are output at different points in time, in the vertical and / or horizontal direction, directly, without dividing the image of the base layer based on the attributes (parity) of the fields (S155 ) to build an enlarged image with the same resolution (i.e. size) as the resolution of the image of the enhanced layer. An enlarged image constructed in this way is used to perform interlayer texture prediction of the current progressive image of the enhanced layer (S156).

FIG. 15a shows, at the image level, an interpolation procedure for an image containing even and odd fields by means of an image based on field attributes. However, the EL encoder 20 can obtain the same results as those shown in FIG. 15a by performing the procedure shown in FIG. 15a at the macroblock level. In particular, when a base layer containing even and odd fields is encoded in MBAFF mode, a pair of vertically adjacent macroblocks in image 1501 that are aligned with a pair of macroblocks in the enhanced layer image, which should currently be encoded with texture prediction, may contain video signals components of even and odd fields, as in figa or 16b. Fig. 16a shows a transmission mode of pairs of macroblock frames in which the even and odd field components are interleaved in each of the pair of macroblocks A and B, and Fig. 16b shows a transmission mode of pairs of macroblock fields in which each of the pair of macroblocks A and B contains scan lines with the same field attribute.

In the case shown in FIG. 16a, in order to apply the method shown in FIG. 15a, even lines of each of the pair of macroblocks A and B are selected to construct an even field block A ′, and its odd lines are selected to construct an odd field block B ′, which divides the pair of macroblocks containing the components of the even and odd fields, interleaved in each macroblock, into two blocks A 'and B', containing, respectively, the components of the even and odd fields. Interpolation is performed on each of the two macroblocks A ′ and B ′ obtained by dividing in the aforementioned manner to construct an enlarged block. Texture prediction is performed using data in the region in the enlarged block, which corresponds to the macroblock in the intra-frame or inter-frame prediction mode, respectively, according to the base layer (intra_BL) or residual data (residual_prediction) in the image of the enhanced layer, which should currently be encoded with texture prediction. Although not shown in FIG. 16a, combining separately enlarged blocks based on field attributes partially constructs enlarged even and odd images 1502a and 1502b in FIG. 15a, and therefore enlarged even and odd images 1502a and 1502b in FIG. 15a can be constructed by repeating the above operations for each pair of macroblocks.

In the case where the pair of macroblocks is separated based on the field attributes for constructing each macroblock, as in FIG. 16b, the above separation procedure is the process of simply copying each of the pair of macroblocks to build two separate macroblocks. The following procedure is similar to the procedure described with reference to figa.

T_B). The case when the enhanced layer is interlaced and the base layer is progressive

On figa presents an embodiment of a method of using the image of the base layer for interlayer prediction of the texture in this case. As shown, first, the EL encoder 20 constructs two images for the image 1700 of the current layer (S171). In an exemplary method for constructing two images, even lines of the corresponding image 1701 are selected to build one image 1701a, and its odd lines are selected to build another image 1701b. Then, the EL encoder 20 interpolates said two constructed images 1701a and 1701b in the vertical and / or horizontal direction to create two enlarged images 1702a and 1702b (S172). With such interpolation, one of a number of known methods is used, for example, 6-point filtering and binary-linear filtering, as in the case of T_A). Relations for increasing resolution are the same as relations described in case of T_A).

After the interpolation is completed, the two enlarged fields 1702a and 1702b are combined to create an image 1703 (S173). With this combination, the lines of the two enlarged fields 1702a and 1702b are selected alternately (1702a -> 1702b -> 1702a -> l702b -> ...) and then line up in the selected order to build the combined image 1703. Since the combined image 1703 constructed in this way has spatial resolution equal to the spatial resolution of the current image 1700 enhanced layer, texture prediction (for example, inter-frame texture prediction of inter-macroblocks or texture prediction described with reference to fig.4g) macroblocks in the current h the interlacing image 1700 is performed based on the corresponding macroblocks of the combined image 1703 (S174).

On fig.17b presents a method of using images of the base layer in the interlayer prediction of the texture in accordance with another embodiment of the present invention. As shown, in this embodiment, the base layer image is interpolated in the vertical and / or horizontal direction, directly, without dividing the base layer image into two images, (S175) to construct an enlarged image of the same resolution (i.e., size) as image resolution of the enhanced layer. An enlarged image constructed in this way is used to perform interlayer prediction of the texture of the current interlaced image of the enhanced layer (S176).

Although FIG. 17a is also described at the image level, the EL encoder 20 may perform an image splitting process at the macroblock level, as described above in the case of T_A). The method shown in FIG. 17b is similar to the separation and interpolation procedure shown in FIG. 17a when a single image 1701 is considered to be a pair of vertically adjacent macroblocks. A detailed description of this procedure is not provided in this application because it is intuitive from FIG.

T_C). The case when both the enhanced and the base layers are interlaced

On Fig presents an embodiment of a method of using the image of the base layer for interlayer prediction of the texture in this case. In this case, as shown, the EL encoder 20 divides the base layer image 1801 corresponding to the time image 1800 of the current layer into even and odd fields (S181) in the same manner as in the case of T_A). Then, the EL encoder 20 interpolates the obtained separation fields 1801a and 1801b in the vertical and / or horizontal direction to create enlarged even and odd images 1802a and 1802b (S182). Then, the EL encoder 20 combines the enlarged even and odd images 1802a and 1802b to construct an image 1803 (S182). Then, the EL encoder 20 performs inter-layer texture prediction (for example, inter-frame texture prediction of inter-macroblocks or texture prediction described with reference to Fig. 4g) of macroblocks (pairs of macroblock frames encoded in MBAFF mode) in the current interlaced image 1800 based on the corresponding macroblocks the combined image 1803 (S184).

Although both layers have the same image format, the EL encoder 20 separates the base layer image 1801 based on the field attributes (S181) and separately enlarges the fields obtained by dividing (S182) and then combines the enlarged images (S183), because if the image 1801 combines even and the odd field, interpolate directly when it is characterized by the fact that the video signals of the even and odd fields change significantly, then the enlarged image may contain a distorted image (for example, an image containing lived borders) compared to the interlaced image 1800, containing alternated even and odd fields of the improved layer. Accordingly, even when both layers are interlaced, the EL encoder 20 uses the image of the base layer after it is divided based on the field attributes to obtain two fields and separately increase the two fields, and then combine the enlarged fields in accordance with the present invention.

Of course, instead of using the method shown in Fig. 18, always when the images of both layers are interlaced, the method can be used selectively, depending on the characteristics of the video signal of the images.

On Fig shows, at the image level, the procedure for dividing and enlarging an image containing even and odd fields based on the attributes of the fields, in accordance with the present invention. However, as described above in T_A), the EL encoder 20 can obtain the same results as those shown in FIG. 18 by performing, at the macroblock level, the procedure shown in FIG. 18, which contains the separation and interpolation processes for macroblocks described with reference to figa and 16b, (in particular, the separation of a pair of macroblock frames into blocks with even and odd lines and the separate increase obtained by the separation of blocks) and the processes of combining and interlayer texture prediction (in particular, alternately selecting rows of enlarged blocks for swarming pair of enlarged blocks and performing pair prediction texture of the current layer frame macroblocks using the constructed pair of enlarged blocks).

T_D). The case when both the advanced and the base layers are progressive

In this case, the image of the base layer is enlarged to the same size as the image size of the enhanced layer, and the enlarged image is used to interlayerly predict the texture of the current image of the enhanced layer having the same image format.

Although embodiments of texture prediction are described above for the case where the base and enhanced layers have the same time resolution, the two layers may have different time resolutions, i.e. different frame rates. If the images of the layers differ in the type of image scan, then even when the layers have the same temporal resolution, the images can contain video signals with different output times, even if they are images with the same POC (i.e., images corresponding in time to one another). The following is a description of the interlayer texture prediction method for such a case. In the following description, it is assumed that both layers initially have the same spatial resolution. If the layers have different spatial resolutions, then after oversampling each image of the base layer to ensure that its spatial resolution is equal to the spatial resolution of the enhanced layer, as described above, the methods described below are applied.

a) The case where the enhanced layer contains progressive frames, the base layer contains MBAFF frames, and the time resolution of the enhanced layer is twice as high

On figa presents a method of inter-layer prediction of motion for such a case. As shown, each MBAFF frame of the base layer contains even and odd fields that differ in output time, and therefore, the EL encoder 20 divides each MBAFF frame into even and odd fields (S191). The EL encoder 20 separates even field components (e.g., even lines) and odd field components (e.g., odd lines) of each MBAFF frame into an even field and an odd field, respectively. After dividing the MBAFF frame into two fields in the aforementioned manner, the EL encoder 20 interpolates each field in the vertical direction so that its resolution is doubled (S192). With such interpolation, one of a number of known methods is used, for example, 6-point filtering, binary-linear filtering and simple addition with insignificant lines. After the interpolation is completed, each frame of the enhanced layer contains a time-matching image in the base layer, and therefore, the EL encoder 20 performs the known inter-layer texture prediction (e.g., inter-frame inter-macroblock prediction) on the macroblocks of each frame of the enhanced layer (S193 )

The above procedure is also applicable to interlayer motion prediction. In this case, when dividing the MBAFF frame into two fields, the EL encoder 20 copies the information about the movement of each pair of field macroblocks in the MBAFF frame as information about the movement of a macroblock with the same attribute (parity) of the field in order to use it for interlayer motion prediction. Using this method, it is possible to create a time-consistent image in accordance with the above method for performing interlayer motion prediction, even when there is no time-matching image in the base layer (in the case of t1, t3, ...).

The above method can be applied directly when the resolution of one of the two layers is two times higher than the resolution of the other layer, as in the example of FIG. 19a, and even when the resolution is N (three or more) times higher. For example, when the resolution is three times higher, one of the two fields obtained by separation can be additionally copied to build and use three fields, and when the resolution is four times higher, each of the two fields obtained by separation can be copied again to build and use four fields . Obviously, for any difference in time resolution, those skilled in the art will be able to perform inter-layer prediction by simply applying the principles of the present invention, without any creative strain. Therefore, it is natural that any prediction method between layers with different temporal resolutions, not considered in the present description, is not beyond the scope of the present invention. The same is true for the other cases described below.

If the base layer is encoded in an image adaptive field coding (PAFF) mode, instead of MBAFF frames, both layers may have the same temporal resolution, as in FIG. 19b. Therefore, in this case, the interlayer texture prediction is performed after constructing an image having the same temporal resolution as the resolution of the current layer by direct frame interpolation, without the process of dividing the frame into two fields.

b) The case where the enhanced layer contains MBAFF frames, the base layer contains progressive frames, and the temporal resolution of the enhanced layer is half the temporal resolution of the base layer

20 shows an interlayer texture prediction method for such a case. As shown, each MBAFF frame of the enhanced layer contains even and odd fields with different output times, and therefore, the EL encoder 20 divides each MBAFF frame into even and odd fields (S201). The EL encoder 20 separates even field components (e.g., even lines) and odd field components (e.g., odd lines) of each MBAFF frame into an even field and an odd field, respectively. The EL encoder 20 sub-samples each frame of the base layer in the vertical direction to construct an image with a resolution reduced by half (S202). Mentioned downsampling may use line subsampling or one of many other known downsampling techniques. In the example of FIG. 20, the EL encoder 20 selects even lines of images with even image indices (images t0, t2, t4, ...) to obtain images halved in size, and selects odd lines of images with odd image indices ( images t1, t3, t5, ...) to obtain images reduced in size by half. Separation (S201) and downsampling (S202) of the frame may also be performed in the reverse order.

After the two processes S201 and S202 are completed, the fields 2001 extracted from the frames of the enhanced layer contain images that coincide in time with the fields 2001 and have the same spatial resolution as the resolution of the fields 2001 in the base layer, and therefore EL the encoder 20 performs known interlayer texture prediction (for example, frame-by-frame prediction of inter-macroblocks) on macroblocks in each field (S203).

The above procedure is also applicable to interlayer motion prediction. In this case, when acquiring a reduced size image from each frame of the base layer by downsampling (S202), the EL encoder 20 may obtain motion information of the corresponding macroblock from the motion information of each of a pair of vertically adjacent macroblocks in accordance with a suitable method (e.g., the method of selecting information about the movement of the block, which was not completely broken) and then can use the obtained information about the movement for inter-layer prediction of motion.

In this case, the enhanced layer images are encoded in PAFF mode for transmission, since interlayer prediction is performed on each field image 2001 extracted from MBAFF frames.

c) The case where the enhanced layer contains MBAFF frames, the base layer contains progressive frames, and both layers have the same temporal resolution

On Fig presents a method for interlayer texture prediction for such a case. As shown, each MBAFF frame of the enhanced layer contains even and odd fields that differ in output time, and therefore, the EL encoder 20 divides each MBAFF frame into even and odd fields (S211). The EL encoder 20 separates even field components (e.g., even lines) and odd field components (e.g., odd lines) of each MBAFF frame into an even field and an odd field, respectively. The EL encoder 20 sub-samples each frame of the base layer in the vertical direction to construct an image with a resolution that is halved (S212). Mentioned downsampling may use line subsampling or one of many other known downsampling techniques. Separation (S211) and downsampling (S212) of the frame can also be performed in the reverse order.

The EL encoder 20 may also construct a field (eg, an even field image) from an MBAFF frame, instead of dividing the MBAFF frame into two fields. This is possible due to the fact that both layers have the same temporal resolution, and, therefore, only one of the two field images extracted from one frame (and not both such images) corresponds to a frame in the base layer, which can be used for interlayer prediction.

After the two processes S211 and S212 are completed, the EL encoder 20 performs the known interlayer texture prediction (for example, inter-frame prediction of inter-macroblocks) only on even (odd) fields from fields extracted from frames of the enhanced layer, based on the corresponding sub-sampled images in the base layer (S213).

In this case, with respect to the field separation obtained by the enhanced layer for which the interlayer texture prediction is performed, the interlayer motion prediction can also be performed in the same manner as described in case b).

Although the above descriptions relate to inter-layer prediction operations performed by the EL encoder 20 shown in FIGS. 2a or 2b, all descriptions of inter-layer prediction operations can typically be applied to an EL decoder that receives decoded information from the base layer and decodes the enhanced streams layer. During the encoding and decoding procedures, the interlayer prediction operations described above (including operations for splitting, enlarging, and combining video signals in images or macroblocks) are performed in the same way, while the operations following the interlayer prediction are performed differently. An example of a difference is that, after performing motion and texture prediction, the encoder encodes the predicted information or the difference information between the predicted information and the actual information to transmit it to the decoder, while the decoder receives the actual motion information and texture information by directly applying information obtained by performing the same interlayer motion and texture prediction as the prediction made in the encoder to the current macroblock or by complementary full use of actually adopted information about the coding of macroblocks. The detailed data and principles of the present invention set forth above in terms of encoding are directly applicable to a decoder that decodes the received data streams of two layers.

However, when the EL encoder transmits the enhanced layer of MBAFF frames in PAFF mode after dividing the enhanced layer into a sequence of fields and performing inter-layer prediction, as described with reference to FIGS. 20 and 21, the decoder does not perform the above procedure for dividing MBAFF frames into field images in the current received layer.

In addition, then the decoder, from the received signal, the “field_base_flag” flag, which indicates whether the EL encoder 20 has performed interlayer texture prediction between macroblocks, as shown in FIG. 8d, or as shown in FIG. 8h. Based on the decoded flag value, the decoder decides whether the prediction between the macroblocks is performed, as shown in FIG. 8d or as shown in FIG. 8h, and obtains texture prediction information in accordance with the decision. If the flag "field_base_flag" is not received, then the EL decoder considers that a flag having a value of "0" is received. That is, the EL decoder considers that the texture prediction between the macroblocks is made in accordance with the method shown in FIG. 8d, and obtains the prediction information in the current pair of macroblocks to construct the current macroblock or pair of macroblocks.

At least one of the limited embodiments of the present invention described above can perform inter-layer prediction, even when using video sources in different formats (or modes). Therefore, when encoding multiple layers, the data encoding rate can be increased, regardless of the types of video signals of the images, for example, interlaced video signals, progressive signals, images MBAFF frames and field images. In addition, when one of the two sources is an interlaced video source, the video object in the image for use in prediction can be constructed more similar to the original video object for prediction encoding, which increases the encoding speed of the data.

The present invention has been described above with reference to preferred embodiments, but it will be apparent to those skilled in the art that various improvements, changes, and additions can be made to the present invention without departing from the scope and spirit of the invention. Therefore, it is intended that the invention covers the improvements, changes, substitutions and additions to the invention, provided that they do not go beyond the scope defined by the appended claims and their equivalents.

Claims (8)

1. A method of decoding an image signal, comprising stages in which
receive information on the virtual position of the corresponding pair of macroblock fields, and this corresponding pair of macroblock fields corresponds to a pair of macroblock fields of the improved layer;
receiving position information of a pair of macroblocks of a reference frame based on said virtual position information of a corresponding pair of macroblocks of fields, wherein this pair of macroblocks of a reference frame is on the base layer;
predicting motion information of said pair of macroblock fields of the enhanced layer fields based on motion information of said pair of macroblocks of a reference frame according to said position information of a pair of macroblocks of a reference frame; and
the said pair of macroblock fields of the enhanced layer fields are decoded using said motion information of the pair of macroblock fields of the enhanced layer fields. 2. The method according to claim 1, wherein said corresponding pair of field macroblocks is composed of an upper field macroblock and a lower field macroblock, wherein the upper part of the upper field macroblock is composed of even rows in the upper macroblock of said pair of macroblocks of the reference frame and the lower part of the upper field macroblock composed of even rows in a lower macroblock of said pair of macroblocks of a reference frame, wherein the upper part of a lower macroblock of a field is composed of odd rows in an upper macroblock of said pair of macroblocks of supports frame, and the lower part of the lower macroblock of the field is composed of odd rows in the lower macroblock of the said pair of macroblocks of the reference frame. 3. The method according to claim 1, wherein when said pair of macroblocks of a reference frame is composed of a macroblock encoded by intra-frame coding and a macroblock encoded by inter-frame coding, said motion information of a pair of macroblocks of enhanced layer fields is predicted based on motion information of a given macroblock encoded by inter-frame encoding. 4. The method according to claim 1, wherein said pair of macroblock fields of the enhanced layer fields are adaptively encoded as a macroblock of a frame or a macroblock of a field for each macroblock. 5. A device for decoding an image signal containing a decoding module that receives information about the virtual position of the corresponding pair of macroblock fields; obtains position information of a pair of macroblocks of a reference frame based on said virtual position information of a corresponding pair of macroblocks of fields; predicts motion information of a pair of macroblock fields of the enhanced layer based on motion information of said pair of macroblocks of a reference frame according to said position information of a pair of macroblocks of a reference frame; and decodes said pair of macroblock fields of the enhanced layer using said motion information of the pair of macroblock fields of the enhanced layer,
wherein said corresponding pair of macroblocks of the fields corresponds to said pair of macroblocks of the fields of the enhanced layer, and said pair of macroblocks of the reference frame is on the base layer. 6. The device according to claim 5, wherein said corresponding pair of field macroblocks is composed of an upper field macroblock and a lower field macroblock, wherein the upper part of the upper field macroblock is composed of even rows in the upper macroblock of said pair of macroblocks of the reference frame and the lower part of the upper field macroblock composed of even rows in a lower macroblock of said pair of macroblocks of a reference frame, wherein the upper part of a lower macroblock of a field is composed of odd rows in an upper macroblock of said pair of macroblocks a reference frame, and the lower part of the lower macroblock of the field is composed of odd rows in the lower macroblock of said pair of macroblocks of the reference frame. 7. The device according to claim 5, in which, when said pair of macroblocks of the reference frame is composed of a macroblock encoded by intraframe coding and a macroblock encoded by interframe coding, said motion information of a pair of macroblocks of enhanced layer field fields is predicted based on the motion information of this macroblock encoded by interframe encoding. 8. The device according to claim 5, in which said pair of macroblock fields of the enhanced layer is encoded adaptively as a macroblock of a frame or a macroblock of a field for each macroblock.

Images