KR102722391B1 - Method and apparatus for image encoding/decoding - Google Patents

Abstract

A video encoding/decoding method and device supporting multiple layers are disclosed. The video decoding method supporting multiple layers includes a step of decoding information of a first layer referenced by a picture of a second layer including a current decoding target block, a step of mapping the information of the first layer to a picture size of the second layer, a step of adding the mapped first layer information to configure a reference picture list for a picture of the second layer, and a step of performing prediction on a current decoding target block of the second layer based on the reference picture list to generate prediction samples of the current decoding target block.

Description

METHOD AND APPARATUS FOR IMAGE ENCODING/DECODING

The present invention relates to image encoding and decoding, and more particularly, to a method for predicting, encoding, and decoding an image of an upper layer using information of a lower layer when encoding and decoding an image of a multi-layer structure.

Recently, as the multimedia environment is being built, various terminals and networks are being used, and user demands are also diversifying accordingly.

For example, as the performance and computing capability of terminals become more diverse, the supported performance also becomes more diverse for each device. In addition, the network through which information is transmitted is also becoming more diverse not only in terms of external structure such as wired and wireless networks, but also in terms of functions such as the form, amount, and speed of the information being transmitted. Users select the terminal and network to use according to the desired function, and the spectrum of terminals and networks that companies provide to users is also becoming more diverse.

In this regard, as broadcasting with HD (High Definition) resolution has recently expanded and become available not only domestically but also internationally, many users are becoming accustomed to high-resolution, high-quality images. Accordingly, many video service-related organizations are putting a lot of effort into developing next-generation video devices.

In addition, as interest in UHD (Ultra High Definition), which has a resolution more than four times that of HDTV, increases, the demand for technology that compresses and processes higher resolution, high-definition images is increasing.

In order to compress and process an image, an inter prediction technique that predicts pixel values included in a current picture from temporally previous and/or subsequent pictures, an intra prediction technique that predicts other pixel values included in the current picture using pixel information in the current picture, and an entropy encoding technique that assigns short codes to symbols with a high appearance frequency and long codes to symbols with a low appearance frequency can be used.

As described above, considering the different functions supported by each terminal and network, and the diverse needs of users, the quality, size, and frame of the supported video also need to be diversified accordingly.

In this way, scalability, which supports various image quality, resolution, size, frame rate, etc. due to heterogeneous communication networks and various functions and types of terminals, is becoming an important function of video formats.

Therefore, in order to provide services required by users in various environments based on a highly efficient video encoding method, it is necessary to provide a scalability function to enable efficient video encoding and decoding in terms of time, space, and image quality.

The present invention provides a method and device for encoding/decoding an upper layer using information of a lower layer in scalable video coding.

The present invention provides a method and apparatus for mapping lower layer images in scalable video coding.

The present invention provides a method and device for constructing a reference picture list of an upper layer image and performing prediction using a lower layer image in scalable video coding.

According to one embodiment of the present invention, an image decoding method supporting a plurality of layers is provided. The image decoding method includes a step of decoding information of a first layer referenced by a picture of a second layer including a current decoding target block, a step of mapping the information of the first layer to a picture size of the second layer, a step of adding the mapped information of the first layer to configure a reference picture list for the picture of the second layer, and a step of performing prediction on a current decoding target block of the second layer based on the reference picture list to generate prediction samples of the current decoding target block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, a video decoding device supporting a plurality of layers is provided. The video decoding device includes a decoding unit which decodes information of a first layer referenced by a picture of a second layer including a current decoding target block, and a prediction unit which maps the information of the first layer to a picture size of the second layer, adds the mapped information of the first layer to form a reference picture list for the picture of the second layer, and performs prediction on a current decoding target block of the second layer based on the reference picture list to generate prediction samples of the current decoding target block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, an image encoding method supporting a plurality of layers is provided. The image encoding method includes a step of decoding information of a first layer referenced by a picture of a second layer including a current encoding target block, a step of mapping the information of the first layer to a picture size of the second layer, a step of adding the mapped information of the first layer to configure a reference picture list for the picture of the second layer, and a step of performing prediction on a current encoding target block of the second layer based on the reference picture list to generate prediction samples of the current encoding target block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, an image encoding device supporting a plurality of layers is provided. The image encoding device includes an encoding unit which encodes information of a first layer referenced by a picture of a second layer including a current encoding target block, and a prediction unit which maps the information of the first layer to a picture size of the second layer, adds the mapped information of the first layer to form a reference picture list for the picture of the second layer, and performs prediction on a current encoding target block of the second layer based on the reference picture list to generate prediction samples of the current encoding target block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

In the prior art, there is a problem that an unnecessary motion information mapping process is performed even when temporal motion vector prediction of an upper layer is not performed in the process of mapping motion information of a lower layer. In addition, when motion information mapping of a lower layer is not performed, since a decoded picture of a lower layer can be indicated as a corresponding picture (collocated picture) for a temporal motion vector in the upper layer, there is a problem that temporal motion vector prediction of the upper layer cannot be performed, resulting in a decrease in encoding efficiency.

According to the present invention, by mapping motion information of a lower layer referenced by an upper layer to an image size of the upper layer and then predicting and restoring the upper layer using the image signal of the mapped lower layer, encoding/decoding efficiency can be improved. In addition, by modifying the upper level syntax, the mapping process for unnecessary motion information of the lower layer can be omitted, and complexity can be reduced. In addition, the use of an incorrect corresponding picture can be prevented, thereby preventing a decrease in encoding efficiency.

FIG. 1 is a block diagram showing the configuration of an image encoding device to which the invention is applied according to one embodiment.
FIG. 2 is a block diagram showing the configuration of an image decoding device according to one embodiment of the present invention.
FIG. 3 is a conceptual diagram schematically illustrating one embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied.
FIG. 4 is a flowchart schematically illustrating a method for performing inter-layer prediction (inter-layer prediction) in scalable video coding according to an embodiment of the present invention.
FIGS. 5 and 6 are diagrams illustrating a method for mapping inter-layer motion information according to one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In describing the embodiments of this specification, if it is determined that a specific description of a related known configuration or function may obscure the gist of this specification, such description may be omitted.

When it is mentioned in this specification that a component is “connected” or “connected” to another component, it may mean that it is directly connected or connected to that other component, or it may mean that there are other components in between. In addition, the description in this specification that a specific configuration “includes” does not exclude configurations other than the configuration concerned, and it means that additional configurations may be included in the scope of the implementation of the present invention or the technical idea of the present invention.

The terms first, second, etc. may be used to describe various configurations, but the configurations are not limited by the terms. The terms are used to distinguish one configuration from another. For example, without departing from the scope of the present invention, the first configuration may be referred to as the second configuration, and similarly, the second configuration may also be referred to as the first configuration.

In addition, the components shown in the embodiments of the present invention are independently depicted to indicate different characteristic functions, and do not mean that each component is formed as a separate hardware or software configuration unit. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may form one component, or one component may be divided into multiple components to perform a function. Integrated embodiments and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

In addition, some components may not be essential components that perform essential functions in the present invention, but may be optional components that are merely used to improve performance. The present invention may be implemented by including only essential components for implementing the essence of the present invention, excluding components that are merely used to improve performance, and a structure that includes only essential components, excluding optional components that are merely used to improve performance, is also included in the scope of the present invention.

FIG. 1 is a block diagram showing the configuration of an image encoding device to which the invention is applied according to one embodiment.

A scalable video encoding device supporting a multi-layer structure can be implemented by extending a general video encoding device having a single layer structure. The block diagram of Fig. 1 illustrates an embodiment of a video encoding device that can be the basis of a scalable video encoding device applicable to a multi-layer structure.

Referring to FIG. 1, an image encoding device (100) includes an inter prediction unit (110), an intra prediction unit (120), a switch (115), a subtractor (125), a transform unit (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse transform unit (170), an adder (175), a filter unit (180), and a reference picture buffer (190).

An image encoding device (100) can perform encoding on an input image in intra mode or inter mode and output a bitstream.

In the case of intra mode, the switch (115) may be switched to intra, and in the case of inter mode, the switch (115) may be switched to inter. Intra prediction means prediction within a screen, and inter prediction means prediction between screens. The image encoding device (100) may generate a prediction block for an input block of an input image, and then encode the difference (residual) between the input block and the prediction block. At this time, the input image may mean an original picture.

In the case of intra mode, the intra prediction unit (120) can use the sample values of already encoded/decoded blocks surrounding the current block as reference samples. The intra prediction unit (120) can perform spatial prediction using the reference samples and generate prediction samples for the current block.

In the case of inter mode, the inter prediction unit (110) can obtain a motion vector that specifies a reference block with the smallest difference from the input block (current block) among the reference pictures stored in the reference picture buffer (190) during the motion prediction process. The inter prediction unit (110) can perform motion compensation using the motion vector and the reference picture stored in the reference picture buffer (190) to generate a prediction block for the current block.

In the case of a multi-layer structure, the inter prediction applied in the inter mode may include inter-layer prediction. The inter-prediction unit (110) may sample pictures of a reference layer to configure inter-layer reference pictures, and perform inter-layer prediction by including the inter-layer reference pictures in a reference picture list. The reference relationship between layers may be signaled through information specifying the dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generation of reference samples by copying or interpolating samples from the reference layer picture. When the current layer picture and the reference layer picture have different resolutions, sampling applied to the reference layer picture may mean upsampling.

For example, in cases where resolutions are different between layers, an inter-layer reference picture can be constructed by upsampling the restored picture of the reference layer between layers that support resolution scalability.

A decision on which layer's picture to use to construct an inter-layer reference picture can be made by considering encoding cost, etc. An encoding device can transmit information specifying a layer to which a picture to be used as an inter-layer reference picture belongs to a decoding device.

Additionally, in inter-layer prediction, the referenced layer, that is, the picture used for prediction of the current block within the reference layer, may be a picture of the same AU (Access Unit) as the current picture (the target picture to be predicted within the current layer).

A subtractor (125) can generate a residual block by the difference between an input block and a generated prediction block.

The transform unit (130) can perform a transform on the residual block and output a transform coefficient. Here, the transform coefficient can mean a coefficient value generated by performing a transform on the residual block and/or the residual signal. Hereinafter, in the present specification, a quantized transform coefficient level generated by applying quantization to the transform coefficient can also be called a transform coefficient.

When the transform skip mode is applied, the transform unit (130) may skip the transformation for the residual block.

The quantization unit (140) can quantize the input transform coefficient according to a quantization parameter and output a quantized coefficient. The quantized coefficient may also be called a quantized transform coefficient level. At this time, the quantization unit (140) can quantize the input transform coefficient using a quantization matrix.

The entropy encoding unit (150) can output a bitstream by entropy encoding values produced by the quantization unit (140) or encoding parameter values produced during the encoding process according to a probability distribution. In addition to pixel information of the video, the entropy encoding unit (150) can also entropy encode information for video decoding (e.g., syntax elements, etc.).

Encoding parameters are information required for encoding and decoding, and may include not only information encoded in an encoding device and transmitted to a decoding device, such as syntax elements, but also information that can be inferred during the encoding or decoding process.

For example, encoding parameters may include values or statistics such as intra/inter prediction mode, translation/motion vector, reference picture index, encoded block pattern, presence or absence of residual signal, transform coefficients, quantized transform coefficients, quantization parameters, block size, and block partition information.

The residual signal can mean the difference between the original signal and the predicted signal, or it can mean the signal in the form of a transformed difference between the original signal and the predicted signal, or the signal in the form of a transformed and quantized difference between the original signal and the predicted signal. The residual signal can be called a residual block in block units.

When entropy coding is applied, a small number of bits are allocated to symbols with a high occurrence probability, and a large number of bits are allocated to symbols with a low occurrence probability, thereby representing symbols, so that the size of the bit string for symbols to be encoded can be reduced. Therefore, the compression performance of image encoding can be improved through entropy coding.

The entropy encoding unit (150) may use encoding methods such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) for entropy encoding. For example, the entropy encoding unit (150) may perform entropy encoding using a Variable Length Coding/Code (VLC) table. In addition, the entropy encoding unit (150) may derive a binarization method of a target symbol and a probability model of a target symbol/bin, and then perform entropy encoding using the derived binarization method or probability model.

Since the image encoding device (100) according to the embodiment of Fig. 1 performs inter-prediction encoding, i.e., inter-screen prediction encoding, the currently encoded image needs to be decoded and stored in order to be used as a reference image. Accordingly, the quantized coefficients can be inversely quantized in the inverse quantization unit (160) and inversely transformed in the inverse transformation unit (170). The inversely quantized and inversely transformed coefficients are added to the prediction block through the adder (175), and a reconstructed block is generated.

The restoration block passes through the filter unit (180), and the filter unit (180) can apply at least one of a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) to the restoration block or the restoration picture. The filter unit (180) may also be called an adaptive in-loop filter. The deblocking filter can remove block distortion occurring at the boundary between blocks. The SAO can add an appropriate offset value to a pixel value in order to compensate for a coding error. The ALF can perform filtering based on a value obtained by comparing the restored image with the original image. The restoration block that has passed through the filter unit (180) can be stored in the reference picture buffer (190).

FIG. 2 is a block diagram showing the configuration of an image decoding device according to one embodiment of the present invention.

A scalable video decoding device supporting a multi-layer structure can be implemented by extending a general video decoding device having a single layer structure. The block diagram of Fig. 2 illustrates an embodiment of a video decoding device that can be the basis of a scalable video decoding device applicable to a multi-layer structure.

Referring to FIG. 2, the image decoding device (200) includes an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), an inter prediction unit (250), an adder (255), a filter unit (260), and a reference picture buffer (270).

The image decoding device (200) can receive a bitstream output from an encoder, perform decoding in intra mode or inter mode, and output a reconstructed image, i.e., a restored image.

When in intra mode, the switch can be switched to intra, and when in inter mode, the switch can be switched to inter.

The video decoding device (200) can obtain a reconstructed residual block from an input bitstream, generate a prediction block, and then add the reconstructed residual block and the prediction block to generate a reconstructed block, i.e., a restored block.

The entropy decoding unit (210) can entropy decode the input bitstream according to a probability distribution and output information such as quantized coefficients and syntax elements.

The quantized coefficients are inversely quantized in the inverse quantization unit (220) and inversely transformed in the inverse transformation unit (230). As a result of the inverse quantization/inverse transformation of the quantized coefficients, a restored residual block can be generated. At this time, the inverse quantization unit (220) can apply a quantization matrix to the quantized coefficients.

In the case of intra mode, the intra prediction unit (240) can perform spatial prediction using sample values of already decoded blocks surrounding the current block and generate prediction samples for the current block.

In the inter mode, the inter prediction unit (250) can generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer (270).

In the case of a multi-layer structure, the inter prediction applied in the inter mode may include inter-layer prediction. The inter-prediction unit (250) may sample pictures of a reference layer to configure inter-layer reference pictures, and perform inter-layer prediction by including the inter-layer reference pictures in a reference picture list. The reference relationship between layers may be signaled through information specifying the dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generation of reference samples by copying or interpolating samples from the reference layer picture. When the current layer picture and the reference layer picture have different resolutions, sampling applied to the reference layer picture may mean upsampling.

For example, if inter-layer prediction is applied between layers that support resolution scalability in cases where the resolutions between layers are different, an inter-layer reference picture can be constructed by upsampling the restored picture of the reference layer.

At this time, information specifying the layer to which a picture to be used as an inter-layer reference picture belongs can be transmitted from the encoding device to the decoding device.

Additionally, in inter-layer prediction, the referenced layer, that is, the picture used for prediction of the current block within the reference layer, may be a picture of the same AU (Access Unit) as the current picture (the target picture to be predicted within the current layer).

The restored residual block and the predicted block are added in an adder (255) to generate a restored block. In other words, the residual sample and the predicted sample are added to generate a restored sample or restored picture.

The reconstructed picture is filtered in the filter unit (260). The filter unit (260) can apply at least one of a deblocking filter, SAO, and ALF to the reconstructed block or the reconstructed picture. The filter unit (260) outputs a modified or filtered reconstructed picture. The reconstructed image can be stored in the reference picture buffer (270) and used for inter prediction.

In addition, the image decoding device (200) may further include a parsing unit (not shown) that parses information related to an encoded image included in a bitstream. The parsing unit may include an entropy decoding unit (210) or may be included in the entropy decoding unit (210). This parsing unit may also be implemented as a component of the decoding unit.

In FIGS. 1 and 2, it is described that one encoding/decoding device processes all encoding/decoding for multiple layers, but this is for convenience of explanation, and the encoding/decoding device may be configured for each layer.

In this case, the encoding/decoding device of the upper layer can perform encoding/decoding of the upper layer using information of the upper layer and information of the lower layer. For example, the prediction unit (inter prediction unit) of the upper layer can perform intra prediction or inter prediction for the current block using pixel information or picture information of the upper layer, or can receive picture information reconstructed from the lower layer and perform inter prediction (inter-layer prediction) for the current block of the upper layer using the same. Although only inter-layer prediction is described as an example here, the encoding/decoding device can perform encoding/decoding for the current layer using information of another layer regardless of whether it is configured for each layer or whether one device processes multiple layers.

In the present invention, a layer may include a view. In this case, in the case of inter-layer prediction, rather than simply using information of a lower layer to perform prediction of an upper layer, inter-layer prediction may be performed using information of another layer between layers that are specified as having dependency by information specifying inter-layer dependency.

Fig. 3 is a conceptual diagram schematically illustrating one embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied. In Fig. 3, GOP (Group of Picture) represents a group of pictures, i.e., a group of pictures.

In order to transmit video data, a transmission medium is required, and its performance varies depending on the transmission medium and various network environments. A scalable video coding method can be provided for application to such various transmission media or network environments.

A video coding method supporting scalability (hereinafter referred to as “scalable coding” or “scalable video coding”) is a coding method that removes redundancy between layers by utilizing texture information, motion information, and residual signals between layers to improve encoding and decoding performance. The scalable video coding method can provide various scalabilities in terms of spatial, temporal, picture quality, and view, depending on surrounding conditions such as transmission bit rate, transmission error rate, and system resources.

Scalable video coding can be performed using a multiple layers structure so as to provide a bitstream applicable to various network situations. For example, the scalable video coding structure can include a base layer that compresses and processes video data using a general video decoding method, and an enhancement layer that compresses and processes video data using decoding information of the base layer and a general video decoding method together.

The base layer may be referred to as the base layer or the lower layer. The enhancement layer may be referred to as the enhancement layer or the higher layer. In this case, the lower layer may refer to a layer that supports lower scalability than a specific layer, and the higher layer may refer to a layer that supports higher scalability than a specific layer. In addition, a layer that is referenced for encoding/decoding of another layer may be referred to as a reference layer (reference layer), and a layer that is encoded/decoded using another layer may be referred to as a current layer (current layer). The reference layer may be a lower layer than the current layer, and the current layer may be a higher layer than the reference layer.

Here, a layer refers to a set of images and bitstreams that are distinguished based on spatial (e.g., image size), temporal (e.g., decoding order, image output order, frame rate), image quality, complexity, view, etc.

Referring to FIG. 3, for example, the base layer may be defined as SD (standard definition), a frame rate of 15 Hz, and a bit rate of 1 Mbps, the first enhancement layer may be defined as HD (high definition), a frame rate of 30 Hz, and a bit rate of 3.9 Mbps, and the second enhancement layer may be defined as 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps.

The above format, frame rate, bit rate, etc. are only examples and may be determined differently as needed. Also, the number of layers used is not limited to this example and may be determined differently depending on the situation. For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to 15 Hz or less for transmission.

The scalable video coding method can provide temporal, spatial, image quality, and view scalability by the method described in the embodiment of FIG. 3. In this specification, scalable video coding has the same meaning as scalable video encoding from an encoding perspective, and scalable video decoding from a decoding perspective.

Typically, inter-prediction (hereinafter, inter prediction) uses at least one of the previous picture or the next picture of the current picture as a reference picture, and can perform prediction for the current block based on the reference picture.

The image used to predict the current block is called a reference picture or reference frame.

The region (reference block) used for prediction of the current block within the reference picture can be represented using a reference picture index (refIdx) indicating the reference picture and a motion vector.

Inter prediction can generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block within the reference picture. At this time, inter prediction can generate a prediction block so that the residual signal with respect to the current block is minimized and the motion vector size is also minimized.

In order to utilize information of reference pictures in inter prediction, information of surrounding blocks located around the current block can be utilized. For example, inter prediction can apply skip mode, merge mode, AMVP (Advanced Motion Vector Prediction), etc. depending on the method of utilizing information of surrounding blocks. These skip mode, merge mode, and AMVP mode can generate a prediction block for the current block based on information of surrounding blocks.

Skip mode can use the information of surrounding blocks as is for the current block. Therefore, when skip mode is applied, the encoder only needs to transmit information indicating which surrounding block's motion information to use as the motion information of the current block to the decoder, and does not transmit any syntax information such as residuals to the decoder.

The merge mode can generate a prediction block for the current block by directly using the motion information of the surrounding blocks. When the merge mode is applied, the encoder can transmit information indicating whether to apply the merge mode for the current block, information indicating which motion information of the surrounding block to use, residual information for the current block, etc. to the decoder. The decoder can generate a prediction block for the current block by using the motion information of the surrounding blocks indicated by the encoder, and can reconstruct the current block by adding the generated prediction block and the residual transmitted from the encoder.

AMVP can predict the motion vector of the current block using the motion information of the surrounding blocks. When AMVP is applied, the encoder can transmit to the decoder information indicating which motion information of the surrounding block is used, the difference between the motion vector of the current block and the predicted motion vector, and a reference picture index indicating the reference picture. The decoder can predict the motion vector of the current block using the motion information (motion vector) of the surrounding blocks, and derive the motion vector for the current block using the difference between the motion vectors received from the encoder. The decoder can generate a prediction block for the current block based on the derived motion vector and the reference picture index information received from the encoder.

In the case of inter prediction, the decoder can check the skip flag, merge flag, etc. received from the encoder, and derive motion information (e.g., information about motion vector, reference picture index, etc.) required for inter prediction of the current block based on the checked information.

The processing unit where prediction is performed may be different from the processing unit where the prediction method and specific content are determined. For example, inter prediction or intra prediction may be determined on a coding unit (CU: Coding Unit) basis, and the prediction mode for intra prediction and the prediction mode for inter prediction may be determined on a prediction unit (PU: Prediction Unit) basis. Alternatively, the prediction mode may be determined on a prediction unit basis, and prediction may be performed on a transform unit (TU: Transform Unit) basis.

Meanwhile, in a scalable video coding structure that supports multiple layers, since there is a strong correlation between layers, if prediction is performed using this correlation, redundant elements of data can be removed and the encoding performance of the image can be improved. Therefore, when predicting a picture (image) of the current layer (upper layer) to be encoded/decoded, not only inter prediction or intra prediction using information of the current layer, but also inter-layer prediction (or inter-layer prediction) using information of another layer can be performed.

Since multiple layers may differ in at least one of resolution, frame rate, color format, and viewpoint (i.e., scalability differences between layers), signal distortion may occur or residual signals may increase during inter-layer prediction for the current layer.

Therefore, the present invention provides a method for performing inter prediction for the current layer by mapping motion information of a reference layer (lower layer) referenced by the current layer (upper layer) to the image size of the current layer and then adding the information to the reference picture list of the current layer.

In addition, the present invention relates to encoding and decoding of an image including a plurality of layers or views, wherein the plurality of layers or views can be expressed as a first, a second, a third, an n-th layer or a first, a second, a third, an n-th view.

Hereinafter, in the embodiments of the present invention, for the sake of convenience of explanation, an image having a first layer and a second layer is described as an example, but the same method can be applied to an image having more layers or viewpoints. In addition, the first layer can be expressed as a base layer or a basic layer or a reference layer, and the second layer can be expressed as an enhancement layer or an enhancement layer or a current layer.

FIG. 4 is a flowchart schematically illustrating a method for performing inter-layer prediction (inter-layer prediction) in scalable video coding according to an embodiment of the present invention. The method of FIG. 4 can be performed in the video encoding device of FIG. 1 and the video decoding device of FIG. 2 described above.

Referring to FIG. 4, the encoding/decoding device decodes information of the first layer referenced by the image (picture) of the second layer (S400).

As described above, the second layer refers to a layer that currently performs encoding/decoding, and may be a higher layer that provides higher scalability than the first layer. The first layer may be a layer referenced for encoding/decoding of the second layer, and may be referred to as a reference layer or a lower layer.

The encoding/decoding device can decode and use information of the first layer as a reference signal for predicting a target block to be encoded/decoded within an image of the second layer.

Information of the first layer to be decoded (information on the decoding target of the first layer) may include sample values and motion information of the first layer image (picture). Here, the motion information may be information about a motion vector value, a reference picture index, a prediction direction indicator, a reference picture POC (Picture Order Count), a prediction mode, a reference picture list, a merge flag, a merge index, a picture type of a reference picture (short-term reference picture, long-term reference picture), etc.

The decoded motion information of the first layer can be compressed and stored in unit blocks of NxN size. For example, the decoded motion information of the first layer can be compressed and stored in 16x16 blocks.

The encoding/decoding device maps the decoded first layer information to the image size of the second layer (S410).

Here, mapping may be sampling of images to adjust image sizes or resolutions to be the same when images have different sizes or resolutions, and may include sampling of image sample values and sampling of image motion information.

In other words, the encoding/decoding device can perform sampling on the decoded first layer image to map the decoded first layer image to the size of the second layer image, and can refer to sample values and motion information of the sampled first layer image.

In a scalable video coding structure, since the sizes of images between layers may be different, when the sizes of images between layers (between the first layer and the second layer) are different, the encoding/decoding device may perform resampling on sample values of the decoded first layer image to map the size of the decoded first layer image to the size of the second layer image.

For example, if the image size of the decoded first layer is 960x540 and the image size of the second layer is 1920x1080, the encoding/decoding device can perform upsampling to map the image size of the decoded first layer to the image size of 1920x1080, which is the image size of the second layer. Upsampling can be performed to adjust the image size between layers having different image sizes, and interpolated samples can be derived by applying interpolation to the image of a layer (reference layer or lower layer) having a small image size, so that the image size can be adjusted.

The encoding/decoding device can use the decoded motion information of the first layer by mapping it to the image size of the second layer. The inter-layer motion information mapping that maps the motion information of the decoded first layer image to the image size of the second layer can divide the second layer image into NxN sized unit blocks, and use the motion information of the first layer corresponding to each NxN unit block of the second layer as the motion information of the corresponding block of the second layer.

Hereinafter, a method for mapping inter-layer motion information according to one embodiment of the present invention will be described through the embodiments of FIGS. 5 and 6.

FIGS. 5 and 6 are diagrams illustrating a method for mapping inter-layer motion information according to one embodiment of the present invention.

Figure 5 shows an image of the second layer divided into unit blocks of NxN size.

For example, as illustrated in FIG. 5, when the image size of the second layer is 1920x1080 and N=16, the encoding/decoding device can divide the image of the second layer into a total of 8160 16x16 unit blocks, and can use the motion information of the corresponding first layer block for each unit block of the second layer as the motion information of the corresponding block of the second layer.

At this time, the motion information can be stored for each unit block. Alternatively, the motion information can be stored for each picture in the form of a look-up table so that each unit block can reference and use it.

Fig. 6 shows an NxN block of the second layer, and as an example, a 16x16 block (600) is illustrated when N=16. Each square in the 16x16 block (600) represents one sample, and the location of the uppermost left sample in the 16x16 block (600) is assumed to be (xP, yP).

As described above, when mapping motion information between layers, the motion information of the first layer block corresponding to each NxN unit block of the second layer is mapped and used as motion information of the second layer. At this time, the motion information of the first layer block corresponding to each NxN block of the second layer may be motion information of the first layer sample position corresponding to the reference sample position of the second layer NxN block.

For example, as illustrated in FIG. 6, the encoding/decoding device may determine a location (xP+8, yP+8) within a 16x16 block (600) of the second layer as a reference sample location, and perform inter-layer motion information mapping using motion information of a first layer sample location corresponding to the reference sample location (xP+8, yP+8).

Alternatively, the encoding/decoding device may determine the location (xP, yP) within the 16x16 block (600) of the second layer as a reference sample location and perform inter-layer motion information mapping using motion information of the first layer sample location corresponding to the reference sample location (xP, yP).

Alternatively, the encoding/decoding device may determine the location (xP+15, yP) within the 16x16 block (600) of the second layer as a reference sample location, and perform inter-layer motion information mapping using motion information of the first layer sample location corresponding to the reference sample location (xP+15, yP).

Alternatively, the encoding/decoding device may determine a location (xP, yP+15) within a 16x16 block (600) of the second layer as a reference sample location, and perform inter-layer motion information mapping using motion information of a first layer sample location corresponding to the reference sample location (xP, yP+15).

Alternatively, the encoding/decoding device may determine a location (xP+15, yP+15) within a 16x16 block (600) of the second layer as a reference sample location, and perform inter-layer motion information mapping using motion information of a first layer sample location corresponding to the reference sample location (xP+15, yP+15).

Alternatively, the encoding/decoding device may determine a location (xP+1, yP+1) within a 16x16 block (600) of the second layer as a reference sample location, and perform inter-layer motion information mapping using motion information of a first layer sample location corresponding to the reference sample location (xP+1, yP+1).

The encoding/decoding device can perform inter-layer motion information mapping using motion information of first layer sample positions corresponding to other reference sample positions in addition to the above-mentioned reference sample positions.

The sample position of the first layer corresponding to the reference sample position of the second layer can be calculated as in the following mathematical expression 1 by considering the size of the image between layers.

Here, xRef and yRef are sample locations of the first layer, xP and yP are reference sample locations of the second layer, ScaledW and ScaledH are the width and height sizes of the scaled first layer picture, and picWRL and picHRL are the width and height sizes of the first layer picture.

For example, if the motion information of the first layer is compressed and stored in 16x16 units, the sample positions of the first layer calculated by the mathematical expression 1 can be adjusted to use the motion information of 16x16 units through the mathematical expression 2 below.

As another example, if the motion information of the first layer is compressed and stored in 8x8 units, the sample positions of the first layer calculated by the mathematical expression 1 can be adjusted to use the motion information of the 8x8 units through the mathematical expression 3 below.

If the prediction mode of the first layer sample position corresponding to the reference sample position of the second layer is the intra prediction mode, the encoding/decoding device may use (0, 0) as the motion vector value of the corresponding block of the second layer, and may assign a value of -1 to the reference picture index and reference picture POC of the corresponding block of the second layer.

Alternatively, if the prediction mode of the first layer sample position corresponding to the reference sample position of the second layer is the intra prediction mode, the encoding/decoding device may use (0, 0) as the motion vector value of the second layer corresponding block, and may assign specific values (for example, 0 is assigned to the reference picture index and the reference picture POC is assigned as the POC of the picture indicated by the reference picture index 0) to the reference picture index and the reference picture POC of the second layer corresponding block.

When the prediction mode of the first layer sample position corresponding to the reference sample position of the second layer is the inter prediction mode, the encoding/decoding device can use the motion vector value, reference picture index, and reference picture POC value of the first layer sample position as the motion information of the corresponding block of the second layer.

When using the motion vector of the first layer, the encoding/decoding device can use the motion vector of the first layer by reflecting the size of the image between layers as shown in the mathematical expression 4 below.

The encoding/decoding device can use the picture type information (picture type information on whether the reference picture is a short-term reference picture or a long-term reference picture) of the reference picture referenced by the corresponding block of the first layer corresponding to the block of the second layer as information of the corresponding block of the second layer.

The above-described inter-layer motion information mapping can be determined at a higher level, and the encoding device can transmit information related to the inter-layer motion information mapping at a higher level. The decoding device can obtain information related to the inter-layer motion information mapping from a higher level.

The upper level can be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, etc.

Below, a method for signaling information related to motion information mapping at a high level according to an embodiment of the present invention is described.

According to one embodiment of the present invention, information related to motion information mapping of a corresponding layer in a VPS can be signaled as shown in Table 1.

Referring to Table 1, if inter_layer_mfm_enable_flag is 0, motion information mapping of the i-th layer may not be performed.

If inter_layer_mfm_enable_flag is 1, motion information mapping of the i-th layer can be performed.

According to another embodiment of the present invention, information related to motion information mapping of a corresponding layer can be signaled in an extension of SPS as in Table 2.

Referring to Table 2, if sps_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

If sps_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer can be performed.

*According to another embodiment of the present invention, it is possible to determine whether to transmit information (e.g., sps_inter_layer_mfm_enable_flag) indicating whether to perform motion information mapping of a corresponding layer according to information (e.g., sps_temporal_mvp_enabled_flag) indicating whether to use temporal motion vector prediction transmitted from SPS as in Table 3.

Referring to Table 3, when sps_temporal_mvp_enabled_flag, which indicates whether to use temporal motion vector prediction, is 1, sps_inter_layer_mfm_enable_flag, which indicates whether to perform motion information mapping of the first layer, can be transmitted.

When sps_temporal_mvp_enabled_flag is 1 and sps_inter_layer_mfm_enable_flag is 0, the motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used as the TMVP (temporal motion vector predictor) of the second layer.

According to another embodiment of the present invention, whether to perform motion information mapping of a corresponding layer can be determined by the sps_temporal_mvp_enabled_flag value without transmitting sps_inter_layer_mfm_enable_flag as in Table 3. For example, if sps_temporal_mvp_enabled_flag is 1, sps_inter_layer_mfm_enable_flag is considered to be 1, and motion information mapping of the corresponding layer can be performed. If sps_temporal_mvp_enabled_flag is 0, sps_inter_layer_mfm_enable_flag is considered to be 0, and motion information mapping of the corresponding layer can be performed.

According to another embodiment of the present invention, information related to motion information mapping of a corresponding layer can be signaled in an extension of PPS as in Table 4.

Referring to Table 4, if pps_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

If pps_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer can be performed.

According to another embodiment of the present invention, information related to motion information mapping of a corresponding layer can be signaled in an extension of a slice segment header as in Table 5.

Referring to Table 5, if slice_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

If slice_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer can be performed.

According to another embodiment of the present invention, information related to motion information mapping of a corresponding layer can be signaled in an extension of a slice segment header as in Table 6.

Referring to Table 6, when slice_temporal_mvp_enabled_flag, which indicates whether to use temporal motion vector prediction, is 1, slice_inter_layer_mfm_enable_flag, which indicates whether to perform motion information mapping of the first layer, can be transmitted. slice_inter_layer_mfm_enable_flag is transmitted only in a layer other than the base layer (i.e., the enhancement layer), and in the case of the enhancement layer, if a layer used for motion prediction does not exist, slice_inter_layer_mfm_enable_flag may not be transmitted. If the flag does not exist, the flag value can be inferred as 0.

If slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 0, the motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used for the Temporal Motion Vector Predictor (TMVP) of the second layer.

If slice_inter_layer_mfm_enable_flag is transmitted in a non-base layer (i.e., enhancement layer), slice_inter_layer_mfm_enable_flag can be present above slice_segement_header_extension_present_flag in the slice segment header of the non-base layer (i.e., enhancement layer), as shown in Table 7.

Referring to Table 7, when slice_temporal_mvp_enabled_flag, which indicates whether to use temporal motion vector prediction, is 1, slice_inter_layer_mfm_enable_flag, which indicates whether to perform motion information mapping of the first layer, can be transmitted.

If slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 0, the motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used as the TMVP (temporal motion vector predictor) of the second layer.

When slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 1, the motion information of the first layer can be used in the second layer. That is, after mapping the motion information of the first layer to the size of the second layer, the restored image of the first layer can be used as a corresponding picture (ColPic, Collocated picture) for TMVP of the second layer.

When slice_inter_layer_mfm_enable_flag is 1, information about corresponding pictures (ColPic, Collocated pictures) of the first layer for TMVP of the second layer can be signaled through additional syntax information as shown in Table 8. For example, information about the layer to be referenced can be known through collocated_ref_layer_idx, and motion information of the corresponding picture of the reference layer can be mapped to the image size of the second layer. In addition, in case of B slice, the direction of the reference picture list where pictures of the reference layer that can be used as corresponding pictures are located can be determined through collocated_ref_layer_from_l0_flag.

Referring to Table 8, collocated_ref_layer_idx is an indicator that provides information on the reference layer used for motion prediction when the number of reference layers used for motion prediction is 2 or more. If the number of reference layers used for motion prediction is 1, collocated_ref_layer_idx can be omitted. In this case, a constraint can be imposed that all slices in the picture must have the same value.

collocated_ref_layer_from_l0_flag is transmitted in the B slice. If the collocated_ref_layer_from_l0_flag value is 1, the picture of the reference layer existing in LIST0 is used as the corresponding picture (ColPic). If the collocated_ref_layer_from_l0_flag value is 0, the picture of the reference layer existing in LIST1 is used as the corresponding picture (ColPic). Thereafter, in the process of obtaining the motion vector of the corresponding block (ColPb) in the corresponding picture (ColPic), the collocated_ref_layer_from_l0_flag value can be used with the same meaning instead of collocated_from_l0_flag. At this time, a constraint that all slices in the picture must have the same value can be imposed. Alternatively, in the process of obtaining the motion vector of the corresponding block (ColPb) within the corresponding picture (ColPic), if the corresponding block has motion information in both the LIST0 and LIST1 directions, motion information in the same direction as the direction pointed by the current encoding/decoding target picture can be obtained.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture within the second layer can be indicated using collocated_from_l0_flag and collocated_ref_idx without using the motion information of the reference layer.

When slice_inter_layer_mfm_enable_flag is 1, information about corresponding pictures (ColPic, Collocated pictures) of the first layer for TMVP of the second layer and reference picture list direction information can be signaled using additional syntax information as shown in Table 9. For example, when slice_temporal_mvp_enabled_flag is 1, for B slice, the reference picture list direction of the corresponding picture can be determined from the collocated_from_l0_flag value, and the referencing layer can be known from the collocated_ref_layer_idx value.

Referring to Table 9, collocated_from_l0_flag indicates the direction of the reference picture list to which the corresponding picture belongs when slice_temporal_mvp_enabled_flag is 1 and it is a B slice. If collocated_from_l0_flag does not exist, collocated_from_l0_flag can be inferred to be 1. In this case, a constraint can be imposed that all slices in the picture must have the same value.

collocated_ref_layer_idx is an indicator that provides information on the reference layers used for motion prediction when there are two or more reference layers used for motion prediction. If there is only one reference layer used for motion prediction, collocated_ref_layer_idx can be omitted. In this case, a constraint that all slices in a picture must have the same value can be applied. The list direction for the corresponding picture can be determined from the collocated_from_l0_flag transmitted from above.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture within the second layer can be indicated using collocated_from_l0_flag and collocated_ref_idx without using the motion information of the reference layer.

When slice_inter_layer_mfm_enable_flag is 1, information on the referenced layer can be known through collocated_ref_layer_idx as in Table 10, and motion information of the corresponding picture of the reference layer can be mapped to the image size of the second layer. At this time, depending on the slice type, in the case of a P slice, a picture of the reference layer existing in LIST0 is used as the corresponding picture, and in the case of a B slice, a convention is made in the direction of either LIST0 or LIST1, and the corresponding picture of the reference layer is used as the corresponding picture so that the encoding/decoding device can use it identically.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture within the second layer can be indicated using collocated_from_l0_flag and collocated_ref_idx without using the motion information of the reference layer.

According to another embodiment of the present invention, when a picture is divided into N independent slices, slice_inter_layer_mfm_enable_flag in each slice segment header must have the same value, and a motion information mapping process can be performed once per picture according to the slice_inter_layer_mfm_enable_flag value in the first independent slice segment header.

In the embodiments of the present invention described above, the ‘sps_inter_layer_mfm_enable_flag’ transmitted at a higher level was used as information indicating whether motion information mapping is performed, but this information can be extended and utilized as information indicating whether inter-layer syntax prediction is performed. For example, when sps_inter_layer_mfm_enable_flag is 1, not only motion information mapping is performed, but also inter-layer syntax (motion information) prediction can be performed.

In the embodiments of the present invention described above, a separate syntax for indicating whether motion information mapping is performed is used, but motion information mapping may also be performed according to a syntax information value regarding inter-layer prediction performance without transmitting a separate syntax.

In the present invention, it is possible to determine whether motion information mapping is performed using syntax information that indicates whether inter-layer prediction is performed, which is signaled from a higher layer. The syntax information that indicates whether inter-layer prediction is performed can be signaled from a higher level.

Here, the upper level can be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, etc.

According to one embodiment of the present invention, a syntax indicating whether inter-layer prediction is performed in a VPS may be transmitted as in Table 11.

Referring to Table 11, when a syntax indicating whether to perform inter-layer prediction is transmitted in VPS, if no_inter_layer_pred_flag is 1, motion information mapping may not be performed, and if no_inter_layer_pred_flag is 0, motion information mapping may be performed.

In the present invention, it is possible to determine whether motion information mapping is performed using syntax information that indicates whether syntax prediction between layers signaled from a higher layer is performed. The syntax information that indicates whether syntax prediction between layers is performed can be signaled from a higher level.

Here, the upper level can be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, etc.

According to one embodiment of the present invention, syntax indicating whether to perform inter-layer syntax prediction in PPS may be transmitted as in Table 12.

Referring to Table 12, when a syntax indicating whether to perform inter-layer syntax prediction is transmitted in PPS, if no_inter_layer_syntax_pred_flag is 0, motion information mapping can be performed, and if no_inter_layer_syntax_pred_flag is 1, motion information mapping may not be performed.

According to another embodiment of the present invention, syntax indicating whether to perform inter-layer syntax prediction may be transmitted in a slice segment header as in Table 13.

Referring to Table 13, when a syntax indicating whether to perform inter-layer syntax prediction is transmitted in the slice segment header, if no_inter_layer_syntax_pred_flag is 0, motion information mapping can be performed, and if no_inter_layer_syntax_pred_flag is 1, motion information mapping may not be performed.

Referring again to FIG. 4, the encoding/decoding device adds information of the first layer to the reference picture list of the second layer and encodes/decodes an image of the second layer (S420).

That is, the encoding/decoding device can add the decoded sample values and motion information of the first layer, which are mapped to the second layer image size through the steps S400 to S410 described above, to the reference picture list for the current encoding/decoding target image of the second layer, and can configure the reference picture list of the second layer image using the added information of the first layer. In addition, the encoding/decoding device can perform inter prediction to generate a prediction signal for the image of the second layer based on the reference picture list.

When configuring the reference picture list of the second layer, the decoded image of the first layer can be added to the last position of the reference picture list for the current encoding/decoding target image of the second layer.

Alternatively, the decoded image of the first layer can be added to a specific location in the reference picture list for the current encoding/decoding target image of the second layer. At this time, the specific location can be notified at a higher level (e.g., VPS, SPS, PPS, slice segment header, etc.), or the specific location can be specified by a set convention without transmitting additional information.

Alternatively, the decoded image of the first layer may be added to the reference picture lists L0 and L1 for the current encoding/decoding target image of the second layer. In this case, the positions added to the reference picture lists L0 and L1 may be the same or different. For example, the decoded image of the first layer may be added to the first position in the reference picture list L0, and the decoded image of the first layer may be added to the last position in the reference picture list L1.

Alternatively, the decoded image of the first layer can be added to either one of the reference picture list directions (L0 and L1) for the current encoding/decoding target image of the second layer. For example, if the current encoding/decoding target image of the second layer is encoded/decoded with a B slice, the decoded image of the first layer can be added only to the reference picture list L0 or the reference picture list L1.

At this time, when adding the decoded image of the first layer to the reference picture list of a specific direction, the position in the reference picture list of a specific direction to be added can be indicated at a higher level (e.g., VPS, SPS, PPS, slice segment header, etc.), or a specific position can be specified by a set convention without transmitting additional information.

Alternatively, the decoded image of the first layer can be added to different positions in the reference picture list according to the depth using hierarchical depth information of the prediction structure. At this time, information about the position can be notified at a higher level (e.g., VPS, SPS, PPS, slice segment header, etc.). For example, the position at which the decoded image of the first layer is added in the reference picture list can be different based on the temporal level (temporal_Id) value according to the hierarchical depth. For example, when the temporal level (temporal_Id) is 2 or greater, the decoded image of the first layer can be added to the last position of the reference picture list, and when the temporal level (temporal_Id) is less than 2, the decoded image of the first layer can be added to the first position of the reference picture list. At this time, the reference temporal level value can be notified at a higher level (e.g., VPS, SPS, PPS, slice segment header, etc.).

The decoded image of the first layer described above may include not only the decoded sample values of the first layer, but also the decoded motion information of the first layer mapped to the image size of the second layer.

The encoding/decoding device can perform prediction on the current encoding/decoding target image of the second layer using a conventional inter prediction method after adding the decoded image of the first layer to the reference picture list for the current encoding/decoding target image of the second layer in the same manner as described above.

When performing prediction using the decoded image of the first layer as a reference signal of the current encoding target image of the second layer (when performing interlayer prediction), the encoding device can use the first layer sample value at the same position as the prediction block in the current encoding target image as the prediction signal without a motion prediction process. In this case, the encoding device may not transmit motion information of the corresponding block (e.g., motion vector difference value, motion prediction candidate flag, etc.).

If the image indicated by the reference picture index of the second layer is a decoded image of the first layer in the reference picture list, the decoding device may use the first layer sample value at the same position as the prediction block in the current decoding target image as a prediction signal without decoding motion information (e.g., a motion vector difference value, a motion prediction candidate flag, etc.) for the prediction block in the current decoding target image of the second layer.

If the image indicated by the reference picture index of the current decoding target block of the second layer is a decoded image of the first layer in the reference picture list (if interlayer prediction is performed), the decoding device can determine whether to decode motion information (e.g., a motion vector difference value, a motion prediction candidate flag, etc.) for the current decoding target block of the second layer based on information transmitted from a higher level (e.g., VPS, SPS, PPS, slice segment header, etc.) and perform prediction for the current decoding target block of the second layer.

Table 14 shows an example of a syntax in which information (sps_inter_layer_mv_zero_flag) indicating whether the inter-layer motion vector has the value (0, 0) is transmitted in SPS.

Referring to Table 14, if sps_inter_layer_mv_zero_flag transmitted from SPS has a value of 1 and the reference picture index of the current decoding target block indicates a decoded image of the first layer within the reference picture list, the decoding device may use a sample value of the decoded first layer image having the same position as the position of the current decoding target block as a prediction signal for the current decoding target block without decoding the motion vector differential value and the motion prediction candidate flag.

If the sps_inter_layer_mv_zero_flag transmitted from the SPS has a value of 0 and the reference picture index of the current decoding target block indicates a decoded image of the first layer within the reference picture list, the decoding device decodes the motion vector differential value and the motion prediction candidate flag, and then obtains a motion vector value, and can use the sample value of the decoded first layer image at the position of the obtained motion vector value as a prediction signal for the current decoding target block.

Meanwhile, in order to use the motion information of the mapped first layer as a temporal motion vector candidate for the current encoding target block of the second layer, the encoding device may use the decoded image of the first layer as a collocated picture for predicting the temporal motion vector of the second layer. In this case, the encoding device may designate the decoded image of the first layer as the collocated picture through information (collocated_ref_idx) indicating the corresponding picture in the slice segment header.

When information (collocated_ref_idx) indicating a corresponding picture in a slice segment header transmitted from an encoder indicates a decoded image of the first layer, the decoding device can use the mapped motion information of the first layer as a temporal motion vector candidate of the second layer.

When the motion information of the mapped first layer is used as a temporal motion vector candidate of the second layer, the encoding/decoding device can use one or more of the motion information of the mapped first layer, that is, the reference picture POC, the reference picture list, and the picture type (short-term reference picture, long-term reference picture) information of the reference picture, and by using the motion information of one or more mapped first layers, the motion vector of the second layer can be scaled and used according to the temporal distance.

When prediction is performed by adding only reference sample values of the decoded first layer image to the reference picture list without allowing motion information mapping of the first layer at the upper level, a restriction can be placed in the standard document so that the information indicating the corresponding picture (collocated_ref_idx) does not indicate the decoded first layer image as the corresponding picture.

As another example of using the motion information of the mapped first layer as the motion information of the second layer, the motion information of the mapped first layer may be used as an additional candidate mode other than the temporal motion vector candidate of the second layer. In this case, in order to allow the temporal motion vector candidate of the second layer to be derived from the reference picture of the second layer, the information indicating the corresponding picture (collocated_ref_idx) may be restricted in the standard document so as not to indicate the decoded image of the first layer as the corresponding picture. In addition, in order to use the motion information of the first layer mapped as the additional candidate mode, the encoding device may additionally transmit an index (e.g., base_ref_idx) that can indicate the position of the mapped image of the first layer in the reference picture list, such as in the slice segment header. The position of the additional posteriors in the reference picture list may be applied equally to the encoding device and the decoding device.

The method according to the present invention described above can be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and also includes a recording medium implemented in the form of a carrier wave (e.g., transmission via the Internet).

The computer-readable recording medium can be distributed across network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above method can be easily inferred by programmers in the technical field to which the present invention belongs.

In the above-described embodiments, the methods are described based on the flowchart as a series of steps or blocks, but the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. In addition, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention.

The above description is merely an illustrative description of the technical idea of the present invention, and those skilled in the art will appreciate that various modifications and variations may be made without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims (5)

In an image decoding method that performs inter-layer prediction,
Step of decoding the picture of the first layer;
If the picture of the first layer is a picture referenced by a picture of the second layer including the current decoding target block, a step of adding the picture of the first layer to a reference picture list for the picture of the second layer, the first layer being a different layer from the second layer; and
Including a step of selecting one of the reference pictures included in the above reference picture list as a corresponding picture,
The above reference picture list is an L0 reference picture list or an L1 reference picture list,
Based on the flag decoded through the bitstream, it is determined whether the corresponding picture is included in the L0 reference picture list or the L1 reference picture list,
If the picture of the first layer and the picture of the second layer have different sizes, the picture of the first layer is not determined as the corresponding picture.
A video decoding method, characterized in that the information indicating the corresponding picture does not indicate a picture of the first layer among the reference pictures included in the reference picture list. In paragraph 1,
A video decoding method, characterized in that a picture of the first layer is added to the reference picture list when the picture has the same POC (Picture Order Count) as a picture of the second layer. In paragraph 1,
A video decoding method, characterized in that whether the picture of the first layer is a picture referenced by the picture of the second layer is determined based on the layer ID of the picture of the second layer. In an image encoding method for performing inter-layer prediction,
Step of restoring the picture of the first layer;
If the picture of the first layer is a picture referenced by a picture of the second layer including the current encoding target block, a step of adding the picture of the first layer to a reference picture list for the picture of the second layer, the first layer being a different layer from the second layer; and
Including a step of selecting one of the reference pictures included in the above reference picture list as a corresponding picture,
The above reference picture list is an L0 reference picture list or an L1 reference picture list,
A flag indicating whether the corresponding picture is included in the L0 reference picture list or the L1 reference picture list is encoded in the bitstream,
If the picture of the first layer and the picture of the second layer have different sizes, the picture of the first layer is not determined as the corresponding picture.
A video encoding method, characterized in that the information indicating the corresponding picture does not indicate a picture of the first layer among the reference pictures included in the reference picture list. In a recording medium storing a bitstream generated by a video encoding method,
The above image encoding method is,
Step of restoring the picture of the first layer;
If the picture of the first layer is a picture referenced by a picture of the second layer including the current encoding target block, a step of adding the picture of the first layer to a reference picture list for the picture of the second layer, the first layer being a different layer from the second layer; and
Including a step of selecting one of the reference pictures included in the above reference picture list as a corresponding picture,
The above reference picture list is an L0 reference picture list or an L1 reference picture list,
A flag indicating whether the corresponding picture is included in the L0 reference picture list or the L1 reference picture list is encoded in the bitstream,
If the picture of the first layer and the picture of the second layer have different sizes, the picture of the first layer is not determined as the corresponding picture.
A recording medium storing a bitstream generated by a video encoding method, characterized in that the information indicating the corresponding picture does not indicate a picture of the first layer among the reference pictures included in the reference picture list.