WO2017043734A1 - Method for processing image on basis of inter prediction mode and device therefor - Google Patents

Abstract

A method for processing an image on the basis of an inter prediction mode and a device therefor are disclosed. Particularly, the method for processing an image on the basis of inter prediction comprises the steps of: configuring motion information candidate lists by adding, to the motion information candidate lists of a current block, according to a predetermined order, spatial motion information candidates derived from blocks spatially neighboring the current block and temporal motion information candidates derived from blocks temporally neighboring the current block; deriving motion information of the current block from a motion information candidate selected among the motion information candidates added to the motion information candidate list; and generating a prediction block of the current block by using the motion information of the current block, wherein the order of the motion information candidates can be sorted according to occurrence frequency of the motion information candidates added to the motion information candidate list.

Description

Inter prediction mode based image processing method and apparatus therefor

The present invention relates to a still image or moving image processing method, and more particularly, to a method for encoding / decoding a still image or moving image based on an inter prediction mode and an apparatus supporting the same.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or for storing in a form suitable for a storage medium. Media such as an image, an image, an audio, and the like may be a target of compression encoding. In particular, a technique of performing compression encoding on an image is called video image compression.

Next-generation video content will be characterized by high spatial resolution, high frame rate and high dimensionality of scene representation. Processing such content would result in a tremendous increase in terms of memory storage, memory access rate, and processing power.

Accordingly, there is a need to design coding tools for more efficiently processing next generation video content.

When performing inter-picture prediction during encoding / decoding of video, list motion information around the current block in order to encode motion information efficiently, and select a suitable item to represent motion of the current block and use it for motion compensation. do. At this time, the neighboring motion information belonging to the list must be arranged in the same order of high probability as the motion information of the current block to obtain high coding efficiency.

Accordingly, the present invention proposes a method for efficiently encoding motion information by constructing a list in consideration of the frequency of the motion information generated in each item of the motion information list around the current block.

In addition, the present invention proposes a method of considering long term motion information when listing surrounding motion information.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

An aspect of the present invention is a method for processing an image based on inter prediction, comprising: a spatial motion information candidate derived from a block spatially neighboring a current block and a block temporally neighboring the current block Constructing the motion information candidate list by adding the derived temporal motion information candidate to the motion information candidate list of the current block in a predetermined order, from the motion information candidate selected from the motion information candidates added to the motion information candidate list; Deriving motion information of a current block and generating a prediction block of the current block by using motion information of the current block, wherein the motion information candidate is added to the motion information candidate list according to a frequency of occurrence of the motion information candidate added to the motion information candidate list. Sequence of motion information candidates Can be aligned.

An aspect of the present invention is an apparatus for processing an image based on inter prediction, comprising: a spatial motion information candidate derived from a block spatially neighboring a current block and a block temporally neighboring the current block A motion information candidate list constructing unit constituting the motion information candidate list by adding the derived temporal motion information candidates to the motion information candidate list of the current block in a predetermined order, and selecting among motion information candidates added to the motion information candidate list A motion information derivation unit for deriving motion information of the current block from a motion information candidate; and a prediction block generator for generating a prediction block of the current block by using motion information of the current block, and adding to the motion information candidate list. After the movement information The motion information candidates may be ordered according to the frequency of occurrence of beams.

Preferably, only the order of motion information candidates in which the difference in frequency of occurrence is greater than or equal to a predetermined level may be changed.

Preferably, a block that is spatially neighboring to the left of the current block, a block that is spatially neighboring to the upper left of the current block, a block that is spatially neighboring to the upper right of the current block, and spatially neighboring to the lower left of the current block The motion information candidate may be derived in the order of a block and a block temporally neighboring the current block.

Preferably, after motion information candidates are alternately derived from a block adjacent to the left side of the current block and a block adjacent to the top of the current block, the block next to the left side of the left side of the current block, The motion information candidate may be derived in the order of a block adjacent to the upper right end of the current block, a block adjacent to the lower left end of the current block, and a block temporally adjacent to the current block.

Preferably, when the depth at which the current block is divided is equal to or larger than a predetermined value or when the size of the current block is smaller than a predetermined size, the motion information candidate list may be sorted.

Preferably, when the number of motion information candidates added to the motion information candidate list is smaller than the number of predetermined items of the motion information candidate list, motion information generated by combining the motion information candidates added to the motion information candidate list. A candidate may be added to the motion information candidate list.

Preferably, when the number of motion information candidates added to the motion information candidate list is smaller than the number of predetermined items of the motion information candidate list, a zero motion vector may be added to the motion information candidate list. have.

Preferably, the long-term motion information candidate selected from the long-term motion information list including motion information of a block spaced apart from the current block by a predetermined number of blocks or more and / or global motion information of a picture to which the current block belongs. The information may be added to the candidate list.

Preferably, when the number of motion information candidates added to the motion information candidate list is smaller than the number of predetermined items of the motion information candidate list, the long-term motion information candidate may be added to the motion information candidate list.

Preferably, when the long-term motion information candidate is added to the motion information candidate list, the long-term motion information candidate may be added so as not to overlap with the motion information candidate added to the motion information candidate list.

Preferably, when motion information is inserted into the long-term motion list, the motion information may be inserted into the highest item of the long-term motion list.

Preferably, when the first motion information is inserted in the long-term motion list, if second motion information identical to the first motion information exists in the long-term motion list, the second motion information is deleted and the first motion is deleted. Information may be inserted at the top of the long-term movement list.

Preferably, the global motion information is a mean of absolute difference between the current picture and the reference picture from a top-left pixel of the current picture before moving when the current picture is moved with respect to a reference picture. Or a difference between the top-left pixel and the direction of the current picture at a location that minimizes Mean of Squared Error (MSE).

Preferably, motion information having the highest frequency in the picture to which the current block belongs in the long-term motion information list may be added to the motion information candidate list.

According to an embodiment of the present invention, by sorting items using a frequency of overlapping motion information in a motion information list of a current block, a probability model is reflected when encoding a list index selected from a motion information list, thereby improving encoding efficiency. have.

In addition, according to the embodiment of the present invention, when using the motion information list in the inter prediction mode, since the non-overlapping peripheral motion information can be used in various ways for higher coding efficiency, encoding performance can be improved.

In addition, according to an embodiment of the present invention, various motion information can be applied to the motion information list, thereby improving the accuracy of inter prediction.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, included as part of the detailed description in order to provide a thorough understanding of the present invention, provide embodiments of the present invention and together with the description, describe the technical features of the present invention.

1 is a schematic block diagram of an encoder in which encoding of a still image or video signal is performed according to an embodiment to which the present invention is applied.

2 is a schematic block diagram of a decoder in which encoding of a still image or video signal is performed according to an embodiment to which the present invention is applied.

3 is a diagram for describing a division structure of a coding unit that may be applied to the present invention.

4 is a diagram for describing a prediction unit that may be applied to the present invention.

5 is a diagram illustrating a direction of inter prediction as an embodiment to which the present invention may be applied.

6 illustrates integer and fractional sample positions for quarter sample interpolation, as an embodiment to which the present invention may be applied.

7 illustrates a position of a spatial candidate as an embodiment to which the present invention may be applied.

8 is a diagram illustrating an inter prediction method as an embodiment to which the present invention is applied.

9 is a diagram illustrating a motion compensation process as an embodiment to which the present invention may be applied.

10 is a diagram illustrating a process of constructing a merge candidate list.

11 illustrates an example of a configuration of a motion information candidate list according to an embodiment of the present invention.

12 is a diagram illustrating a method of sorting a motion information candidate list according to an embodiment of the present invention.

13 is a diagram illustrating a method of sorting a motion information candidate list according to an embodiment of the present invention.

14 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

15 is a diagram illustrating a spatial / temporal neighbor block that can be referred to to generate a motion information candidate list according to an embodiment of the present invention.

16 is a diagram illustrating a spatial neighboring block that can be referred to for generating a motion information candidate list according to an embodiment of the present invention.

17 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

18 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

19 illustrates a method of constructing a motion information candidate list using long-term motion information according to an embodiment of the present invention.

20 illustrates global motion information of a picture as long-term motion information according to an embodiment of the present invention.

21 illustrates a method of deriving global motion information of a picture as long-term motion information according to an embodiment of the present invention.

22 illustrates a method of constructing a motion information candidate list using long-term motion information according to an embodiment of the present invention.

23 is a diagram illustrating an inter prediction based image processing method according to an embodiment of the present invention.

24 is a diagram illustrating an inter predictor according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description, which will be given below with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, one of ordinary skill in the art appreciates that the present invention may be practiced without these specific details.

In some instances, well-known structures and devices may be omitted or shown in block diagram form centering on the core functions of the structures and devices in order to avoid obscuring the concepts of the present invention.

In addition, the terminology used in the present invention was selected as a general term widely used as possible now, in a specific case will be described using terms arbitrarily selected by the applicant. In such a case, since the meaning is clearly described in the detailed description of the part, it should not be interpreted simply by the name of the term used in the description of the present invention, and it should be understood that the meaning of the term should be interpreted. .

Specific terms used in the following description are provided to help the understanding of the present invention, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present invention. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately replaced and interpreted in each coding process.

Hereinafter, in the present specification, a 'processing unit' refers to a unit in which an encoding / decoding process such as prediction, transformation, and / or quantization is performed. Hereinafter, for convenience of description, a processing unit may be referred to as a 'processing block' or 'block'.

The processing unit may be interpreted to include a unit for a luma component and a unit for a chroma component. For example, the processing unit may correspond to a Coding Tree Unit (CTU), a Coding Unit (CU), a Prediction Unit (PU), or a Transform Unit (TU).

In addition, the processing unit may be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit may be a coding tree block (CTB), a coding block (CB), a prediction block (PU), or a transform block (TB) for a luma component. May correspond to. Or, it may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PU), or a transform block (TB) for a chroma component. In addition, the present invention is not limited thereto, and the processing unit may be interpreted to include a unit for a luminance component and a unit for a chroma component.

In addition, the processing unit is not necessarily limited to a square block, but may also be configured in a polygonal form having three or more vertices.

1 is a schematic block diagram of an encoder in which encoding of a still image or video signal is performed according to an embodiment to which the present invention is applied.

Referring to FIG. 1, the encoder 100 may include an image divider 110, a subtractor 115, a transform unit 120, a quantizer 130, an inverse quantizer 140, an inverse transform unit 150, and a filtering unit. 160, a decoded picture buffer (DPB) 170, a predictor 180, and an entropy encoder 190. The predictor 180 may include an inter predictor 181 and an intra predictor 182.

The image divider 110 divides an input video signal (or a picture or a frame) input to the encoder 100 into one or more processing units.

The subtractor 115 subtracts the difference from the prediction signal (or prediction block) output from the prediction unit 180 (that is, the inter prediction unit 181 or the intra prediction unit 182) in the input image signal. Generate a residual signal (or difference block). The generated difference signal (or difference block) is transmitted to the converter 120.

The transform unit 120 may convert a differential signal (or a differential block) into a transform scheme (eg, a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), and a karhunen-loeve transform (KLT)). Etc.) to generate transform coefficients. In this case, the transform unit 120 may generate transform coefficients by performing a transform using a transform mode determined according to the prediction mode applied to the difference block and the size of the difference block.

The quantization unit 130 quantizes the transform coefficients and transmits the transform coefficients to the entropy encoding unit 190, and the entropy encoding unit 190 entropy codes the quantized signals and outputs them as bit streams.

Meanwhile, the quantized signal output from the quantization unit 130 may be used to generate a prediction signal. For example, the quantized signal may recover the differential signal by applying inverse quantization and inverse transformation through an inverse quantization unit 140 and an inverse transformation unit 150 in a loop. A reconstructed signal may be generated by adding the reconstructed difference signal to a prediction signal output from the inter predictor 181 or the intra predictor 182.

Meanwhile, in the compression process as described above, adjacent blocks are quantized by different quantization parameters, thereby causing deterioration of the block boundary. This phenomenon is called blocking artifacts, which is one of the important factors in evaluating image quality. In order to reduce such deterioration, a filtering process may be performed. Through this filtering process, the image quality can be improved by removing the blocking degradation and reducing the error of the current picture.

The filtering unit 160 applies filtering to the reconstruction signal and outputs it to the reproduction apparatus or transmits the decoded picture buffer to the decoding picture buffer 170. The filtered signal transmitted to the decoded picture buffer 170 may be used as the reference picture in the inter prediction unit 181. As such, by using the filtered picture as a reference picture in the inter prediction mode, not only image quality but also encoding efficiency may be improved.

The decoded picture buffer 170 may store the filtered picture for use as a reference picture in the inter prediction unit 181.

The inter prediction unit 181 performs temporal prediction and / or spatial prediction to remove temporal redundancy and / or spatial redundancy with reference to a reconstructed picture.

Here, since the reference picture used to perform the prediction is a transformed signal that has been quantized and dequantized in units of blocks at the time of encoding / decoding in the previous time, blocking artifacts or ringing artifacts may exist. have.

Accordingly, the inter prediction unit 181 may interpolate the signals between pixels in sub-pixel units by applying a lowpass filter to solve performance degradation due to discontinuity or quantization of such signals. Herein, the sub-pixels mean virtual pixels generated by applying an interpolation filter, and the integer pixels mean actual pixels existing in the reconstructed picture. As the interpolation method, linear interpolation, bi-linear interpolation, wiener filter, or the like may be applied.

The interpolation filter may be applied to a reconstructed picture to improve the precision of prediction. For example, the inter prediction unit 181 generates an interpolation pixel by applying an interpolation filter to integer pixels, and uses an interpolated block composed of interpolated pixels as a prediction block. You can make predictions.

The intra predictor 182 predicts the current block by referring to samples in the vicinity of the block to which the current encoding is to be performed. The intra prediction unit 182 may perform the following process to perform intra prediction. First, reference samples necessary for generating a prediction signal may be prepared. The prediction signal may be generated using the prepared reference sample. Then, the prediction mode is encoded. In this case, the reference sample may be prepared through reference sample padding and / or reference sample filtering. Since the reference sample has been predicted and reconstructed, there may be a quantization error. Accordingly, the reference sample filtering process may be performed for each prediction mode used for intra prediction to reduce such an error.

The prediction signal (or prediction block) generated by the inter prediction unit 181 or the intra prediction unit 182 is used to generate a reconstruction signal (or reconstruction block) or a differential signal (or differential block). It can be used to generate.

2 is a schematic block diagram of a decoder in which encoding of a still image or video signal is performed according to an embodiment to which the present invention is applied.

2, the decoder 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, and a decoded picture buffer (DPB). Buffer Unit (250), the prediction unit 260 may be configured. The predictor 260 may include an inter predictor 261 and an intra predictor 262.

The reconstructed video signal output through the decoder 200 may be reproduced through the reproducing apparatus.

The decoder 200 receives a signal (ie, a bit stream) output from the encoder 100 of FIG. 1, and the received signal is entropy decoded through the entropy decoding unit 210.

The inverse quantization unit 220 obtains a transform coefficient from the entropy decoded signal using the quantization step size information.

The inverse transform unit 230 applies an inverse transform scheme to inverse transform the transform coefficients to obtain a residual signal (or a differential block).

The adder 235 outputs the obtained difference signal (or difference block) from the prediction unit 260 (that is, the prediction signal (or prediction block) output from the inter prediction unit 261 or the intra prediction unit 262. ) Generates a reconstructed signal (or a reconstruction block).

The filtering unit 240 applies filtering to the reconstructed signal (or the reconstructed block) and outputs the filtering to the reproduction device or transmits the decoded picture buffer unit 250 to the reproduction device. The filtered signal transmitted to the decoded picture buffer unit 250 may be used as a reference picture in the inter predictor 261.

In the present specification, the embodiments described by the filtering unit 160, the inter prediction unit 181, and the intra prediction unit 182 of the encoder 100 are respectively the filtering unit 240, the inter prediction unit 261, and the decoder of the decoder. The same may be applied to the intra predictor 262.

Processing unit partition structure

In general, a still image or video compression technique (eg, HEVC) uses a block-based image compression method. The block-based image compression method is a method of processing an image by dividing the image into specific block units, and may reduce memory usage and calculation amount.

3 is a diagram for describing a division structure of a coding unit that may be applied to the present invention.

The encoder splits one image (or picture) into coding units of a coding tree unit (CTU) in a rectangular shape. In addition, one CTU is sequentially encoded according to a raster scan order.

In HEVC, the size of the CTU may be set to any one of 64 × 64, 32 × 32, and 16 × 16. The encoder may select and use the size of the CTU according to the resolution of the input video or the characteristics of the input video. The CTU includes a coding tree block (CTB) for luma components and a CTB for two chroma components corresponding thereto.

One CTU may be divided into a quad-tree structure. That is, one CTU has a square shape and is divided into four units having a half horizontal size and a half vertical size to generate a coding unit (CU). have. This partitioning of the quad-tree structure can be performed recursively. That is, a CU is hierarchically divided into quad-tree structures from one CTU.

CU refers to a basic unit of coding in which an input image is processed, for example, intra / inter prediction is performed. The CU includes a coding block (CB) for a luma component and a CB for two chroma components corresponding thereto. In HEVC, the size of a CU may be set to any one of 64 × 64, 32 × 32, 16 × 16, and 8 × 8.

Referring to FIG. 3, the root node of the quad-tree is associated with the CTU. The quad-tree is split until it reaches a leaf node, which corresponds to a CU.

More specifically, the CTU corresponds to a root node and has a smallest depth (ie, depth = 0). The CTU may not be divided according to the characteristics of the input image. In this case, the CTU corresponds to a CU.

The CTU may be divided into quad tree shapes, resulting in lower nodes having a depth of 1 (depth = 1). In addition, a node that is no longer divided (ie, a leaf node) in a lower node having a depth of 1 corresponds to a CU. For example, in FIG. 3 (b), CU (a), CU (b), and CU (j) corresponding to nodes a, b, and j are divided once in the CTU and have a depth of one.

At least one of the nodes having a depth of 1 may be split into a quad tree again, resulting in lower nodes having a depth of 1 (ie, depth = 2). In addition, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 2 corresponds to a CU. For example, in FIG. 3 (b), CU (c), CU (h) and CU (i) corresponding to nodes c, h and i are divided twice in the CTU and have a depth of two.

In addition, at least one of the nodes having a depth of 2 may be divided into quad tree shapes, resulting in lower nodes having a depth of 3 (ie, depth = 3). And, a node that is no longer partitioned (ie, a leaf node) in a lower node having a depth of 3 corresponds to a CU. For example, in FIG. 3 (b), CU (d), CU (e), CU (f), and CU (g) corresponding to nodes d, e, f, and g are divided three times in the CTU, Has depth.

In the encoder, the maximum size or the minimum size of the CU may be determined according to characteristics (eg, resolution) of the video image or in consideration of encoding efficiency. Information about this or information capable of deriving the information may be included in the bitstream. A CU having a maximum size may be referred to as a largest coding unit (LCU), and a CU having a minimum size may be referred to as a smallest coding unit (SCU).

In addition, a CU having a tree structure may be hierarchically divided with predetermined maximum depth information (or maximum level information). Each partitioned CU may have depth information. Since the depth information indicates the number and / or degree of division of the CU, the depth information may include information about the size of the CU.

Since the LCU is divided into quad tree shapes, the size of the SCU can be obtained by using the size and maximum depth information of the LCU. Or conversely, using the size of the SCU and the maximum depth information of the tree, the size of the LCU can be obtained.

For one CU, information indicating whether the corresponding CU is split (for example, a split CU flag split_cu_flag) may be transmitted to the decoder. This split mode is included in all CUs except the SCU. For example, if the flag indicating whether to split or not is '1', the CU is divided into 4 CUs again. If the flag indicating whether to split or not is '0', the CU is not divided further. Processing may be performed.

As described above, a CU is a basic unit of coding in which intra prediction or inter prediction is performed. HEVC divides a CU into prediction units (PUs) in order to code an input image more effectively.

The PU is a basic unit for generating a prediction block, and may generate different prediction blocks in PU units within one CU. However, PUs belonging to one CU are not mixed with intra prediction and inter prediction, and PUs belonging to one CU are coded by the same prediction method (ie, intra prediction or inter prediction).

The PU is not divided into quad-tree structures, but is divided once in a predetermined form in one CU. This will be described with reference to the drawings below.

4 is a diagram for describing a prediction unit that may be applied to the present invention.

The PU is divided differently according to whether an intra prediction mode or an inter prediction mode is used as a coding mode of a CU to which the PU belongs.

FIG. 4A illustrates a PU when an intra prediction mode is used, and FIG. 4B illustrates a PU when an inter prediction mode is used.

Referring to FIG. 4 (a), assuming that a size of one CU is 2N × 2N (N = 4,8,16,32), one CU has two types (ie, 2N × 2N or N). XN).

Here, when divided into 2N × 2N type PU, it means that only one PU exists in one CU.

On the other hand, when divided into N × N type PU, one CU is divided into four PUs, and different prediction blocks are generated for each PU unit. However, the division of the PU may be performed only when the size of the CB for the luminance component of the CU is the minimum size (that is, the CU is the SCU).

Referring to FIG. 4 (b), assuming that a size of one CU is 2N × 2N (N = 4,8,16,32), one CU has 8 PU types (ie, 2N × 2N). , N × N, 2N × N, N × 2N, nL × 2N, nR × 2N, 2N × nU, 2N × nD).

Similar to intra prediction, PU partitioning in the form of N × N may be performed only when the size of the CB for the luminance component of the CU is the minimum size (that is, the CU is the SCU).

In inter prediction, 2N × N splitting in the horizontal direction and N × 2N splitting in the vertical direction are supported.

In addition, it supports PU partitions of nL × 2N, nR × 2N, 2N × nU, and 2N × nD types, which are Asymmetric Motion Partition (AMP). Here, 'n' means a 1/4 value of 2N. However, AMP cannot be used when the CU to which the PU belongs is a CU of the minimum size.

In order to efficiently encode an input image in one CTU, an optimal partitioning structure of a coding unit (CU), a prediction unit (PU), and a transformation unit (TU) is performed through the following process to achieve the minimum rate-distortion. It can be determined based on the value. For example, looking at the optimal CU partitioning process in 64 × 64 CTU, rate-distortion cost can be calculated while partitioning from a 64 × 64 CU to an 8 × 8 CU. The specific process is as follows.

1) The partition structure of the optimal PU and TU that generates the minimum rate-distortion value is determined by performing inter / intra prediction, transform / quantization, inverse quantization / inverse transform, and entropy encoding for a 64 × 64 CU.

2) Divide the 64 × 64 CU into four 32 × 32 CUs and determine the optimal PU and TU partitioning structure that generates the minimum rate-distortion value for each 32 × 32 CU.

3) The 32 × 32 CU is subdivided into four 16 × 16 CUs, and a partition structure of an optimal PU and TU that generates a minimum rate-distortion value for each 16 × 16 CU is determined.

4) Subdivide the 16 × 16 CU into four 8 × 8 CUs and determine the optimal PU and TU partitioning structure that generates the minimum rate-distortion value for each 8 × 8 CU.

5) 16 × 16 blocks by comparing the sum of the rate-distortion values of the 16 × 16 CUs calculated in 3) above with the rate-distortion values of the four 8 × 8 CUs calculated in 4) above. Determine the partition structure of the optimal CU within. This process is similarly performed for the remaining three 16 × 16 CUs.

6) 32 × 32 block by comparing the sum of the rate-distortion values of the 32 × 32 CUs calculated in 2) above with the rate-distortion values of the four 16 × 16 CUs obtained in 5) above. Determine the partition structure of the optimal CU within. Do this for the remaining three 32x32 CUs.

7) Finally, compare the sum of the rate-distortion values of the 64 × 64 CUs calculated in step 1) with the rate-distortion values of the four 32 × 32 CUs obtained in step 6). The partition structure of the optimal CU is determined within the x64 block.

In the intra prediction mode, a prediction mode is selected in units of PUs, and prediction and reconstruction are performed in units of actual TUs for the selected prediction mode.

TU means a basic unit in which actual prediction and reconstruction are performed. The TU includes a transform block (TB) for a luma component and a TB for two chroma components corresponding thereto.

In the example of FIG. 3, as one CTU is divided into quad-tree structures to generate CUs, the TUs are hierarchically divided into quad-tree structures from one CU to be coded.

Since the TU is divided into quad-tree structures, the TU divided from the CU can be further divided into smaller lower TUs. In HEVC, the size of the TU may be set to any one of 32 × 32, 16 × 16, 8 × 8, and 4 × 4.

Referring again to FIG. 3, it is assumed that a root node of the quad-tree is associated with a CU. The quad-tree is split until it reaches a leaf node, which corresponds to a TU.

In more detail, a CU corresponds to a root node and has a smallest depth (that is, depth = 0). The CU may not be divided according to the characteristics of the input image. In this case, the CU corresponds to a TU.

The CU may be divided into quad tree shapes, resulting in lower nodes having a depth of 1 (depth = 1). In addition, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 1 corresponds to a TU. For example, in FIG. 3B, TU (a), TU (b), and TU (j) corresponding to nodes a, b, and j are divided once in a CU and have a depth of 1. FIG.

At least one of the nodes having a depth of 1 may be split into a quad tree again, resulting in lower nodes having a depth of 1 (ie, depth = 2). In addition, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 2 corresponds to a TU. For example, in FIG. 3B, TU (c), TU (h), and TU (i) corresponding to nodes c, h, and i are divided twice in a CU and have a depth of two.

In addition, at least one of the nodes having a depth of 2 may be divided into quad tree shapes, resulting in lower nodes having a depth of 3 (ie, depth = 3). And, a node that is no longer partitioned (ie, a leaf node) in a lower node having a depth of 3 corresponds to a CU. For example, in FIG. 3 (b), TU (d), TU (e), TU (f), and TU (g) corresponding to nodes d, e, f, and g are divided three times in a CU. Has depth.

A TU having a tree structure may be hierarchically divided with predetermined maximum depth information (or maximum level information). Each divided TU may have depth information. Since the depth information indicates the number and / or degree of division of the TU, it may include information about the size of the TU.

For one TU, information indicating whether the corresponding TU is split (for example, split TU flag split_transform_flag) may be delivered to the decoder. This partitioning information is included in all TUs except the smallest TU. For example, if the value of the flag indicating whether to split is '1', the corresponding TU is divided into four TUs again. If the value of the flag indicating whether to split is '0', the corresponding TU is no longer divided.

Prediction

The decoded portion of the current picture or other pictures containing the current processing unit may be used to restore the current processing unit in which decoding is performed.

Intra picture or I picture (slice) using only the current picture for reconstruction, i.e. performing only intra-picture prediction, and picture (slice) using up to one motion vector and reference index to predict each unit A picture using a predictive picture or P picture (slice), up to two motion vectors, and a reference index (slice) may be referred to as a bi-predictive picture or a B picture (slice).

Intra prediction means a prediction method that derives the current processing block from data elements (eg, sample values, etc.) of the same decoded picture (or slice). That is, a method of predicting pixel values of the current processing block by referring to reconstructed regions in the current picture.

Hereinafter, the inter prediction will be described in more detail.

Inter  Inter prediction (or inter screen prediction)

Inter prediction means a prediction method of deriving a current processing block based on data elements (eg, sample values or motion vectors, etc.) of pictures other than the current picture. That is, a method of predicting pixel values of the current processing block by referring to reconstructed regions in other reconstructed pictures other than the current picture.

Inter prediction (or inter picture prediction) is a technique for removing redundancy existing between pictures, and is mostly performed through motion estimation and motion compensation.

5 is a diagram illustrating a direction of inter prediction as an embodiment to which the present invention may be applied.

Referring to FIG. 5, inter prediction includes uni-directional prediction that uses only one past picture or a future picture as a reference picture on a time axis with respect to one block, and bidirectional prediction that simultaneously refers to past and future pictures. Bi-directional prediction).

In addition, uni-directional prediction includes forward direction prediction using one reference picture displayed (or output) before the current picture in time and 1 displayed (or output) after the current picture in time. It can be divided into backward direction prediction using two reference pictures.

The motion parameter (or information) used to specify which reference region (or reference block) is used to predict the current block in the inter prediction process (i.e., unidirectional or bidirectional prediction) is an inter prediction mode (where The inter prediction mode may indicate a reference direction (i.e., unidirectional or bidirectional) and a reference list (i.e., L0, L1 or bidirectional), a reference index (or reference picture index or reference list index), Contains motion vector information. The motion vector information may include a motion vector, a motion vector prediction (MVP), or a motion vector difference (MVD). The motion vector difference value means a difference value between the motion vector and the motion vector prediction value.

For unidirectional prediction, motion parameters for one direction are used. That is, one motion parameter may be needed to specify the reference region (or reference block).

Bidirectional prediction uses motion parameters for both directions. In the bidirectional prediction method, up to two reference regions may be used. The two reference regions may exist in the same reference picture or may exist in different pictures, respectively. That is, up to two motion parameters may be used in the bidirectional prediction scheme, and two motion vectors may have the same reference picture index or different reference picture indexes. In this case, all of the reference pictures may be displayed (or output) before or after the current picture in time.

The encoder performs motion estimation to find the reference region most similar to the current processing block from the reference pictures in the inter prediction process. In addition, the encoder may provide a decoder with a motion parameter for the reference region.

The encoder / decoder may obtain a reference region of the current processing block using the motion parameter. The reference region exists in a reference picture having the reference index. In addition, the pixel value or interpolated value of the reference region specified by the motion vector may be used as a predictor of the current processing block. That is, using motion information, motion compensation is performed to predict an image of a current processing block from a previously decoded picture.

In order to reduce the amount of transmission associated with the motion vector information, a method of acquiring a motion vector prediction value mvp using motion information of previously coded blocks and transmitting only a difference value mvd thereof may be used. That is, the decoder obtains a motion vector prediction value of the current processing block using motion information of other decoded blocks, and obtains a motion vector value for the current processing block using the difference value transmitted from the encoder. In obtaining the motion vector prediction value, the decoder may obtain various motion vector candidate values by using motion information of other blocks that are already decoded, and obtain one of them as the motion vector prediction value.

Reference picture set and reference picture list

To manage multiple reference pictures, a set of previously decoded pictures are stored in a decoded picture buffer (DPB) for decoding the remaining pictures.

The reconstructed picture used for inter prediction among the reconstructed pictures stored in the DPB is referred to as a reference picture. In other words, a reference picture refers to a picture including a sample that can be used for inter prediction in a decoding process of a next picture in decoding order.

A reference picture set (RPS) refers to a set of reference pictures associated with a picture, and is composed of all pictures previously associated in decoding order. The reference picture set may be used for inter prediction of an associated picture or a picture following an associated picture in decoding order. That is, reference pictures maintained in the decoded picture buffer DPB may be referred to as a reference picture set. The encoder may provide the decoder with reference picture set information in a sequence parameter set (SPS) (ie, a syntax structure composed of syntax elements) or each slice header.

A reference picture list refers to a list of reference pictures used for inter prediction of a P picture (or slice) or a B picture (or slice). Here, the reference picture list may be divided into two reference picture lists, and may be referred to as reference picture list 0 (or L0) and reference picture list 1 (or L1), respectively. Also, a reference picture belonging to reference picture list 0 may be referred to as reference picture 0 (or L0 reference picture), and a reference picture belonging to reference picture list 1 may be referred to as reference picture 1 (or L1 reference picture).

In the decoding process of P pictures (or slices), one reference picture list (i.e., reference picture list 0) is used, and in the decoding process of B pictures (or slices), two reference picture lists (i.e., reference) Picture list 0 and reference picture list 1) may be used. Such information for distinguishing a reference picture list for each reference picture may be provided to the decoder through reference picture set information. The decoder adds the reference picture to the reference picture list 0 or the reference picture list 1 based on the reference picture set information.

A reference picture index (or reference index) is used to identify any one specific reference picture in the reference picture list.

Fractional sample interpolation

 A sample of the prediction block for the inter predicted current processing block is obtained from the sample value of the corresponding reference region in the reference picture identified by the reference picture index. Here, the corresponding reference region in the reference picture represents the region of the position indicated by the horizontal component and the vertical component of the motion vector. Fractional sample interpolation is used to generate predictive samples for noninteger sample coordinates, except when the motion vector has an integer value. For example, a motion vector of one quarter of the distance between samples may be supported.

For HEVC, fractional sample interpolation of luminance components applies an 8-tap filter in the horizontal and vertical directions, respectively. In addition, fractional sample interpolation of the color difference component applies a 4-tap filter in the horizontal direction and the vertical direction, respectively.

6 illustrates integer and fractional sample positions for quarter sample interpolation, as an embodiment to which the present invention may be applied.

Referring to FIG. 6, the shaded block in which the upper-case letter (A_i, j) is written indicates the integer sample position, and the shaded block in which the lower-case letter (x_i, j) is written is the fractional sample position. Indicates.

Fractional samples are generated by applying interpolation filters to integer sample values in the horizontal and vertical directions, respectively. For example, in the horizontal direction, an 8-tap filter may be applied to four integer sample values on the left side and four integer sample values on the right side based on the fractional sample to be generated.

Inter prediction mode

In HEVC, a merge mode and advanced motion vector prediction (AMVP) may be used to reduce the amount of motion information.

1) Merge Mode

Merge mode refers to a method of deriving a motion parameter (or information) from a neighboring block spatially or temporally.

The set of candidates available in merge mode is composed of spatial neighbor candidates, temporal candidates and generated candidates.

7 illustrates a position of a spatial candidate as an embodiment to which the present invention may be applied.

Referring to FIG. 7A, it is determined whether each spatial candidate block is available according to the order of {A1, B1, B0, A0, B2}. In this case, when the candidate block is encoded in the intra prediction mode and there is no motion information, or when the candidate block is located outside the current picture (or slice), the candidate block is not available.

After determining the validity of the spatial candidate, the spatial merge candidate can be constructed by excluding unnecessary candidate blocks from candidate blocks of the current processing block. For example, when the candidate block of the current prediction block is the first prediction block in the same coding block, the candidate block having the same motion information may be excluded except for the corresponding candidate block.

When the spatial merge candidate configuration is completed, the temporal merge candidate configuration process is performed in the order of {T0, T1}.

In the temporal candidate configuration, when the right bottom block T0 of the collocated block of the reference picture is available, the block is configured as a temporal merge candidate. The colocated block refers to a block existing at a position corresponding to the current processing block in the selected reference picture. On the other hand, otherwise, the block T1 located at the center of the collocated block is configured as a temporal merge candidate.

The maximum number of merge candidates may be specified in the slice header. If the number of merge candidates is larger than the maximum number, the number of spatial candidates and temporal candidates smaller than the maximum number is maintained. Otherwise, additional merge candidates (ie, combined bi-predictive merging candidates) are generated by combining the candidates added to date. Even if the merge candidate list is constructed using the combined bi-predictive merge candidates, if the number of merge candidates is still smaller than the maximum number, the number of merge candidates is zero movement while changing the reference picture index until the number of candidate candidates is the maximum number. Zero motion vector merging candidates are added to the merge candidate list.

The encoder constructs a merge candidate list in the above manner and performs motion estimation to merge candidate block information selected from the merge candidate list into a merge index (for example, merge_idx [x0] [y0] '). Signal to the decoder. In FIG. 7B, the B1 block is selected from the merge candidate list. In this case, “index 1” may be signaled to the decoder as a merge index.

The decoder constructs a merge candidate list similarly to the encoder, and derives the motion information of the current block from the motion information of the candidate block corresponding to the merge index received from the encoder in the merge candidate list. The decoder generates a prediction block for the current processing block based on the derived motion information (ie, motion compensation).

2) Advanced Motion Vector Prediction (AMVP) Mode

The AMVP mode refers to a method of deriving a motion vector prediction value from neighboring blocks. Thus, horizontal and vertical motion vector difference (MVD), reference index, and inter prediction modes are signaled to the decoder. The horizontal and vertical motion vector values are calculated using the derived motion vector prediction value and the motion vector difference (MVD) provided from the encoder.

That is, the encoder constructs a motion vector predictor candidate list and performs motion estimation to perform a motion estimation flag (ie, candidate block information) selected from the motion vector predictor candidate list (for example, mvp_lX_flag [x0] [y0). ] ') Is signaled to the decoder. The decoder constructs a motion vector predictor candidate list similarly to the encoder, and derives a motion vector predictor of the current processing block using the motion information of the candidate block indicated by the motion reference flag received from the encoder in the motion vector predictor candidate list. The decoder obtains a motion vector value for the current processing block by using the derived motion vector prediction value and the motion vector difference value transmitted from the encoder. The decoder generates a prediction block for the current processing block based on the derived motion information (ie, motion compensation).

In the AMVP mode, two spatial motion candidates are selected from among the five available candidates in FIG. 7. The first spatial motion candidate is selected from the set of {A0, A1} located on the left side, and the second spatial motion candidate is selected from the set of {B0, B1, B2} located above. At this time, when the reference index of the neighboring candidate block is not the same as the current prediction block, the motion vector is scaled.

If the number of candidates selected as a result of the search for the spatial motion candidate is two, the candidate configuration is terminated, but if less than two, the temporal motion candidate is added.

8 is a diagram illustrating an inter prediction method as an embodiment to which the present invention is applied.

Referring to FIG. 8, a decoder (in particular, the inter prediction unit 261 of the decoder in FIG. 2) decodes a motion parameter for a processing block (eg, a prediction unit) (S801).

For example, if the processing block has a merge mode applied, the decoder may decode the merge index signaled from the encoder. The motion parameter of the current processing block can be derived from the motion parameter of the candidate block indicated by the merge index.

In addition, when the processing block is applied with the AMVP mode, the decoder may decode horizontal and vertical motion vector difference (MVD), reference index, and inter prediction mode signaled from the encoder. The motion vector prediction value may be derived from the motion parameter of the candidate block indicated by the motion reference flag, and the motion vector value of the current processing block may be derived using the motion vector prediction value and the received motion vector difference value.

The decoder performs motion compensation on the prediction unit by using the decoded motion parameter (or information) (S802).

That is, the encoder / decoder performs motion compensation that predicts an image of a current unit from a previously decoded picture by using the decoded motion parameter.

9 is a diagram illustrating a motion compensation process as an embodiment to which the present invention may be applied.

9 illustrates a case in which a motion parameter for a current block to be encoded in a current picture is unidirectional prediction, a second picture in LIST0, LIST0, and a motion vector (-a, b). do.

In this case, as shown in FIG. 9, the current block is predicted using values of positions (ie, sample values of reference blocks) that are separated from the current block by (-a, b) in the second picture of LIST0.

In the case of bidirectional prediction, another reference list (eg, LIST1), a reference index, and a motion vector difference value are transmitted so that the decoder derives two reference blocks and predicts the current block value based on the reference block.

Inter  Prediction-based Image Processing Method

As described above, the merge mode used in HEVC collects and lists neighboring (block) motion information of the current block to be encoded, selects one of them, and transmits only an index indicating the corresponding block.

Referring to FIG. 7 again, the order of collecting (listing) peripheral motion information of the current block is shown in Table 1 below.

Referring to Table 1, in order to construct a merge mode motion information candidate list, the encoder / decoder includes five spatial peripheral motion information (A1, B1, B0, A0, B2), and two temporal peripheral motion information (T0, T1). The process of selecting a total of five motion information in order of a combined bi-predicted candidate, a non-scaled bi-predicted candidate, and a zero vector. Perform.

The duplicate candidate is checked while constructing the merge candidate list, and if duplicate motion information exists, the next candidate is considered as a list insertion target. This will be described with reference to the drawings below.

10 is a diagram illustrating a process of constructing a merge candidate list.

Referring to FIG. 10, the encoder / decoder selects up to four pieces of spatial peripheral motion information and adds them to the merge candidate list except for duplication (S1001).

Here, the spatial peripheral motion information refers to the motion information of the A1, B1, B0, A0, and B2 blocks (ie, parts of the prediction unit) that are spatially neighbored to the current block in the example of FIG. 7.

Not available in the motion information of a block spatially neighboring the current block (e.g., when a spatially neighboring block is encoded in intra prediction mode) and / or in the order of A1, B1, B0, A0, B2 If the motion information of the neighboring block is searched along with the motion information of the previous neighboring block (ie, motion information added to the merge candidate list), the motion information of the neighboring block is not added to the merge list. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder may add motion information of neighboring blocks available in the order of spatially neighboring A1, B1, B0, A0, and B2 (see FIG. 7) to the merge candidate list so as not to overlap.

Here, motion information added to a merge candidate list from a spatially neighboring block may be referred to as a spatial merge candidate (or a spatial motion information candidate).

The encoder / decoder selects one of the temporal peripheral motion information except for duplication and adds it to the merge candidate list (S1002).

Here, the temporal peripheral motion information refers to motion information of the T0 and T1 blocks (ie, parts of the prediction unit) that are temporally neighbored to the current block in the example of FIG. 7.

If the motion information of the T0 block that is temporally neighbored to the current block is available, the motion information of the T0 block may be added to the merge candidate list. On the other hand, when the motion information of the T0 block is not available, if the motion information of the T1 block is available, the motion information of the T1 block may be added to the merge candidate list. Even in this case, if the motion information of each neighboring block is the same as the motion information (ie, the motion information added to the merge candidate list) of the previous neighboring block, it may not be added to the merge candidate list.

That is, the encoder / decoder may add motion information of neighboring blocks available in the order of neighboring T0 and T1 (see FIG. 7) to the merge candidate list so as not to overlap.

Here, the motion information added to the merge candidate list from the temporal neighboring block may be referred to as a temporal merge candidate (or spatial motion information candidate).

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1003). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If there are five items in the merge candidate list, the encoder / decoder completes the merge candidate list construction.

On the other hand, if there are not five items in the merge candidate list, the encoder / decoder combines all of the current merge candidates in the merge list (ie, spatial merge candidates and / or temporal merge candidates). (Ie, the combined bidirectional motion information candidates) are added to the merge candidate list (S1004).

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1005). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If there are five items in the merge candidate list, the encoder / decoder completes the merge candidate list construction.

On the other hand, if there are not five merge candidate list items, the encoder / decoder adds non-scaled bidirectional prediction candidates (ie, non-scaled motion information candidates) to the merge candidate list (S1006).

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1007). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If there are five merge candidate list items, the encoder / decoder completes the merge candidate list construction.

On the other hand, if there are not five merge candidate list items, the encoder / decoder adds a zero vector to the merge candidate list (S1008).

That is, the encoder / decoder merges zero motion vector merging candidates (ie, zero motion vector candidates) while changing the reference picture index until the number of merge candidates reaches the maximum number of candidates. Add to

Meanwhile, in FIG. 10, the process of adding the non-scaled bidirectional prediction candidate to the list may be omitted. In this case, steps S1006 and S1007 are deleted, and if there are not five list items in step S1005, step S1008 may be performed.

In general, since the neighboring motion information has a high probability of being identical to the motion information of the current block, and then the motion information around the temporal neighbors and the combined bidirectional prediction candidates, the merge candidate list construction method described above is simple. High efficiency

However, in some cases, the aforementioned order does not always match, and since the index of the merge mode is encoded by truncated rice code, it is optimal to construct the merge candidate list in the order of most likely candidates. Will be shown.

Accordingly, the present invention proposes a method of constructing a motion information candidate list of the current block.

Hereinafter, a case of constructing a merge candidate list will be described for convenience of description. However, the present invention is not limited thereto, and the same method may be applied to the case of constructing a motion vector prediction value candidate list by applying an AMVP mode to a current block. have. That is, the motion information candidate list is collectively referred to herein as a merge candidate list and a motion vector predictor candidate list.

Hereinafter, the motion information may include a reference picture index and a motion vector indicating the reference picture. In addition, it may further include an inter prediction mode and a motion vector prediction value indicating the direction of the reference picture.

Example  One

According to an embodiment of the present invention, a method for constructing a motion information list using a frequency of occurrence of a motion information candidate added to a motion information candidate list of a current block is proposed.

The method proposed in the present embodiment proceeds in the same manner as the motion information candidate list construction order described with reference to FIG. 10, but there is a difference in that the number of overlaps is recorded.

According to the example of FIG. 10, when duplication occurs while generating the motion information candidate list (that is, the same motion information as the previous motion information candidate), the subsequent procedure is not inserted into the motion information candidate list.

On the other hand, according to the present embodiment, when duplicated motion information is identified (that is, the motion information of the neighboring block is searched in a predetermined order while the motion information of the neighboring block is added to the previous neighboring block (ie, motion added to the motion information candidate list) Information) is not inserted into the motion information candidate list, but the frequency of occurrence is increased to the same existing motion information item (ie, motion information added to the motion information candidate list) and the subsequent procedure is performed. Will be performed.

For example, the encoder / decoder may search for motion information of spatially neighboring blocks of the current prediction unit in the order of A1, B1, B0, A0, and B2 (see FIG. 7). First, when motion information of the A1 block is available, the encoder / decoder may add motion information of the A1 block to the motion information candidate list. At this time, if the motion information of the B1 block is the same as the motion information of A1, the frequency of the motion information (ie, duplicated motion information) of the already added A1 block is not added to the motion information candidate list without adding the motion information of the B1 block to the motion information candidate list. Increment frequency by 1 Thereafter, the same method may be applied to the motion information of other spatial / temporal neighboring blocks.

In this case, when the coordinate of the top-left pixel of the current block is (x, y), the width of the current block is W, and the height of the current block is H, the A0 block is the coordinate (x-1, y + H). Block A1 is a block containing pixels having coordinates (x-1, y + H-1), and block B0 includes pixels having coordinates (x + W, y-1). The block B1 may be a block including a pixel having coordinates (x + W-1, y-1), and the block B2 may represent a block including a pixel having coordinates (x-1, y-1).

In addition, the T0 block is a block including pixels having coordinates (x + W, y + H) in a picture different from the picture to which the current block belongs, and the T1 block is a coordinate (x +) in a picture different from the picture to which the current block belongs. A block including a pixel of W / 2, y + H / 2) may be represented.

In this case, as shown in FIG. 10, the sequence of the spatial motion information candidate, the temporal motion information candidate, the combined bidirectional prediction candidate (ie, the combined motion information candidate), and the zero motion vector prediction candidate (ie, the zero motion information candidate) are shown. It can be added to the motion information candidate list. However, the present invention is not limited thereto, and the motion information candidate list may be configured using only some of the above four motion information candidates, or the motion information candidate list may be configured including other motion information. Also, the motion information candidate list may be configured in a different order from that in FIG.

After all procedures are completed (that is, the construction of the motion information candidate list is completed), the motion information candidate list can be sorted according to the frequency of occurrence of each motion information candidate. Accordingly, the motion information candidate close to the actual occurrence probability may be located above the motion information candidate list.

11 illustrates an example of a configuration of a motion information candidate list according to an embodiment of the present invention.

11 shows an example of a motion information candidate list (eg, merge candidate list) generated through the present embodiment.

Referring to FIG. 11, the motion information candidate list includes motion information of the A1 block, motion information of the B0 block, motion information of the B1 block, motion information of the T0 block, and combined motion information of the motion information candidate (ie, the merge candidate list). For example, an item added to a list is shown. In this case, it is assumed that the occurrence frequency of the motion information of the A1 block is three times, the occurrence frequency of the motion information of the B0 block is two times, and the occurrence frequency of the remaining motion information is one time.

In this case, unlike the order of the motion information candidate list applied in the merge mode according to the example of FIG. 10, it may be confirmed that the order of B0 and B1 is changed.

In other words, the encoder / decoder may add motion information of neighboring blocks available in the order of spatially neighboring A1, B1, B0, A0, and B2 (see FIG. 7) to the merge candidate list so as not to overlap. However, since the motion information of the B0 block is twice and the frequency of occurrence of the motion information of the B1 block is one time, the motion information candidate list is located such that the motion information of the B0 block is located above the motion information candidate list than the motion information of the B1 block. The items of (ie, motion information candidates) may be sorted.

In addition, in the process of sorting the motion information candidate list, as described above, list items (ie, motion information candidates) may be simply sorted according to the frequency of occurrence. In this case, the sorting may be performed only (ie, the order of motion information candidates) when the difference in the frequency of occurrence is more than a predetermined level. For example, assuming that a predetermined predetermined level is 2 in the example of FIG. 11, the order of the motion information of the B0 block and the motion information of the B1 block is not changed, and the motion information of the B1 block is the motion information candidate rather than the motion information of the B0 block. It can be located above the list.

A process of sorting the motion information candidate list will be described in more detail with reference to the following drawings.

12 is a diagram illustrating a method of sorting a motion information candidate list according to an embodiment of the present invention.

In FIG. 12, it is assumed that the number of items (items) of the motion information candidate list is five, and the motion information candidate list is configured as shown in FIG. 12 (a).

The encoder / decoder may determine the position (or position) of each motion information candidate in the order from the top to the bottom in the motion information candidate list. The position of each motion information candidate can be determined by comparing the occurrence frequency of each motion information candidate with the occurrence frequency of the motion information candidate located one step higher. In this case, when the position of the corresponding motion information candidate is changed, the position of the corresponding motion information candidate may be determined by comparing again with the occurrence frequency of the motion information candidate located one step higher than the changed position.

First, the encoder / decoder compares the occurrence frequency of the motion information candidate 2 with the occurrence frequency of the motion information candidate 1 located higher than itself, and if the occurrence frequency of the motion information candidate 2 is greater than the occurrence frequency of the motion information candidate 1, FIG. As shown in 12 (b), the position of the motion information candidate 2 may be replaced with the position of the motion information candidate 1.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 3 with the occurrence frequency of the motion information candidate 1 located one step higher than its current position, and the occurrence frequency of the motion information candidate 3 is equal to that of the motion information candidate 1. If it is larger than the occurrence frequency, the position of the motion information candidate 3 can be replaced with the position of the motion information candidate 1 as shown in FIG.

At this time, since the position of the motion information candidate 3 is changed, the encoder / decoder compares the occurrence frequency of the motion information candidate 3 with the occurrence frequency of the motion information candidate 2 located one step higher than the current position, and generates the motion information candidate 3. If the frequency is greater than the occurrence frequency of the motion information candidate 2, the position of the motion information candidate 3 can be replaced with the position of the motion information candidate 2 as shown in FIG.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 4 with the occurrence frequency of the motion information candidate 1 located above the current position, and the occurrence frequency of the motion information candidate 4 is greater than the occurrence frequency of the motion information candidate 1. Otherwise, the position of the motion information candidate 4 may not be changed.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 5 with the occurrence frequency of the motion information candidate 4 located above the current position, and if the occurrence frequency of the motion information candidate 5 is greater than the occurrence frequency of the motion information candidate 4 12, the position of the motion information candidate 5 may be replaced with the position of the motion information candidate 4. As shown in FIG.

At this time, since the position of the motion information candidate 5 is changed, the encoder / decoder compares the occurrence frequency of the motion information candidate 5 with the occurrence frequency of the motion information candidate 1 located one step higher than the current position, and generates the motion information candidate 5. If the frequency is greater than the occurrence frequency of the motion information candidate 1, the position of the motion information candidate 5 can be replaced with the position of the motion information candidate 1 as shown in FIG.

At this time, since the position of the motion information candidate 5 is changed again, the encoder / decoder compares the occurrence frequency of the motion information candidate 5 with the occurrence frequency of the motion information candidate 2 located one step higher than the current position, If the occurrence frequency is not greater than the occurrence frequency of the motion information candidate 2, the position of the motion information candidate 5 may not be changed.

13 is a diagram illustrating a method of sorting a motion information candidate list according to an embodiment of the present invention.

In FIG. 13, it is assumed that the number of items (items) of the motion information candidate list is 5, and that the motion information candidate list is configured as shown in FIG. 13 (a). The level for determining whether to change the position of the motion information candidate is' Assume that 2 'is predetermined.

First, the encoder / decoder compares the occurrence frequency of the motion information candidate 2 with the occurrence frequency of the motion information candidate 1 located higher than itself, and if the occurrence frequency of the motion information candidate 2 is two or more greater than the occurrence frequency of the motion information candidate 1 As shown in FIG. 13B, the position of the motion information candidate 2 may be replaced with the position of the motion information candidate 1.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 3 with the occurrence frequency of the motion information candidate 1 located one step higher than its current position, and the occurrence frequency of the motion information candidate 3 is equal to that of the motion information candidate 1. If the frequency is greater than 2, the position of the motion information candidate 3 can be replaced with the position of the motion information candidate 1 as shown in FIG.

At this time, since the position of the motion information candidate 3 is changed, the encoder / decoder compares the occurrence frequency of the motion information candidate 3 with the occurrence frequency of the motion information candidate 2 located one step higher than the current position, and generates the motion information candidate 3. If the frequency is not greater than two or more than the occurrence frequency of the motion information candidate 2, the position of the motion information candidate 3 may not be changed.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 4 with the occurrence frequency of the motion information candidate 1 located above the current position, and the occurrence frequency of the motion information candidate 4 is 2 higher than the occurrence frequency of the motion information candidate 1. If it is not large, the position of motion information candidate 4 may not be changed.

Next, the encoder / decoder compares the occurrence frequency of the motion information candidate 5 with the occurrence frequency of the motion information candidate 4 located above the current position, and the occurrence frequency of the motion information candidate 5 is greater than the occurrence frequency of the motion information candidate 4. If it is not large, the position of the motion information candidate 5 may not be changed.

14 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

In FIG. 14, for convenience of description, it is assumed that the total number of items (ie, motion information candidates) of the merge candidate list (ie, motion information candidate list) is 5, similar to FIG. 10, and is spatial in the merge candidate list. Up to four merge candidates may be added, and assuming that one merge candidate may be added. However, the present invention is not limited thereto. That is, the present invention can be applied equally even if the number of items of the predefined motion information candidate list and the number of spatial motion information candidates and temporal motion information candidates included in the motion information candidate list are different from each other.

Referring to FIG. 14, the encoder / decoder selects up to four of spatial peripheral motion information except for overlap and adds the merge candidate list to the merge candidate list, but increases the frequency in the case of duplication (S1401).

Here, the spatial peripheral motion information may refer to motion information of blocks A1, B1, B0, A0, and B2 (that is, part of a prediction unit) that are spatially neighbored to the current block in the example of FIG. 7.

Motion information that is not available in the motion information of a block that is spatially neighboring to the current block may not be added to the merge list (eg, when the spatially neighboring block is encoded in the intra prediction mode).

Also, as motion information is searched in the order of A1, B1, B0, A0, and B2, motion information of a neighboring block currently being searched for is motion information of a previous neighboring block (ie, motion information added to a merge candidate list). If it is equal to and is not added to the merge list, the frequency of occurrence of motion information (ie, duplicate motion information) of a previous neighboring block may be increased by one. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder adds motion information of neighboring blocks available in the order of spatially neighboring A1, B1, B0, A0, and B2 blocks (see FIG. 7) to the merge candidate list so as not to overlap, but duplicate motion information. Is generated, the frequency of occurrence of duplication of the corresponding motion information may be increased by one.

The encoder / decoder selects one of the temporal peripheral motion information except for duplication and adds it to the merge candidate list, but increases the frequency in the case of duplication (S1402).

Here, the temporal peripheral motion information may refer to motion information of the T0 and T1 blocks (ie, parts of the prediction unit) that are temporally neighbored to the current block in the example of FIG. 7.

If the motion information of the T0 block that is temporally neighbored to the current block is available, the motion information of the T0 block may be added to the merge candidate list. On the other hand, when the motion information of the T0 block is not available, if the motion information of the T1 block is available, the motion information of the T1 block may be added to the merge candidate list. Even in this case, if the motion information of the neighboring block currently being searched for is the same as the motion information or the spatial merge candidate of the previous neighboring block, the motion information of the previous neighboring block is not added to the merge candidate list, that is, the duplicated motion information. ), The frequency of occurrence may be increased by one.

That is, the encoder / decoder adds motion information of neighboring blocks available in the order of neighboring T0 and T1 (see FIG. 7) to the merge candidate list so as not to overlap, but duplicates the motion information when duplicate motion information is generated. The frequency of occurrence can be increased by one.

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1403). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If it is determined in step S1403 that there are five items in the merge candidate list, the encoder / decoder sorts the items in the merge candidate list based on the frequency (S1409).

That is, merge candidates of the merge candidate list are sorted in ascending order, and merge candidates having the same occurrence frequency are arranged according to a predetermined merge candidate list construction order (i.e., search order of motion information of spatial / temporal neighboring blocks). Can be aligned.

In this case, as described above, when sorting in the order of occurrence frequency, the merge candidate list may be sorted only when the difference in occurrence frequency is greater than or equal to a predetermined level.

For example, the merge candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1403 that the merge candidate list has not five items (ie, if there are less than five), the encoder / decoder indicates that all current merge candidates (ie, spatial merge candidates and / or temporal merge candidates) in the merge list. ), The combined bidirectional prediction candidates (ie, the combined bidirectional motion information candidates) are added to the merge candidate list (S1404).

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1405). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If it is determined in step S1405 that there are five items of the merge candidate list, the encoder / decoder sorts the items of the merge candidate list based on the frequency (S1409).

That is, merge candidates of the merge candidate list are sorted in ascending order, and merge candidates having the same occurrence frequency are arranged in a predetermined merge candidate list structure (that is, motion information of spatial / temporal neighboring blocks and combined bidirectional prediction candidates). Search order).

In this case, as described above, when sorting in the order of occurrence frequency, the merge candidate list may be sorted only when the difference in occurrence frequency is greater than or equal to a predetermined level.

For example, the merge candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1405 that there are not five items in the merge candidate list (that is, if there are less than five), the non-scaled bidirectional prediction candidates (that is, the non-scaled bidirectional motion information candidates). ) Is added to the merge candidate list (S1406).

The encoder / decoder determines whether there are five items (or items) in the merge candidate list (S1407). That is, it is determined whether the number of merge candidates added to the merge candidate list is five.

If it is determined in step S1407 that there are five items of the merge candidate list, the encoder / decoder sorts the items of the merge candidate list based on the frequency (S1409).

That is, merge candidates of the merge candidate list are sorted in ascending order, and merge candidates having the same occurrence frequency are arranged in a predetermined merge candidate list order (i.e., motion information of spatial / temporal neighboring blocks, combined bidirectional prediction candidates). And search order of non-scaled bidirectional prediction candidates).

In this case, as described above, when sorting in the order of occurrence frequency, the merge candidate list may be sorted only when the difference in occurrence frequency is greater than or equal to a predetermined level.

For example, the merge candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1407 that the merge candidate list has not five items (i.e., less than five), the encoder / decoder inserts a zero vector (i.e., zero motion vector candidate) into the merge candidate list. It adds (S1408).

That is, the encoder / decoder zero motion vector merge candidate (zero motion) while changing the reference picture index until the number of merge candidates added to the merge candidate list is a predetermined number (for example, five) of the merge candidate list. Add vector merging candidates to the merge candidate list.

The encoder / decoder sorts the items of the merge candidate list based on the frequency (S1409).

That is, merge candidates of the merge candidate list are sorted in ascending order, and merge candidates having the same occurrence frequency are arranged in a predetermined merge candidate list order (i.e., motion information of spatial / temporal neighboring blocks, combined bidirectional prediction candidates). , Search order of non-scaled bidirectional prediction candidates, zero motion vector candidates).

In this case, as described above, when sorting in the order of occurrence frequency, the merge candidate list may be sorted only when the difference in occurrence frequency is greater than or equal to a predetermined level.

For example, the merge candidate list may be sorted according to the process of FIGS. 12 and / or 13.

Meanwhile, motion information (ie, spatial motion information candidate, temporal motion information candidate, combined bidirectional prediction candidate, and non-scaled bidirectional prediction) that are searched for constructing the motion information candidate list in FIG. The candidate, zero vector) is just one example, and the search may be performed on only some of the motion information in the example of FIG. 14, or motion information not illustrated in FIG. 14 may be further searched.

In particular, the process of adding a non-scaled bidirectional prediction candidate to the list can be omitted. In this case, steps S1406 and S1407 are deleted, and if there are not five list items in step S1405, step S1408 may be performed.

In addition, in FIG. 14, the construction order of the motion information candidate list (ie, spatial motion information candidate, temporal motion information candidate, combined bidirectional prediction candidate, non-scaled bidirectional prediction candidate, and zero vector) is shown. Order) is just one example, and may proceed in a different predetermined order.

Meanwhile, the above-described embodiment has described a case in which a motion information candidate list is generated while updating the frequency of occurrence of duplicated motion information only for positions of spatial / temporal neighboring blocks referred to in the existing merge candidate list generation process. The invention is not limited thereto.

Accordingly, the present embodiment proposes a method of using spatial / temporal surrounding information of various positions to generate a neighboring motion list (ie, motion information candidate list) of the current block in consideration of the occurrence frequency. That is, in this embodiment, the occurrence frequency of the motion information may be checked using the motion information of various spatial / temporal neighboring blocks, and a motion information candidate list may be generated based on the occurrence frequency.

15 is a diagram illustrating a spatial / temporal neighbor block that can be referred to to generate a motion information candidate list according to an embodiment of the present invention.

In FIG. 15, neighboring blocks adjacent to the left of the current block in space are referred to as L0, L1, ..., and blocks adjacent to the top of the current block in space are referred to as A0, A1, .... Also, a block adjacent to the upper left end of the current block is called AL, a block adjacent to the upper right end is referred to as AR, and a block adjacent to the lower left end is referred to as BL. Also, T4 is the block corresponding to the upper left portion of the current block in another picture temporally, T3 is the upper left block at the position corresponding to the center of the current block in another picture temporally, and the center of the current block in another picture temporally. T2 is the right bottom block at the position corresponding to T2, T1 is the block corresponding to the bottom right portion of the current block in another picture temporally, T0 is the block adjacent to the bottom right end of the block corresponding to the current block in another picture temporally. It is called.

At this time, assuming that the coordinates of the top-left pixel of the current block is (x, y), the width of the current block is W, and the height of the current block is H, each block may be represented as follows. .

The A0 block may be a block including a pixel having coordinates (x, y-1), and the A1 block may be a block including a pixel having coordinates (x + A0 width, y-1). Similarly, the A2 block may be a block including a pixel having (x + A0 width + A1 width, y-1).

The L0 block may be a block including pixels having coordinates (x-1, y), and the L1 block may be a block including pixels having coordinates (x-1, y + L0 heights). Similarly, the L2 block may be a block including a pixel of (x-1, y + L0 height + L1 height).

The AL block is a block containing pixels with coordinates (x-1, y-1), the AR block is a block containing pixels with coordinates (x + W, y-1), and the BL block is a coordinate (x-1 , y + H).

In addition, the T0 block is a block including pixels having coordinates (x + W, y + H) in a picture different from the picture to which the current block belongs, and the T1 block is a coordinate (x +) in a picture different from the picture to which the current block belongs. W-1, y + H-1), and the block T2 includes pixels having coordinates (x + W / 2, y + H / 2) in a picture different from the picture to which the current block belongs. Block, and the T3 block is a block containing pixels with coordinates (x + W / 2-1, y + H / 2-1) within a picture different from the picture to which the current block belongs, and the T4 block is a picture to which the current block belongs. It may be a block including a pixel having a coordinate (x, y) in a picture different from.

The encoder / decoder may generate a motion information candidate list of the current block by referring to motion information of a block (for example, a part of a prediction unit) corresponding to each position spatially and temporally.

In this case, the encoder / decoder may generate a motion information candidate list using the surrounding motion information that can be referenced in a predetermined order. For example, the motion information candidate list may be generated in the following order.

1) Refer to motion information of left block, AL, motion information of upper block, AR, BL, and temporal peripheral motion information

In this case, the encoder / decoder searches for motion information in the order of L0, L1, L2, ..., AL, A0, A1, A2, ..., AR, BL, T0, T1, ..., T4 blocks. While new items (ie, motion information) are inserted into the motion information candidate list, the duplicated motion information can be updated (that is, increased by 1) without being inserted into the motion information candidate list. The encoder / decoder may terminate the motion information candidate list generation process when the number of items of the predefined motion information candidate list is reached.

2) Alternately reference the motion information of the left block and the motion information of the upper block, and refer to AL, AR, BL, and temporal surrounding motion information.

In this case, the encoder / decoder searches for motion information in the order of L0, A0, L1, A1, ..., AL, AR, BL, T0, T1, ...., T4 blocks, Information) can be inserted into the motion information candidate list, and the frequency can be updated (ie, increased by 1) without inserting duplicate motion information into the motion information candidate list. The encoder / decoder may terminate the motion information candidate list generation process when the number of items of the predefined motion information candidate list is reached.

In addition to the above two methods, there may be a reference order for generating various motion information candidate lists according to the order in which the positions of the AL, AR, and BR and the temporal peripheral motion information are considered.

16 is a diagram illustrating a spatial neighboring block that can be referred to for generating a motion information candidate list according to an embodiment of the present invention.

Although only spatial neighboring blocks are illustrated in FIG. 16, available neighboring blocks may be defined as temporal neighboring blocks as illustrated in FIG. 7 or FIG. 16.

Assuming that the coordinates of the top-left pixel of the current block is (x, y), the width of the current block is W, and the height of the current block is H, each block may be represented as follows.

The A0 block may be a block including a pixel having coordinates (x, y-1), and the A1 block may be a block including a pixel having coordinates (x + W / 2, y-1).

The L0 block may be a block including a pixel having coordinates (x-1, y), and the L1 block may be a block including a pixel having coordinates (x-1, y + H / 2).

The AL block is a block containing pixels with coordinates (x-1, y-1), the AR block is a block containing pixels with coordinates (x + W, y-1), and the BL block is a coordinate (x-1 , y + H).

In this case, the encoder / decoder may generate a motion information candidate list using the surrounding motion information that can be referenced in a predetermined order. For example, the motion information candidate list may be generated in the following order.

1) As shown in FIG. 15, the encoder / decoder may generate a motion information candidate list by referring to motion information of a left block, AL, motion information of an upper block, AR, BL, and temporal peripheral motion information.

In this case, the encoder / decoder searches for motion information in the order of L0, L1, AL, A0, A1, AR, BL, and temporally neighboring blocks, and inserts a new item (i.e., motion information) into the motion information candidate list. The frequency information can be updated (that is, increased by 1) without overlapping the motion information candidate list. The encoder / decoder may terminate the motion information candidate list generation process when the number of items of the predefined motion information candidate list is reached.

2) Alternatively, refer to alternating motion information of the left block and motion information of the upper block, AL, AR, BL, and temporal peripheral motion information.

In this case, the encoder / decoder searches for motion information in the order of L0, A0, L1, A1, AL, AR, BL, and temporally neighboring blocks, and inserts a new item (ie, motion information) into the motion information candidate list in the case of a new item (ie, motion information). The frequency information can be updated (that is, increased by 1) without overlapping the motion information candidate list. The encoder / decoder may terminate the motion information candidate list generation process when the number of items of the predefined motion information candidate list is reached.

The encoder / decoder may refer to the motion information of all the spatial / temporal neighboring blocks, which can be referred to as shown in FIG. 15, but may refer to only the motion information of the spatial / temporal neighboring block of a specific location as shown in FIG. 16.

Alternatively, the encoder / decoder may generate a list by referring to motion information of a block selected at predetermined intervals from all the spatial / temporal neighboring blocks that can be referred to as shown in FIG. 15. For example, the encoder / decoder may refer to the blocks of A0, A2, A4, ... by selecting in two block intervals for blocks adjacent to the top of the current block and / or for blocks adjacent to the left of the current block. It is also possible to refer to blocks L0, L2, ... by selecting at intervals of two blocks.

In addition, the encoder / decoder of each block may be used only when the left and / or top blocks of the current block are divided into blocks smaller than the size of the current block by using divided information (eg, prediction unit division). The motion information candidate list may be generated with reference to the motion information (or motion information of blocks selected at predetermined intervals).

17 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

In FIG. 17, it is assumed that the number of items (items) of the predetermined motion information candidate list is N.

Referring to FIG. 17, the encoder / decoder adds spatial peripheral motion information to a motion information candidate list by a predetermined number, and updates (ie, increases) the frequency when duplicates (S1701).

Here, the spatial peripheral motion information is previously shown in the examples of FIGS. 15 and / or 16. AL, AR, BL, A0, ..., L0, ... blocks (ie, parts of prediction units) spatially neighboring the current block. ) May mean motion information.

Motion information that is not available in the motion information of a block spatially neighboring the current block may not be added to the motion information candidate list (eg, when the spatially neighboring block is encoded in the intra prediction mode).

In addition, when motion information of a neighboring block is searched according to a predetermined order, the motion information of the neighboring block currently being searched for is the same as the motion information of the previous neighboring block (ie, motion information added to the motion information candidate list). Not added to the information candidate list, the frequency of occurrence of motion information (ie, duplicate motion information) of a previous neighboring block may be increased by one. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder adds the motion information of neighboring blocks to the motion information candidate list so as not to be overlapped spatially in a predetermined order, and increases the frequency of duplication of the corresponding motion information by 1 when duplicate motion information is generated. Can be.

The encoder / decoder adds temporal peripheral motion information to the motion information candidate list by a predetermined number, but updates (ie, increases) the frequency in the case of overlapping (S1702).

Here, the temporal peripheral motion information may refer to motion information of the T0, ... block (ie, part of the prediction unit) that is temporally neighbored to the current block in the example of FIG. 15 and / or 16.

Motion information that is not available in the motion information of a block temporally neighboring to the current block may not be added to the motion information candidate list (eg, when a spatially neighboring block is encoded in an intra prediction mode).

In addition, when motion information of a neighboring block is searched according to a predetermined order, the motion information of the neighboring block currently being searched for is the same as the motion information of the previous neighboring block (ie, motion information added to the motion information candidate list). Not added to the information candidate list, the frequency of occurrence of motion information (ie, duplicate motion information) of a previous neighboring block may be increased by one. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder adds motion information of neighboring blocks to the motion information candidate list in a predetermined order so as not to overlap, and increases the frequency of duplication of the corresponding motion information by 1 when duplicate motion information is generated. Can be.

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1703).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1703 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder sorts the items of the motion information candidate list based on the frequency number (S1709).

That is, motion information candidates of the motion information candidate list are arranged in order of occurrence frequency, and motion information candidates having the same occurrence frequency are arranged in a predetermined motion information candidate configuration order (i.e., search order of motion information of spatial / temporal neighboring blocks). Can be sorted according to

In this case, as described above, when the frequency of occurrence is sorted in ascending order, the motion information candidate list may be sorted only when the difference in the frequency of occurrence is greater than or equal to a predetermined level.

For example, the motion information candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1703 that the number of motion information candidates added to the motion information candidate list is not N (that is, if it is less than N), the encoder / decoder adds all motion information candidates added to the present in the motion information candidate list. By combining the spatial motion information candidates and / or the temporal motion information candidates, the combined bidirectional prediction candidates (ie, the combined motion information candidates) are added to the motion information candidate list (S1704).

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1705).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1705 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder sorts the items of the motion information candidate list based on the frequency number (S1709).

That is, motion information candidates of the motion information candidate list are arranged in order of occurrence frequency, and motion information candidates having the same occurrence frequency are arranged in a predetermined order of motion information candidate configuration (i.e., motion information of spatial / temporal neighboring blocks and combined). Search order of bidirectional prediction candidates).

In this case, as described above, when the frequency of occurrence is sorted in ascending order, the motion information candidate list may be sorted only when the difference in the frequency of occurrence is greater than or equal to a predetermined level.

For example, the motion information candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1705 that the number of motion information candidates added to the motion information candidate list is not N (that is, less than N), the encoder / decoder is a non-scaled bidirectional prediction candidate ( That is, a non-scaled bidirectional motion information candidate) is added to the motion information candidate list (S1706).

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1707).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1707 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder sorts the items of the motion information candidate list based on the frequency number (S1709).

That is, motion information candidates of the motion information candidate list are arranged in order of occurrence frequency, and motion information candidates having the same occurrence frequency are arranged in a predetermined order of motion information candidate configuration (that is, motion information of spatial / temporal neighboring blocks, combined Search order of bidirectional prediction candidates and non-scaled prediction candidates).

In this case, as described above, when the frequency of occurrence is sorted in ascending order, the motion information candidate list may be sorted only when the difference in the frequency of occurrence is greater than or equal to a predetermined level.

For example, the motion information candidate list may be sorted according to the process of FIGS. 12 and / or 13.

On the other hand, if it is determined in step S1707 that the number of motion information candidates added to the motion information candidate list is not N (that is, less than N), the encoder / decoder is a zero vector (that is, zero motion vector candidates). ) Is added to the motion information candidate list (S1708).

That is, the encoder / decoder changes the reference picture index until the number of motion information candidates added to the motion information candidate list becomes the number of items (items) of the predetermined motion information candidate list, and selects zero motion vector candidates. Can be added to the motion information candidate list.

The encoder / decoder sorts the items of the motion information candidate list based on the frequency (S1409).

That is, the motion information candidates of the motion information candidate list are arranged in the order of occurrence frequency, and the motion information candidates having the same occurrence frequency are arranged in the order of the predetermined motion information candidate list construction (that is, the motion information and the combination of the spatial / temporal neighboring blocks). Ordered bidirectional prediction candidates, non-scaled bidirectional predictive candidates, and search order of zero motion vector candidates).

In this case, as described above, when the frequency of occurrence is sorted in ascending order, the motion information candidate list may be sorted only when the difference in the frequency of occurrence is greater than or equal to a predetermined level.

For example, the motion information candidate list may be sorted according to the process of FIGS. 12 and / or 13.

Meanwhile, motion information (ie, spatial motion information candidate, temporal motion information candidate, combined bidirectional motion information candidate, and non-scaled bidirectional) that are searched for constructing the motion information candidate list in FIG. The motion information candidate and the zero vector candidate) are just one example. In the example of FIG. 17, a search may be performed on only some of the motion information, or motion information not illustrated in FIG. 17 may be further searched. It may be.

In particular, the process of adding a non-scaled bidirectional prediction candidate to the list can be omitted. In this case, steps S1706 and S1707 are deleted, and if there are not N items (that is, motion information candidates) added to the motion information candidate list in step S1705, step S1708 may be performed.

In addition, in FIG. 17, the construction order of the motion information candidate list (ie, spatial motion information candidate, temporal motion information candidate, combined bidirectional motion information candidate, non-scaled bidirectional motion information candidate, zero vector (zero) vector) the order of the candidates is only one example, and may proceed in a different predetermined order.

In the present embodiment, the following method may be performed for additional encoding gain.

1) A method of constructing an existing motion information list according to the degree of partitioning of a current block (eg, a partition depth of a coding unit or a partition mode of a prediction unit) (eg, an existing merge candidate list construction method and / or AMVP motion) Vector prediction value candidate list construction method) or the present invention proposal method may be determined.

If the size of the block is large, even if the surrounding motion information shows a high frequency, the influence on the current block may be small. The smaller the block size is, the higher the probability that the neighboring motion information is similar to the motion information of the current block.

Accordingly, a method of constructing a motion information candidate list may be set differently according to the split depth (or block size or split mode) of the current block (eg, coding unit and / or prediction unit).

In this case, the motion information configuration method according to each division depth or block size or division mode may be transmitted through each header for each sequence unit and slice unit.

For example, in the sequence and slice header, split depth information of a block (for example, split depth information of a coding unit) to which the method of constructing the motion information candidate list proposed in this embodiment is applied, or a block size (for example, , Coding units and / or prediction unit sizes) may be transmitted. In this case, when the partition depth of the current block is larger than the partition depth information of the transmitted block (that is, the partition is smaller) or the size of the current block is smaller than the size of the transmitted block, the present embodiment proposes A method of constructing a motion information candidate list may be applied.

Alternatively, a division mode (eg, a division mode of a prediction unit) to which a method of constructing a motion information candidate list proposed in this embodiment is applied in a sequence and a slice header may be signaled. At this time, when the current block is divided according to the partitioning mode of the transmitted block, the method of constructing the motion information candidate list proposed in this embodiment may be applied.

Alternatively, split depth information (eg, split depth information of a coding unit) or block size (eg, coding unit and / or prediction) of a block to which the method for constructing a motion information candidate list proposed in this embodiment is applied. The size of the unit) or the splitting mode of the block (eg, the splitting mode of the prediction unit) may be predefined.

In this case, if the partition depth of the current block is larger than the partition depth information of the predetermined block (i.e., the partition is smaller) or if the size of the current block is smaller than the size of the predetermined block, the motion proposed in this embodiment is proposed. The construction method of the information candidate list may be applied. Alternatively, if the current block is partitioned according to the partition mode of the predetermined block, the method of constructing the motion information candidate list proposed in this embodiment may be applied.

2) The motion information candidate list construction method proposed in the present embodiment can be applied not only to the motion information list construction for the merge mode but also to the list construction process for the AMVP mode as described above. Even when applied to the list construction process for the AMVP mode, the motion information candidate list construction method proposed in this embodiment is applied according to the partition depth (or block size or partition mode) of the current block as described in 1) above. Whether or not applied may be determined.

3) After the construction of the motion information candidate list in consideration of the frequency number is completed, the method of encoding / decoding the motion information list index (for example, the merge index or the motion reference flag) in consideration of the distribution of the frequencies may be changed. Can be. When the frequency of a particular motion information candidate is very high, a context model (ie, information on occurrence probability of a symbol) or a code word for an index indicating the motion information candidate may be changed.

For example, if five motion information candidates are allowed, the motion information index may be binarized with truncated rice (e.g., variable cMax = 4, parameter cRiceParam = 0 for truncated rice binarization). binarization) is performed to encode up to four symbols, and the context model can be applied only to the first symbol. For example, the initial value of the context is 122 for a P slice and 137 for a B slice, and all of the Most Probable Symbols (MPSs) may be set to have a probability that a symbol is 0 (70-85%).

In this case, when the frequency of occurrence of a specific motion candidate is overwhelmingly high, the context for the index indicating the motion information candidate may be set to have a higher probability of being 0 (eg, 90% or more).

In addition, when the binarization of changing codewords has a uniform frequency of motion candidates, for example, when five motion information candidates are allowed, the motion information index is (00, 01, 10, 110, 111) for each motion candidate. Binarization may be performed in the form of. In this case, when the frequency of occurrence of a specific motion candidate is overwhelmingly high, a codeword different from the above may be assigned to the index indicating the motion information candidate (for example, shorter codeword allocation).

In this case, in order to perform parsing at the decoder without dependency of a decoding process, a separate flag for indicating this when another context or binarization is performed on an index indicating a specific motion information candidate may be transmitted from an encoder.

Example  2

The present invention proposes a method of defining long term motion information when listing the motion information of the surroundings and considering this when constructing a motion information candidate list.

In the present specification, long-term motion information refers to motion information that cannot be obtained in the vicinity of the current block (ie, a spatially adjacent block). That is, motion information of blocks that are spatially separated from the current block by two or more blocks, motion information that exhibits overall tendency of the picture to which the current block belongs, and the like may belong to this.

The long-term motion information may include at least one of the following motion information.

-Motion information recently used but not present in the current motion information candidate list (ie motion information of blocks more than M blocks away from the current block regardless of spatial direction)

Global motion information of the picture

Motion information with the highest frequency in the picture

18 is a diagram illustrating a method of constructing a motion information candidate list according to an embodiment of the present invention.

In FIG. 18, it is assumed that the number of items (items) of the predetermined motion information candidate list is N.

Referring to FIG. 18, the encoder / decoder adds spatial peripheral motion information to a motion information candidate list as a predetermined number, except for duplicate motion information (S1801).

Here, the spatial peripheral motion information may mean motion information of blocks A1, B1, B0, A0, and B2 (that is, a part of a prediction unit) in the example of FIG. 7, or may be previously described with reference to FIGS. 15 and / or 16. In the example, this may mean motion information of AL, AR, BL, A0, ..., L0, ... blocks (that is, parts of prediction units) spatially neighboring the current block.

Motion information that is not available in the motion information of a block spatially neighboring the current block may not be added to the motion information candidate list (eg, when the spatially neighboring block is encoded in the intra prediction mode).

In addition, when motion information of a neighboring block is searched according to a predetermined order, the motion information of the neighboring block currently being searched for is the same as the motion information of the previous neighboring block (ie, motion information added to the motion information candidate list). It is not added to the information candidate list. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder may add the motion information of the neighboring block in the predetermined order to the motion information candidate list so as not to overlap, except for the overlapped motion information.

The encoder / decoder adds temporal peripheral motion information to the motion information candidate list in a predetermined number, except for duplicate motion information (S1802).

In this case, the temporal peripheral motion information may mean motion information of the T0 and T1 blocks (that is, a part of the prediction unit) in the example of FIG. 7, or may be temporal to the current block in the examples of FIGS. 15 and / or 16. This may mean motion information of neighboring T0, ... blocks (that is, parts of prediction units).

Motion information that is not available in the motion information of a block temporally neighboring to the current block may not be added to the motion information candidate list (eg, when a spatially neighboring block is encoded in an intra prediction mode).

In addition, when motion information of a neighboring block is searched according to a predetermined order, the motion information of the neighboring block currently being searched for is the same as the motion information of the previous neighboring block (ie, motion information added to the motion information candidate list). It is not added to the information candidate list. In this case, if the motion vector and the reference index are the same, it may be determined that the motion information is the same.

That is, the encoder / decoder may add the motion information of the neighboring block to the motion information candidate list so as not to overlap in time available in a predetermined order.

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1803).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1803 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder ends the construction of the motion information candidate list.

On the other hand, if it is determined in step S1803 that the number of motion information candidates added to the motion information candidate list is not N (that is, less than N), the encoder / decoder excludes the long motion information candidates except for duplicate motion information. It is added to the motion information candidate list (S1804).

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1805).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1805 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder ends the construction of the motion information candidate list.

On the other hand, if it is determined in step S1805 that the number of motion information candidates added to the motion information candidate list is not N (that is, if it is less than N), the encoder / decoder indicates all the motion information candidates added to the present time in the motion information candidate list. (I.e., combining spatial motion information candidates, temporal motion information candidates, and / or long-term motion information candidates) to add a combined bidirectional prediction candidate (i.e., combined motion information candidates) to the motion information candidate list (S1806). ).

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1807).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1807 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder ends the construction of the motion information candidate list.

On the other hand, if it is determined in step S1807 that the number of motion information candidates added to the motion information candidate list is not N (that is, less than N), the encoder / decoder is a non-scaled bidirectional prediction candidate ( That is, the non-scaled bidirectional motion information candidate) is added to the motion information candidate list (S1808).

The encoder / decoder determines whether the number of motion information candidates added to the motion information candidate list is N (S1809).

That is, it may be determined whether the number of motion information candidates added to the motion information candidate list is equal to the number of items (items) of the predetermined motion information candidate list.

If it is determined in step S1809 that the number of motion information candidates added to the motion information candidate list is N, the encoder / decoder ends the construction of the motion information candidate list.

On the other hand, if it is determined in step S1809 that the number of motion information candidates added to the motion information candidate list is not N (that is, less than N), the encoder / decoder is a zero vector (that is, zero motion vector candidates). ) Is added to the motion information candidate list (S1810).

That is, the encoder / decoder changes the reference picture index until the number of motion information candidates added to the motion information candidate list becomes the number of items (items) of the predetermined motion information candidate list, and selects zero motion vector candidates. Can be added to the motion information candidate list.

Meanwhile, motion information (ie, spatial motion information candidate, temporal motion information candidate, combined bidirectional motion information candidate, and non-scaled bidirectional) that are searched for constructing the motion information candidate list in FIG. 18. The motion information candidate and the zero vector candidate) are just one example. In the example of FIG. 18, a search may be performed on only some of the motion information, or motion information not illustrated in FIG. 18 may be further searched. It may be.

In particular, the process of adding a non-scaled bidirectional prediction candidate to the list can be omitted. In this case, steps S1808 and S1809 are deleted, and if there are not N items (that is, motion information candidates) added to the motion information candidate list in step S1807, step S1810 may be performed.

In addition, in FIG. 18, the construction order of the motion information candidate list (ie, spatial motion information candidate, temporal motion information candidate, long-term motion information candidate, combined bidirectional motion information candidate, and non-scaled bidirectional motion information candidate). , Order of zero vector candidates) is just one example, and may be differently performed in a predetermined order.

In particular, the process of adding the long-term motion information to the motion information candidate list may be performed at a different step from FIG. 18. For example, long-term motion information may be added to the motion information candidate list after the spatial motion information candidate is added to the motion information candidate list, or long-term motion information after the combined bidirectional motion information candidate is added to the motion information candidate list. May be added to the motion information candidate list, or long-term motion information may be added to the motion information candidate list after a non-scaled bidirectional motion information candidate is added to the motion information candidate list, and a zero vector After the (zero vector) candidate is added to the motion information candidate list, the long-term motion information may be added to the motion information candidate list.

Hereinafter, the long term motion information will be described in more detail.

For convenience of explanation, it is assumed that the long-term motion information candidate is added to the motion information candidate list after the spatial motion information candidate and the temporal motion information candidate are added to the motion information candidate list as shown in the example of FIG. 18.

First, as an example of the long-term motion information, motion information that is recently used but does not exist in the current motion information candidate list (that is, motion information of blocks more than M blocks spatially independent of the current block) is used. Examine the case.

The encoder / decoder may be used in the process of constructing a motion information candidate list for the current block after storing recently used motion information that is not inserted in the current motion information list as long-term motion info. This will be described in more detail with reference to the drawings below.

19 illustrates a method of constructing a motion information candidate list using long-term motion information according to an embodiment of the present invention.

FIG. 19A illustrates a motion information candidate list configured in a coding / decoding process of a previous block (for example, a previous CU), and FIG. 19B illustrates a current block (example). For example, a motion information candidate list configured in a coding / decoding process of a current coding unit (Current CU) is illustrated.

In FIG. 19, it is assumed that a previous block and a current block are encoded in a merge mode, but the present invention is not limited thereto.

The motion information applied in the encoding / decoding process of the previous block is stored in a long term motion information list, and the information can be used when the motion information candidate list of the current block is constructed.

In this case, the previous block refers to a block that is spaced apart from the current block by at least M blocks regardless of a direction.

Referring to FIG. 19A, a motion information candidate index 1 selected in a motion information candidate list (ie, a merge candidate list) of a previous CU (ie, a merge candidate list) The case where the motion information of index 1) is inserted into the third item of the long term motion information list is illustrated.

In addition, as shown in FIG. 19B, even after the spatial motion information candidate and the temporal motion information candidate are inserted in the process of generating the motion information candidate list (that is, the merge candidate list) of the current CU, N is inserted. If the list items (that is, the number of items of the predetermined motion information candidate list) are not completed, the long-term motion information of the long term motion information list is selected and inserted into the motion information candidate list of the current block. .

In this case, items of the long-term motion information list may also be inserted in consideration of overlap with items (ie, motion information) added to the motion information candidate list of the current block. Therefore, the second item or the subsequent item of the long-term motion information list may be inserted into the motion information candidate list of the current block.

In FIG. 19, it is assumed that the motion information of the first item (LT motion info. 1) of the long-term motion information list is the same as the motion information candidate present in the motion information candidate list of the current block. For example, the motion information of the second item (LT motion info. 2) is inserted into the motion information candidate list of the current block.

The size of the long-term motion information list (that is, the number of items) may be fixed, and the process of managing the long-term motion information list in the encoder / decoder is as follows.

1) The most recently selected (or used) motion information may be inserted at the top of the long-term motion information list. That is, when long-term motion information is inserted into the long-term motion list, the long-term motion information may be inserted at the highest item. For example, in the process of encoding / decoding each block by the z scan method, the motion information of the current block may be inserted at the top of the long-term motion information list for the blocks to be encoded / decoded next.

2) When inserted into the long-term motion information list, if the same motion information exists in the long-term motion information list, the item (ie motion information) at the corresponding position is deleted and the motion information is inserted at the top of the long-term motion information list.

For example, when the first motion information is inserted into the long-term motion information list, if the second motion information identical to the first motion information exists in a specific item of the long-term motion information list, the second motion information is deleted and then the first motion information is deleted. One motion information may be inserted at the highest item of the long-term motion information lease.

In this case, whether the motion information is the same may be determined whether the reference picture and the motion vector are the same.

Alternatively, even when the reference picture is different, it may be determined that the motion information is the same when the motion vector has the same value by scaling. In this case, the motion picture closest to the current picture and the picture order count (POC) may be inserted into the upper item. For example, when motion information A is inserted in the long-term motion information list, it is assumed that motion information A 'having a different reference picture but the same motion vector exists due to scaling in the third item of the long-term motion information list. At this time, the POC difference (a) between the reference picture of the motion information A and the current picture and the POC difference (a ') between the reference picture of the motion information A' and the current picture are compared. If the POC difference (a ') between the reference picture of the motion information A' and the current picture is smaller, the motion information A 'rather than the motion information A may be inserted at the top of the long-term motion information list.

3) When the motion information is inserted, if the item (i.e., motion information candidate) is full in the long-term motion information list, the lowest motion information is deleted and the new motion information is inserted in the highest item of the long-term motion information list. Can be.

4) When the motion information candidate list of the current block (for example, the current coding unit or the prediction unit) is generated, when the motion information is obtained from the long-term motion information list, the motion information of the uppermost item is the motion information candidate list of the current block. By comparing the motion information of all items of the can determine whether or not. When there is no identical motion information, corresponding motion information of the long-term motion information list may be inserted into the motion information candidate list of the current block.

That is, it is determined whether each item in the long-term motion information list is identical to the motion information candidate added to the motion information candidate list of the current block sequentially, and if the same motion information is not in the motion information candidate list of the current block, It may be added to the motion information candidate list.

More specifically, the encoder / decoder determines whether motion information identical to the first item of the long-term motion information list is added to the motion information candidate list of the current block. If the same motion information as the first item of the long-term motion information list is not added to the motion information candidate list of the current block, the motion information of the first item may be added to the motion information candidate list of the current block. On the other hand, when the same motion information as the first item of the long-term motion information list is added to the motion information candidate list of the current block, whether the same motion information as the second item of the long-term motion information list is added to the motion information candidate list of the current block. You can judge. The same is true for subsequent steps.

19 illustrates a case where five long-term motion information lists are set, but the present invention is not limited thereto.

That is, the long-term motion information list is set to a predetermined size (that is, the number of items) and can be equally applied to the receiving end (decoder) and the transmitting end (encoder).

In addition, the size of the long-term motion information list may be individually reset for each sequence, each picture, or each slice, and may be transmitted from a transmitting end (encoder) to a receiving end (decoder).

Next, as an example of long-term motion information, a case in which global motion information of a picture is used will be described.

20 illustrates global motion information of a picture as long-term motion information according to an embodiment of the present invention.

Global motion information / vector of a picture means that the previous picture and the current picture tend to move as a whole as shown in FIG. 20. For example, images captured by panning (horizontal shooting) or tilting (vertical shooting) may exhibit higher efficiency than images taken with the camera fixed. have.

To represent (derive) global motion information, the current picture and the reference picture (for example, the reference picture) are superimposed at the same position, and then moved within the search range, and the Mean of Absolute Difference (MAD) between the current picture and another picture. Alternatively, a method of calculating a mean of squared error (MSE) and calculating a difference and direction between a position showing the smallest value and an initial start position may be used. Global motion information may be derived by calculating the MAD or MSE between the current picture and the reference picture. For example, the global motion information is used to minimize the MAD or MSE between the current picture and the reference picture from the top-left pixel of the current picture before moving the current picture when the current picture is moved relative to the reference picture. It can be derived from the direction and the difference between the position and the top-left pixel of the current picture at the position.

In this way, information about the distance and direction in which the reference picture and the current picture have moved can be obtained, and the distance moved in accordance with the resolution of the motion vector used in the video encoder / decoder can be expressed.

Since the global motion information is a relationship between the current picture and the reference picture, the encoder / decoder may have global motion information with the current picture for each reference picture present in the reference list. That is, the encoder / decoder may derive global motion information of the current picture based on the reference picture.

If it is difficult to calculate the global motion information by the calculation between the current picture and the reference picture described above, the global motion information is calculated based on a few reference pictures and then the virtual POC distance between the current picture and other reference pictures is taken into consideration. Global motion information can be generated. This will be described with reference to the drawings below.

21 illustrates a method of deriving global motion information of a picture as long-term motion information according to an embodiment of the present invention.

In FIG. 21, it is assumed that a reference picture of a current picture T0 is T-2.

As described above, after overlapping at the same position between the current picture (T0) and the reference picture (T-2) and moving within the search range and calculating the MAD or MSE between the current picture and the reference picture, The global motion information for the reference picture T-2 may be derived by calculating a difference and a direction from the initial start position.

In this case, the global motion information between the current picture T0 and the reference picture T-2 may be derived by considering the POC distance from the global motion information between the current picture T0 and another reference picture T1. For example, the global motion information between the current picture T0 and the reference picture T-2 may include a difference in POC between the current picture T0 and the reference picture T-2, and a reference picture T1 different from the current picture T0. Can be derived by scaling global motion information between the current picture T0 and another reference picture T1 as a ratio of the POC difference between the < RTI ID = 0.0 >

Alternatively, the global motion information between the current picture T0 and the reference picture T-2 may be derived by considering the POC distance relatively from the global motion information between the current picture T0 and another reference picture T2. For example, the global motion information between the current picture T0 and the reference picture T-2 includes a POC difference between the current picture T0 and the reference picture T-2, and a reference picture T2 different from the current picture T0. ) Can be derived by scaling global motion information between the current picture T0 and another reference picture T2 as a ratio of POC differences between the < RTI ID = 0.0 >

Alternatively, the global motion information between the current picture T0 and the reference picture T-2 may include first global motion information between the current picture T0 and another reference picture 1 (T1) and another reference picture 2 different from the current picture T0. It can be derived by considering the POC distance relatively from the second global motion information between (T2). For example, the global motion information between the current picture T0 and the reference picture T-2 may include a difference in POC between the current picture T0 and the reference picture T-2, and another reference picture 1 different from the current picture T0. First motion information that scales first global motion information between the current picture T0 and another reference picture 1 (T1) as a ratio of the POC difference between T1), and between the current picture T0 and the reference picture T-2. Averaging the second motion information by scaling the second global motion information between the current picture (T0) and another reference picture (T1) as a ratio of the POC difference and the POC difference between the current picture (T0) and another reference picture 2 (T2). (Or weighted).

Alternatively, a method of directly transmitting global motion information by a transmitting end (ie, an encoder) in a slice header (or a sequence header) may be used without predicting global motion information at a receiving end (ie, a decoder). .

In this case, when the transmitting end (ie, the encoder) calculates the global motion information by using the input current picture, the reference picture for calculating the global motion information of the current picture is a picture that was encoded just before the encoding order. A picture that is used or has a value of POC-1 of the current picture on a POC basis may be used, or the first picture of a group of pictures (GOP) of the current picture may be used.

When the encoded global motion information is used as the long-term motion information, the above-described reference picture may be used as a reference picture, or a specific picture (eg, the first picture) of the list 0 LIST0 may be used. Alternatively, in the case of bidirectional prediction, a specific picture (eg, the first picture) of List0 (LIST0) and a specific picture (eg, the first picture) of List1 (LIST1) may be applied simultaneously.

In this case, when the reference picture is different from the picture for reference, the global motion information derived based on the reference picture may be inserted in the long-term motion information list by applying scaling in proportion to the POC distance. That is, global motion information derived based on the reference picture is scaled as a ratio of the POC difference between the current picture and the reference picture and the POC difference between the current picture and the reference picture, and then inserted into the long-term motion information list.

Next, as an example of the long-term motion information, the case where the motion information showing the highest frequency in the picture is used will be described.

Since the motion information having the highest frequency in the picture reflects the characteristics of the image and may not exist in the surrounding motion information, it may contribute to the diversity of the motion information list when it is managed as long-term motion information.

Accordingly, a specific number of motion information storage spaces may be created to find the highest frequency of motion information in the picture, and the motion information selected to represent the motion of the current block may be stored in the storage space. That is, the long-term motion information list may be constructed to find the highest frequency of motion information in the picture. At this time, when the motion information selected to represent the motion of the current block is stored in the storage space, if the same motion information exists in the storage space, the process of updating the occurrence frequency of the motion information is performed.

22 illustrates a method of constructing a motion information candidate list using long-term motion information according to an embodiment of the present invention.

FIG. 22A illustrates a motion information candidate list configured in a coding / decoding process of a previous block (eg, a previous CU), and FIG. 22B illustrates a current block (eg, a CU). For example, a motion information candidate list configured in a coding / decoding process of a current coding unit (Current CU) is illustrated.

In FIG. 22, it is assumed that a previous block and a current block are encoded in a merge mode, but the present invention is not limited thereto.

The motion information applied in the encoding / decoding process of the previous block is stored in a long term motion information list, and the information can be used when the motion information candidate list of the current block is constructed.

In this case, the previous block refers to a block that is spaced apart from the current block by at least M blocks regardless of a direction.

Referring to FIG. 22A, a motion information candidate index 0 (ie, a merge candidate list index) selected from a motion information candidate list (ie, merge candidate list) of a previous CU (previous CU) For example, the motion information of index 0) is inserted as a second item of a long term motion information list.

When the motion information selected in the previous block is the merge candidate list index 0 as shown in FIG. 22, since the corresponding motion information is inserted into the long-term motion information list and is the same as the second item of the long-term motion information list, the frequency of the second item is 6 Can be updated to 7.

The motion information candidate list of the current block is generated in consideration of the updated long-term motion information list. That is, the long-term motion information list is composed of long-term motion information and occurrence frequency, and motion information having the highest occurrence frequency in the long-term motion information list may be inserted as a motion information candidate of the motion information candidate list of the current block.

As the first item of the long-term motion information list as shown in FIG. 22 has the highest occurrence frequency, it may be inserted as the fifth item of the motion information candidate list of the current block.

In this case, items of the long-term motion information list may also be inserted in consideration of overlap with items (ie, motion information) added to the motion information candidate list of the current block. Therefore, even if the first item of the long-term motion information list is the most frequent, if it is the same as the motion information added to the motion information candidate list of the current block, the second item or a subsequent item is added to the motion information candidate list of the current block. Can be inserted.

When updating the frequency of the long-term motion information (that is, adding the motion information of the current block selected from the motion information candidate list of the current block to the long-term motion information list), it is determined whether or not the item exists in the existing long-term motion information list. That is, when determining whether or not the same as the motion information added to the existing long-term motion information list) the following methods can be used.

1) It is determined that the items of the existing long-term motion information list and the motion information of the current block are the same as the existing items only when the motion vector reference picture list and the reference picture index are the same, and the corresponding item of the long-term motion information list. You can update the frequency of.

2) The items of the existing long-term motion information list and the motion information of the current block are determined to be the same when the reference picture list and the reference picture index are the same and the difference between the motion vectors is equal to or less than a certain level. You can update the frequency of.

3) Although the items of the existing long-term motion information list and the motion information of the current block are different from the reference picture list or the reference picture index, the result of scaling the motion vector in consideration of the POC difference with the current picture is the same or different. If it is below a certain level, it is determined that the same motion information, and the frequency of the corresponding item of the long-term motion information list can be updated.

When the third method is applied, various pieces of information may be generated according to the reference picture. Therefore, the reference picture that is closest to the current picture may be selected and the motion vector according to the picture may be calculated and managed as a long-term motion information list. have.

For example, when motion information A is inserted into the long-term motion information list, a motion information A 'having a different reference picture list or reference picture but having the same motion vector due to scaling exists in the third item of the long-term motion information list. Assume At this time, the POC difference (a) between the reference picture of the motion information A and the current picture and the POC difference (a ') between the reference picture of the motion information A' and the current picture are compared. If the POC difference a 'between the reference picture of the motion information A' and the current picture is smaller, the motion information A 'instead of the motion information A may be replaced with a corresponding third item of the long-term motion information list.

Meanwhile, Embodiment 1 and Embodiment 2 described above may be used in combination. For example, after the long-term motion information candidate according to the second embodiment is added to the motion information candidate list of the current block, the order of the motion information candidates in the motion information candidate list of the current block can be sorted according to the first embodiment. have.

23 is a diagram illustrating an inter prediction based image processing method according to an embodiment of the present invention.

Referring to FIG. 23, the encoder / decoder constructs a motion information candidate list for deriving motion information of the current block (S2301).

In this case, the encoder / decoder may configure the motion information candidate list according to the first and / or second embodiments described above.

In other words, the encoder / decoder may assign, in a predetermined order, one or more spatial motion information candidates derived from the current block and one or more neighboring blocks spatially and / or one or more temporal motion information candidates derived from one or more temporally neighboring blocks. By adding accordingly, a motion information candidate list can be constructed.

In this case, as described with reference to FIG. 7, 15, or 16, the encoder / decoder uses motion information candidates derived from blocks spatially neighboring the current block and / or blocks temporally neighboring the current block. Lists can be constructed.

In addition, as in the first embodiment described above, the encoder / decoder may sort the motion information candidates according to the frequency of occurrence of the motion information candidates added to the motion information candidate list (that is, the frequency of occurrence in descending order). . For example, the motion information candidate list may be sorted as shown in the example of FIG. 12.

When sorting in this order of occurrence frequency, the encoder / decoder may sort the motion information candidate list only when the difference in frequency of occurrence is more than a predetermined level. For example, the motion information candidate list may be sorted as in the example of FIG. 13 described above.

As in Embodiment 1 described above, after the spatial motion information candidate and the temporal motion information candidate are added to the motion information candidate list, the number of motion information candidates added to the current motion information candidate list is determined by a predetermined item of the motion information candidate list. If smaller than the number, the combined bidirectional motion information candidate may be added to the motion information candidate list by combining the motion information added to the motion information candidate list. In particular, a combined bidirectional motion information candidate may be added to the motion information candidate list when the slice (or picture) to which the current block belongs is a bidirectional slice (or picture).

As in Embodiment 1 described above, after the combined bidirectional motion information candidate is added to (or may not be added to) the motion information candidate list, the number of motion information candidates added to the current motion information candidate list is added to the motion information candidate list. If less than a predetermined number of items, a non-scaled bidirectional motion information candidate may be added to the motion information candidate list. In particular, when the slice (or picture) to which the current block belongs is a bidirectional slice (or picture), a non-scaled bidirectional motion information candidate may be added to the motion information candidate list.

As in Embodiment 1 described above, after the non-scaled bidirectional motion information candidate is added to (or may not be added to) the motion information candidate list, the motion information candidate added to the current motion information candidate list. If the number of times is smaller than the number of predetermined items in the motion information candidate list, a zero motion vector candidate may be added to the motion information candidate list. In this case, the zero motion vector candidate may be added to the motion information candidate list until the number of motion information candidates added to the current motion information candidate list is equal to the number of predetermined items of the motion information candidate list. .

In addition, as in the second embodiment described above, the encoder / decoder may generate and manage a long-term motion information list including long-term motion information. Here, as described above, the long-term motion information candidate is motion information that is recently used but does not exist in the current motion information candidate list (that is, motion information of M blocks or more away from the current block regardless of spatial direction), At least one of the global motion information of the picture and the motion information showing the highest frequency in the picture.

The encoder / decoder may add the long-term motion information to the motion information candidate list of the current block as the long-term motion information candidate.

In this case, the long-term motion information candidate may be added to the motion information candidate list before the spatial motion information candidate is added (that is, the first priority), or may be added to the motion information candidate list after the spatial motion information candidate is added. , Or may be added to the motion information candidate list after the temporal motion information candidate is added, the combined bidirectional motion information candidate may be added to the motion information candidate list, and then added to the motion information candidate list, and may be added to a non-scaled ( non-scaled) The bidirectional motion information candidate may be added to the motion information candidate list after being added to the motion information candidate list, and the zero motion vector candidate may be added to the motion information candidate list after being added.

The encoder / decoder derives the motion information of the current block from the motion information candidate selected from the motion information candidates added to the motion information candidate list (S2302).

In this case, the decoder may receive a motion information candidate index for specifying motion information of the current block from the motion information candidate list of the current block from the encoder. In this case, the motion information of the current block may be derived from the motion information candidate indicated by the received motion information candidate index.

For example, when the merge mode is applied to the current block, the decoder may receive the merge index from the encoder and decode the received merge index. The motion information indicated by the merge index in the merge candidate index may be derived as the motion information of the current block.

In addition, when the AMVP mode is applied to the current block, the decoder may receive a motion reference flag from the encoder and decode the received motion reference flag. The motion information indicated by the motion reference flag in the motion vector predictor candidate list can be derived as the motion vector predictor of the current block. The decoder may decode the motion vector difference value, the reference picture index, and the inter prediction mode received from the encoder, and derive the motion vector of the current block by adding the derived motion vector prediction value and the decoded motion vector difference value.

The encoder / decoder generates the prediction block of the current block by using the derived motion information of the current block (S2303).

The encoder / decoder may generate the predictive block of the current block from the block specified by the motion information (especially the motion vector) in the reference picture specified by the motion information (especially the reference picture index).

In other words, the encoder / decoder may generate (derive) the sample value of the prediction unit of the current block from the sample value of the block (region) specified by the motion vector in the reference picture.

24 is a diagram illustrating an inter predictor according to an embodiment of the present invention.

In FIG. 24, the inter prediction unit 181 (see FIG. 1; see 261 and FIG. 2) is shown as one block for convenience of description, but the inter prediction units 181 and 261 are included in the encoder and / or the decoder. It can be implemented as.

Referring to FIG. 24, the inter prediction units 181 and 261 implement the functions, processes, and / or methods proposed in FIGS. 1 to 23. In detail, the inter predictors 181 and 261 may include a motion information candidate list constructer 2401, a subblock divider 2402, and a predictive block generator 2403. In addition, the long-term motion information list construction unit 2404 may be further included.

The motion information candidate list constructing unit 2401 constructs a motion information candidate list for deriving motion information of the current block.

The motion information candidate list constructing unit 2401 may construct a motion information candidate list according to the first and / or second embodiments described above.

In other words, the motion information candidate list constructer 2401 may include one or more spatial motion information candidates derived from the current block and one or more neighboring blocks spatially and / or one or more temporal motion information derived from one or more temporally neighboring blocks. The motion information candidate list can be constructed by adding candidates in a predetermined order.

In this case, as described with reference to FIG. 7, FIG. 15, or FIG. 16, the motion information candidate list constructer 2401 is a motion information candidate derived from a block spatially neighboring the current block and / or a block temporally neighboring the current block. The motion information candidate list can be constructed using.

In addition, as in the first embodiment described above, the motion information candidate list constructing unit 2401 sorts the order of the motion information candidates according to the frequency of occurrence of the motion information candidate added to the motion information candidate list (that is, in descending order of occurrence frequency). Sorting). For example, the motion information candidate list may be sorted as shown in the example of FIG. 12.

As such, when the frequency of occurrence is sorted in ascending order, the motion information candidate list construction unit 2401 may sort the motion information candidate list only when the difference in the frequency of occurrence between the motion information candidates is equal to or greater than a predetermined level. That is, only the order of motion information candidates whose difference in frequency of occurrence in the motion information candidate list is greater than or equal to a predetermined level may be changed. For example, the motion information candidate list may be sorted as in the example of FIG. 13 described above.

As in Embodiment 1 described above, after the spatial motion information candidate and the temporal motion information candidate are added to the motion information candidate list, the number of motion information candidates added to the current motion information candidate list is determined by a predetermined item of the motion information candidate list. If less than the number, the combined bidirectional motion information candidate generated by combining the motion information candidates added to the motion information candidate list may be added to the motion information candidate list. In particular, a combined bidirectional motion information candidate may be added to the motion information candidate list when the slice (or picture) to which the current block belongs is a bidirectional slice (or picture).

As in Embodiment 1 described above, after the combined bidirectional motion information candidate is added to (or may not be added to) the motion information candidate list, the number of motion information candidates added to the current motion information candidate list is added to the motion information candidate list. If less than a predetermined number of items, a non-scaled bidirectional motion information candidate may be added to the motion information candidate list. In particular, when the slice (or picture) to which the current block belongs is a bidirectional slice (or picture), a non-scaled bidirectional motion information candidate may be added to the motion information candidate list.

As in Embodiment 1 described above, after the non-scaled bidirectional motion information candidate is added to (or may not be added to) the motion information candidate list, the motion information candidate added to the current motion information candidate list. If the number of times is smaller than the number of predetermined items in the motion information candidate list, a zero motion vector candidate may be added to the motion information candidate list. In this case, the zero motion vector candidate may be added to the motion information candidate list until the number of motion information candidates added to the current motion information candidate list is equal to the number of predetermined items of the motion information candidate list. .

The long-term motion information list constructer 2404 may generate and manage a long-term motion information list composed of long-term motion information candidates. Here, as described above, the long-term motion information candidate is motion information that is recently used but does not exist in the current motion information candidate list (that is, motion information of M blocks or more away from the current block regardless of spatial direction), At least one of the global motion information of the picture and the motion information showing the highest frequency in the picture.

In this case, as in the above-described second embodiment, the motion information candidate list configuring unit 2401 may configure the motion information candidate list of the current block by using the long-term motion information list generated by the long-term motion information list forming unit 2404. have. That is, the motion information candidate list generator 2401 may add the long-term motion information in the long-term motion information list generated by the long-term motion information list generator 2404 to the motion information candidate list of the current block as the long-term motion information candidate. .

In this case, the long-term motion information candidate may be added to the motion information candidate list before the spatial motion information candidate is added (that is, the first priority), or may be added to the motion information candidate list after the spatial motion information candidate is added. , Or may be added to the motion information candidate list after the temporal motion information candidate is added, the combined bidirectional motion information candidate may be added to the motion information candidate list, and then added to the motion information candidate list, and may be added to a non-scaled ( non-scaled) The bidirectional motion information candidate may be added to the motion information candidate list after being added to the motion information candidate list, and the zero motion vector candidate may be added to the motion information candidate list after being added.

The subblock dividing unit 2402 derives motion information of the current block from the motion information candidate selected from the motion information candidates added to the motion information candidate list.

In this case, the sub-block dividing unit 2402 may derive the motion information of the current block from the motion information candidate indicated by the motion information candidate index (for example, the merge index or the motion reference flag) received from the encoder.

The prediction block generator 2403 generates a prediction block of the current block by using the derived motion information of the current block.

The prediction block generator 2403 may generate the prediction block of the current block from the block specified by the motion information (particularly, the motion vector) in the reference picture specified by the motion information (particularly, the reference picture index).

In other words, the prediction block generator 2403 may generate (derive) the sample value of the prediction unit of the current block from the sample value of the block (region) specified by the motion vector in the reference picture.

The embodiments described above are the components and features of the present invention are combined in a predetermined form. Each component or feature is to be considered optional unless stated otherwise. Each component or feature may be embodied in a form that is not combined with other components or features. It is also possible to combine some of the components and / or features to form an embodiment of the invention. The order of the operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is obvious that the claims may be combined to form an embodiment by combining claims that do not have an explicit citation relationship in the claims or as new claims by post-application correction.

Embodiments according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of a hardware implementation, an embodiment of the present invention may include one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code may be stored in memory and driven by the processor. The memory may be located inside or outside the processor, and may exchange data with the processor by various known means.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential features of the present invention. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

As mentioned above, preferred embodiments of the present invention are disclosed for purposes of illustration, and those skilled in the art can improve and change various other embodiments within the spirit and technical scope of the present invention disclosed in the appended claims below. , Replacement or addition would be possible.

Claims (15)

In the method for processing an image based on inter prediction, The motion by adding spatial motion information candidates derived from a block spatially neighboring the current block and temporal motion information candidates derived from a block temporally neighboring the current block to the motion information candidate list of the current block in a predetermined order; Constructing an information candidate list; Deriving motion information of the current block from the motion information candidate selected from the motion information candidates added to the motion information candidate list; And Generating a prediction block of the current block by using motion information of the current block, And an order of the motion information candidates according to the frequency of occurrence of the motion information candidates added to the motion information candidate list. The method of claim 1, And only a sequence of motion information candidates whose difference in frequency is greater than or equal to a predetermined level is changed. The method of claim 1, A block neighboring to the left of the current block spatially, a block neighboring to the upper left of the current block, a block neighboring to the upper right of the current block spatially, a block spatially neighboring to the lower left of the current block, the An inter prediction mode based image processing method in which motion information candidates are derived in an order of a neighboring block in time with a current block. The method of claim 1, After motion information candidates are alternately derived from a block adjacent to the left side of the current block and a block adjacent to the top of the current block, the block next to the left side of the current block and the current block An inter prediction mode based image processing method in which motion information candidates are derived in the order of a spatially neighboring right uppermost block, the current block and a spatially lower left neighboring block, and the current block and a temporal neighboring block. The method of claim 1, And the motion information candidate list is arranged when the depth at which the current block is divided is equal to or larger than a predetermined value or when the size of the current block is smaller than a predetermined size. The method of claim 1, If the number of motion information candidates added to the motion information candidate list is smaller than the number of predetermined items of the motion information candidate list, the motion information candidates generated by combining the motion information candidates added to the motion information candidate list are determined. An inter prediction mode based image processing method added to a motion information candidate list. The method of claim 1, When the number of motion information candidates added to the motion information candidate list is smaller than the number of predetermined items of the motion information candidate list, a zero motion vector is added to the motion information candidate list. Image processing method. The method of claim 1, The long-term motion information candidate selected from the long-term motion information list including motion information of a block spaced apart from the current block by a predetermined number of blocks or more and global motion information of a picture to which the current block belongs is included in the motion information candidate list. Inter prediction mode based image processing method added to the. The method of claim 8, When the number of motion information candidates added to the motion information candidate list is smaller than a predetermined number of items of the motion information candidate list, the long-term motion information candidate is added to the motion information candidate list. . The method of claim 9, And when the long-term motion information candidate is added to the motion information candidate list, the long-term motion information candidate is added not to overlap with the motion information candidate added to the motion information candidate list. The method of claim 8, When the motion information is inserted into the long-term motion list, the inter prediction mode based image processing method is inserted into the highest item of the long-term motion list. The method of claim 11, When the first motion information is inserted into the long-term motion list, if second motion information identical to the first motion information exists in the long-term motion list, the second motion information is deleted and the first motion information is determined. An inter prediction mode based image processing method inserted into an uppermost item of a long-term motion list. The method of claim 8, The global motion information is a mean of absolute difference (MAD) or MSE between the current picture and the reference picture from a top-left pixel of the current picture before moving when the current picture is moved relative to the reference picture. An inter prediction mode based image processing method derived from a position and a difference between a position and a top-left pixel of the current picture at a position of minimizing a mean of squared error. The method of claim 8, And the motion information having the highest frequency in the picture to which the current block belongs in the long-term motion information list is added to the motion information candidate list. An apparatus for processing an image based on inter prediction, The motion by adding spatial motion information candidates derived from a block spatially neighboring the current block and temporal motion information candidates derived from a block temporally neighboring the current block to the motion information candidate list of the current block in a predetermined order; A motion information candidate list constructing unit constituting the information candidate list; A motion information derivation unit for deriving motion information of the current block from a motion information candidate selected from the motion information candidates added to the motion information candidate list; And A prediction block generation unit generating a prediction block of the current block by using motion information of the current block, And ordering the motion information candidates according to a frequency of occurrence of the motion information candidates added to the motion information candidate list.

Images