CN110024402B - Image encoding/decoding method and apparatus, and recording medium storing bit stream - Google Patents

Abstract

The present invention relates to an image encoding/decoding method. To this end, the image decoding method may include the steps of: generating a first prediction block of the current block by using motion information of the current block; generating a second prediction block of the at least one current subordinate block by using motion information of at least one neighboring subordinate block of the current subordinate block; and generating a final prediction block based on a weighted sum of the first prediction block of the current block and the second prediction block of the at least one current lower block.

Description

Image encoding/decoding method and device, and recording medium storing bitstream

technical field

The present invention relates to a method and apparatus for encoding/decoding images and a recording medium storing a bit stream. More particularly, the present invention relates to a method and apparatus for encoding/decoding images using overlapping block motion compensation.

Background technique

Recently, demands for high-resolution quality images such as high-definition (HD) images or ultra-high-definition (UHD) images have increased in various application fields. However, the data volume of image data of higher resolution and quality has increased compared with conventional image data. Therefore, when image data is transmitted by using a medium such as a conventional wired broadband network or a wireless broadband network, or when image data is stored in a conventional storage medium, transmission costs and storage costs increase. In order to solve these problems that arise as the resolution and quality of image data increase, efficient image encoding/decoding techniques are required.

Image compression techniques include various techniques including: an inter-frame prediction technique for predicting pixel values included in a current picture from previous or subsequent pictures of the current picture; predicting pixel values included in the current picture by using pixel information in the current picture Intra-prediction techniques for pixel values; entropy coding techniques that assign short codes to high-occurrence values and long codes to low-occurrence values; and the like. By using such an image compression technique, image data can be efficiently compressed, and the compressed image data is transmitted or stored.

Conventional image encoding/decoding methods and apparatuses use motion information of only a limited number of neighboring blocks for overlapping block motion compensation. Therefore, there is a limit to the improvement of coding efficiency.

Contents of the invention

technical problem

An object of the present invention is to provide a method and apparatus for performing overlapping block motion compensation using motion information of an increased number of adjacent blocks to improve image encoding/decoding efficiency.

solution

The present invention provides an image decoding method. The image decoding method includes: using the motion information of the current block to generate a first prediction block of the current block; using the motion information of at least one adjacent sub-block of the current sub-block of the current block to generate at least one second prediction block of the current sub-block of the current block; and generating a final prediction block of the current block based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current sub-block.

In the image decoding method, the at least one adjacent sub-block may include an adjacent sub-block of a sub-block of a co-located block temporally corresponding to the current block.

In the image decoding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the The at least one second prediction block is generated by using motion information of at least one adjacent sub-block of the sub-block of the co-located block.

In the image decoding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the The motion information included in at least one of the list and the motion vector list of the current block is combined to generate the at least one second prediction block.

In the image decoding method, the step of generating at least one second predicted block may include: only when the current sub-block is included in at least one of the left boundary area and the upper boundary area of the current block, using at least one adjacent sub-block motion information to generate the at least one second prediction block.

In the image decoding method, the step of generating at least one second predicted block may include: when the current subblock is included in the left boundary area of the current block, using a left adjacent subblock among adjacent subblocks of the current subblock , the motion information of at least one of the upper-left adjacent sub-block and the lower-left adjacent sub-block to generate the at least one second prediction block; and when the current sub-block is included in the upper boundary area of the current block, using the current sub-block The at least one second prediction block is generated by using motion information of at least one of an upper adjacent sub-block, an upper left adjacent sub-block, and an upper right adjacent sub-block among adjacent sub-blocks of the block.

In the image decoding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the current of the subblock's upper, left, lower, right, upper left, lower left, lower right, and upper right adjacent subblocks of the subblock At least one piece of motion information is used to generate the at least one second prediction block.

In the image decoding method, the step of generating at least one second prediction block may include: deriving motion information from at least one adjacent sub-block of the current sub-block in a predetermined order of the adjacent sub-blocks; and using at least one piece of derived motion information to generate the at least one second prediction block.

In the image decoding method, the step of generating the final prediction block may include: applying different weighting factors to samples in the first prediction block and the second prediction block according to the positions of adjacent sub-blocks used to generate the second prediction block The weighted sum of the first prediction block and the second prediction block is obtained in the case of a point.

In the image decoding method, in the step of generating the final predicted block, when the number of the second predicted block of the current sub-block is 2 or more, the first predicted block of the current block and the number of the current sub-block can be calculated as a whole The weighted sum of the second prediction block to produce the final prediction block.

The present invention provides an image coding method. The image coding method includes: using motion information of the current block to generate a first prediction block of the current block; using motion information of at least one adjacent sub-block of the current sub-block of the current block to generate the current block at least one second prediction block of the current sub-block; and generating a final prediction block of the current block based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current sub-block.

In the image encoding method, the at least one adjacent sub-block may include an adjacent sub-block of a sub-block of a co-located block temporally corresponding to the current block.

In the image coding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the The at least one second prediction block is generated by using motion information of at least one adjacent sub-block of the sub-block of the co-located block.

In the image coding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the The motion information included in at least one of the list and the motion vector list of the current block is combined to generate the at least one second prediction block.

In the image coding method, the step of generating at least one second prediction block may include: using at least one adjacent subblock only when the current subblock is included in at least one of the left boundary area and the upper boundary area of the current block. motion information to generate the at least one second prediction block.

In the image coding method, the step of generating at least one second predicted block may include: when the current subblock is included in the left boundary area of the current block, using a left adjacent subblock among adjacent subblocks of the current subblock , the motion information of at least one of the upper-left adjacent sub-block and the lower-left adjacent sub-block to generate the at least one second prediction block; and when the current sub-block is included in the upper boundary area of the current block, using the current sub-block The at least one second prediction block is generated by using motion information of at least one of an upper adjacent sub-block, an upper left adjacent sub-block, and an upper right adjacent sub-block among adjacent sub-blocks of the block.

In the image coding method, the step of generating at least one second prediction block may include: when the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the current of the subblock's upper, left, lower, right, upper left, lower left, lower right, and upper right adjacent subblocks of the subblock At least one piece of motion information is used to generate the at least one second prediction block.

In the image coding method, the step of generating at least one second prediction block may include: deriving motion information from at least one adjacent sub-block of the current sub-block in a predetermined order of the adjacent sub-blocks; and using at least one piece of derived motion information to generate the at least one second prediction block.

In the image coding method, the step of generating the final prediction block may include: applying different weighting factors to samples in the first prediction block and the second prediction block according to the positions of adjacent sub-blocks used to generate the second prediction block The weighted sum of the first prediction block and the second prediction block is obtained in the case of a point.

In the image coding method, in the step of generating the final prediction block, when the number of the second prediction blocks of the current sub-block is 2 or more, the first prediction block of the current block and the number of the current sub-block can be calculated as a whole The weighted sum of the second prediction block to produce the final prediction block.

The present invention provides a recording medium for storing a bit stream generated by an image coding method. The image coding method includes: using motion information of the current block to generate a first prediction block of the current block; using at least generating at least one second prediction block of the current subblock of the current block based on motion information of a neighboring subblock; and generating based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current subblock The final predicted block of the current block.

Beneficial effect

According to the present invention, it is possible to provide a method and apparatus for encoding/decoding images with improved compression efficiency.

According to the present invention, image encoding/decoding efficiency can be improved.

According to the present invention, the computational complexity of an image encoder and an image decoder can be reduced.

Description of drawings

FIG. 1 is a block diagram showing the configuration of an encoding device according to an embodiment to which the present invention is applied;

FIG. 2 is a block diagram showing a configuration of a decoding device according to an embodiment to which the present invention is applied;

3 is a diagram schematically showing a partition structure of an image used when encoding or decoding an image;

4 is a diagram illustrating an embodiment of an inter prediction process;

FIG. 5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating an image decoding method according to an embodiment of the present invention;

7 is a flowchart illustrating an image encoding method according to another embodiment of the present invention;

8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention;

9 is a diagram illustrating an example of deriving a spatial motion vector candidate of a current block;

10 is a diagram illustrating an example of deriving a temporal motion vector candidate of a current block;

FIG. 11 is a diagram illustrating an example of adding spatial merging candidates to a merging candidate list;

FIG. 12 is a diagram illustrating an example of adding temporal merge candidates to a merge candidate list;

13 is a diagram illustrating an example of performing overlapping block motion compensation on a sub-block basis;

14 is a diagram illustrating an example of performing overlapping block motion compensation using motion information of sub-blocks of co-located blocks;

15 is a diagram illustrating an example of performing overlapping block motion compensation using motion information of blocks adjacent to a boundary of a reference block;

FIG. 16 is a diagram illustrating an example of performing overlapping block motion compensation on a subblock group basis;

17 is a diagram illustrating an example of the number of pieces of motion information used for overlapping block motion compensation;

18 and 19 are diagrams illustrating an order of deriving motion information for generating a second prediction block;

FIG. 20 is a diagram illustrating determining whether motion information of an adjacent subblock is information that can be used to generate a second prediction block by comparing the POC of the reference picture of the current subblock with the POC of the reference pictures of the adjacent subblocks of the current subblock. An illustration of an example;

21 is a diagram illustrating an embodiment of applying a weight factor when calculating a weighted sum of a first prediction block and a second prediction block;

22 is a diagram illustrating an embodiment of applying different weight factors to samples according to their positions in the block when calculating a weighted sum of a first prediction block and a second prediction block;

23 is a diagram illustrating an embodiment in which weighted sums of a first prediction block and a second prediction block are sequentially and cumulatively calculated in a predetermined order during motion compensation of overlapping blocks;

24 is a diagram illustrating an embodiment of calculating a weighted sum of a first prediction block and a second prediction block during overlapping block motion compensation;

FIG. 25 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

Detailed ways

Various modifications can be made to the invention, and there are various embodiments of the invention, examples of which will now be provided with reference to the accompanying drawings and will be described in detail. However, the present invention is not limited thereto although the exemplary embodiments can be construed to include all modifications, equivalents or replacements within the technical idea and technical scope of the present invention. Like reference numerals refer to identical or similar functions in various respects. In the drawings, the shapes and sizes of elements may be exaggerated for clarity. In the following detailed description of the invention, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. It should be understood that the various embodiments of the present disclosure, although different, are not necessarily mutually exclusive. For example, certain features, structures and characteristics described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of the present disclosure. In addition, it is to be understood that the location or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the disclosure. Accordingly, the following detailed description is not to be taken in a limiting sense, with the scope of the present disclosure being defined only by the appended claims (together, where properly interpreted, along with the full scope of equivalents to which the claims claim).

The terms 'first', 'second', etc. used in the specification may be used to describe various components, but the components are not construed as being limited to the terms. The terms are only used to distinguish one component from other components. For example, a 'first' component may be called a 'second' component, and a 'second' component may be similarly called a 'first' component, without departing from the scope of the present invention. The term "and/or" includes a combination of multiple items or any one of multiple items.

It will be understood that, in this specification, when an element is simply referred to as being "connected to" or "coupled to" another element rather than "directly connected to" or "directly coupled to" another element, it may be "directly connected to" or "directly coupled to" another element. connected to" or "directly coupled to" another element, or is connected or coupled to another element with other elements interposed therebetween. In contrast, it will be understood that when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intervening elements present.

Furthermore, constituent elements shown in the embodiments of the present invention are shown independently so as to exhibit characteristic functions different from each other. Therefore, this does not mean that each constituent element is constituted as a constituent unit of separate hardware or software. In other words, each constituent part includes each of the enumerated constituent parts for convenience. Accordingly, at least two of each constituent part may be combined to form one constituent part, or one constituent part may be divided into a plurality of constituent parts to perform each function. An embodiment in which each constituent part is combined and an embodiment in which one constituent part is divided are also included in the scope of the present invention without departing from the essence of the present invention.

The terms used in this specification are for describing specific embodiments only, and are not intended to limit the present invention. An expression used in the singular includes a plural expression unless it has a clearly different meaning in the context. In this specification, it will be understood that terms such as "comprising of", "having" and the like are intended to indicate the features, numbers, steps, actions, elements, parts, or combinations thereof disclosed in the specification It is not intended to exclude the possibility that one or more other features, quantities, steps, actions, elements, components, or combinations thereof may exist or may be added. In other words, when specific elements are said to be "included", elements other than the corresponding elements are not excluded, but additional elements may be included in the embodiments of the present invention or within the scope of the present invention .

In addition, some constituent elements may not be indispensable constituent elements to perform essential functions of the present invention, but optional constituent elements that merely enhance performance thereof. The present invention can be implemented by including only essential constituent elements for carrying out the essence of the present invention and excluding constituent elements used in enhancing performance. A structure including only the indispensable constituent parts and excluding optional constituent parts used only to improve performance is also included in the scope of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing exemplary embodiments of the present invention, well-known functions or constructions will not be described in detail since they would unnecessarily obscure the understanding of the invention. The same constituent elements in the drawings are denoted by the same reference numerals, and repeated description of the same elements will be omitted.

Also, hereinafter, an image may mean a picture constituting a video, or may mean the video itself. For example, "encode or decode an image or both" could mean "encode or decode a video or both" and could mean "encode or decode one of the images of a video Or do both". Here, a picture and an image may have the same meaning.

term description

Encoder: Means a device that performs encoding.

Decoder: Means a device that performs decoding.

Block: is the sample point of M×N matrix. Here, M and N mean positive integers, and a block may mean a sample matrix in a two-dimensional form. A block may refer to a unit. The current block may mean an encoding target block targeted when encoding, or a decoding target block targeted when decoding. Also, the current block may be at least one of a coding block, a prediction block, a residual block, and a transformation block.

Sample point: It is the basic unit that constitutes a block. A sample may be represented as a value from 0 to 2 ^Bd −1 according to the bit depth ( _Bd ). In the present invention, a sample point can be used as a meaning of a pixel.

Unit: Refers to the encoding and decoding unit. When encoding and decoding an image, a unit may be a region generated by partitioning a single image. Also, a unit may mean a sub-split unit when a single image is partitioned into a plurality of sub-split units during encoding or decoding. When encoding and decoding an image, predetermined processing for each unit may be performed. A single unit can be partitioned into subunits that are smaller in size than the unit. Depending on function, a unit may mean a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transformation unit, a transformation block, and the like. Also, in order to distinguish a unit from a block, a unit may include a luma component block, a chroma component block associated with the luma component block, and a syntax element for each color component block. The cells may have various sizes and shapes, in particular, the shape of the cells may be a two-dimensional geometric figure such as a rectangular shape, a square shape, a trapezoidal shape, a triangular shape, a pentagonal shape, and the like. Also, the unit information may include at least one of a unit type (indicating a coding unit, a prediction unit, a transformation unit, etc.), a unit size, a unit depth, an order of encoding and decoding a unit, and the like.

Coding Tree Unit: A single coding treeblock configured with the luma component Y and two coding treeblocks associated with the chroma components Cb and Cr. Additionally, a coding tree unit may be meant to include blocks and syntax elements for each block. Lower units, such as coding units, prediction units, transformation units, etc., may be constructed by partitioning each coding tree unit using at least one of a quadtree partitioning method and a binary tree partitioning method. A coding tree unit may be used as a term for designating a pixel block that becomes a processing unit when encoding/decoding an image that is an input image.

Coding tree block: Can be used as a term for designating any one of Y coding tree block, Cb coding tree block, and Cr coding tree block.

Adjacent block: means the block adjacent to the current block. A block adjacent to the current block may mean a block in contact with a boundary of the current block, or a block located within a predetermined distance from the current block. A neighboring block may mean a block adjacent to a vertex of a current block. Here, a block adjacent to the vertex of the current block may mean a block vertically adjacent to a neighboring block horizontally adjacent to the current block, or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

Reconstructed adjacent blocks: means adjacent blocks that are adjacent to the current block and have been temporally/spatially encoded or decoded. Here, reconstructing neighboring blocks may mean reconstructing neighboring cells. The reconstructed spatial neighboring blocks may be blocks within the current picture that have been reconstructed by encoding or decoding, or both. A reconstructed temporal neighboring block is a block within the reference picture that is co-located with the current block of the current picture, or a neighboring block of the block.

Cell Depth: Means the degree to which a cell is partitioned. In a tree structure, the root node can be the highest node and the leaf nodes can be the lowest nodes. Also, when a unit is represented as a tree structure, the level at which a unit exists may mean a unit depth.

Bitstream: means a bitstream including coded image information.

Parameter Set: Corresponds to the header information in the structure within the bitstream. At least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set may be included in the parameter set. Also, the parameter set may include slice header and tile header information.

Parsing: may mean determining the value of a syntax element by performing entropy decoding, or may mean entropy decoding itself.

Symbol: may mean at least one of a syntax element, an encoding parameter, and a transform coefficient value of an encoding/decoding target unit. Additionally, a symbol may mean an entropy encoded target or an entropy decoded result.

Prediction unit: means a basic unit when performing prediction such as inter prediction, intra prediction, inter compensation, intra compensation, and motion compensation. A single prediction unit may be partitioned into a plurality of partitions having a small size, or may be partitioned into lower-level prediction units.

PU partition: means a shape obtained by partitioning a prediction unit.

Reference picture list: means a list including one or more reference pictures for inter-picture prediction or motion compensation. LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), L3 (List 3), etc. are types of reference picture lists. One or more reference picture lists may be used for inter prediction.

Inter-prediction indicator: may mean the inter-prediction direction (unidirectional prediction, bidirectional prediction, etc.) of the current block. Alternatively, the inter-picture prediction indicator may mean the number of reference pictures used to generate the prediction block of the current block. Further optionally, the inter-picture prediction indicator may mean the number of prediction blocks for performing inter-picture prediction or motion compensation for the current block.

Reference picture index: means an index indicating a specific reference picture in the reference picture list.

Reference picture: may mean a picture that a particular block is referenced for inter-picture prediction or motion compensation.

Motion vector: is a two-dimensional vector used for inter-picture prediction or motion compensation, and can mean an offset between a reference picture and an encoding/decoding target picture. For example, (mvX, mvY) may represent a motion vector, mvX may represent a horizontal component, and mvY may represent a vertical component.

Motion vector candidate: may mean a block that becomes a prediction candidate when a motion vector is predicted, or may mean a motion vector of the block. Motion vector candidates may be listed in a motion vector candidate list.

Motion Vector Candidate List: May mean a list of motion vector candidates.

Motion Vector Candidate Index: May mean an indicator indicating a motion vector candidate in a motion vector candidate list. The motion vector candidate index is also referred to as the index of the motion vector predictor.

Motion information: may mean information including motion vectors, reference picture indexes, inter-picture prediction indicators, and at least any one of: reference picture list information, reference pictures, motion vector candidates, motion vector candidate indexes, merge candidates and merged indexes.

Merge Candidate List: means a list consisting of merge candidates.

Merge candidate: means a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-predictive merge candidate, a zero merge candidate, and the like. A merge candidate may have an inter-picture prediction indicator, a reference picture index for each list, and motion information such as a motion vector.

Merge index: means information indicating merge candidates within the merge candidate list. The merge index may indicate a block used to derive a merge candidate among reconstructed blocks spatially and/or temporally adjacent to the current block. The merge index may indicate at least one item of motion information possessed by the merge candidate.

Transform unit: means a basic unit when encoding/decoding such as transform, inverse transform, quantization, inverse quantization, and transform coefficient encoding/decoding is performed on a residual signal. A single transformation unit may be partitioned into a plurality of transformation units having a small size.

Scaling: Means the process of multiplying transform coefficient levels with factors. The transform coefficients may be generated by scaling the transform coefficient levels. Scaling may also be referred to as inverse quantization.

Quantization parameter: may mean the value used when generating the transform coefficient levels of the transform coefficients during quantization. A quantization parameter may also mean a value used when generating transform coefficients by scaling the transform coefficient levels during inverse quantization. A quantization parameter may be a value mapped on a quantization step size.

Variable delta quantization parameter: means the difference between the quantization parameter of the encoding/decoding target unit and the predicted quantization parameter.

Scan: Means a method of sorting coefficients within a block or matrix. For example, an operation of changing a two-dimensional matrix of coefficients into a one-dimensional matrix may be called scanning, and an operation of changing a one-dimensional matrix of coefficients into a two-dimensional matrix may be called scanning or inverse scanning.

Transform Coefficients: May mean coefficient values resulting after a transform is performed in an encoder. The transform coefficient may mean a coefficient value generated after at least one of entropy decoding and inverse quantization is performed in a decoder. Quantized levels obtained by quantizing transform coefficients or residual signals or quantized transform coefficient levels may also fall within the meaning of transform coefficients.

Level of quantization: means a value generated by quantizing transform coefficients or residual signals in an encoder. Alternatively, the level of quantization may mean a value that is an inverse quantization target subjected to inverse quantization in a decoder. Similarly, quantized transform coefficient levels as a result of transform and quantization may also fall within the meaning of quantized levels.

Non-Zero Transform Coefficient: Means a transform coefficient whose value is not 0, or means a transform coefficient level whose value is not 0.

Quantization matrix: means a matrix used in quantization processing or dequantization processing performed in order to improve subjective image quality or objective image quality. Quantization matrices may also be referred to as scaling lists.

Quantization matrix coefficient: means each element in the quantization matrix. Quantization matrix coefficients may also be referred to as matrix coefficients.

Default matrix: means a predetermined quantization matrix initially defined in an encoder or decoder.

Non-default matrix: means a quantization matrix that is not initially defined in the encoder or decoder but is signaled by the user.

FIG. 1 is a block diagram showing the configuration of an encoding device according to an embodiment to which the present invention is applied.

The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include at least one image. The encoding apparatus 100 may sequentially encode the at least one image.

Referring to FIG. 1 , an encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, and an inverse quantization unit 160. , an inverse transform unit 170 , an adder 175 , a filter unit 180 and a reference picture buffer 190 .

The encoding apparatus 100 may perform encoding on an input image by using an intra mode or an inter mode, or both of the intra mode and the inter mode. Also, the encoding apparatus 100 may generate a bitstream by encoding an input image, and may output the generated bitstream. The generated bit stream may be stored in a computer-readable recording medium, or may be streamed through a wired/wireless transmission medium. When the intra mode is used as the prediction mode, the switcher 115 may switch to intra. Alternatively, when the inter mode is used as the prediction mode, the switcher 115 may switch to the inter mode. Here, the intra mode may mean an intra prediction mode, and the inter mode may mean an inter prediction mode. The encoding apparatus 100 may generate a prediction block of an input block of an input image. Also, after generating the prediction block, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input image may be referred to as a current image that is a current encoding target. The input block may be referred to as a current block that is a current encoding target or may be referred to as an encoding target block.

When the prediction mode is the intra mode, the intra prediction unit 120 may use pixel values of encoded/decoded blocks adjacent to the current block as reference pixels. The intra prediction unit 120 may perform spatial prediction by using reference pixels, or may generate prediction samples of an input block by performing spatial prediction. Here, intra prediction may mean prediction within a frame.

When the prediction mode is the inter mode, the motion prediction unit 111 may search for an area that best matches the input block from a reference image when performing motion prediction, and may derive a motion vector by using the searched area. The reference picture may be stored in the reference picture buffer 190 .

The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector. Here, inter prediction may mean prediction or motion compensation between frames.

When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial region of a reference picture. In order to perform inter picture prediction or motion compensation on a coding unit, it may be determined which mode among skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and current picture reference mode is used for prediction included in the corresponding coding unit Unit motion prediction and motion compensation. Then, inter prediction or motion compensation may be performed differently according to the determined mode.

The subtracter 125 may generate a residual block by using a residual between the input block and the prediction block. A residual block may be referred to as a residual signal. The residual signal may mean the difference between the original signal and the predicted signal. Also, the residual signal may be a signal generated by transforming or quantizing, or transforming and quantizing, a difference between an original signal and a predicted signal. The residual block may be a residual signal in block units.

The transform unit 130 may generate transform coefficients by performing transform on the residual block, and may output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block. When the transform skip mode is applied, the transform unit 130 may skip transforming the residual block.

The level of quantization may be generated by applying quantization to the transform coefficients or to the residual signal. Hereinafter, in an embodiment, quantized levels may also be referred to as transform coefficients.

The quantization unit 140 may generate quantized levels by quantizing transform coefficients or residual signals according to parameters, and may output the generated quantized levels. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding on a value calculated by the quantization unit 140 or on an encoding parameter value calculated when encoding is performed according to a probability distribution, and may output the generated bitstream. The entropy encoding unit 150 may perform entropy encoding on pixel information of an image and information for decoding the image. For example, information used to decode an image may include syntax elements.

When entropy coding is applied, symbols can be represented by assigning a smaller number of bits to symbols with high chances of generation and a larger number of bits to symbols with low chances of generation, thereby reducing the number of bits used to be encoded The size of the symbolic bitstream. The entropy encoding unit 150 may use an encoding method for entropy encoding such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), and the like. For example, the entropy encoding unit 150 may perform entropy encoding by using a variable length code/code (VLC) table. Also, the entropy encoding unit 150 may derive a binarization method of a target symbol and a probability model of a target symbol/bin, and may perform arithmetic coding by using the derived binarization method and context model.

In order to encode transform coefficient levels, the entropy encoding unit 150 may change coefficients in a two-dimensional block form into a one-dimensional vector form by using a transform coefficient scanning method.

The encoding parameters may include information such as syntax elements (flags, indices, etc.) encoded in the encoder and signaled to the decoder, and information derived when encoding or decoding is performed. Encoding parameters may mean information required when encoding or decoding an image. For example, the encoding parameters may include at least one value or a combination of the following items: unit/block size, unit/block depth, unit/block partition information, unit/block partition structure, whether to perform quadtree partitioning, whether to perform Partition in binary tree form, partition direction in binary tree form (horizontal direction or vertical direction), partition form in binary tree form (symmetric partition or asymmetric partition), intra prediction mode/direction, reference sample filtering method, prediction block filtering method, Prediction Block Filter Taps, Prediction Block Filter Coefficients, Inter Prediction Modes, Motion Information, Motion Vectors, Reference Picture Indexes, Inter Prediction Angles, Inter Prediction Indicators, Reference Picture Lists, Reference Pictures, Motion Vector Predictor Candidates , motion vector candidate list, whether to use merge mode, merge candidate, merge candidate list, whether to use skip mode, interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, representation accuracy of motion vector, transformation Type, transformation size, whether to use the information of the first (first) transformation, whether to use the information of the second transformation, the index of the first transformation, the index of the second transformation, the information of whether the residual signal exists, the coding block style, the coding block flag (CBF) , quantization parameter, quantization matrix, whether to apply in-loop filter, in-loop filter coefficient, in-loop filter tap, in-loop filter shape/form, whether to apply deblocking filter, deblocking filter coefficient , DF taps, DF strength, DF shape/form, ASO applied or not, ASO value, ASO category, ASO offset type, adaptive in-loop filter applied or not, adaptive in-loop filter coefficients, adaptive in-loop filter taps, adaptive in-loop filter shape/form, binarization/debinarization method, context Model determination method, context model update method, whether to execute normal mode, whether to execute bypass mode, context bin, bypass bin, transformation coefficient, transformation coefficient level, transformation coefficient level scanning method, image display/output order, banding identification information, slice type, slice partition information, tile identification information, tile type, tile partition information, picture type, bit depth, and information of luma signal or chrominance signal.

Here, signaling a flag or an index may mean that the corresponding flag or index is entropy-encoded by an encoder and included in a bitstream, and may mean that a corresponding flag or index is entropy-decoded from a bitstream by a decoder.

When the encoding apparatus 100 performs encoding through inter prediction, an encoded current image may be used as a reference image for another image that is subsequently processed. Accordingly, the encoding apparatus 100 may reconstruct or decode the encoded current image, or may store the reconstructed or decoded image as a reference image.

The quantized levels may be inversely quantized in the inverse quantization unit 160 or may be inversely transformed in the inverse transform unit 170 . Inverse quantized or inverse transformed coefficients, or both inverse quantized and inverse transformed coefficients may be added to the prediction block by the adder 175 . A reconstructed block may be generated by adding dequantized or inverse transformed coefficients, or both dequantized and inverse transformed coefficients, to the prediction block. Here, dequantized or inversely transformed coefficients or both dequantized and inversely transformed coefficients may mean coefficients on which at least one of inverse quantization and inverse transform is performed, and may mean reconstructed residual blocks.

The reconstructed block may pass through a filter unit 180 . The filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed image. The filter unit 180 may be referred to as an in-loop filter.

A deblocking filter may remove block distortion generated at boundaries between blocks. In order to determine whether to apply the deblocking filter, whether to apply the deblocking filter to the current block may be determined based on pixels included in several rows or columns included in the block. When a deblocking filter is applied to a block, another filter may be applied depending on the desired deblocking filtering strength.

To compensate for coding errors, appropriate offset values can be added to pixel values by using sample adaptive offset. The sample adaptive offset can correct the offset between the deblocked image and the original image according to the pixel unit. A method of applying offset in consideration of edge information about each pixel may be used, or a method of partitioning pixels of an image into a predetermined number of regions, determining regions to which offset is applied, and applying offset to the determined regions may be used.

The adaptive loop filter may perform filtering based on a comparison between the filtered reconstructed image and the original image. Pixels included in an image may be partitioned into predetermined groups, a filter to be applied to each group may be determined, and different filtering may be performed for each group. Information on whether to apply the ALF may be signaled per coding unit (CU), and the shape and coefficient of the ALF to be applied to each block may vary.

The reconstructed block or reconstructed image passed through the filter unit 180 may be stored in the reference picture buffer 190 . FIG. 2 is a block diagram showing the configuration of a decoding device to which the present invention is applied according to an embodiment.

The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

Referring to FIG. 2 , the decoding apparatus 200 may include an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensation unit 250 , an adder 255 , a filter unit 260 , and a reference picture buffer 270 .

The decoding device 200 may receive the bitstream output from the encoding device 100 . The decoding device 200 may receive a bit stream stored in a computer-readable recording medium, or may receive a bit stream streamed through a wired/wireless transmission medium. The decoding apparatus 200 may decode a bitstream by using an intra mode or an inter mode. Also, the decoding apparatus 200 may generate a reconstructed image or a decoded image generated by performing decoding, and may output the reconstructed image or the decoded image.

When the prediction mode used in decoding is the intra mode, the switcher may be switched to intra. Alternatively, when the prediction mode used at the time of decoding is the inter mode, the switcher may be switched to the inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block by decoding an input bitstream, and may generate a prediction block. When the reconstructed residual block and the predicted block are obtained, the decoding apparatus 200 may generate a reconstructed block which becomes a decoding target by adding the reconstructed residual block to the predicted block. A decoding target block may be referred to as a current block.

The entropy decoding unit 210 may generate symbols by entropy decoding a bitstream according to a probability distribution. The generated symbols may include symbols in quantized hierarchical form. Here, the entropy decoding method may be an inverse process of the above-mentioned entropy encoding method.

In order to decode transform coefficient levels, the entropy decoding unit 210 may change coefficients in a one-dimensional vector form into a two-dimensional block form by using a transform coefficient scanning method.

The quantized levels may be inversely quantized in the inverse quantization unit 220 or may be inversely transformed in the inverse transform unit 230 . The quantized level may be a result of performing inverse quantization or inverse transformation, or both, and may be generated as a reconstructed residual block. Here, the inverse quantization unit 220 may apply a quantization matrix to quantized levels.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of already-decoded blocks adjacent to the decoding target block.

When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a reference image stored in the reference picture buffer 270 and a motion vector.

The adder 255 may generate a reconstructed block by adding the reconstructed residual block to the predicted block. The filter unit 260 may apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 may output a reconstructed image. A reconstructed block or reconstructed image may be stored in the reference picture buffer 270, and may be used when inter prediction is performed.

FIG. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded. FIG. 3 schematically shows an example of partitioning a single unit into multiple lower-level units.

In order to efficiently partition an image, a coding unit (CU) may be used when encoding and decoding. A coding unit may be used as a basic unit in encoding/decoding an image. Also, a coding unit may be used as a unit for distinguishing an intra mode from an inter mode when encoding/decoding an image. A coding unit may be a basic unit for predicting, transforming, quantizing, inverse transforming, inverse quantizing, or encoding/decoding processing on a transform coefficient.

Referring to FIG. 3 , an image 300 is sequentially partitioned by largest coding unit (LCU), and the LCU unit is determined as a partition structure. Here, an LCU may be used in the same meaning as a Coding Tree Unit (CTU). Cell partitioning may mean partitioning blocks associated with a cell. In the block partition information, information of a cell depth may be included. Depth information may represent the number of times or the degree to which a cell is partitioned or both. A single unit may be partitioned in layers associated with depth information based on a tree structure. Each partitioned lower unit may have depth information. Depth information may be information representing the size of a CU, and may be stored in each CU.

The partition structure may mean distribution of coding units (CUs) in the LCU 310 . Such a distribution may be determined according to whether a single CU is partitioned into multiple (a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc.) CUs. The horizontal size and vertical size of the CU generated by performing partitioning may be half of the horizontal size and vertical size of the CU before partitioning, respectively, or may have a smaller horizontal size and vertical size than the horizontal size and vertical size before partitioning according to the number of times partitioning is performed, respectively. size. A CU can be recursively partitioned into multiple CUs. Partitioning may be performed recursively on a CU up to a predefined depth or a predefined size. For example, the depth of an LCU may be 0, and the depth of a minimum coding unit (SCU) may be a predefined maximum depth. Here, the LCU may be a coding unit having the largest coding unit size, and the SCU may be a coding unit having the smallest coding unit size as described above. Partitioning is performed starting from the LCU 310 , and when the horizontal size or the vertical size or both of the horizontal size and the vertical size of the CU is reduced by performing partitioning, the depth of the CU is increased by 1.

In addition, information on whether a CU is partitioned may be represented by using partition information of the CU. Partition information may be 1-bit information. All CUs except the SCU may include partition information. For example, when the value of the partition information is the first value, the CU may not be partitioned, and when the value of the partition information is the second value, the CU may be partitioned.

Referring to FIG. 3 , an LCU having a depth of 0 may be a 64×64 block. 0 can be the minimum depth. An SCU with depth 3 may be an 8x8 block. 3 can be the maximum depth. A CU of a 32×32 block and a 16×16 block may be represented as depth 1 and depth 2, respectively.

For example, when a single coding unit is partitioned into four coding units, the horizontal size and the vertical size of the partitioned four coding units may be half of the horizontal size and the vertical size of the CU before being partitioned. In one embodiment, when a coding unit having a size of 32×32 is partitioned into four coding units, each of the partitioned four coding units may have a size of 16×16. When a single coding unit is partitioned into four coding units, it can be said that the coding unit can be partitioned into a quadtree form.

For example, when a single coding unit is partitioned into two coding units, the horizontal or vertical size of the two coding units may be half the horizontal or vertical size of the coding unit before being partitioned. For example, when a coding unit having a size of 32×32 is partitioned in a vertical direction, each of two partitioned coding units may have a size of 16×32. When a single coding unit is partitioned into two coding units, it can be said that the coding unit is partitioned in a binary tree form. The LCU 320 of FIG. 3 is an example of an LCU to which both quadtree-form partitioning and binary-tree-form partitioning are applied.

FIG. 4 is a diagram illustrating an embodiment of an inter-picture prediction process.

In FIG. 4, a rectangle may represent a screen. In Fig. 4, arrows indicate prediction directions. A picture may be classified into an intra picture (I picture), a predictive picture (P picture), and a bidirectional predictive picture (B picture) according to an encoding type of the picture.

I-pictures can be coded by intra-prediction instead of inter-picture prediction. A P picture may be encoded through inter-picture prediction by using a reference picture existing in one direction (ie, forward or backward) with respect to a current block. The B picture may be encoded via inter-picture prediction by using reference pictures preset in two directions (ie, forward and backward) with respect to the current block. When inter-picture prediction is used, the encoder may perform inter-picture prediction or motion compensation, and the decoder may perform corresponding motion compensation.

Hereinafter, an embodiment of inter-picture prediction will be described in detail.

Inter prediction or motion compensation may be performed using reference pictures and motion information.

Motion information of a current block may be derived by each of the encoding apparatus 100 and the decoding apparatus 200 during inter-picture prediction. Motion information of a current block may be derived by using motion information of a reconstructed neighboring block, a co-located block (also referred to as a col block or co-located block), and/or motion information of a block adjacent to the co-located block. A co-located block may mean a block that is spatially co-located with a current block within a previously reconstructed co-located picture (also referred to as a col picture or a co-located picture). The co-located picture may be one picture among one or more reference pictures included in the reference picture list.

A method of deriving motion information of a current block may vary according to a prediction mode of the current block. For example, as prediction modes for inter-picture prediction, there may be AMVP mode, merge mode, skip mode, current picture reference mode, and the like. Merge mode may be referred to as motion merge mode.

For example, when AMVP is used as the prediction mode, at least one of the motion vector of the reconstructed adjacent block, the motion vector of the co-located block, the motion vector of the block adjacent to the co-located block, and the motion vector (0,0) may be determined is a motion vector candidate for a current block, and a motion vector candidate list is generated by using the motion vector candidate. Motion vector candidates of the current block may be derived by using the generated motion vector candidate list. Motion information for the current block may be determined based on the derived motion vector candidates. A motion vector of a co-located block or a motion vector of a block adjacent to the co-located block may be referred to as a temporal motion vector candidate, and a motion vector of a reconstructed neighboring block may be referred to as a spatial motion vector candidate.

The encoding apparatus 100 may calculate a motion vector difference (MVD) between a motion vector of a current block and a motion vector candidate, and may perform entropy encoding on the motion vector difference (MVD). Also, the encoding apparatus 100 may perform entropy encoding on the motion vector candidate index and generate a bitstream. The motion vector candidate index may indicate the best motion vector candidate among motion vector candidates included in the motion vector candidate list. The decoding apparatus may perform entropy decoding on a motion vector candidate index included in a bitstream, and may select a motion vector candidate of a decoding target block among motion vector candidates included in a motion vector candidate list by using the entropy-decoded motion vector candidate index. . Also, the decoding apparatus 200 may add the entropy-decoded MVD to the motion vector candidate extracted by the entropy decoding, thereby deriving the motion vector of the decoding target block.

The bitstream may include a reference picture index indicating a reference picture. The reference picture index may be entropy-encoded by the encoding apparatus 100 and then signaled to the decoding apparatus 200 as a bitstream. The decoding apparatus 200 may generate a prediction block of a decoding target block based on the derived motion vector and reference picture index information.

Another example of a method of deriving motion information of a current block may be a merge mode. Merge mode may mean a method of merging motions of multiple blocks. The merge mode may mean a mode in which motion information of a current block is derived from motion information of neighboring blocks. When the merge mode is applied, the motion information of reconstructed neighboring blocks and/or the motion information of co-located blocks may be used to generate a merge candidate list. The motion information may include at least one of a motion vector, a reference picture index, and an inter-picture prediction indicator. The prediction indicator may indicate unidirectional prediction (L0 prediction or L1 prediction) or bidirectional prediction (L0 prediction and L1 prediction).

The merge candidate list may be a list of stored motion information. The motion information included in the merging candidate list may be at least one of zero merging candidates and new motion information, wherein the new motion information is motion information (spatial merging candidates) of a neighboring block adjacent to the current block, the current A combination of motion information (temporal merging candidates) of a block including a co-located block in a reference picture, and motion information present in a merging candidate list.

The encoding apparatus 100 may generate a bitstream by performing entropy encoding on at least one of a merge flag and a merge index, and may signal the bitstream to the decoding apparatus 200 . The merge flag may be information indicating whether to perform a merge mode for each block, and the merge index may be information indicating which adjacent block among adjacent blocks of the current block is a merge target block. For example, adjacent blocks of the current block may include a left adjacent block on the left side of the current block, an above adjacent block located above the current block, and a temporally adjacent block temporally adjacent to the current block.

The skip mode may be a mode in which motion information of neighboring blocks is applied to the current block as it is. When the skip mode is applied, the encoding device 100 may perform entropy encoding on information of the fact of which block's motion information will be used as the motion information of the current block to generate a bitstream, and may signal the bitstream to the decoding device 200. The encoding apparatus 100 may not signal syntax elements regarding at least any one of motion vector difference information, encoded block flags, and transform coefficient levels to the decoding apparatus 200 .

A current picture reference mode may mean a prediction mode in which a previously reconstructed area to which a current block within a current picture belongs is used for prediction. Here, a vector may be used to designate the previously reconstructed region. Information indicating whether a current block is to be encoded in a current picture reference mode may be encoded by using a reference picture index of the current block. A flag or index indicating whether the current block is a block coded in the current picture reference mode may be signaled, and may be derived based on the reference picture index of the current block. In case the current block is coded in the current picture reference mode, the current picture may be added to the reference picture list for the current block so as to be located at a fixed position or a random position in the reference picture list. The fixed position may be eg the position indicated by reference picture index 0 or the last position in the list. When the current picture is added to the reference picture list so as to be located at the random position, a reference picture index indicating the random position may be signaled.

Based on the above description, an image encoding method and an image decoding method according to an embodiment of the present invention will be described in detail below.

FIG. 5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention, and FIG. 6 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

Referring to FIG. 5, the encoding apparatus may derive motion vector candidates (step S501), and may generate a motion vector candidate list based on the derived motion vector candidates (step S502). After the motion vector candidate list is generated, a motion vector may be determined based on the generated motion vector candidate list (step S503), and motion compensation may be performed based on the determined motion vector (step S504). Thereafter, the encoding device may encode information associated with motion compensation (step S505).

Referring to FIG. 6 , the decoding device may perform entropy decoding on information associated with motion compensation received from the encoding device (step S601 ), and may derive motion vector candidates (step S602 ). The decoding apparatus may generate a motion vector candidate list based on the derived motion vector candidates (step S603), and determine a motion vector using the generated motion vector candidate list (step S604). Thereafter, the decoding device may perform motion compensation by using the determined motion vector (step S605).

FIG. 7 is a flowchart illustrating an image encoding method according to another embodiment of the present invention, and FIG. 8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

Referring to FIG. 7, the encoding apparatus may derive a merge candidate (step S701), and generate a merge candidate list based on the derived merge candidate. After the merge candidate list is generated, the encoding apparatus may determine motion information using the generated merge candidate list (step S702), and may perform motion compensation on a current block using the determined motion information (step S703). Thereafter, the encoding apparatus may perform entropy encoding on information associated with motion compensation (step S704).

Referring to FIG. 8 , the decoding device may perform entropy decoding on information associated with motion compensation received from the encoding device ( S801 ), derive merge candidates ( S802 ), and generate a merge candidate list based on the derived merge candidates. After the merge candidate list is generated, the decoding apparatus may determine motion information of the current block by using the generated merge candidate list (S803). Thereafter, the decoding apparatus may perform motion compensation using the motion information (S804).

5 and 6 show examples in which the AMVP shown in FIG. 4 is applied, and FIGS. 7 and 8 show examples in which the merge mode shown in FIG. 4 is applied.

Hereinafter, each step in FIGS. 5 and 6 will be described, and then each step in FIGS. 7 and 8 will be described. However, the motion compensation steps corresponding to S504, S605, S703, and S804 and the entropy encoding/decoding steps corresponding to S505, S601, S704, and S801 will be collectively described.

Hereinafter, each step in FIGS. 5 and 6 will be described in detail below.

First, the steps of deriving motion vector candidates (S501, S602) will be described in detail.

The motion vector candidates of the current block may include one of the spatial motion vector candidates and the temporal motion vector candidates, or both of the spatial motion vector candidates and the temporal motion vector candidates.

A spatial motion vector for the current block may be derived from reconstructed blocks neighboring the current block. For example, motion vectors of reconstructed blocks adjacent to the current block may be determined as spatial motion vector candidates of the current block.

FIG. 9 is a diagram illustrating an example of deriving a spatial motion vector candidate of a current block.

Referring to FIG. 9 , spatial motion vector candidates of a current block X may be derived from neighboring blocks adjacent to the current block X. Referring to FIG. Adjacent blocks to the current block X include block B1 adjacent to the upper end of the current block, block A1 adjacent to the left end of the current block, block B0 adjacent to the upper right corner of the current block, and block B0 adjacent to the upper left corner of the current block At least one of the adjacent block B2, and the block A0 adjacent to the lower left corner of the current block. Neighboring blocks adjacent to the current block may have a square shape or a non-square shape. When one neighboring block among a plurality of neighboring blocks adjacent to the current block has a motion vector, the motion vector of the neighboring block may be determined as a spatial motion vector candidate of the current block. Whether the neighboring block has a motion vector or whether the motion vector of the neighboring block can be used as the spatial motion vector of the current block may be determined based on the determination of whether the neighboring block exists or whether the neighboring block has been coded through the inter prediction process candidate. The determination of whether a specific neighboring block has a motion vector or whether the motion vector of a neighboring block can be used as a spatial motion vector candidate of the current block may be performed in a predetermined order. For example, as shown in FIG. 9 , availability determination of motion vectors may be performed in the order of blocks A0, A1, B0, B1, and B2.

When a reference picture of a current block and a reference picture of a neighboring block having a motion vector are different from each other, the motion vector of the neighboring block is scaled, and then the scaled motion vector may be used as a spatial motion vector candidate of the current block. The motion vector scaling may be performed based on at least any one of the distance between the current picture and the reference picture of the current block and the distance between the current picture and the reference pictures of the neighboring blocks. Here, the spatial motion vector candidate of the current block may be derived by scaling the motion vectors of the neighboring blocks according to the ratio of the distance between the current picture and the reference picture of the current block to the distance between the current picture and the reference pictures of the neighboring blocks.

However, when the reference picture index of the current block and the reference picture index of the neighboring block having the motion vector are different, the scaled motion vector of the neighboring block may be determined as a spatial motion vector candidate of the current block. Even in this case, scaling may be performed based on at least one of the distance between the current picture and the reference picture of the current block and the distance between the current picture and the reference picture of the neighboring blocks.

Regarding scaling, motion vectors of neighboring blocks may be scaled based on a reference picture indicated by a reference picture index having a predefined value, and the scaled motion vector may be determined as a spatial motion vector candidate of a current block. The predefined value can be zero or a positive integer. For example, based on the ratio of the distance between the current picture and the reference picture indicated by the reference picture index with the predefined value of the current block to the distance between the current picture and the reference picture of the neighboring block with the predefined value The motion vectors of neighboring blocks are scaled to derive spatial motion vector candidates for the current block.

Optionally, the spatial motion vector candidate of the current block may be derived based on at least one of the encoding parameters of the current block.

A temporal motion vector candidate of a current block may be derived from a reconstructed block included in a co-located picture of the current picture. The co-located picture is a picture encoded/decoded before the current picture, and may be different from the current picture in time order.

FIG. 10 is a diagram illustrating an example of deriving temporal motion vector candidates of a current block.

Referring to FIG. 10 , a temporal motion vector candidate of the current block may be derived from a block at a position outside of a block spatially co-located with the current block X within a co-located picture (also referred to as a co-located picture) including the current picture, or from a block including The temporal motion vector candidate of the current block is derived from a block at an inner position of a block spatially co-located with the current block X. Here, the temporal motion vector candidate may mean a motion vector of a co-located block of the current block. For example, a temporal motion vector candidate for the current block X may be derived from a block H adjacent to the lower right corner of a block C that is spatially co-located with the current block X, or from a block C3 that includes the middle position of the block C The temporal motion vector candidate for X. Block H, block C3, etc. used to derive temporal motion vector candidates for the current block are referred to as co-located blocks.

Optionally, at least one of a temporal motion vector candidate, a co-located picture, a co-located block, a prediction list utilization flag, and a reference picture index may be derived based on at least one of the encoding parameters.

When the distance between the current picture including the current block and the reference picture of the current block is different from the distance between the co-located picture including the co-located block and the reference picture of the co-located block, the current frame can be obtained by scaling the motion vector of the co-located block. Temporal motion vector candidates for blocks. Here, scaling may be performed based on at least one of a distance between the current picture and the reference picture of the current block and a distance between the co-located picture and the reference picture of the co-located block. For example, the temporal motion vector candidate of the current block may be derived by scaling the motion vector of the co-located block according to the ratio of the distance between the current picture and the reference picture of the current block to the distance between the co-located picture and the reference picture of the co-located block.

Next, the steps (S502, S503) of generating a motion vector candidate list based on the derived motion vector candidates will be described.

The step of generating the motion vector candidate list may include a process of adding or removing a motion vector candidate to or from the motion vector candidate list, and a process of adding a combined motion vector candidate to the motion vector candidate list.

First, a process of adding a derived motion vector candidate to a motion vector candidate list or removing a derived motion vector candidate from the motion vector candidate list will be described. The encoding device and the decoding device may add the derived motion vector candidates to the motion vector candidate list in the order in which the motion vector candidates are derived.

It is assumed that the motion vector candidate list mvpListLX may mean a motion vector candidate list corresponding to the reference picture lists L0, L1, L2, and L3. That is, the motion vector candidate list corresponding to the reference picture list L0 can be represented by mvpListL0.

In addition to the spatial motion vector candidates and the temporal motion vector candidates, a motion vector having a predetermined value may be added to the motion vector candidate list. For example, when the number of motion vector candidates in the motion vector candidate list is less than the maximum number of motion vector candidates that can be included in the motion vector candidate list, a motion vector having a value of 0 may be added to the motion vector candidate list.

Next, a process of adding combined motion vector candidates to the motion vector candidate list will be described.

When the number of motion vector candidates in the motion vector candidate list is less than the maximum number of motion vector candidates that can be included in the motion vector candidate list, one or more motion vector candidates in the motion vector candidate list are combined to generate one or More combined motion vector candidates, and the resulting combined motion vector candidates may be added to the motion vector candidate list. For example, at least one or more of spatial motion vector candidates, temporal motion vector candidates, and zero motion vector candidates included in the motion vector candidate list are used to generate a combined motion vector candidate, and the generated combined motion vector candidate may be Add to motion vector candidate list.

Alternatively, a combined motion vector candidate may be generated based on at least one of encoding parameters, and the combined motion vector candidate generated based on at least one of the encoding parameters may be added to the motion vector candidate list.

Next, the step of selecting the predicted motion vector of the current block from the motion vector candidate list (S503, S604) will be described below.

Among the motion vector candidates included in the motion vector candidate list, the motion vector candidate indicated by the motion vector candidate index may be determined as the predicted motion vector of the current block.

The encoding apparatus may calculate a difference between a predicted motion vector and a motion vector of a current block, thereby generating a motion vector difference. The decoding apparatus may generate the motion vector of the current block by adding the predicted motion vector and the motion vector difference.

The steps of performing motion compensation (S504, S605) and the steps of entropy encoding/decoding information associated with motion compensation (S505, S601) shown in FIGS. 7 and 8 shown in the steps of performing motion compensation (S703, S804) and the steps of entropy encoding/decoding (S704, S801).

Next, each step shown in FIGS. 7 and 8 will be described in detail.

First, the steps of deriving merge candidates (S701, S802) will be described.

The merging candidates of the current block may include at least one of spatial merging candidates, temporal merging candidates, and additional merging candidates. Here, the expression "deriving a spatial merging candidate" means a process of deriving a spatial merging candidate and adding the derived merging candidate to the merging candidate list.

Referring to FIG. 9 , spatial merging candidates of a current block X may be derived from neighboring blocks adjacent to the current block X. Referring to FIG. Adjacent blocks adjacent to the current block X may include block B1 adjacent to the upper end of the current block, block A1 adjacent to the left end of the current block, block B0 adjacent to the upper right corner of the current block, and block B0 adjacent to the upper left end of the current block. At least one of the corner adjacent block B2, and the block A0 adjacent to the lower left corner of the current block.

In order to derive spatial merging candidates for the current block, it is determined whether each neighboring block adjacent to the current block is available for derivation of spatial merging candidates for the current block. Such a determination may be made for neighboring blocks in a predetermined priority order. For example, in the example of FIG. 9 , the availability of spatial merging candidates may be determined in the order of blocks A1 , B1 , B0 , A0 , and B2 . The spatial merging candidates determined based on the availability determination order may be sequentially added to the merging candidate list of the current block.

FIG. 11 is a diagram illustrating an example of a process of adding spatial merging candidates to a merging candidate list.

Referring to FIG. 11 , four spatial merging candidates are derived from four neighboring blocks A1, B0, A0, and B2, and the derived spatial merging candidates may be sequentially added to a merging candidate list.

Optionally, spatial merging candidates may be derived based on at least one of the encoding parameters.

Here, the motion information of spatial merging candidates may include three or more pieces of motion information, wherein the three or more pieces of motion information include L2 motion information and L3 motion information in addition to L0 motion information and L1 motion information. . Here, there may be at least one reference picture list including, for example, L0, L1, L2, and L3.

Next, a method of deriving temporal merging candidates of the current block will be described.

A temporal merging candidate of a current block may be derived from a reconstructed block included in a co-located picture of the current picture. The co-located picture may be a picture encoded/decoded before the current picture, and may be different from the current picture in time order.

The expression "deriving a temporal merging candidate" means a process of deriving a temporal merging candidate and adding the derived temporal merging candidate to the merging candidate list.

Referring to FIG. 10 , a temporal merge candidate of the current block may be derived from a block at a position outside of a block spatially co-located with the current block X among co-located pictures (also referred to as co-located pictures) including the current picture, or may be derived from Temporal merging candidates of the current block are derived from blocks including positions inside blocks spatially co-located with the current block X in co-located pictures of the current picture. The term "temporal merge candidate" may mean motion information of a co-located block. For example, the temporal merge candidate of the current block X can be derived from the block H adjacent to the lower right corner of the block C that is spatially co-located with the current block X, or the current block X can be derived from the block C3 including the middle position of the block C time to merge candidates. The blocks H, C3, etc. used to derive temporal merging candidates for the current block are called co-located blocks (also called co-located blocks).

When a temporal merging candidate of the current block can be derived from a block H including a position outside the block C, the block H is set as a co-located block of the current block. In this case, temporal merging candidates of the current block may be derived based on motion information of the block H. On the contrary, when a temporal merge candidate of the current block cannot be derived from the block H, a block C3 including a position located inside the block C may be set as a co-located block of the current block. In this case, temporal merging candidates of the current block may be derived based on motion information of the block C3. When no temporal merging candidate for the current block can be derived from neither block H nor block C3 (e.g., block H and block C3 are both intra-coded blocks), it may not be possible to derive a temporal merging candidate for the current block at all, Or the temporal merging candidates of the current block may be derived from blocks other than block H and block C3.

Alternatively, for example, multiple temporal merging candidates for the current block may be derived from multiple blocks included in the co-located picture. That is, multiple temporal merging candidates for the current block may be derived from block H, block C3, and so on.

FIG. 12 is a diagram showing an example of a process of adding temporal merging candidates to a merging candidate list.

Referring to FIG. 12 , when a temporal merging candidate is derived from a co-located block located at a position H1, the derived temporal merging candidate may be added to a merging candidate list.

When the distance between the current picture including the current block and the reference picture of the current block is different from the distance between the co-located picture including the co-located block and the reference picture of the co-located block, the current frame can be obtained by scaling the motion vector of the co-located block. The motion vectors of the temporal merging candidates for the block. Here, the scaling of the motion vector may be performed based on at least one of the distance between the current picture and the reference picture of the current block and the distance between the co-located picture and the reference picture of the co-located block. For example, the motion of the temporal merging candidate of the current block can be derived by scaling the motion vector of the co-located block according to the ratio of the distance between the current picture and the reference picture of the current block to the distance between the co-located picture and the reference picture of the co-located block vector.

In addition, at least one of a temporal merging candidate, a co-located picture, a co-located block, a prediction list utilization flag, and a reference picture index may be derived based on at least one of encoding parameters of a current block, a neighboring block, or a co-located block.

The merge candidate list may be generated by generating at least one of the spatial merge candidates and the temporal merge candidates and sequentially adding the derived merge candidates to the merge candidate list in the order of derivation.

Next, a method of deriving additional merging candidates of the current block will be described.

The term 'additional merge candidate' may mean at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a predetermined merge candidate having a predetermined motion information value. Here, the expression "deriving additional merging candidates" may mean a process of deriving additional merging candidates and adding the derived additional merging candidates to the merging candidate list.

The modified spatial merging candidate may mean a merging candidate obtained by modifying at least one item of motion information of the derived spatial merging candidates.

The modified temporal merge candidate may mean a modified merge candidate obtained by modifying at least one item of motion information of the derived temporal merge candidates.

Combining a merge candidate may mean a motion by at least one of a spatial merge candidate, a temporal merge candidate, a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a predetermined merge candidate having a predetermined motion information value. Merge candidates obtained by combining information, wherein the spatial merge candidates, temporal merge candidates, modified spatial merge candidates, modified temporal merge candidates, combined merge candidates, and predetermined merge candidates with predetermined motion information values are all included in Merge candidate list.

Alternatively, the combined merge candidate may mean a merge candidate derived by combining motion information of at least one of the merge candidates not included in the merge candidate list but from which a spatial merge candidate and a temporal merge candidate can be derived a spatial merging candidate and a temporal merging candidate derived from at least one of the blocks; a modified spatial merging candidate and a modified temporal merging candidate derived based on the spatial merging candidate and the temporal merging candidate derived from the block; combining merge candidates; and predetermined merge candidates having predetermined motion information values.

Alternatively, combined merging candidates may be derived using motion information obtained by performing entropy decoding on a bitstream in a decoder. In this case, the motion information used to derive the combined merging candidates may be entropy encoded into a bitstream in the encoder.

A combined merge candidate may mean a combined bi-predictive merge candidate. The combined bi-predictive merging candidate is a merging candidate using bi-prediction, and the combined bi-predictive merging candidate may be a merging candidate having L0 motion information and L1 motion information.

A merge candidate having a predetermined motion information value may be a zero merge candidate having a motion vector (0,0). A merge candidate having a predetermined motion information value may be set such that the merge candidate has the same value in the encoding device and the decoding device.

At least one of a modified spatial merging candidate, a modified temporal merging candidate, a combined merging candidate, and a merging candidate having a predetermined motion information value may be derived or generated based on at least one of encoding parameters of a current block, an adjacent block, or a co-located block. one. In addition, at least one of a modified spatial merging candidate, a modified temporal merging candidate, a combined merging candidate, and a merging candidate with a predetermined motion information value may be combined based on at least one of encoding parameters of the current block, an adjacent block, or a co-located block. Add to merge candidate list.

The size of the merge candidate list may be determined based on encoding parameters of the current block, neighboring blocks, or co-located blocks, and may vary according to the encoding parameters.

Next, the step of determining motion information of the current block using the generated merging candidate list (S702, S803) will be described.

The encoder may select a merge candidate to be used for motion compensation of a current block from a merge candidate list through motion estimation, and may encode a merge candidate index merge_idx indicating the determined merge candidate to a bitstream.

In order to generate a prediction block of a current block, the encoder may select a merging candidate from a merging candidate list by using a merging candidate index, and determine motion information of the current block. Then, the encoder may perform motion compensation based on the determined motion information, thereby generating a prediction block of the current block.

The decoder may decode the merge candidate index in the received bitstream and determine a merge candidate indicated by the merge candidate index included in the merge candidate list. The determined merging candidates may be determined as motion information of the current block. The determined motion information is used for motion compensation of the current block. Here, the term "motion compensation" may have the same meaning as inter prediction.

Next, steps of performing motion compensation using motion vectors or motion information (S504, S605, S703, S804) will be described.

The encoding device and the decoding device may calculate the motion vector of the current block by using the predicted motion vector and the motion vector difference. After the motion vector is calculated, the encoding device and the decoding device may perform inter prediction or motion compensation using the calculated motion vector (S504, S605).

The encoding device and the decoding device may perform inter prediction or motion compensation using the determined motion information (S703, S804). Here, the current block may have motion information of the determined merging candidates.

Depending on the prediction direction of the current block, the current block may have one (minimum) to N (maximum) motion vectors. One (minimum value) to N (maximum value) prediction blocks may be generated using the one to N motion vectors, and a final prediction block may be selected among the generated prediction blocks.

For example, when the current block has one motion vector, the prediction block generated using the motion vector (or motion information) is determined as the final prediction block of the current block.

In addition, when the current block has a plurality of motion vectors (or pieces of motion information), the plurality of motion vectors (or the pieces of motion information) are used to generate a plurality of prediction blocks, and based on the plurality of prediction blocks to determine the final prediction block of the current block. A plurality of reference pictures respectively including a plurality of prediction blocks respectively indicated by a plurality of motion vectors (or pieces of motion information) may be listed in different reference picture lists or in one reference picture list.

For example, a plurality of prediction blocks of the current block may be generated based on at least one of a spatial motion vector candidate, a temporal motion vector candidate, a motion vector having a predetermined value, and a combined motion vector candidate, and then may be based on weighted sum to determine the final prediction block for the current block.

Optionally, for example, a plurality of prediction blocks of the current block may be generated based on motion vector candidates indicated by a preset motion vector candidate index, and then the final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks . In addition, a plurality of prediction blocks may be generated based on motion vector candidates indicated by indexes within a predetermined motion vector candidate index range, and then a final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks.

A weighting factor for each prediction block may be equally 1/N (here, N is the number of generated prediction blocks). For example, when two prediction blocks are generated, the weight factor for each prediction block is 1/2. Similarly, when three prediction blocks are generated, the weighting factor for each prediction block is 1/3. When four prediction blocks are generated, the weighting factor for each prediction block may be 1/4. Optionally, the final prediction block of the current block may be determined in a manner of applying different weight factors to each prediction block.

Weighting factors for predicting blocks may not be fixed but variable. The weighting factors for predicting blocks may not be equal but different. For example, when two prediction blocks are generated, the weighting factors for the two prediction blocks may be equal, such as (1/2, 1/2), or may not be equal, such as (1/3, 2/3) , (1/4,3/4), (2/5,3/5) or (3/8,5/8). Weighting factors can be positive real-valued or negative real-valued. That is, the value of the weighting factor may include a negative real value, such as (-1/2, 3/2), (-1/3, 4/3) or (1-1/4, 5/4).

In order to apply variable weight factors, one or more pieces of weight factor information for the current block may be signaled through the bitstream. The weighting factor information may be signaled on a prediction block-by-prediction-block basis, or may be signaled on a reference-picture-by-reference-picture basis. Optionally, multiple prediction blocks can share one weighting factor.

The encoding device and the decoding device may determine whether to use a predicted motion vector (or predicted motion information) using a flag based on the predicted block list. For example, for each reference picture list, when the predicted block list utilization flag has a first value of one (1), the encoding apparatus and the decoding apparatus may perform inter prediction or motion compensation on the current block using the predicted motion vector of the current block. However, when the predicted block list utilization flag has the second value of zero (0), the encoding apparatus and the decoding apparatus may not perform inter prediction or motion compensation on the current block using the predicted motion vector of the current block. The first and second values of the prediction block list utilization flag may be set to 0 and 1, respectively, instead. Expression 1 to Expression 3 are examples of methods of generating the final prediction block of the current block when the inter prediction indicator of the current block is PRED_BI, PRED_TRI, or PRED_QUAD and when the prediction direction of each reference picture list is unidirectional .

[expression1]

P_BI＝(WF_LO*P_LO+OFFSET_LO+WF_L1*P_L1+OFFSET_L1+RF)》1

[expression2]

P_TRI＝(WF_LO*P_LO+OFFSET_LO+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+RF)/3

[expression 3]

P_QUAD＝(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+WF_L3*P_L3+OFFSET_L3+RF)＞＞2

In Expression 1 to Expression 3, each of P_BI, P_TRI, and P_QUAD represents a final prediction block of a current block, and LX (X=0, 1, 2, 3) represents a reference picture list. WF_LX indicates the weight factor of the prediction block generated using the LX reference picture list. OFFSET_LX indicates an offset value for a prediction block generated using the LX reference picture list. P_LX indicates a prediction block generated using a motion vector (or motion information) of the LX reference picture list of the current block. RF means rounding factor, and it can be set to 0, a positive integer or a negative integer. The LX reference picture list may include at least one of the following reference pictures: long-term reference picture, reference picture without filter, reference picture without sample adaptive offset, reference picture without adaptive loop filter picture, reference picture with DF and AO only, reference picture with DF and ALF only, reference with SAO and ALF picture, and all reference pictures that have passed through DF, SAO and ALF. In this case, the LX reference picture list may be at least any one of an L2 reference picture list and an L3 reference picture list.

Even if a plurality of prediction directions exist for a predetermined reference picture list, a final prediction block of a current block may be obtained based on a weighted sum of prediction blocks. In this case, weighting factors of a plurality of prediction blocks derived using one reference picture list may be equal or may be different from each other.

At least a weight factor WF_LX or an offset OFFSET_LX of a plurality of prediction blocks may be an encoding parameter to be entropy encoded/decoded. Alternatively, weighting factors and offsets may be derived from previously encoded/decoded neighboring blocks adjacent to the current block, for example. Here, the neighboring blocks adjacent to the current block may include at least one of a block for deriving a spatial motion vector candidate of the current block and a block for deriving a temporal motion vector candidate of the current block.

Further alternatively, for example, the weighting factor and the offset may be determined based on the display order of the current picture (picture order count (POC)) and the POC of each reference picture. In this case, when the distance between the current picture and the reference picture increases, the value of the weighting factor or offset may decrease. That is, when the current picture and the reference picture are close to each other, a larger value may be set as the weight factor or offset. For example, when the difference between the POC of the current picture and the POC of the L0 reference picture is 2, the value of the weighting factor applied to the prediction block generated using the L0 reference picture may be set to 1/3. Also, when the difference between the POC of the current picture and the POC of the L0 reference picture is 1, the value of the weighting factor applied to the prediction block generated using the L0 reference picture may be set to 2/3. As described above, the weighting factor or offset may be inversely proportional to the difference between the display order (POC) of the current picture and the display order (POC) of the reference picture. Alternatively, the weighting factor or offset may be proportional to the difference between the presentation order (POC) of the current picture and the presentation order (POC) of the reference picture.

Optionally, for example, at least one of the weight factor and the offset may be entropy encoded/decoded based on at least one encoding parameter. In addition, a weighted sum of prediction blocks may be calculated based on at least one encoding parameter.

A weighted sum of a plurality of prediction blocks may be applied to only a partial area of the prediction block. The partial area may be a boundary area adjacent to a boundary of each prediction block. In order to apply the weighted sum to only the partial region as described above, the weighted sum may be calculated subblock by subblock in each prediction block.

In a block having a block size indicated by region information, inter prediction or motion compensation may be performed on a subblock smaller than the block by using the same prediction block or the same final prediction block.

In a block having a block depth indicated by the region information, inter prediction or motion compensation may be performed on a subblock having a block depth deeper than that of the block by using the same prediction block or the same final prediction block.

Also, when the weighted sum of the prediction block is calculated by motion vector prediction, the weighted sum may be calculated using at least one of the motion vector candidates included in the motion vector candidate list, and the calculation result may be used as the final prediction of the current block piece.

For example, a prediction block may be generated using only spatial motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using spatial motion vector candidates and temporal motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only combined motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only a motion vector candidate indicated by a specific index, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only motion vector candidates indicated by indexes within a predetermined index range, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

When the weighted sum of the predicted block is calculated using the merge mode, the weighted sum may be calculated using at least one merge candidate among merge candidates included in the merge candidate list, and the calculation result may be used as a final predicted block of the current block.

For example, a prediction block may be generated using only spatial merging candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using spatial merging candidates and temporal merging candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only combined merging candidates, a weighted sum of the prediction blocks may be generated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only a merging candidate indicated by a specific index, a weighted sum of the prediction blocks may be generated, and the calculated weighted sum may be used as a final prediction block of the current block.

For example, a prediction block may be generated using only merge candidates indicated by indexes within a predetermined index range, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.

In an encoder and a decoder, motion compensation may be performed using a motion vector or motion information of a current block. At this time, at least one prediction block may be used to determine a final prediction block as a result of motion compensation. Here, the current block may mean at least one of a current encoding block and a current prediction block.

The final prediction block of the current block may be generated by performing overlapping block motion compensation on the boundary area of the current block.

The boundary area of the current block may be an area located within the current block and adjacent to a boundary between the current block and neighboring blocks of the current block. The boundary area of the current block may include at least one of an upper boundary area, a left boundary area, a lower boundary area, a right boundary area, an upper right area, a lower right area, an upper left area, and a lower left area. The boundary area of the current block may be an area corresponding to a portion of the prediction block of the current block.

The overlapped block motion compensation may mean a process of performing motion compensation by calculating a weighted sum of a prediction block corresponding to a boundary region of a current block and a prediction block generated using motion information of encoded/decoded blocks adjacent to the current block.

Calculation of the weighted sum may be performed sub-block by sub-block by dividing the current block into a plurality of sub-blocks. That is, motion compensation of the current block may be performed subblock by subblock using motion information of encoded/decoded subblocks adjacent to the current block. A sub-block may mean a lower-level block of a current block.

In addition, when calculating the weighted sum, a first prediction block generated for each sub-block of the current block using motion information of the current block, and a first prediction block generated using motion information of adjacent sub-blocks spatially adjacent to the current block may be used. Two prediction blocks. In this case, the expression "use motion information" means "deduce motion information". The first prediction block may mean a prediction block generated by using motion information of an encoding/decoding target sub-block within a current block. The second prediction block may be a prediction block generated by using motion information of adjacent subblocks spatially adjacent to the encoding/decoding target subblock within the current block.

A final prediction block of the current block may be generated using a weighted sum of the first prediction block and the second prediction block. That is, overlapping block motion compensation consists in finding the final prediction block of the current block using the motion information of the current block and the motion information of another block.

In addition, when at least one of Advanced Motion Vector Prediction (AMVP) mode, merge mode, affine motion compensation mode, decoder side motion vector derivation mode, adaptive motion vector resolution mode, local illumination compensation mode, bi-directional optical flow mode When used, the current block may be divided into sub-blocks, and overlapping block motion compensation may be performed on a sub-block basis.

When the merge mode is used for motion compensation, overlapping block motion compensation may be performed on at least one of an improved temporal motion vector predictor (ATMVP) candidate and a space-temporal motion vector predictor (STMVP) candidate.

Details of overlapping block motion compensation will be described later with reference to FIGS. 13 to 24 .

Next, a process of performing entropy encoding/entropy decoding on information associated with motion compensation ( S505 , S601 , S704 , S801 ) will be described.

The encoding device may entropy encode information associated with motion compensation to a bitstream, and the decoding device may decode information associated with motion compensation included in the bitstream. The information associated with motion compensation that is the target of entropy encoding or entropy decoding may include at least one of the following: inter prediction indicator inter_pred_idc, reference picture indices ref_idx_l0, ref_idx_l1, ref_idx_l2, and ref_idx_l3, motion vector candidate index mvp_l0_idx, mvp_l1_idx, mvp_l2_idx and mvp_l3_idx, motion vector difference, skip mode use/unuse information cu_skip_flag, merge mode use/unuse information merge_flag, merge index information merge_index, weight factors wf_l0, wf_l1, wf_l2 and wf_l3, and offset value offset_10, offset_11, offset_12, and offset_13.

The inter prediction indicator may mean a prediction direction of inter prediction, the number of prediction directions, or both of the prediction direction and the number of prediction directions of inter prediction when the current block is encoded/decoded by inter prediction. For example, an inter-prediction indicator may indicate uni-directional prediction or multi-directional prediction such as bi-prediction, tri-prediction, and quad-prediction. The inter prediction indicator may indicate the number of reference pictures used to generate the prediction block of the current block. Optionally, one reference picture can be used for prediction in multiple directions. In this case, M reference pictures are used to perform prediction in N directions (where N>M). The inter prediction indicator may also mean the number of prediction blocks used for inter prediction or motion compensation of the current block.

According to the number of prediction directions of the current block, the reference picture indicator may indicate one direction PRED_LX, two directions PRED_BI, three directions PRED_TRI, four directions PRED_QUAD, or more directions.

The prediction list utilization flag for a particular reference picture list indicates whether the reference picture list is used to generate the prediction block.

For example, when the prediction list utilization flag of a specific reference picture list has a first value of one (1), it means that the prediction block is generated using the reference picture list. When the prediction list utilization flag has a second value of zero (0), it means that the prediction block is not generated using the reference picture list. Here, the first and second values of the prediction list utilization flag may be set to 0 and 1, respectively, instead.

That is, when the prediction list utilization flag of a specific reference picture list has the first value, motion information corresponding to the reference picture list may be used to generate a prediction block of the current block.

The reference picture index may indicate a specific reference picture that exists in the reference picture list and is referred to by the current block. For each reference picture list, one or more reference picture indices may be entropy encoded/decoded. The current block may be motion compensated using one or more reference picture indices.

The motion vector candidate index indicates a motion vector candidate of a current block among motion vector candidates included in a motion vector candidate list prepared for each reference picture list or each reference picture index. At least one or more motion vector candidate indices may be entropy encoded/decoded for each motion vector candidate list. The current block may be motion compensated using at least one or more motion vector candidate indices.

The motion vector difference represents the difference between the current motion vector and the predicted motion vector. For each motion vector candidate list generated for each reference picture list or each reference picture index for the current block, one or more motion vector differences may be entropy encoded/decoded. The current block may be motion compensated using one or more motion vector differences.

Regarding the skip mode use/non-use information cu_skip_flag, when the skip mode use/non-use information cu_skip_flag has a first value of one (1), the skip mode may be used. On the contrary, when the skip mode use/non-use information cu_skip_flag has a second value of zero (0), the skip mode may not be used. Motion compensation may be performed on the current block using the skip mode according to the skip mode use/non-use information.

Regarding the merge mode use/non-use information merge_flag, when the merge mode use/non-use information merge_flag has a first value of one (1), the merge mode may be used. On the contrary, when the merge mode use/non-use information merge_flag has a second value of zero (0), the merge mode may not be used. Motion compensation may be performed on a current block using a merge mode according to merge mode use/non-use information.

The merge index information merge_index may mean information indicating merge candidates within the merge candidate list.

Alternatively, the merged index information may mean information about the merged index.

Also, the merge index information may indicate a reconstructed block used to derive a merge candidate among reconstructed blocks spatially/temporally adjacent to the current block.

Merge index information may indicate one or more pieces of motion information that a merge candidate has. For example, when the merge index information has a first value of zero (0), the merge index information may indicate the first merge candidate listed as the first entry in the merge candidate list; when the merge index information has a second value of one (1), , the merge index information may indicate the second merge candidate listed as the second entry in the merge candidate list; when the merge index information has a third value of two (2), the merge index information indicates that the merge candidate list is listed as the third entry Destination third merge candidate. Similarly, when the merge index information has values from the fourth value to the Nth value, the merge index information may indicate the merge candidates listed in the merge candidate list at positions according to the order of the values. Here, N can be 0 or a positive integer.

Motion compensation may be performed on the current block using a merge mode based on the merge index information.

When two or more prediction blocks are generated during motion compensation of the current block, a final prediction block of the current block may be determined based on a weighted sum of the prediction blocks. When calculating the weighted sum, a weight factor, an offset, or both a weight factor and an offset may be applied to each prediction block. The weighted sum factors (for example, weight factors and offsets) used to calculate the weighted sum may be entropy encoded/entropy decoded by an amount corresponding to at least one of the following items or may be entropy encoded by an amount corresponding to at least one of the following items Coding/entropy decoding: reference picture list, reference picture, motion vector candidate index, motion vector difference, motion vector, skip mode use/unuse information, merge mode use/unuse information, merge index information. In addition, the weight sum factor for each prediction block may be entropy encoded/entropy decoded based on the inter prediction indicator. The weighted sum factor may include at least one of a weight factor and an offset.

Information associated with motion compensation may be entropy encoded/decoded block by block, or may be entropy encoded/entropy decoded in units of higher-level units. For example, information associated with motion compensation may be entropy encoded/decoded on a block-by-block basis (eg, CTU-by-CTU, CU-by-CU, or PU-by-PU). Alternatively, information associated with motion compensation may be entropy encoded/decoded in units of higher-level units such as video parameter sets, sequence parameter sets, picture parameter sets, adaptation parameter sets, or slice headers.

The information associated with motion compensation may be entropy encoded/decoded based on a motion compensation information difference indicating a difference between the information associated with motion compensation and a predicted value of the information associated with motion compensation .

Information associated with motion compensation of already encoded/decoded blocks adjacent to the current block may be used as information associated with motion compensation of the current block instead of entropy encoding/ Entropy decoding.

At least one piece of information associated with motion compensation may be derived based on at least one of the encoding parameters.

At least one piece of information associated with motion compensation may be generated by decoding the bitstream based on at least one of the encoding parameters. Instead, at least one piece of information associated with motion compensation may be entropy encoded into the bitstream based on at least one of the encoding parameters.

At least one piece of information associated with motion compensation may include motion vector, motion vector candidate, motion vector candidate index, motion vector difference, motion vector predictor, skip mode use/unuse information skip_flag, merge mode use/unuse information merge_flag , merge index information merge_index, motion vector resolution information, overlapping block motion compensation information, local illumination compensation information, affine motion compensation information, decoder side motion vector derivation information, and bidirectional optical flow information. Here, decoder-side motion vector derivation may mean pattern-matched motion vector derivation.

The motion vector resolution information may be information indicating which specific resolution is used for at least one of the motion vector and the motion vector difference. Here, resolution may mean precision. The specified resolution can be set in 16-pixel (16-pel) units, 8-pixel (8-pel) units, 4-pixel (4-pel) units, integer-pixel (integer-pel) units, 1 /2-pixel (1/2-pel) unit, 1/4-pixel (1/4-pel) unit, 1/8-pixel (1/8-pel) unit, 1/16-pixel (1/16 -pel) unit, 1/32-pixel (1/32-pel) unit, and 1/64-pixel (1/64-pel) unit.

The overlapping block motion compensation information may be information indicating whether motion vectors of neighboring blocks spatially adjacent to the current block will be additionally used to calculate a weighted sum of prediction blocks of the current block during motion compensation of the current block.

The local illumination compensation information may be information indicating whether any one of a weight factor and an offset is applied when generating a prediction block of a current block. Here, at least one of the weight factor and the offset may be a value calculated based on the reference block.

The affine motion compensation information may be information indicating whether an affine motion model is used for motion compensation of the current block. Here, the affine motion model may be a model that divides one block into a plurality of sub-blocks using a plurality of parameters and calculates motion vectors of the sub-blocks using representative motion vectors.

The decoder-side motion vector derivation information may be information indicating whether a motion vector necessary for motion compensation is derived by the decoder and then used in the decoder. The information associated with the motion vector may not be entropy encoded/entropy decoded by deriving the information from the decoder side motion vector. When the decoder-side motion vector derivation information indicates that the motion vector is derived by the decoder and then used in the decoder, the information associated with the merge mode may be entropy encoded/entropy decoded. That is, the decoder-side motion vector derivation information may indicate whether merge mode is used in the decoder.

The bidirectional optical flow information may be information indicating whether a motion vector is modified pixel by pixel or subblock by subblock and then the modified motion vector is used for motion compensation. According to bidirectional optical flow information, motion vectors may not be entropy encoded/entropy decoded pixel by pixel or subblock by subblock. The modification of the motion vector means converting the value of the block-based motion vector to the value of the pixel-based motion vector or the value of the sub-block based motion vector.

Motion compensation may be performed on the current block based on at least one piece of information associated with motion compensation, and entropy encoding/decoding may be performed on the at least one piece of information associated with motion compensation.

When information associated with motion compensation is entropy encoded/entropy decoded, a binarization method such as a truncated Rice binarization method, a K-order exponential Golomb binarization method, a limited K-order exponential Golomb binarization method may be used , fixed-length binarization method, unary binarization method, and truncated unary binarization method.

When information associated with motion compensation is entropy encoded/entropy decoded, the context model may be determined based on at least one of information associated with motion information of neighboring blocks adjacent to the current block or region information of neighboring blocks information; previously encoded/decoded information associated with motion compensation or previously encoded/decoded region information; information about the depth of the current block; information about the size of the current block.

Alternatively, when information associated with motion compensation is entropy encoded/entropy decoded, entropy encoding/decoding may be performed by using at least one of the following information as a predictive value of information associated with motion compensation of the current block Entropy decoding: information associated with motion compensation of neighboring blocks, previously encoded/decoded information associated with motion compensation, information about the depth of the current block, and information about the size of the current block.

Hereinafter, details of overlapping block motion compensation may be described with reference to FIGS. 13 to 24 .

FIG. 13 is a diagram illustrating an example of performing overlapping block motion compensation on a sub-block basis.

Referring to FIG. 13 , shaded blocks are regions where overlapping block motion compensation will be performed. The shaded block may include a sub-block located on a boundary of the current block or a sub-block in the current block. An area marked by a thick solid line may be the current block.

Arrows indicate that motion information of neighboring sub-blocks is used for motion compensation of the current sub-block. Here, the area where the tail of the arrow is located may mean (1) a neighboring sub-block adjacent to the current block or a neighboring sub-block adjacent to the current sub-block within the current block. The area where the head of the arrow is located may mean the current sub-block within the current block.

For shaded blocks, a weighted sum of the first prediction block and the second prediction block may be calculated. Motion information of a current sub-block within the current block may be used as motion information for generating the first prediction block. At least one or both of motion information of adjacent subblocks adjacent to the current block and motion information of adjacent subblocks adjacent to the current subblock and included in the current block may be used as a parameter for generating the second prediction block. Sports information.

In addition, in order to improve coding efficiency, the motion information used to generate the second predicted block may include the upper block, left block, lower block, right block, upper right block, lower right block, and upper left block of the current sub-block in the current block and motion information of at least one of the lower left blocks. Neighboring subblocks that can be used to generate the second prediction block may be determined according to the location of the current subblock. For example, when the current subblock is located at the upper boundary of the current block, at least one adjacent subblock among adjacent subblocks located on the upper side, upper right side, and upper left side of the current subblock may be used. When the current subblock is located at the left boundary of the current block, at least one adjacent subblock among adjacent subblocks located on the left, upper left, and lower left of the current subblock may be used.

Here, blocks located on the upper side, left side, lower side, right side, upper right side, lower right side, upper left side, and lower left side of the current sub-block may be called upper adjacent sub-blocks, left adjacent sub-blocks, and lower adjacent sub-blocks, respectively. subblock, right adjacent subblock, upper right adjacent subblock, lower right adjacent subblock, upper left adjacent subblock, and lower left adjacent subblock.

In addition, in order to reduce the computational complexity, the motion information used to generate the second prediction block can be determined according to the size of the motion vector of the adjacent sub-block adjacent to the current block or the adjacent sub-block adjacent to the current sub-block within the current block. Variety.

For example, when the neighboring subblock is a bidirectionally predicted subblock, the magnitudes of the motion vector in the L0 direction and the L1 direction are compared, and only the motion information of the larger magnitude direction may be used to generate the second prediction block.

Alternatively, for example, the sum of the absolute values of the x and y components in the L0 direction of the motion vector and the sum of the absolute values of the x and y components in the L1 direction of the motion vector are calculated. Then, only motion vectors equal to or greater than a predetermined value may be used to generate the second prediction block. Here, the predetermined value may be 0 or a positive integer. The predetermined value may be a value determined based on information signaled from the encoder to the decoder. Alternatively, the predetermined value may not be signaled, but may be a value set identically in the encoder and decoder.

In addition, in order to reduce computational complexity, the motion information used to generate the second prediction block may vary according to the magnitude and direction of the motion vector of the current sub-block.

For example, the absolute values of the x-component and y-component of the motion vector of the current sub-block may be compared. When the absolute value of the x component is large, motion information of at least one of the left and right subblocks of the current subblock may be used to generate the second prediction block.

Optionally, for example, the absolute values of the x-component and y-component of the motion vector of the current sub-block may be compared. When the absolute value of the y component is large, motion information of at least one of the upper subblock and the lower subblock of the current subblock may be used to generate the second prediction block.

Optionally, for example, when the absolute value of the x component of the motion vector of the current subblock is equal to or greater than a predetermined value, motion information of at least one of the left subblock and the right subblock of the current subblock may be used to generate the second Two prediction blocks. Here, the predetermined value may be zero (0) or a positive integer. The predetermined value may be determined based on information signaled from the encoder to the decoder, or may be set identically in the encoder and decoder.

Further optionally, for example, when the absolute value of the y component of the motion vector of the current subblock is equal to or greater than a predetermined value, the motion information of at least one of the upper subblock and the lower subblock of the current subblock may be used to generate the second prediction block. Here, the predetermined value may be zero (0) or a positive integer. The predetermined value may be determined based on information signaled from the encoder to the decoder, or may be set identically in the encoder and decoder.

Here, the sub-block may have a size of N×M, where N and M are positive integers. N and M may or may not be equal. For example, the size of a sub-block may be 4x4 or 8x8. The information of the size of the sub-block may be entropy-encoded/entropy-decoded at a sequence unit level.

The size of the sub-block may be determined based on the size of the current block. For example, when the size of the current block is K samples or less, the size of the sub-block may be 4×4. In addition, when the size of the current block is greater than K samples, the size of the sub-block may be 8×8. Here, K is a positive integer such as 256.

Here, the information of the size of the sub-block may be entropy-encoded/entropy-decoded in units of at least any one of sequence, picture, slice, tile, CTU, CU, and PU. In addition, the size of the sub-block may be a predetermined size preset in the encoder and the decoder.

A sub-block may have a square shape or a rectangular shape. For example, when the current block has a square shape or a rectangular shape, the sub-block may have a square shape.

For example, when the current block has a rectangular shape, the sub-block may have a rectangular shape.

Here, the information of the shape of the sub-block may be entropy-encoded/entropy-decoded in units of at least one of sequence, picture, slice, tile, CTU, CU, and PU. In addition, the shape of the sub-block may be a predetermined shape preset in the encoder and the decoder.

FIG. 14 is a diagram illustrating an example of performing overlapping block motion compensation using motion information of a sub-block of a co-located block. In order to improve coding efficiency, the motion information of the co-located sub-block spatially co-located with the current block in the co-located picture or the reference picture can be used to generate the second prediction block.

Referring to FIG. 14 , motion information of a subblock temporally adjacent to a current block within a co-located block may be used to perform overlapping block motion compensation on the current subblock. The area where the tail of the arrow is located may be a sub-block within the co-located block. The area where the head of the arrow is located may be a current sub-block within the current block.

In addition, motion information of at least one of the co-located sub-blocks in the co-located picture, the spatially adjacent sub-blocks adjacent to the current block, and the spatially adjacent sub-blocks within the current block and the current sub-block may be used to generate the second prediction block.

FIG. 15 is a diagram illustrating an example of performing overlapping block motion compensation using motion information of blocks adjacent to a boundary of a reference block. In order to improve coding efficiency, a reference block in a reference picture can be identified by using at least one of the motion vector of the current block and the reference picture index, and the motion information of adjacent blocks adjacent to the boundary of the identified reference block can be used to generate the second prediction block. Here, the adjacent block may include an encoded/decoded block adjacent to a sub-block of the reference block located at a right boundary or a left boundary.

Referring to FIG. 15 , motion information of an encoded/decoded block adjacent to a lower boundary or a right boundary of a reference block may be used for overlapping block motion compensation for a current subblock.

In addition, the motion information of the coded/decoded block adjacent to the lower boundary or the right boundary of the reference block, the motion information of the adjacent sub-blocks spatially adjacent to the current block, and the spatial distance between the current block and the current sub-block At least one motion information among the motion information of adjacent adjacent sub-blocks may be used to generate the second prediction block.

In order to improve encoding efficiency, motion information of at least one merging candidate among a plurality of merging candidates included in the merging candidate list may be used to generate a second prediction block. Here, the merge candidate list may be a list used in a merge mode among a plurality of inter prediction modes.

For example, spatial merging candidates in the merging candidate list may be used as motion information for generating the second prediction block.

Alternatively, for example, temporal merging candidates in the merging candidate list may be used as motion information for generating the second prediction block.

Further optionally, for example, combined merging candidates in the merging candidate list may be used as motion information for generating the second prediction block.

Optionally, in order to improve encoding efficiency, at least one motion vector among a plurality of motion vector candidates included in the motion vector candidate list may be used as a motion vector for generating the second prediction block. Here, the motion vector candidate list may be a list used in AMVP mode among a plurality of inter prediction modes.

For example, spatial motion vector candidates in the motion vector candidate list may be used as motion information for generating the second prediction block.

Alternatively, for example, temporal motion vector candidates in the motion vector candidate list may be used as motion information for generating the second prediction block.

When at least one of the merge candidate and the motion vector candidate is used as motion information required to generate the second prediction block, a region to which the overlapping block motion compensation is applied may be set differently. The area to which overlapping block motion compensation is applied can be an area of the block that is adjacent to the boundary (i.e., a sub-block of the block that lies on the boundary) or an area of the block that is not adjacent to the boundary (i.e., a sub-block that is not located on the boundary of the block). piece).

When overlapping block motion compensation is applied to a region of a block that is not adjacent to a boundary, at least one of a merge candidate and a motion vector candidate may be used as motion information required to generate a second prediction block.

For example, overlapping block motion compensation may be performed for an area of a block that is not adjacent to a boundary by using a spatial merging candidate or a spatial motion vector candidate as motion information.

Alternatively, overlapping block motion compensation may be performed for an area of a block that is not adjacent to a boundary, for example, by using a temporal merging candidate or a temporal motion vector candidate as motion information.

Further alternatively, for example, overlapping block motion compensation may be performed for an area of a block adjacent to a lower boundary or a right boundary by using a spatial merging candidate or a spatial motion vector candidate as motion information.

Further alternatively, for example, overlapping block motion compensation may be performed for an area of a block adjacent to a lower boundary or a right boundary by using a temporal merging candidate or a temporal motion vector candidate as motion information.

In addition, in order to improve encoding efficiency, motion information derived from a specific block within a merge candidate list or a motion vector candidate list may be used for overlapping block motion compensation for a specific region.

For example, when the motion information of the upper right neighboring block of the current block is included in the merge candidate list or the motion vector candidate list, the motion information may be used for overlapping block motion compensation of the right boundary area of the current block.

Optionally, for example, when the motion information of the lower left neighboring block of the current block is included in the merge candidate list or the motion vector candidate list, the motion information may be used for overlapping block motion compensation of the lower boundary area of the current block.

FIG. 16 is a diagram illustrating an example of performing overlapping block motion compensation on a subblock group basis. In order to reduce computational complexity, subblock-based overlapping block motion compensation may be performed in units of subblock sets including one or more subblocks. A unit of a subchunk set may mean a unit of a subchunk group.

Referring to FIG. 16, hatched areas demarcated with lines may be referred to as sub-block groups. Arrows mean that the motion information of adjacent adjacent sub-blocks can be used for motion compensation of the current sub-block group. The area where the tail of the arrow is located can be (1) a neighboring subblock adjacent to the current block, (2) a group of neighboring subblocks adjacent to the current block, or (3) a group of adjacent subblocks adjacent to the current block within the current block neighboring subblocks. In addition, the area where the head of the arrow is located may mean the current sub-block group within the current block.

For each subblock group, a weighted sum of the first prediction block and the second prediction block may be calculated. Motion information of a current subblock group within a current block may be used as motion information for generating the first prediction block. Here, the motion information of the current sub-block group in the current block may be any one of the average value, median value, minimum value, maximum value and weighted sum of the motion information of each sub-block in the current sub-block group. At least one of motion information of adjacent subblocks adjacent to the current block, motion information of adjacent subblock groups adjacent to the current block, and motion information of adjacent subblocks within the current block adjacent to the current subblock may be Used as motion information for generating the second prediction block. Here, the motion information of the adjacent sub-block group adjacent to the current block may be any one of the average value, median value, minimum value, maximum value, and weighted sum of the motion information of each sub-block included in the adjacent sub-block group. one.

Here, the current block may include one or more sub-block groups. The horizontal size of a subblock group may be equal to or smaller than the horizontal size of a current subblock. In addition, the vertical size of a subblock group may be equal to or smaller than the vertical size of a current subblock. In addition, overlapping block motion compensation may be performed on at least one sub-block of a plurality of sub-blocks located at the upper boundary and the left boundary of the current block.

Since blocks adjacent to the lower boundary and the right boundary within the current block have not been encoded/decoded, overlapping block motion compensation may not be performed on at least one subblock among the plurality of subblocks located at the lower boundary and the right boundary within the current block. Optionally, since the blocks adjacent to the lower boundary and the right boundary in the current block have not been coded/decoded, the motion information of the upper block of the current sub-block, the motion information of the left block, and the motion information of the upper left block can be used. information, motion information of the lower left block, and motion information of the upper right block, perform overlapping block motion compensation for at least any one of the plurality of sub-blocks located at the left boundary and the right boundary in the current block.

In addition, when the current block is to be predicted in merge mode and has at least one of an improved temporal motion vector prediction candidate and a space-temporal motion vector prediction candidate, for a plurality of sub-blocks located at the lower boundary and the right boundary within the current block At least one sub-block of , overlapping block motion compensation may not be performed.

In addition, when the current block is to be predicted in the decoder-side motion vector derivation mode or the affine motion compensation mode, for at least one sub-block among the plurality of sub-blocks located at the lower boundary and the right boundary within the current block, overlapping blocks may not be performed motion compensation.

In addition, overlapping block motion compensation may be performed for at least one of color components of the current block. The color components may include at least one of a luma component and a chroma component.

Optionally, overlapping block motion compensation may be performed according to the inter prediction indicator of the current block. That is, overlapping block motion compensation may be performed when the current block is to be uni-predicted, bi-predicted, tri-predicted, and/or quad-predicted. Alternatively, overlapping block motion compensation may be performed only when the current block is unidirectionally predicted. Further optionally, overlapping block motion compensation may be performed only when the current block is bi-predicted.

FIG. 17 is a diagram illustrating an example of pieces of motion information used for overlapping block motion compensation.

The maximum number of pieces of motion information for generating the second prediction block may be K. That is, a maximum of K second prediction blocks can be generated and used for overlapping block motion compensation. Here, K may be zero (0) or a positive integer such as 1, 2, 3 or 4.

For example, when the second prediction block is generated using motion information of neighboring sub-blocks adjacent to the current block, at most two pieces of motion information may be derived from at least one of the upper block and the right block. When the second prediction block is generated based on the motion information of adjacent sub-blocks adjacent to the current sub-block in the current block, the upper block, left block, right block, upper-left block, upper-right block, and lower-left block of the current sub-block can be selected. At least one of the square and the bottom right square derives up to four pieces of motion information. Here, the expression "deriving motion information" may mean a process of generating a second prediction block using the derived motion information and then performing overlapping block motion compensation using the generated second prediction block.

Referring to FIG. 17 , in order to improve encoding efficiency, when motion compensation is performed on at least one of a plurality of subblocks located at an upper boundary and a left boundary within a current block, up to three pieces of motion information may be derived for generating a second prediction block. That is, motion information for generating the second prediction block may be derived based on 3-connection.

For example, when motion compensation is performed on a subblock located on the upper boundary within the current block, it may be derived from at least one of the upper adjacent block, the upper left adjacent block, and the upper right adjacent block among the adjacent subblocks adjacent to the current block. Sports information.

For example, when motion compensation is performed on a subblock located at the left boundary within the current block, at least one of the left adjacent block, the upper left adjacent block, and the lower left adjacent block among the adjacent subblocks adjacent to the current block may be Deriving motion information.

In addition, when motion compensation is performed on a sub-block located at the upper-left boundary within the current block, it may be derived from at least one of an upper adjacent block, a left adjacent block, and an upper-left adjacent block among adjacent sub-blocks adjacent to the current block. Sports information.

In addition, when motion compensation is performed on a subblock located at the upper right boundary within the current block, it may be derived from at least one of an upper adjacent block, an upper left adjacent block, and an upper right adjacent block among adjacent subblocks adjacent to the current block. Sports information.

In addition, when motion compensation is performed on a sub-block located at the lower-left boundary within the current block, at least one of a left adjacent block, an upper-left adjacent block, and a lower-left adjacent block among adjacent sub-blocks adjacent to the current block may be Deriving motion information.

Optionally, in order to improve coding efficiency, when motion compensation is performed on a sub-block in the current block that is not at least one of the sub-blocks located on the upper boundary and the left boundary, a maximum of 8 pieces of motion information can be derived to generate the first Two prediction blocks. That is, motion information for generating the second prediction block may be derived based on 8-connection.

For example, for a sub-block within the current block, it can be selected from the upper adjacent block, the left adjacent block, the lower adjacent block, the right adjacent block, the upper left At least one of the neighboring block, the lower left neighboring block, the lower right neighboring block, and the upper right neighboring block derives motion information.

Additionally, the motion information used to generate the second predictive block may be derived from co-located sub-blocks within the co-located picture. In addition, motion information for generating the second prediction block may be derived from encoded/decoded blocks adjacent to the lower boundary and the right boundary of the reference block within the reference picture.

In addition, in order to improve encoding efficiency, the number of pieces of motion information used to generate the second prediction block may be determined according to the magnitude and direction of the motion vector.

For example, when the sum of the absolute values of the x-component and y-component of the motion vector is equal to or greater than J, up to L pieces of motion information can be used. On the contrary, when the sum of the absolute values of the x component and the y component of the motion vector is less than J, a maximum of P pieces of motion information can be used. In this case, J, L and P are zero or positive integers. L and P are preferably different values. However, L and P may be equal to each other.

In addition, when the current block is to be predicted in merge mode and when at least one of the improved temporal motion vector prediction candidate and the space-temporal motion vector prediction candidate is used, up to K pieces of motion information may be used to generate the second prediction piece. Here, K can be zero or a positive integer, such as 4.

In addition, when the current block is to be predicted in the decoder-side motion vector derivation mode, at most K pieces of motion information may be used to generate the second prediction block. Here, K can be zero or a positive integer, such as 4.

In addition, when the current block is to be predicted in the affine motion compensation mode, up to K pieces of motion information may be used to generate the second prediction block. Here, K can be zero or a positive integer, such as 4.

18 and 19 are diagrams illustrating an order of deriving motion information for generating a second prediction block. Motion information for generating the second prediction block may be derived in a predetermined order preset in the encoder and the decoder.

Referring to FIG. 18 , motion information may be derived from neighboring blocks of a current block in the order of an upper block, a left block, a lower block, and a right block.

Referring to FIG. 19 , in order to improve encoding efficiency, an order of deriving motion information for generating a second prediction block may be determined based on a position of a current subblock.

For example, when deriving the motion information for the current sub-block located on the upper boundary within the current block, the motion information can be derived according to (1) the upper adjacent block, (2) the upper-left adjacent block, and (3) the adjacent sub-blocks adjacent to the current block. The sequence of top right neighbors derives motion information from neighboring sub-blocks.

In addition, when deriving motion information for a current subblock located at the left boundary within the current block, it may be based on (1) the left adjacent block, (2) the upper left adjacent block, and (3) the adjacent subblocks adjacent to the current block. ) order to the lower left neighboring block derives motion information from neighboring sub-blocks.

In addition, when the motion information is derived for the current sub-block located at the upper-left boundary within the current block, it may be based on (1) the upper adjacent block, (2) the left adjacent block, and (3) the adjacent sub-blocks adjacent to the current block. The order of the top-left neighbors derives motion information from neighboring sub-blocks.

In addition, when deriving the motion information for the current subblock located at the upper right boundary within the current block, it can be based on (1) the upper adjacent block, (2) the upper left adjacent block, and (3) the adjacent subblocks adjacent to the current block. The sequence of top right neighbors derives motion information from neighboring sub-blocks.

In addition, when the motion information is derived for the current sub-block located at the lower right boundary of the current block, it can be derived according to (1) the left adjacent block, (2) the upper left adjacent block and ( 3) The order of the lower left neighboring blocks derives motion information from neighboring sub-blocks.

As in the example of FIG. 19 , the following subblocks adjacent to the current subblock can be selected in order of (1) upper adjacent block, (2) left adjacent block, (3) lower adjacent block, (4) right adjacent block block, (5) upper-left adjacent block, (6) lower-left adjacent block, (7) lower-right adjacent block, and (8) upper-right adjacent block to derive the motion information of the current sub-block within the current block. Alternatively, motion information may be derived in an order different from that shown in FIG. 19 .

On the other hand, motion information of a collocated sub-block within a co-located picture may be subsequently derived after motion information of neighboring sub-blocks spatially adjacent to the current sub-block is derived. Optionally, the motion information of the co-located sub-blocks within the co-located picture may be derived before deriving the motion information of neighboring sub-blocks spatially adjacent to the current sub-block.

In addition, motion information of encoded/decoded blocks located at the lower and right boundaries of the reference block within the reference picture may be subsequently derived after the motion information of neighboring sub-blocks spatially adjacent to the current sub-block is derived. Optionally, the motion information of encoded/decoded blocks located at the lower and right boundaries of the reference block within the reference picture may be derived before deriving the motion information of neighboring sub-blocks spatially adjacent to the current sub-block.

Only when a predetermined condition is satisfied, motion information of neighboring subblocks adjacent to the current block or motion information of neighboring subblocks adjacent to the current subblock may be derived as motion information for generating the second prediction block.

For example, when at least one of adjacent subblocks adjacent to the current block or adjacent subblocks adjacent to the current subblock within the current block exists, motion information of the existing adjacent subblocks may be derived as 2. Predict the motion information of the block.

Further optionally, for example, when at least one of the adjacent sub-blocks adjacent to the current block or the adjacent sub-blocks within the current block is predicted in the inter-frame prediction mode, the inter-frame prediction is used Motion information of the predicted neighboring sub-blocks may be derived as motion information for generating the second predicted block. In addition, when at least one of the adjacent subblocks adjacent to the current block or the adjacent subblocks within the current block is predicted in the intra prediction mode, the adjacent subblocks predicted in the intra prediction mode Motion information for a block may not be derived as motion information for generating the second prediction block because the neighboring sub-block does not have motion information.

In addition, when the inter prediction indicator of at least one of the adjacent subblocks adjacent to the current block or the adjacent subblocks adjacent to the current subblock within the current block does not indicate L0 prediction, L1 prediction, L2 prediction, L3 prediction When at least one of unidirectional prediction, bidirectional prediction, tridirectional prediction and quadridirectional prediction, motion information for generating the second prediction block may not be derived.

Also, when the inter prediction indicator used to generate the second prediction block is different from the inter prediction indicator used to generate the first prediction block, motion information for generating the second prediction block may be derived.

Also, when the motion vector used to generate the second prediction block is different from the motion vector used to generate the first prediction block, motion information required to generate the second prediction block may be derived.

In addition, when the reference picture index used to generate the second prediction block is different from the reference picture index used to generate the first prediction block, motion information required to generate the second prediction block may be derived.

In addition, when at least one of the motion vector and the reference picture index used to generate the second predictive block is different from at least one of the motion vector and the reference picture index used to generate the first predictive block, generating the second predictive block may be derived desired exercise information.

In addition, in order to reduce computational complexity, in the case where the inter prediction indicator used to generate the first prediction block indicates unidirectional prediction, when the motion vector sum of the L0 prediction direction and the L1 prediction direction used to generate the second prediction block When at least one of the reference picture index is different from at least one of the motion vector and the reference picture index used to generate the first prediction block, motion information required to generate the second prediction block may be derived.

In addition, in order to reduce the computational complexity, based on the inter prediction indicator used to generate the first prediction block, in the case where the inter prediction indicator indicates bi-directional prediction, when the L0 prediction direction and the L1 prediction direction used to generate the second prediction block When at least one set of motion vectors and reference picture indexes in the prediction direction is different from at least one set of motion vectors and reference picture indexes in the L0 prediction direction and L1 prediction direction used to generate the first prediction block, the second prediction block can be derived desired exercise information.

In addition, in order to reduce computational complexity, when at least one piece of motion information used to generate the second predicted block is different from at least one piece of motion information used to generate the first predicted block, motion information required to generate the second predicted block may be derived.

FIG. 20 is a diagram illustrating determining whether motion information of a specific neighboring sub-block can be used as an index for generating a second prediction block by comparing the POC of the reference picture of the current sub-block with the POC of the reference picture of the specific neighboring sub-block. A diagram of an example of motion information.

Referring to FIG. 20 , in order to reduce computational complexity, when the POC of the reference picture of the current subblock is equal to the POC of the reference picture of the neighboring subblock, the motion information of the neighboring subblock may be used to generate the second prediction block of the current subblock.

In addition, in order to reduce computational complexity, as in the example of FIG. 20 , when the POC of the reference picture used to generate the second prediction block is different from the POC of the reference picture used to generate the first prediction block, the generation of the second prediction block can be derived. Motion information needed to predict a block.

Specifically, when the POC of the reference picture used to generate the second prediction block is different from the POC of the reference picture used to generate the first prediction block, it can be used to generate the first prediction block based on the reference picture or the POC pair of the reference picture The motion vector for generating the second prediction block is derived by scaling the motion vector.

FIG. 21 is a diagram illustrating an example of using a weight factor when calculating a weighted sum of a first prediction block and a second prediction block.

When calculating the weighted sum of the first prediction block and the second prediction block, different weighting factors may be applied to the samples in the block used according to their position in the block. In addition, a weighted sum of samples co-located in the first prediction block and the second prediction block may be calculated. In this case, when the weighted sum is calculated to produce the final prediction block, at least one of the weight factor and the offset may be used in the calculation.

Here, the weighting factor may be a negative value less than zero or a positive value greater than zero. The offset can be zero, a negative value less than zero, or a positive value greater than zero.

When the weighted sum of the first prediction block and the second prediction block is calculated, the same weight factor may be applied to all samples in each prediction block.

Referring to FIG. 21, for example, weighting factors {3/4, 7/8, 15/16, and 31/32} may be applied to respective rows or columns of the first prediction block, and weighting factors {1/4, 1 /8, 1/16 and 1/32} apply to each row or column of the second prediction block. In this case, samples in the same row or the same column may be applied with the same weighting factor.

The value of the weighting factor increases as the distance from the boundary of the current sub-block decreases. Additionally, weighting factors may be applied to all samples within a sub-block.

In Fig. 21, (a), (b), (c) and (d) show that by using the motion information of the upper neighboring block, the motion information of the lower neighboring block, the motion information of the left neighboring block and the right neighboring block respectively, The motion information of neighboring blocks is used to generate the second prediction block. Here, the upper second prediction block, the lower second prediction block, the left second prediction block, and the right second prediction block may mean motion information based on the upper adjacent block, the motion information of the lower adjacent block, and the left adjacent block, respectively. The second prediction block generated by the motion information of the right adjacent block and the motion information of the right adjacent block.

FIG. 22 is a diagram illustrating an embodiment in which different weight factors are applied to samples according to positions in a block when calculating a weighted sum of a first prediction block and a second prediction block. In order to improve coding efficiency, when the weighted sum of the first prediction block and the second prediction block is calculated, the weight factor can be changed according to the positions of the samples in the block. That is, the weighted sum may be calculated using weight factors that differ according to positions of samples spatially adjacent to the current subblock. In addition, a weighted sum may be calculated for co-located samples in the first prediction block and the second prediction block.

Referring to FIG. 22, in the first prediction block, weighting factors {1/2, 3/4, 7/8, 15/16, 31/32, 63/64, 127/128, 255/256, 511/512 and 1023/ 1024} applied to each sample point, and in the second prediction block, weighting factors {1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128 can be applied according to the position , 1/256, 1/512 and 1/1024} are applied to each sample point. Here, the weighting factor used in at least one of the upper second prediction block, the left second prediction block, the lower second prediction block, and the right second prediction block may be larger than that used in the upper left second prediction block, the lower left prediction block, and the lower left prediction block. A weighting factor used in at least one of the second prediction block, the second lower right prediction block, and the second upper right prediction block.

Also, the weighting factor used in at least one of the upper second prediction block, left second prediction block, lower second prediction block, and right second prediction block may be equal to that used in the upper left second prediction block, lower left second prediction block A weighting factor used in at least one of the second prediction block, the second lower right prediction block, and the second upper right prediction block.

In addition, the weighting factors of all samples in the second prediction block generated by using the motion information of the co-located sub-blocks in the co-located picture may be equal.

In addition, the weight factor of the samples in the second prediction block generated by using the motion information of the co-located sub-blocks in the co-located picture may be equal to the weight factor of the samples in the first prediction block.

In addition, weight factors for all samples in the second prediction block generated using motion information of encoded/decoded blocks adjacent to the lower and right boundaries of the reference block within the reference picture may be equal.

In addition, the weight factor of the samples in the second prediction block generated using the motion information of the coded/decoded blocks adjacent to the lower boundary and the right boundary of the reference block in the reference picture may be equal to the weight factor of the samples in the first prediction block. Point weighting factor.

In order to reduce computational complexity, the weighting factor may vary according to the magnitude of the motion vectors of adjacent subblocks adjacent to the current block or adjacent subblocks adjacent to the current subblock within the current block.

For example, {1/2, 3/4, 7/8, 15/16} may be used as the current sub-block when the sum of the absolute values of the x-component and y-component of the motion vector of the adjacent sub-block is equal to or greater than a predetermined value. The weighting factor for the block. On the contrary, when the sum of the absolute values of the x component and the y component of the motion vector of the neighboring subblock is less than the predetermined value, {7/8, 15/16, 31/32, 63/64} can be used as the current subblock weight factor. In this case, the predetermined value may be zero or a positive integer.

In addition, in order to reduce the computational complexity, the weight factor may vary according to the magnitude or direction of the motion vector of the current sub-block.

For example, when the absolute value of the x component of the motion vector of the current subblock is equal to or greater than a predetermined value, {1/2, 3/4, 7/8, 15/16} can be used as the left adjacent subblock and the right Weighting factor for side adjacent subblocks. On the contrary, when the absolute value of the x component of the motion vector of the current subblock is smaller than the predetermined value, {7/8, 15/16, 31/32, 63/64} can be used as the left adjacent subblock and the right Weighting factor for neighboring subblocks. In this case, the predetermined value may be zero or a positive integer.

For example, when the absolute value of the y component of the motion vector of the current subblock is equal to or greater than a predetermined value, {1/2, 3/4, 7/8, 15/16} can be used as the upper and lower adjacent subblocks Weight factor for subblocks. On the contrary, when the absolute value of the y component of the motion vector of the current subblock is smaller than the predetermined value, {7/8, 15/16, 31/32, 63/64} can be used as the upper and lower adjacent subblocks The weighting factor for the block. In this case, the predetermined value may be zero or a positive integer.

For example, {1/2, 3/4, 7/8, 15/16} may be used as the current sub-block when the sum of the absolute values of the x-component and y-component of the motion vector of the current sub-block is equal to or greater than a predetermined value. The weighting factor for the block. On the contrary, when the sum of the absolute values of the x component and the y component of the motion vector of the current subblock is less than the predetermined value, {7/8, 15/16, 31/32, 63/64} can be used as the current subblock weight factor. In this case, the predetermined value may be zero or a positive integer.

The weighted sum may not be calculated for all samples within a sub-block, but only for some samples in the K rows/columns adjacent to each block boundary. In this case, K can be zero or a positive integer such as 1 or 2.

In addition, when the size of the current block is smaller than N×M, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary. Also, when a current block is divided into sub-blocks and motion compensation is performed on a sub-block basis, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary. Here, K can be zero or a positive integer such as 1 or 2. Also, N and M may be positive integers. For example, N and M may be 4 or greater and 8 or greater. N and M may or may not be equal.

Optionally, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the color component type of the current block. In this case, K can be zero or a positive integer such as 1 or 2. When the current block is a luma component block, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary. On the other hand, when the current block is a chroma component block, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary.

In addition, when the current block is to be predicted in merge mode and has at least one of improved temporal motion vector prediction candidates and space-temporal motion vector prediction candidates, only K rows/columns adjacent to each block boundary can be Calculate the weighted sum of the samples in .

In addition, when the current block is to be predicted in decoder-side motion vector derivation mode, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary. In addition, when the current block is to be predicted in affine motion compensation mode, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary. In these cases, K can be zero or a positive integer, such as 1 or 2.

In addition, in order to reduce computational complexity, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the sub-block size of the current block.

For example, when a sub-block of the current block has a size of 4x4, a weighted sum may be calculated for samples in one, two, three or four rows/columns adjacent to each block boundary. Optionally, when the sub-block of the current block has a size of 8×8, one, two, three, four, five, six, seven or eight adjacent to each block boundary can be Samples in row/column compute weighted sum. In this case, K can be zero or a positive integer. The maximum value of K may correspond to the number of rows or columns included in a subblock.

In addition, to reduce computational complexity, a weighted sum may be calculated for samples in one or two rows/columns adjacent to each block boundary within a sub-block.

In addition, in order to reduce computational complexity, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the number of pieces of motion information used to generate the second prediction block. Here, K may be zero or a positive integer.

For example, when the number of pieces of motion information is less than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary.

In addition, when the number of pieces of motion information is equal to or greater than the predetermined value, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary.

In addition, in order to reduce computational complexity, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the inter prediction indicator of the current block. K can be zero or a positive integer.

For example, when the inter prediction indicator indicates unidirectional prediction, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary. Furthermore, when the inter-prediction indicator indicates bi-prediction, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary.

In addition, in order to reduce computational complexity, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the POC of the reference picture of the current block. Here, K may be zero or a positive integer.

For example, when the difference between the POC of the current picture and the POC of the reference picture is less than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary. On the contrary, when the difference between the POC of the current picture and the POC of the reference picture is equal to or greater than the predetermined value, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary.

In addition, in order to reduce the computational complexity, according to the size of the motion vector of the adjacent sub-block adjacent to the current block or the motion vector of the adjacent sub-block adjacent to the current sub-block within the current block, for each block boundary The weighted sum is calculated for the sample points in the adjacent K rows/columns. Here, K may be zero or a positive integer.

For example, when the sum of the absolute values of x-components and y-components of motion vectors of adjacent sub-blocks is equal to or greater than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary. On the contrary, when the sum of the absolute values of the x-components and y-components of the motion vectors of adjacent sub-blocks is smaller than the predetermined value, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary. In this case, the predetermined value may be zero or a positive integer.

In addition, in order to reduce computational complexity, a weighted sum may be calculated for samples in K rows/columns adjacent to each block boundary according to the magnitude or direction of the motion vector of the current sub-block. Here, K may be zero or a positive integer.

For example, when the absolute value of the x component of the motion vector of the current subblock is equal to or greater than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each of the left and right boundaries. On the contrary, when the absolute value of the x component of the motion vector of the current sub-block is smaller than the predetermined value, a weighted sum may be calculated for samples in one row/column adjacent to each of the left boundary and the right boundary. In this case, the predetermined value may be zero or a positive integer.

For example, when the absolute value of the y component of the motion vector of the current subblock is equal to or greater than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each of the upper boundary and the lower boundary. On the contrary, when the absolute value of the y component of the motion vector of the current sub-block is smaller than the predetermined value, a weighted sum may be calculated for samples in a row/column adjacent to the upper boundary and the lower boundary. In this case, the predetermined value may be zero or a positive integer.

For example, when the sum of the absolute values of the x-component and y-component of the motion vector is equal to or greater than a predetermined value, a weighted sum may be calculated for samples in two rows/columns adjacent to each block boundary. Conversely, when the sum of the absolute values of the x-component and y-component of the motion vector is smaller than the predetermined value, a weighted sum may be calculated for samples in one row/column adjacent to each block boundary. In this case, the predetermined value may be zero or a positive integer.

FIG. 23 is a diagram illustrating an embodiment in which a weighted sum of a first prediction block and a second prediction block is sequentially and cumulatively calculated in a predetermined order during overlapping block motion compensation. The weighted sums of the first prediction block and the second prediction block may be added in a predetermined order preset in the encoder and the decoder.

Referring to FIG. 23 , motion information may be derived from adjacent subblocks in the order of the upper block, left block, and right block adjacent to the current subblock; the derived motion information may be used in this order to generate a second prediction block; A weighted sum of the first prediction block and the second prediction block may be calculated. When the weighted sums are calculated in a predetermined order, the weighted sums may be accumulated in the above-mentioned order, and thus a final prediction block of the current block may be derived.

As in the example of FIG. 23 , a weighted sum of a first predicted block and a second predicted block generated using motion information of an upper block is calculated so that a first weighted sum resultant block may be generated. Then, a weighted sum of the first weighted-sum result block and a second prediction block generated using the motion information of the left block may be calculated so that the second weighted-sum result block may be generated. Then, a weighted sum of the second weighted sum result block and the second prediction block generated using the motion information of the lower block may be calculated so that a third weighted sum result block may be generated. Finally, a weighted sum of the third weighted sum result block and the second predicted block generated using the motion information of the right block may be calculated so that a final predicted block may be generated.

On the other hand, the order of deriving the motion information for generating the second prediction block and the order of the second prediction block for calculating the weighted sum of the first prediction block and the second prediction block may be different.

FIG. 24 is a diagram illustrating an embodiment of calculating a weighted sum of a first prediction block and a second prediction block during overlapping block motion compensation. In order to improve coding efficiency, when the weighted sum is calculated, the weighted sum is not sequentially and cumulatively calculated, but the first prediction block can be calculated regardless of the order in which the second prediction block is generated using the upper block, left block , the weighted sum of the second prediction block generated by the motion information of at least one of the lower block and the right block.

In this case, weighting factors for the second prediction block generated using motion information of at least one of the upper block, the left block, the lower block, and the right block may be equal to each other. Optionally, the weighting factor for the second prediction block and the weighting factor for the first prediction block may be equal.

Referring to FIG. 24 , a plurality of recording spaces corresponding to the total number of the first prediction block and the second prediction block are prepared, and when generating the final prediction block, the second prediction block may be calculated using equal weighting factors for all second prediction blocks. A weighted sum of a prediction block and each second prediction block.

In addition, even for a second prediction block generated using motion information of a co-located sub-block within a co-located picture, a weighted sum of the first prediction block and the second prediction block can be calculated.

When the size of the current block is K samples or less, information for determining whether to perform overlapping block motion compensation on the current block may be entropy encoded/decoded. Here, K may be a positive integer, such as 256.

When the size of the current block is larger than K samples or when the current block is to be predicted in a specific inter prediction mode (e.g., merge mode or advanced motion vector prediction mode), it is optional to determine whether to perform overlapping block motion compensation on the current block Entropy encoding/entropy decoding of the information is performed, but overlapping block motion compensation can be substantially performed.

The encoder may perform prediction after subtracting the second prediction block from the original signal of the boundary area of the current block when performing motion prediction. In this case, when the second prediction block is subtracted from the original signal, a weighted sum of the second prediction block and the original signal may be calculated.

Enhanced Multiple Transform (EMT), in which Discrete Cosine Transform (DCT) and Discrete Sine Transform (DST) are applied to the vertical/ Horizontal transformation. That is, enhanced multiple transform may be applied only to a current block on which motion compensation of overlapping blocks is performed.

FIG. 25 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

Referring to FIG. 25 , a first prediction block of the current block may be generated using motion information of the current block (step S2510).

Next, at least one second prediction block of the current subblock may be generated using motion information of at least one neighboring subblock of the current subblock (step S2520). The adjacent sub-blocks may include adjacent sub-blocks of the sub-blocks of the co-located block temporally corresponding to the current block.

When the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, motion information of at least one adjacent sub-block of the sub-block of the co-located block may be used to generate at least one the second prediction block.

When the current subblock is neither included in the left boundary area nor in the upper boundary area of the current block, the current subblock included in at least one of the merge list and the motion vector list may be used motion information to generate a second prediction block.

Only when the current subblock is included in at least one of the left boundary area and the upper boundary area of the current block, at least one second prediction block may be generated using motion information of at least one neighboring subblock of the current subblock.

When the current subblock is included in the left neighboring region of the current block, at least one the second prediction block. When the current sub-block is included in the upper boundary region of the current block, at least one first adjacent sub-block may be generated using motion information of at least one of an upper adjacent sub-block, an upper-left adjacent sub-block, and an upper-right adjacent sub-block of the current sub-block. Two prediction blocks.

In addition, when the current subblock is neither included in the left boundary area nor in the upper boundary area of the current block, the upper adjacent subblock, the left adjacent subblock, the lower adjacent subblock of the current subblock may be used. The motion information of at least one of the adjacent sub-block, the right adjacent sub-block, the upper-left adjacent sub-block, the lower-left adjacent sub-block, the upper-right adjacent sub-block, and the lower-right adjacent sub-block is used to generate at least one second prediction block.

Motion information of neighboring subblocks of the current subblock may be derived in a predetermined order, and at least one second prediction block may be generated using the derived at least one piece of motion information.

Next, a final prediction block of the current block may be generated based on a weighted sum of the first prediction block of the current block and at least one second prediction block of the current sub-block (step S2530).

In this case, the calculated weight factor can be calculated by applying different weighting factors to the samples in the first prediction block and the samples in the second prediction block according to the position of the neighboring sub-blocks used to generate the second prediction block. The above weighted sum.

In the step S2530 of generating the final prediction block, when multiple second prediction blocks are generated for the current sub-block, it can be generated entirely by the weighted sum of the first prediction block of the current block and all second prediction blocks of the current sub-block The final predicted block of the current block.

Each step of the image decoding method of FIG. 25 may be substantially the same as each step of the image encoding method according to the present invention.

A bit stream generated by the image encoding method according to the present invention can be stored in a recording medium.

The above embodiments can be implemented in the same way in the encoder and decoder.

The order applied to the above embodiments may be different between an encoder and a decoder, or the order applied to the above embodiments may be the same in an encoder and a decoder.

The above embodiments may be performed on each of a luminance signal and a chrominance signal, or may be performed identically on a luminance signal and a chrominance signal.

The block shape to which the above embodiments of the present invention are applied may have a square shape or a non-square shape.

The above embodiments of the present invention may be applied according to a size of at least one of a coding block, a prediction block, a transformation block, a block, a current block, a coding unit, a prediction unit, a transformation unit, a unit, and a current unit. Here, the size may be defined as a minimum size or a maximum size or both such that the above embodiments are applied, or may be defined as a fixed size to which the above embodiments are applied. In addition, in the above embodiments, the first embodiment can be applied to the first size, and the second embodiment can be applied to the second size. In other words, the above embodiments may be applied in combination according to size. In addition, the above embodiments are applicable when the size is equal to or larger than the minimum size and equal to or smaller than the maximum size. In other words, the above embodiments are applicable when the block size is included within a certain range.

For example, when the size of the current block is 8x8 or larger, the above embodiments are applicable. For example, when the size of the current block is 4x4 or larger, the above embodiments are applicable. For example, when the size of the current block is 16×16 or more, the above embodiments are applicable. For example, when the size of the current block is equal to or greater than 16×16 and equal to or less than 64×64, the above embodiments are applicable.

The above embodiments of the present invention can be applied according to temporal layers. In order to identify a temporal layer to which the above embodiments are applicable, a corresponding identifier may be signaled, and the above embodiment may be applied to a specific temporal layer identified by the corresponding identifier. Here, the identifier may be defined as the lowest layer or the highest layer or both the lowest layer and the highest layer to which the above embodiments are applied, or may be defined as indicating a specific layer to which the embodiments are applied. In addition, fixed time horizons to which the embodiments are applied may be defined.

For example, when the temporal layer of the current image is the lowest layer, the above embodiments are applicable. For example, when the temporal layer identifier of the current image is 1, the above embodiments are applicable. For example, when the temporal layer of the current image is the highest layer, the above embodiments are applicable.

A stripe type to which the above embodiments of the present invention are applied may be defined, and the above embodiments may be applied according to the corresponding stripe type.

In the above-mentioned embodiments, the method has been described based on a flowchart having a series of steps or units, but the present invention is not limited to the order of the steps, but some steps may be performed simultaneously with other steps, or may be combined with other steps. The steps are performed in a different order. Furthermore, those of ordinary skill in the art should understand that the steps in the flowchart are not mutually exclusive, and that other steps may be added to the flowchart, or some steps may be copied from the flowchart, without affecting the scope of the present invention. delete.

Embodiments include various aspects of examples. All possible combinations for the various aspects may not be described, but those skilled in the art will be able to recognize different combinations. Accordingly, the present invention may include all alternatives, modifications and changes within the scope of the claims.

Embodiments of the present invention may be implemented in the form of program instructions, which may be executed by various computer components and recorded on a computer-readable recording medium. A computer-readable recording medium may include program instructions, data files, data structures, etc. alone, or a combination of program instructions, data files, data structures, and the like. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or known to those of ordinary skill in the field of computer software technology. Examples of computer-readable recording media include: magnetic recording media (such as hard disks, floppy disks, and magnetic tapes); optical data storage media (such as CD-ROM or DVD-ROM); magneto-optical media (such as floppy disks); and Hardware devices (such as read only memory (ROM), random access memory (RAM), flash memory, etc.) for storing and implementing program instructions. Examples of program instructions include not only machine language codes formed by a compiler but also high-level language codes that can be implemented by a computer using an interpreter. A hardware device may be configured to be operated by one or more software modules for processing according to the present invention, and vice versa.

Although the present invention has been described in terms of specific terms such as detailed elements and limited embodiments and drawings, they are provided only to help understand the present invention more generally and the present invention is not limited to the above-described embodiments. Those skilled in the art to which the invention pertains will appreciate that various modifications and changes can be made from the above description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the full scope of the appended claims and their equivalents will fall within the scope and spirit of the present invention.

industrial availability

The present invention can be applied to an apparatus for encoding/decoding images.

Claims (20)

1. A method for decoding an image, the method comprising: generating a first prediction block of the current block using motion information of the current block; generating at least one second prediction block of the current subblock of the current block using motion information of at least one neighboring subblock of the current subblock of the current block; and generating a final prediction block based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current sub-block; wherein the first predicted block is generated using the first merging candidate in the merging candidate list for the current block, said at least one second predicted block is generated using a second merge candidate in said merge candidate list for the current block, and The first merge candidate and the second merge candidate are different from each other. 2. The method of claim 1, wherein the at least one neighboring sub-block comprises a neighboring sub-block of a sub-block of a co-located block corresponding in time to the current block. 3. The method according to claim 2, wherein the step of generating at least one second prediction block comprises: When the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the motion information of at least one adjacent sub-block of the sub-block of the co-located block to The at least one second prediction block is generated. 4. The method according to claim 1, wherein the step of generating at least one second prediction block comprises: used in at least one of the merge candidate list and the motion vector list of the current block when the current subblock is neither included in the left border area nor in the upper border area of the current block The included motion information is used to generate the at least one second prediction block. 5. The method according to claim 1, wherein the step of generating at least one second prediction block comprises: The at least one second prediction block is generated using motion information of at least one neighboring subblock only when the current subblock is included in at least one of a left boundary region and an upper boundary region of the current block. 6. The method according to claim 1, wherein the step of generating at least one second prediction block comprises: When the current subblock is included in the left boundary area of the current block, using the motion of at least one of the left adjacent subblock, the upper left adjacent subblock, and the lower left adjacent subblock among the adjacent subblocks of the current subblock information to generate the at least one second prediction block; and When the current subblock is included in the upper boundary area of the current block, motion information of at least one of an upper adjacent subblock, an upper left adjacent subblock, and an upper right adjacent subblock among adjacent subblocks of the current subblock is used to generate the at least one second prediction block. 7. The method according to claim 1, wherein the step of generating at least one second prediction block comprises: When the current subblock is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, the upper adjacent subblock, the left adjacent subblock, and the lower adjacent subblock of the current subblock are used , the right adjacent sub-block, the upper left adjacent sub-block, the lower left adjacent sub-block, the lower right adjacent sub-block and the upper right adjacent sub-block to generate the at least one second prediction block. 8. The method according to claim 1, wherein the step of generating at least one second prediction block comprises: deriving motion information of at least one neighboring sub-block of the current sub-block in a predetermined order of the neighboring sub-blocks; and The at least one second prediction block is generated using at least one piece of derived motion information. 9. The method of claim 1, wherein the step of generating the final prediction block comprises: The first is obtained when different weighting factors are applied to samples within the first predictive block and the at least one second predictive block according to the positions of neighboring sub-blocks used to generate the at least one second predictive block. a weighted sum of the prediction block and the at least one second prediction block, Wherein, the weighted sum is only applied to a partial area of the current block. 10. The method according to claim 1, wherein, in the step of generating the final prediction block, when the number of the at least one second prediction block of the current sub-block is 2 or more, by calculating the current block as a whole A weighted sum of the first prediction block of the current sub-block and the at least one second prediction block of the current sub-block to generate a final prediction block. 11. A method for encoding an image, the method comprising: generating a first prediction block of the current block using motion information of the current block; generating at least one second prediction block of the current subblock of the current block using motion information of at least one neighboring subblock of the current subblock of the current block; and generating a final prediction block based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current sub-block, wherein a weight pair is selected for said weighted sum from among a plurality of weight pairs, the first weight of the selected pair of weights is applied to each sample of the first prediction block, and The second weight of the selected pair of weights is applied to each sample of the second prediction block. 12. The method of claim 11, wherein the at least one neighboring sub-block comprises a neighboring sub-block of a sub-block of a co-located block corresponding in time to the current block. 13. The method according to claim 12, wherein the step of generating at least one second prediction block comprises: When the current sub-block is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, using the motion information of at least one adjacent sub-block of the sub-block of the co-located block to The at least one second prediction block is generated. 14. The method according to claim 11, wherein the step of generating at least one second prediction block comprises: When the current sub-block is neither included in the left boundary area nor in the upper boundary area of the current block, use the one included in at least one of the merge list and the motion vector list of the current block motion information to generate the at least one second prediction block. 15. The method according to claim 11, wherein the step of generating at least one second prediction block comprises: The at least one second prediction block is generated using motion information of at least one neighboring subblock only when the current subblock is included in at least one of a left boundary region and an upper boundary region of the current block. 16. The method according to claim 11, wherein the step of generating at least one second prediction block comprises: When the current subblock is included in the left boundary area of the current block, using the motion of at least one of the left adjacent subblock, the upper left adjacent subblock, and the lower left adjacent subblock among the adjacent subblocks of the current subblock information to generate the at least one second prediction block; and When the current subblock is included in the upper boundary area of the current block, motion information of at least one of an upper adjacent subblock, an upper left adjacent subblock, and an upper right adjacent subblock among adjacent subblocks of the current subblock is used to generate the at least one second prediction block. 17. The method according to claim 11, wherein the step of generating at least one second prediction block comprises: When the current subblock is neither included in the left boundary area of the current block nor in the upper boundary area of the current block, the upper adjacent subblock, the left adjacent subblock, and the lower adjacent subblock of the current subblock are used , the right adjacent sub-block, the upper left adjacent sub-block, the lower left adjacent sub-block, the lower right adjacent sub-block and the upper right adjacent sub-block to generate the at least one second prediction block. 18. The method of claim 11 , wherein the step of generating the final prediction block comprises: The first is obtained when different weighting factors are applied to samples within the first predictive block and the at least one second predictive block according to the positions of neighboring sub-blocks used to generate the at least one second predictive block. A weighted sum of the prediction block and the at least one second prediction block. 19. The method according to claim 11, wherein, in the step of generating the final prediction block, when the number of the at least one second prediction block of the current sub-block is 2 or more, by calculating the current block as a whole A weighted sum of the first prediction block of the current sub-block and the at least one second prediction block of the current sub-block to generate a final prediction block. 20. A readable recording medium storing a bitstream comprising computer executable code which, when executed by a processor of a video decoding device, causes the processor to perform the following steps: generating a first prediction block of the current block using motion information of the current block; generating at least one second prediction block of the current subblock of the current block using motion information of at least one neighboring subblock of the current subblock of the current block; and generating a final prediction block based on a weighted sum of the first prediction block of the current block and the at least one second prediction block of the current sub-block, Wherein, the first merge index in the computer-executable code indicates the first merge candidate in the merge candidate list for the current block, the first predicted block is generated using a first merging candidate in said merging candidate list for the current block, a second merge index in the computer-executable code indicates a second merge candidate in the list of merge candidates for the current block, said at least one second predicted block is generated using a second merge candidate in said merge candidate list for the current block, and The first merge candidate and the second merge candidate are different from each other.